[llvm] [X86][SATCVT] Reduce MIN/MAXSS/D by conversion instruction result (PR #136471)

[Nikita Popov via llvm-commits]

================
@@ -21851,6 +21851,15 @@ X86TargetLowering::LowerFP_TO_INT_SAT(SDValue Op, SelectionDAG &DAG) const {
   assert(SatWidth <= DstWidth && SatWidth <= TmpWidth &&
          "Expected saturation width smaller than result width");
 
+  // AVX512 provides VCVTSS/D2USI which return INT_MAX/LONG_MAX when overflow
+  // happens. X86ISD::FMAX makes sure negative value and NaN return 0.
+  if (Subtarget.hasAVX512() && !IsSigned && SatWidth == DstWidth &&
+      (DstVT == MVT::i32 || (Subtarget.is64Bit() && DstVT == MVT::i64))) {
+    SDValue MinFloatNode = DAG.getConstantFP(0.0, dl, SrcVT);
+    SDValue Clamped = DAG.getNode(X86ISD::FMAX, dl, SrcVT, Src, MinFloatNode);
+    return DAG.getNode(ISD::FP_TO_UINT, dl, DstVT, Clamped);
----------------
nikic wrote:

If this is relying on properties of the ultimately selected instruction, doesn't this need a custom ISD node?

https://github.com/llvm/llvm-project/pull/136471