[llvm] [X86] Fix miscompile of fptosi.sat.iN.f16 for NaN under avx512fp16 (PR #210556)

[Phoebe Wang via llvm-commits]

================
@@ -0,0 +1,217 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512fp16 | FileCheck %s
+
+define i16 @fptosi_sat_i16_f16_nan(half %x) nounwind {
+; CHECK-LABEL: fptosi_sat_i16_f16_nan:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vcvttsh2si %xmm0, %eax
+; CHECK-NEXT:    vmovsh {{.*#+}} xmm1 = [-3.2768E+4,0.0E+0,0.0E+0,0.0E+0,0.0E+0,0.0E+0,0.0E+0,0.0E+0]
+; CHECK-NEXT:    vucomish %xmm0, %xmm1
+; CHECK-NEXT:    movl $32768, %ecx # imm = 0x8000
+; CHECK-NEXT:    cmovbel %eax, %ecx
+; CHECK-NEXT:    vucomish {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-NEXT:    movl $32767, %eax # imm = 0x7FFF
+; CHECK-NEXT:    cmovbel %ecx, %eax
+; CHECK-NEXT:    # kill: def $ax killed $ax killed $eax
+; CHECK-NEXT:    retq
+  %r = call i16 @llvm.fptosi.sat.i16.f16(half %x)
+  ret i16 %r
+}
+declare i16 @llvm.fptosi.sat.i16.f16(half)
+
+define i16 @fptosi_sat_i16_f16_qnan() nounwind {
+; CHECK-LABEL: fptosi_sat_i16_f16_qnan:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vmovsh {{.*#+}} xmm0 = [NaN,0.0E+0,0.0E+0,0.0E+0,0.0E+0,0.0E+0,0.0E+0,0.0E+0]
+; CHECK-NEXT:    vcvttsh2si %xmm0, %eax
+; CHECK-NEXT:    vmovsh {{.*#+}} xmm1 = [-3.2768E+4,0.0E+0,0.0E+0,0.0E+0,0.0E+0,0.0E+0,0.0E+0,0.0E+0]
+; CHECK-NEXT:    vucomish %xmm0, %xmm1
+; CHECK-NEXT:    movl $32768, %ecx # imm = 0x8000
+; CHECK-NEXT:    cmovbel %eax, %ecx
+; CHECK-NEXT:    vucomish {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-NEXT:    movl $32767, %eax # imm = 0x7FFF
+; CHECK-NEXT:    cmovbel %ecx, %eax
+; CHECK-NEXT:    # kill: def $ax killed $ax killed $eax
+; CHECK-NEXT:    retq
+  %r = call i16 @llvm.fptosi.sat.i16.f16(half 0xH7E00)
----------------
phoebewang wrote:

I don't think constant qnan/snan inputs matter here. The codegen is the same as variables.

https://github.com/llvm/llvm-project/pull/210556