[llvm] [AMDGPU] Be more careful about using expandDivRem24 (PR #201186)

[via llvm-commits]

================
@@ -1066,9 +1070,24 @@ Value *AMDGPUCodeGenPrepareImpl::expandDivRem24(IRBuilder<> &Builder,
                                                 BinaryOperator &I, Value *Num,
                                                 Value *Den, bool IsDiv,
                                                 bool IsSigned) const {
-  unsigned DivBits = getDivNumBits(I, Num, Den, 24, IsSigned);
-  if (DivBits > 24)
+  unsigned DivBits;
+  unsigned LHSBits;
+  if (!getDivNumBits(I, Num, Den, 24, IsSigned, DivBits, LHSBits) ||
+      DivBits > 24)
+    return nullptr;
+
+  // v_rcp_f32(float(X)) can have an error of 1 ulp.
+  // This can cause expandDivRem24Impl to sometimes calculate Y/X incorrectly
+  // when Y has bit 23 (i.e. 0x800000) set.
+  // For example,
+  // (0xbf2758/0xbf2759) erroneously produces 1 instead of 0.
+  // (0xe3170d/0x000c32) erroneously produces 4767 instead of 4766.
+  //
+  // Note that for IsSigned, abs(Y) is at most
+  // 0x7fffff so it cannot hit this issue.
+  if (!IsSigned && LHSBits == 24)
----------------
LU-JOHN wrote:

> Isn't this just off by one error? Shouldn't the existing check be 23 bits instead of 24? The mantissa is 23 bits.

Mantissa field of FP32 is 23-bits, but the leading 1 is implicit.  FP32 can represent any integer from [-0x800000:0x800000]  with perfect accuracy.  Thus, FP32 can represent any signed-24 or unsigned-24 with perfect accuracy which is why the original check was 24.  This PR is limiting the transformation to signed-24 and unsigned-23 because any numerator with abs(NUM)>0x800000 can expose a 1ulp accuracy issue with v_rcp_f32.

https://github.com/llvm/llvm-project/pull/201186