[llvm] [ConstantRange] Bail out early in unsignedMulMayOverflow for wide integer types (PR #192275)

[via llvm-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-llvm-ir

Author: Firebear (Firebear518)

<details>
<summary>Changes</summary>

Problem             
                                                                                                                                                       
  ConstantRange::unsignedMulMayOverflow calls APInt::umul_ov on the range bounds to determine overflow. APInt multiplication is O(n²) in the word      
  count, so integer types wider than ~1024 bits cause multi-second hangs in InstCombine.                                                               
                                                                                                                                                       
  Minimal reproducer:                                                                                                                                  
                                                               
  define i5754496 @<!-- -->test(i5754496 %arg) {                                                                                                               
    %add1 = add i5754496 %arg, %arg                            
    %add2 = add i5754496 %arg, %add1
    ret i5754496 %add2                                                                                                                                 
  }
                                                                                                                                                       
  # Before this patch                                          
  $ time opt -passes=instcombine repro.ll -o /dev/null
  real    0m30.2s
                                                                                                                                                       
  # After this patch
  $ time opt -passes=instcombine repro.ll -o /dev/null                                                                                                 
  real    0m0.5s                                               

  Discovered via llvm-opt-fuzzer.

  Solution

  Return a conservative MayOverflow early when getBitWidth() > 1024. MayOverflow is always a valid upper bound — callers that receive it simply forgo  
  adding nuw to the folded instruction, which is safe.
                                                                                                                                                       
  The threshold 1024 covers all integer types that appear in real-world IR (≤512 bit) and matches the practical limit above which APInt word-count
  growth causes quadratic cost.

  The sibling functions (unsignedAddMayOverflow, signedAddMayOverflow, unsignedSubMayOverflow, signedSubMayOverflow) rely only on O(n) APInt           
  addition/subtraction and do not require this guard.
                                                                                                                                                       
  Testing                                                      

  [ RUN ] ConstantRangeTest.UnsignedMulOverflowExhaustive      OK (182 ms)                                                                             
  [ RUN ] ConstantRangeTest.UnsignedMulOverflowWideTypeBailout OK (0 ms)
                                                                                                                                                       
  UnsignedMulOverflowWideTypeBailout is a new regression test that completes in 0 ms with the fix and would hang >30 s without it.                     
                                                                                                                                                       
  cc @<!-- -->nikic @<!-- -->RKSimon @<!-- -->LebedevRI

---
Full diff: https://github.com/llvm/llvm-project/pull/192275.diff


2 Files Affected:

- (modified) llvm/lib/IR/ConstantRange.cpp (+6) 
- (modified) llvm/unittests/IR/ConstantRangeTest.cpp (+23) 


``````````diff
diff --git a/llvm/lib/IR/ConstantRange.cpp b/llvm/lib/IR/ConstantRange.cpp
index 4c10dfb140cbe..9e12ac0c03f3e 100644
--- a/llvm/lib/IR/ConstantRange.cpp
+++ b/llvm/lib/IR/ConstantRange.cpp
@@ -2261,6 +2261,12 @@ ConstantRange::OverflowResult ConstantRange::unsignedMulMayOverflow(
   if (isEmptySet() || Other.isEmptySet())
     return OverflowResult::MayOverflow;
 
+  // Bail out for very wide integer types: APInt multiplication is O(n^2) in
+  // the number of words, so types wider than ~1024 bits can cause multi-second
+  // hangs. Return a conservative result instead.
+  if (getBitWidth() > 1024)
+    return OverflowResult::MayOverflow;
+
   APInt Min = getUnsignedMin(), Max = getUnsignedMax();
   APInt OtherMin = Other.getUnsignedMin(), OtherMax = Other.getUnsignedMax();
   bool Overflow;
diff --git a/llvm/unittests/IR/ConstantRangeTest.cpp b/llvm/unittests/IR/ConstantRangeTest.cpp
index 13712a76d3edf..8c774d522b122 100644
--- a/llvm/unittests/IR/ConstantRangeTest.cpp
+++ b/llvm/unittests/IR/ConstantRangeTest.cpp
@@ -2506,6 +2506,29 @@ TEST_F(ConstantRangeTest, UnsignedMulOverflowExhaustive) {
       });
 }
 
+// Regression test: very wide integer types (>1024 bits) triggered an O(n^2)
+// APInt multiplication inside unsignedMulMayOverflow via APInt::umul_ov,
+// causing multi-second hangs in InstCombine (discovered via llvm-opt-fuzzer
+// with i5754496). The fix returns a conservative MayOverflow early for bit
+// widths above 1024.
+TEST_F(ConstantRangeTest, UnsignedMulOverflowWideTypeBailout) {
+  // 2048 bits is above the 1024-bit bailout threshold.
+  const unsigned W = 2048;
+  ConstantRange WFull = ConstantRange::getFull(W);
+  ConstantRange WOne(APInt(W, 1));
+  ConstantRange WMax(APInt::getMaxValue(W));
+  ConstantRange WEmpty = ConstantRange::getEmpty(W);
+
+  // For types wider than 1024 bits all non-empty pairs return MayOverflow
+  // (conservative but always correct).
+  EXPECT_MAY_OVERFLOW(WFull.unsignedMulMayOverflow(WFull));
+  EXPECT_MAY_OVERFLOW(WOne.unsignedMulMayOverflow(WMax));
+  EXPECT_MAY_OVERFLOW(WMax.unsignedMulMayOverflow(WOne));
+  // Empty-set cases are still handled before the width check.
+  EXPECT_MAY_OVERFLOW(WEmpty.unsignedMulMayOverflow(WFull));
+  EXPECT_MAY_OVERFLOW(WFull.unsignedMulMayOverflow(WEmpty));
+}
+
 TEST_F(ConstantRangeTest, SignedAddOverflowExhaustive) {
   TestOverflowExhaustive(
       [](bool &IsOverflowHigh, const APInt &N1, const APInt &N2) {

``````````

</details>


https://github.com/llvm/llvm-project/pull/192275