[llvm] [ConstantRange] Bail out early in unsignedMulMayOverflow for wide integer types (PR #192275)

[via llvm-commits]

https://github.com/Firebear518 created https://github.com/llvm/llvm-project/pull/192275

Problem             
                                                                                                                                                       
  ConstantRange::unsignedMulMayOverflow calls APInt::umul_ov on the range bounds to determine overflow. APInt multiplication is O(n²) in the word      
  count, so integer types wider than ~1024 bits cause multi-second hangs in InstCombine.                                                               
                                                                                                                                                       
  Minimal reproducer:                                                                                                                                  
                                                               
  define i5754496 @test(i5754496 %arg) {                                                                                                               
    %add1 = add i5754496 %arg, %arg                            
    %add2 = add i5754496 %arg, %add1
    ret i5754496 %add2                                                                                                                                 
  }
                                                                                                                                                       
  # Before this patch                                          
  $ time opt -passes=instcombine repro.ll -o /dev/null
  real    0m30.2s
                                                                                                                                                       
  # After this patch
  $ time opt -passes=instcombine repro.ll -o /dev/null                                                                                                 
  real    0m0.5s                                               

  Discovered via llvm-opt-fuzzer.

  Solution

  Return a conservative MayOverflow early when getBitWidth() > 1024. MayOverflow is always a valid upper bound — callers that receive it simply forgo  
  adding nuw to the folded instruction, which is safe.
                                                                                                                                                       
  The threshold 1024 covers all integer types that appear in real-world IR (≤512 bit) and matches the practical limit above which APInt word-count
  growth causes quadratic cost.

  The sibling functions (unsignedAddMayOverflow, signedAddMayOverflow, unsignedSubMayOverflow, signedSubMayOverflow) rely only on O(n) APInt           
  addition/subtraction and do not require this guard.
                                                                                                                                                       
  Testing                                                      

  [ RUN ] ConstantRangeTest.UnsignedMulOverflowExhaustive      OK (182 ms)                                                                             
  [ RUN ] ConstantRangeTest.UnsignedMulOverflowWideTypeBailout OK (0 ms)
                                                                                                                                                       
  UnsignedMulOverflowWideTypeBailout is a new regression test that completes in 0 ms with the fix and would hang >30 s without it.                     
                                                                                                                                                       
  cc @nikic @RKSimon @LebedevRI

>From 6ac4714876cbbd68ac841c699f4ac607e5fa988a Mon Sep 17 00:00:00 2001
From: Firebear518 <phj040518 at gmail.com>
Date: Thu, 16 Apr 2026 00:15:11 +0900
Subject: [PATCH] [ConstantRange] Bail out early in unsignedMulMayOverflow for
 wide integer types

ConstantRange::unsignedMulMayOverflow calls APInt::umul_ov on the range
bounds to determine overflow. APInt multiplication is O(n^2) in the word
count, so integer types wider than ~1024 bits cause multi-second hangs in
InstCombine.

For example, an IR function with i5754496 arguments and two add instructions
causes opt to hang for 30+ seconds via the add->mul fold path:

  add i5754496 %x, %x  ->  mul i5754496 %x, 2
    -> visitMul -> willNotOverflowUnsignedMul
      -> ConstantRange::unsignedMulMayOverflow
        -> APInt::umul_ov  (89914 words, ~4 billion word ops)

Fix: return a conservative MayOverflow early when getBitWidth() > 1024.
MayOverflow is always a valid upper bound; callers that receive it simply
forgo adding 'nuw' to the folded instruction, which is safe.

The sibling functions (unsignedAddMayOverflow, signedSubMayOverflow, etc.)
are unaffected because APInt addition/subtraction is O(n), not O(n^2).

Add a regression test UnsignedMulOverflowWideTypeBailout that completes in
0 ms with the fix (and would hang >30 s without it).

Discovered via llvm-opt-fuzzer.
---
 llvm/lib/IR/ConstantRange.cpp           |  6 ++++++
 llvm/unittests/IR/ConstantRangeTest.cpp | 23 +++++++++++++++++++++++
 2 files changed, 29 insertions(+)

diff --git a/llvm/lib/IR/ConstantRange.cpp b/llvm/lib/IR/ConstantRange.cpp
index 4c10dfb140cbe..9e12ac0c03f3e 100644
--- a/llvm/lib/IR/ConstantRange.cpp
+++ b/llvm/lib/IR/ConstantRange.cpp
@@ -2261,6 +2261,12 @@ ConstantRange::OverflowResult ConstantRange::unsignedMulMayOverflow(
   if (isEmptySet() || Other.isEmptySet())
     return OverflowResult::MayOverflow;
 
+  // Bail out for very wide integer types: APInt multiplication is O(n^2) in
+  // the number of words, so types wider than ~1024 bits can cause multi-second
+  // hangs. Return a conservative result instead.
+  if (getBitWidth() > 1024)
+    return OverflowResult::MayOverflow;
+
   APInt Min = getUnsignedMin(), Max = getUnsignedMax();
   APInt OtherMin = Other.getUnsignedMin(), OtherMax = Other.getUnsignedMax();
   bool Overflow;
diff --git a/llvm/unittests/IR/ConstantRangeTest.cpp b/llvm/unittests/IR/ConstantRangeTest.cpp
index 13712a76d3edf..8c774d522b122 100644
--- a/llvm/unittests/IR/ConstantRangeTest.cpp
+++ b/llvm/unittests/IR/ConstantRangeTest.cpp
@@ -2506,6 +2506,29 @@ TEST_F(ConstantRangeTest, UnsignedMulOverflowExhaustive) {
       });
 }
 
+// Regression test: very wide integer types (>1024 bits) triggered an O(n^2)
+// APInt multiplication inside unsignedMulMayOverflow via APInt::umul_ov,
+// causing multi-second hangs in InstCombine (discovered via llvm-opt-fuzzer
+// with i5754496). The fix returns a conservative MayOverflow early for bit
+// widths above 1024.
+TEST_F(ConstantRangeTest, UnsignedMulOverflowWideTypeBailout) {
+  // 2048 bits is above the 1024-bit bailout threshold.
+  const unsigned W = 2048;
+  ConstantRange WFull = ConstantRange::getFull(W);
+  ConstantRange WOne(APInt(W, 1));
+  ConstantRange WMax(APInt::getMaxValue(W));
+  ConstantRange WEmpty = ConstantRange::getEmpty(W);
+
+  // For types wider than 1024 bits all non-empty pairs return MayOverflow
+  // (conservative but always correct).
+  EXPECT_MAY_OVERFLOW(WFull.unsignedMulMayOverflow(WFull));
+  EXPECT_MAY_OVERFLOW(WOne.unsignedMulMayOverflow(WMax));
+  EXPECT_MAY_OVERFLOW(WMax.unsignedMulMayOverflow(WOne));
+  // Empty-set cases are still handled before the width check.
+  EXPECT_MAY_OVERFLOW(WEmpty.unsignedMulMayOverflow(WFull));
+  EXPECT_MAY_OVERFLOW(WFull.unsignedMulMayOverflow(WEmpty));
+}
+
 TEST_F(ConstantRangeTest, SignedAddOverflowExhaustive) {
   TestOverflowExhaustive(
       [](bool &IsOverflowHigh, const APInt &N1, const APInt &N2) {