[llvm] [ConstantRange] Bail out early in unsignedMulMayOverflow for wide integer types (PR #192275)

[via llvm-commits]

https://github.com/Firebear518 updated https://github.com/llvm/llvm-project/pull/192275

>From 6ac4714876cbbd68ac841c699f4ac607e5fa988a Mon Sep 17 00:00:00 2001
From: Firebear518 <phj040518 at gmail.com>
Date: Thu, 16 Apr 2026 00:15:11 +0900
Subject: [PATCH] [ConstantRange] Bail out early in unsignedMulMayOverflow for
 wide integer types

ConstantRange::unsignedMulMayOverflow calls APInt::umul_ov on the range
bounds to determine overflow. APInt multiplication is O(n^2) in the word
count, so integer types wider than ~1024 bits cause multi-second hangs in
InstCombine.

For example, an IR function with i5754496 arguments and two add instructions
causes opt to hang for 30+ seconds via the add->mul fold path:

  add i5754496 %x, %x  ->  mul i5754496 %x, 2
    -> visitMul -> willNotOverflowUnsignedMul
      -> ConstantRange::unsignedMulMayOverflow
        -> APInt::umul_ov  (89914 words, ~4 billion word ops)

Fix: return a conservative MayOverflow early when getBitWidth() > 1024.
MayOverflow is always a valid upper bound; callers that receive it simply
forgo adding 'nuw' to the folded instruction, which is safe.

The sibling functions (unsignedAddMayOverflow, signedSubMayOverflow, etc.)
are unaffected because APInt addition/subtraction is O(n), not O(n^2).

Add a regression test UnsignedMulOverflowWideTypeBailout that completes in
0 ms with the fix (and would hang >30 s without it).

Discovered via llvm-opt-fuzzer.
---
 llvm/lib/IR/ConstantRange.cpp           |  6 ++++++
 llvm/unittests/IR/ConstantRangeTest.cpp | 23 +++++++++++++++++++++++
 2 files changed, 29 insertions(+)

diff --git a/llvm/lib/IR/ConstantRange.cpp b/llvm/lib/IR/ConstantRange.cpp
index 4c10dfb140cbe..9e12ac0c03f3e 100644
--- a/llvm/lib/IR/ConstantRange.cpp
+++ b/llvm/lib/IR/ConstantRange.cpp
@@ -2261,6 +2261,12 @@ ConstantRange::OverflowResult ConstantRange::unsignedMulMayOverflow(
   if (isEmptySet() || Other.isEmptySet())
     return OverflowResult::MayOverflow;
 
+  // Bail out for very wide integer types: APInt multiplication is O(n^2) in
+  // the number of words, so types wider than ~1024 bits can cause multi-second
+  // hangs. Return a conservative result instead.
+  if (getBitWidth() > 1024)
+    return OverflowResult::MayOverflow;
+
   APInt Min = getUnsignedMin(), Max = getUnsignedMax();
   APInt OtherMin = Other.getUnsignedMin(), OtherMax = Other.getUnsignedMax();
   bool Overflow;
diff --git a/llvm/unittests/IR/ConstantRangeTest.cpp b/llvm/unittests/IR/ConstantRangeTest.cpp
index 13712a76d3edf..8c774d522b122 100644
--- a/llvm/unittests/IR/ConstantRangeTest.cpp
+++ b/llvm/unittests/IR/ConstantRangeTest.cpp
@@ -2506,6 +2506,29 @@ TEST_F(ConstantRangeTest, UnsignedMulOverflowExhaustive) {
       });
 }
 
+// Regression test: very wide integer types (>1024 bits) triggered an O(n^2)
+// APInt multiplication inside unsignedMulMayOverflow via APInt::umul_ov,
+// causing multi-second hangs in InstCombine (discovered via llvm-opt-fuzzer
+// with i5754496). The fix returns a conservative MayOverflow early for bit
+// widths above 1024.
+TEST_F(ConstantRangeTest, UnsignedMulOverflowWideTypeBailout) {
+  // 2048 bits is above the 1024-bit bailout threshold.
+  const unsigned W = 2048;
+  ConstantRange WFull = ConstantRange::getFull(W);
+  ConstantRange WOne(APInt(W, 1));
+  ConstantRange WMax(APInt::getMaxValue(W));
+  ConstantRange WEmpty = ConstantRange::getEmpty(W);
+
+  // For types wider than 1024 bits all non-empty pairs return MayOverflow
+  // (conservative but always correct).
+  EXPECT_MAY_OVERFLOW(WFull.unsignedMulMayOverflow(WFull));
+  EXPECT_MAY_OVERFLOW(WOne.unsignedMulMayOverflow(WMax));
+  EXPECT_MAY_OVERFLOW(WMax.unsignedMulMayOverflow(WOne));
+  // Empty-set cases are still handled before the width check.
+  EXPECT_MAY_OVERFLOW(WEmpty.unsignedMulMayOverflow(WFull));
+  EXPECT_MAY_OVERFLOW(WFull.unsignedMulMayOverflow(WEmpty));
+}
+
 TEST_F(ConstantRangeTest, SignedAddOverflowExhaustive) {
   TestOverflowExhaustive(
       [](bool &IsOverflowHigh, const APInt &N1, const APInt &N2) {