[llvm] [LAA] Be more careful when evaluating AddRecs at symbolic max BTC. (PR #128061)

[Ramkumar Ramachandra via llvm-commits]

================
@@ -206,11 +206,31 @@ std::pair<const SCEV *, const SCEV *> llvm::getStartAndEndForAccess(
   const SCEV *ScStart;
   const SCEV *ScEnd;
 
+  auto &DL = Lp->getHeader()->getDataLayout();
+  Type *IdxTy = DL.getIndexType(PtrExpr->getType());
+  const SCEV *EltSizeSCEV = SE->getStoreSizeOfExpr(IdxTy, AccessTy);
   if (SE->isLoopInvariant(PtrExpr, Lp)) {
     ScStart = ScEnd = PtrExpr;
   } else if (auto *AR = dyn_cast<SCEVAddRecExpr>(PtrExpr)) {
     ScStart = AR->getStart();
-    ScEnd = AR->evaluateAtIteration(MaxBECount, *SE);
+    if (!isa<SCEVCouldNotCompute>(BTC))
----------------
artagnon wrote:

Okay, just thinking out loud here: for simplicity, let AR = `{0, +, 1}` and let SymbolicMax BTC = INT_MAX. Then, we compute `AddExpr(0, MulExpr(1, INT_MAX))`. I don't think this overflows. Now, let AR = `{0, + 2}`. Then, we compute `AddExpr(0, MulExpr(2, BinomialCoefficient(INT_MAX, 2))` where the binomial coefficient evaluates to `INT_MAX * (INT_MAX - 1) / 2`. Naively doing this would overflow even for BTC equal to `sqrt(INT_MAX)`, but it looks like `BinomialCoefficient` is written carefully, although the final result is truncated (?). In conclusion, it looks like the problem is that `evaluateAtIteration` does not wrap, but rather truncates the result?

https://github.com/llvm/llvm-project/pull/128061