[PATCH] D48283: [SCEV] Properly solve quadratic equations

[Krzysztof Parzyszek via Phabricator via llvm-commits]

kparzysz added a comment.

My approach here, in a single sentence, is to solve the quadratic equation q(n) = 0 using the formula for real numbers.  Doing modulo arithmetic is tricky and it's easy to make a false assumption. For example: in modulo arithmetic there is no linear order, or if someone insists on an order, then we lose n+1>n for each n (so it will have different properties from what holds in R or Z). Additionally, a square root will now have multiple values: in modulo 16, 2^2 = 4, but also 14^2 = 4, because -2 is now "positive" as 14. Because of that I wanted to move away from the modulo arithmetic, and instead operate on integers.

I get the "simulated" integers by extending everything to a bit width that will be sufficient for all required operations.  Let's assume for a moment that we're not interested in inequalities, but only in the exact solutions to q(n) = 0.  In integers there can only be 0, 1 or 2 such solutions, but with the modulo arithmetic that we started with there can be infinitely many solutions.  Those extra solutions show up when we have something like 16*16 = 0 (mod 256).  So instead of solving q(n) = 0 we have to find all solutions to q(n) = k * 2^BW and pick the smallest non-negative one.  A lot of the code in SolveQuadraticEquation tries to figure out how to rewrite q(n) = k * 2^BW as q(n) - p = 0, so we can still solve q'(n) = 0 for some updated q'.  It does that by exploiting the properties of a quadratic function: decreasing before the vertex, increasing afterwards, assuming positive coefficient at n^2.  It handles the case where we start with some, let's say positive value, then the value decreases (but remains positive), then goes back up and hits 2^BW eventually. I'll get back to this case later on.

There is still that momentary assumption that we only solve for exact n for which q(n) = 0, but the analysis that this is a part of is also interested in inequalities. The inequality here has the form of q(n) > 0 or q(n) < 0 (or greater-than-or-equal, etc), since the addrec gets shifted by the values at the ends of the range.  The interesting solution here is the first step where the values change from negative to positive, or from positive to negative.  In the original (modulo) arithmetic this can happen when the values cross the 2^BW boundary (any multiple of it, including 0), or when the values cross the signed overflow (i.e. an odd multiple of 2^(BW-1)).  At the time of solving the equation we don't really know which one is the correct one, but since we already have results for k * 2^BW, we can stick with that.  This is where the signed overflow is ignored.  The returned value may not be the correct solution, but since it is validated by the callers, it is ok to return an "educated guess".

Back to the case with decreasing values---in modulo arithmetic adding -1 would actually be adding 2^BW-1.  That would always be an overflow, so strictly speaking we'd have to stop the first time it happens. The problem is that this is rarely the right thing to do.  To avoid that, the coefficients are treated as signed numbers.


Repository:
  rL LLVM

https://reviews.llvm.org/D48283