[PATCH] D15857: [InstCombine] Defend against worst-case exponential execution time

[Joerg Sonnenberger via llvm-commits]

On Tue, Jan 05, 2016 at 11:17:30AM +0000, James Molloy wrote:
> Hi Joerg,
> 
> Actually on further investigation the simple solution of only visiting each
> OR once isn't going to work.
> 
> There's an "OR of ORs" byte-swap idiom that relies on having the same OR in
> both operands of another OR, the same pattern we're trying to defend
> against. See http://reviews.llvm.org/rL248482 .
> 
> So the only defence I can think of is what I've already implemented, or to
> memoize the tree as you suggested.
> 
> However, memoization seems to go against the spirit of instcombine - there
> are no other similar caches kept that I can see, and any member of the tree
> could be modified or optimized by instcombine in any iteration, so we'd
> have to trawl the tree on every iteration to see if it has changed.

I was thinking of doing it just per instruction, not global. E.g.
protect the current recursion only to keep the individual overhead
linear, making it worst case of quadratic. If that is too much, it might
be a good idea to make it a separate pass?

Joerg