[PATCH] Fix infinite recursion bug in SROA

[James Molloy]

Hi Chandler,

 

The attached patch fixes a bug that could cause SROA to go into infinite
recursion (stack overflow in debug mode, infinite loop in release mode).

 

The function findCommonType() attempts to use a type from a load/store inst,
and if there are non-load-store instructions or their types do not match it
falls back to trying to find an integral type that covers all uses.
Otherwise, it should return failure (NULL).

 

The mechanism to fall back to integral types does not reset the "winning"
type to NULL. This means that depending on the order in which the input uses
are iterated over changes the output of the function. If a non-load-store
instruction comes first, it will return NULL (as Ty is never set). If a
load/store inst comes first, it will return the Ty of that instruction.

 

There is no testcase as the bug depends on memory layout; I could not force
it to fail with opt/llc.

 

Is it OK to commit?

 

Cheers,

 

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140120/5aefa443/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sroa-null.diff
Type: application/octet-stream
Size: 734 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140120/5aefa443/attachment.obj>