[LLVMdev] Stack overflow in Legalize Op

[Villmow, Micah]

This isn't running in a child process, but this is a win32 machine.

 

The problem I'm having is LegalizeOp is recursively calling itself 6
times, followed by 3 calls to ExpandEXTRACT_VECTOR_ELT, with each
LegalizeOp pushing 20KBof data onto the stack and Expand pushing 800B of
data on the stack. So that is 9 function calls pushing ~140KB of data
onto the stack. This set of 9 function calls repeats itself 25 times,
pushing almost 3.5 MB of data onto the stack.

 

This is occurring with v16i32 data type being converted to v4i32. If any
larger vector integer data type was used, the stack usage would probably
be even more extreme.

 

Micah

________________________________

From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu]
On Behalf Of Evan Cheng
Sent: Wednesday, March 11, 2009 9:46 AM
To: LLVM Developers Mailing List
Subject: Re: [LLVMdev] Stack overflow in Legalize Op

 

Are you running with restricted stack size, e.g. in a pthread process?

 

Evan

 

On Mar 10, 2009, at 5:16 PM, Villmow, Micah wrote:





I'm hitting an issue where legalizeOp is overflowing the stack. Are
there any recommended ways of getting around this?

 

The bitcode that causes this issue is attached. 

 

Thanks,

Micah Villmow

Systems Engineer

Advanced Technology & Performance

Advanced Micro Devices Inc.

S1-609 One AMD Place

Sunnyvale, CA. 94085

P: 408-749-3966

 

 

<transpose-opt.bc>_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev



 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20090311/02346f05/attachment.html>