[LLVMbugs] [Bug 12127] New: llvm can inline a large function into a "small" recursive one

[bugzilla-daemon at llvm.org]

http://llvm.org/bugs/show_bug.cgi?id=12127

             Bug #: 12127
           Summary: llvm can inline a large function into a "small"
                    recursive one
           Product: libraries
           Version: trunk
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: enhancement
          Priority: P
         Component: Interprocedural Optimizations
        AssignedTo: unassignedbugs at nondot.org
        ReportedBy: rafael.espindola at gmail.com
                CC: llvmbugs at cs.uiuc.edu
    Classification: Unclassified


Created attachment 8111
  --> http://llvm.org/bugs/attachment.cgi?id=8111
testcase

This is a reduction from the firefox js compiler running out of stack space
when compiled with clang. The attached patch has two important functions.
EmitSwitch  is static and uses a lot of stack space. EmitTree calls EmitSwitch
and is recursive.

Compiling this test with gcc 4.6.2 produces a file where EmitTree has

sub    $0x78,%rsp

and EmitSwitch has

sub    $0x8d8,%rsp

Compiling with clang we inline EmitSwitch in EmitTree and get a function with

0x8c8,%rsp

While we do a better job than gcc at saving stack space inside a function
(0x8c8 X 0x8d8 + 0x78), the fact that EmitTree is recursive and now uses a lot
more stack causes firefox's js compiler to run out of stack.

-- 
Configure bugmail: http://llvm.org/bugs/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.