[LLVMbugs] [Bug 9943] New: x86 sibcall logic mis-understands 'byval'

[bugzilla-daemon at llvm.org]

http://llvm.org/bugs/show_bug.cgi?id=9943

           Summary: x86 sibcall logic mis-understands 'byval'
           Product: new-bugs
           Version: unspecified
          Platform: PC
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P
         Component: new bugs
        AssignedTo: unassignedbugs at nondot.org
        ReportedBy: stuart at apple.com
                CC: llvmbugs at cs.uiuc.edu


A not-yet-applied patch to the x86_64 byval logic will teach it to pass tiny
byval structs in registers as required by the x86_64 ABI.  This has exposed a
misconception in the x86 sibcall logic; it thinks that 'byval' means "passed in
memory."  The upshot is that an incorrect sibcall can be generated if one of
the parameters is a byval struct that is passed in registers.

This is fairly academic, as all existing front-ends I know of will only use
byval on large structs that will never be passed in registers.  However, it's
possible to write a legal LLVM testcase that exposes this flaw.

Here is such a test case:

; rdar://r7717598                                                               
%struct.ns = type { i32, i32 }
%struct.cp = type { float, float }

define %struct.ns* @t13(%struct.cp* %yy) nounwind ssp {
; 32: t13:                                                                      
; 32-NOT: jmp                                                                   
; 32: calll                                                                     
; 32: ret                                                                       

; 64: t13:                                                                      
; 64-NOT: jmp                                                                   
; 64: callq                                                                     
; 64: ret                                                                       
entry:
  %0 = tail call fastcc %struct.ns* @foo7(%struct.cp* byval align 4 %yy, i8
signext 0) nounwind
  ret %struct.ns* %0
}

This was extracted from test/CodeGen/X86/sibcall.ll.

Before I apply my byval patch, I will commit a change to the original test
case; I'll make %struct.cp larger, so it will always be passed in memory.  (The
original version is above.)

Once this flaw in the sibcall logic is fixed, we can revert my workaround patch
to sibcall.ll.

-- 
Configure bugmail: http://llvm.org/bugs/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.