[llvm-bugs] [Bug 45006] New: GHC calling convention: Function prologue frame pointer invalidates the %rbp register

[via llvm-bugs]

https://bugs.llvm.org/show_bug.cgi?id=45006

            Bug ID: 45006
           Summary: GHC calling convention: Function prologue frame
                    pointer invalidates the %rbp register
           Product: libraries
           Version: 9.0
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P
         Component: Backend: X86
          Assignee: unassignedbugs at nondot.org
          Reporter: mail at daniel-mendler.de
                CC: craig.topper at gmail.com, llvm-bugs at lists.llvm.org,
                    llvm-dev at redking.me.uk, spatel+llvm at rotateright.com

Created attachment 23165
  --> https://bugs.llvm.org/attachment.cgi?id=23165&action=edit
Includes avx/rt.bc, avx/rt.s, no-avx/rt.bc, no-avx/rt.s

I am observing a bug on x86-64 using the GHC calling convention (cc 10), which
only
occurs if AVX is enabled.

The register %rbp is reserved by the GHC calling convention, however LLVM
creates
a frame pointer in the prologue, invalidating the %rbp register if AVX is
enabled.
The %rbp register is only restored in the epilogue of the function.

This bug has been observed previously in GHC. However it seems the bug was only
triggered
by GHC with LLVM 3.2. The bug is still present in LLVM 9, which I am using
right now.
Note that I am not using GHC for code generation, but another compiler, which
produces similar code
and relies on the GHC calling convention.
See https://gitlab.haskell.org/ghc/ghc/issues/9391 for the original bug report
on the GHC bugtracker.

I am appending the following test files:

avx/rt.bc
avx/rt.s generated via llc-9 from avx/rt.bc
no-avx/rt.bc
no-avx/rt.s generated via llc-9 from avx/rt.bc

The miscompiled function is interpInitModuleCont:

~~~
define internal ghccc i32 @interpInitModuleCont(%struct._ChiRegStore_* noalias,
i64* noalias, i64* noalias, i64* noalias, i64, i64, i64, i64, i64, i64)
~~~

Compiled with AVX enabled:

~~~
interpInitModuleCont:
        pushq   %rbp
        movq    %rsp, %rbp <~~~ %rbp is invalidated
        ...
        movq    %rbp, %rsp
        popq    %rbp       <~~~ %rbp is restored
        vzeroupper
        jmpq    *%rax                   # TAILCALL
~~~

If AVX is disabled using -mno-avx in the frontend, the invalid
prologue/epilogue is not generated, the %rbp register is not invalidated
and things work as expected. The resulting program runs fine.
I did not test other architectures supporting the GHC calling convention,
but I suspect that the bug is x86 specific due the logic in
X86FrameLowering::hasFP.

$ llc-9 -version
LLVM (http://llvm.org/):
  LLVM version 9.0.1

  Optimized build.
  Default target: x86_64-pc-linux-gnu
  Host CPU: skylake

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20200224/98b7e1b1/attachment.html>