[llvm-bugs] [Bug 41123] New: X86: Underflow error in stack probe emitting

[via llvm-bugs]

https://bugs.llvm.org/show_bug.cgi?id=41123

            Bug ID: 41123
           Summary: X86: Underflow error in stack probe emitting
           Product: libraries
           Version: trunk
          Hardware: All
                OS: All
            Status: NEW
          Severity: normal
          Priority: P
         Component: Backend: X86
          Assignee: unassignedbugs at nondot.org
          Reporter: CFSworks at gmail.com
                CC: craig.topper at gmail.com, llvm-bugs at lists.llvm.org,
                    llvm-dev at redking.me.uk, spatel+llvm at rotateright.com

In X86FrameLowering::emitPrologue, there's some logic that checks if NumBytes
(the number of bytes that need to be reserved on the stack for the local stack
frame) exceeds some "stack probe size" and, if so, emits a stack probe call.

The stack probe call expects to find the size of stack space to be probed in
EAX/RAX. If that register is already alive, it's first pushed to the stack, and
the effect on ESP/RSP from that extra push is negated from NumBytes.

This means that if NumBytes is 0, the value passed to the stack probe is
actually an underflowed value, which the stack probe may interpret as an
extremely large number, causing a stack overflow at runtime.

Normally, this isn't a problem - the default "stack probe size" is 4096 bytes,
meaning NumBytes must already be sufficiently large that this subtraction
cannot underflow. However, if the user (or, rather, the project the user is
trying to compile) overrides the stack probe size to 0 ("emit probes
unconditionally"), then the underflow may occur and produce an invalid stack
probe call.

I haven't checked if this bug exists in any other code generators; but I found
this one in the wild whilst trying to run an LLVM-produced X86 program.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20190318/eae6b3ea/attachment-0001.html>