<table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Issue</th>
<td>
<a href=https://github.com/llvm/llvm-project/issues/60972>60972</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>
[LLC] Tail-call optimization on arm64 macOS generates incorrect spill & reload
</td>
</tr>
<tr>
<th>Labels</th>
<td>
new issue
</td>
</tr>
<tr>
<th>Assignees</th>
<td>
</td>
</tr>
<tr>
<th>Reporter</th>
<td>
Baltoli
</td>
</tr>
</table>
<pre>
# Description
I maintain a language runtime that relies on guaranteed tail call optimizations provided by `llc`. On arm64 Macs, a seemingly correct program that executed successfully on other platforms (X86 Linux & macOS) exhibited segmentation faults due to steps executed earlier in the program[^1].
The issue can be narrowed down - I believe - to incorrect stack-spilling code when calling a function with more than 8 arguments. I have minimized the issue to two LLVM programs that differ only in the number of arguments passed to a tail-called `fastcc` function (8 / 9 respectively). When passing 8 arguments, the program executes correctly, but segfaults for 9.
With reference to the (heavily minimized) LLVM code and reproduction steps below, I observe the following snippets of assembly code being generated for the failing case:
```asm
_foo: ; @foo
add sp, sp, #16
ret
_baz:
... ; abbreviated prelude
bl _foo
str x0, [sp, #8] ; 8-byte Folded Spill
sub sp, sp, #16
bl _bar
ldr x0, [sp, #8] ; 8-byte Folded Reload
... ; abbreviated postlude
```
Here, the return value of `_foo` is being spilled to the stack at address `sp + 8`, which the compiler then expects to reload into `x0` after the call to `_bar`. **However**, the stack adjustment to undo the change made by `_foo` is between the spill and reload, meaning that the address `sp + 8` is different for each.
In the successful program (8 args) below, `_foo` and `_baz` do not perform any stack adjustment, but the spill and reload offset assumptions are the same:
```asm
_foo: ; @foo
ret
_baz:
... ; abbreviated prelude
bl _foo
str x0, [sp, #8] ; 8-byte Folded Spill
bl _bar
ldr x0, [sp, #8] ; 8-byte Folded Reload
... ; abbreviated postlude
```
Please let me know if there is any more information it would be helpful for me to provide.
# Reproduction
The problem can be reproduced with the LLVM files and shell commands below. My machine is an M1 Mac Mini (full system information below), but I believe the issue can be reproduced on any arm64 Mac.
## Instructions
```console
$ llc -O0 -tailcallopt bad.ll -o bad.s
$ clang bad.s -o bad
$ ./bad
[1] 68198 segmentation fault ./bad
$ echo $?
139
```
```console
$ llc -O0 -tailcallopt good.ll -o good.s
$ clang good.s -o good
$ ./good
$ echo $?
0
```
## Code
### Failing: `bad.ll`
```llvm
target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
target triple = "arm64-apple-macosx12.0.0"
define void @bar() {
ret void
}
define fastcc i64 @foo(i64 %0, i64 %1, i64 %2, i64 %3, i64 %4, i64 %5, i64 %6, i64 %7, i64 %8) {
ret i64 %0
}
define fastcc i64 @baz() {
entry:
%0 = tail call fastcc i64 @foo(i64 0, i64 0, i64 0, i64 0, i64 0, i64 0, i64 0, i64 0, i64 0)
call void @bar()
ret i64 %0
}
define i32 @main() {
entry:
%0 = call i64 @baz()
%1 = trunc i64 %0 to i32
ret i32 %1
}
```
### Successful: `good.ll`
```llvm
target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
target triple = "arm64-apple-macosx12.0.0"
define void @bar() {
ret void
}
define fastcc i64 @foo(i64 %0, i64 %1, i64 %2, i64 %3, i64 %4, i64 %5, i64 %6, i64 %7) {
ret i64 %0
}
define fastcc i64 @baz() {
entry:
%0 = tail call fastcc i64 @foo(i64 0, i64 0, i64 0, i64 0, i64 0, i64 0, i64 0, i64 0)
call void @bar()
ret i64 %0
}
define i32 @main() {
entry:
%0 = call i64 @baz()
%1 = trunc i64 %0 to i32
ret i32 %1
}
```
## Environment
<details>
<summary>System Information</summary>
```console
$ system_profiler SPSoftwareDataType SPHardwareDataType
Software:
System Software Overview:
System Version: macOS 12.3 (21E230)
Kernel Version: Darwin 21.4.0
Boot Volume: Macintosh HD
Boot Mode: Normal
Computer Name: 44634
User Name: administrator (administrator)
Secure Virtual Memory: Enabled
System Integrity Protection: Enabled
Time since boot: 38 days 17:45
Hardware:
Hardware Overview:
Model Name: Mac mini
Model Identifier: Macmini9,1
Chip: Apple M1
Total Number of Cores: 8 (4 performance and 4 efficiency)
Memory: 8 GB
System Firmware Version: 7459.121.3
OS Loader Version: 7459.101.2
Serial Number (system): H2WDQ1K6Q6NV
Hardware UUID: 92B3F9F2-F010-5367-8CAF-7716809951EE
Provisioning UDID: 00008103-001908282440291E
Activation Lock Status: Disabled
```
</details>
<details>
<summary>LLVM Versions</summary>
```console
$ clang --version
Homebrew clang version 15.0.7
Target: arm64-apple-darwin21.4.0
Thread model: posix
InstalledDir: /opt/homebrew/opt/llvm/bin
$ llc --version
Homebrew LLVM version 15.0.7
Optimized build.
Default target: arm64-apple-darwin21.4.0
Host CPU: apple-m1
Registered Targets:
aarch64 - AArch64 (little endian)
aarch64_32 - AArch64 (little endian ILP32)
aarch64_be - AArch64 (big endian)
amdgcn - AMD GCN GPUs
arm - ARM
arm64 - ARM64 (little endian)
arm64_32 - ARM64 (little endian ILP32)
armeb - ARM (big endian)
avr - Atmel AVR Microcontroller
bpf - BPF (host endian)
bpfeb - BPF (big endian)
bpfel - BPF (little endian)
hexagon - Hexagon
lanai - Lanai
mips - MIPS (32-bit big endian)
mips64 - MIPS (64-bit big endian)
mips64el - MIPS (64-bit little endian)
mipsel - MIPS (32-bit little endian)
msp430 - MSP430 [experimental]
nvptx - NVIDIA PTX 32-bit
nvptx64 - NVIDIA PTX 64-bit
ppc32 - PowerPC 32
ppc32le - PowerPC 32 LE
ppc64 - PowerPC 64
ppc64le - PowerPC 64 LE
r600 - AMD GPUs HD2XXX-HD6XXX
riscv32 - 32-bit RISC-V
riscv64 - 64-bit RISC-V
sparc - Sparc
sparcel - Sparc LE
sparcv9 - Sparc V9
systemz - SystemZ
thumb - Thumb
thumbeb - Thumb (big endian)
ve - VE
wasm32 - WebAssembly 32-bit
wasm64 - WebAssembly 64-bit
x86 - 32-bit X86: Pentium-Pro and above
x86-64 - 64-bit X86: EM64T and AMD64
xcore - XCore
```
</details>
<details>
<summary>Homebrew</summary>
```console
$ brew info llvm
==> Downloading https://formulae.brew.sh/api/cask.json
######################################################################## 100.0%
==> llvm: stable 15.0.7 (bottled), HEAD [keg-only]
Next-gen compiler infrastructure
https://llvm.org/
/opt/homebrew/Cellar/llvm/15.0.7_1 (6,411 files, 1.3GB)
Poured from bottle using the formulae.brew.sh API on 2023-02-17 at 11:24:42
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/llvm.rb
License: Apache-2.0 with LLVM-exception
==> Dependencies
Build: cmake ✔, swig ✘
Required: python@3.11 ✘, six ✔, z3 ✔, zstd ✔
==> Options
--HEAD
Install HEAD version
==> Caveats
To use the bundled libc++ please add the following LDFLAGS:
LDFLAGS="-L/opt/homebrew/opt/llvm/lib/c++ -Wl,-rpath,/opt/homebrew/opt/llvm/lib/c++"
llvm is keg-only, which means it was not symlinked into /opt/homebrew,
because macOS already provides this software and installing another version in
parallel can cause all kinds of trouble.
If you need to have llvm first in your PATH, run:
echo 'export PATH="/opt/homebrew/opt/llvm/bin:$PATH"' >> ~/.zshrc
For compilers to find llvm you may need to set:
export LDFLAGS="-L/opt/homebrew/opt/llvm/lib"
export CPPFLAGS="-I/opt/homebrew/opt/llvm/include"
==> Analytics
install: 34,000 (30 days), 133,673 (90 days), 518,755 (365 days)
install-on-request: 29,851 (30 days), 115,410 (90 days), 396,557 (365 days)
build-error: 163 (30 days)
```
</details>
[^1]: Concretely, a function in our language can be observed to return a correct / expected value, but when the result is stored in a data structure and later retrieved, it no longer points to a valid value, and the program segfaults in library code whose invariants have been violated.
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJzsOk1z27iSv4a5dFFFgvo85CBL1tj17EQvdjKpvaRAsiVhAgJ8AChbOexv32qAlCjZyWTm7exh9rlcKpLobvQ3GkBza8VWIb6NRlfRaPmGN26nzdsrLp2W4k2uy8PbiGWwRFsYUTuhVZQso2Qefm-h4kI5LhRwkFxtG75FMI1yokJwO-7AoBRoQSvYNtxw5RBLcFxIKLiUoGsnKvGNE2ULtdF7UWIJ-QGicSJlEY2TAbxXwE01HsI9L2zEFsDBIlZCbeUBCm0MFo5wt4ZXYVZ8xqJxWIJtigKt3TRSHogJ7XZooJbcbbSpLERs-nk6hjuhmmeI2BgqXrx_iNgM8HkncuFp4LZC5TyPsOGNdBbKBsFpsA5re5oNuZECDQgFbocdS16312k0Wg76ynvcIQhrG4SCK8gRFDdGP2EJpX5SEMMt5KS8PUJMcwnViWodL77GthZSCrWFQpcITztUXqX0hcOmUYXn90m4HVTaeHMomAI324bEsQO4hR3fI1RCkQ3ILkeWnAb3pOHu7tN9J4YNqi3FZoMGtJKHTk7VVDl92pyIQ82tJYoauDd3TLxhSWbdcOsKsuyJy4hNpxCxFczAoK2xcGKP8hCx2QB-JcmIHEnW458coaflzgq28whCX0DeODJga7aNNjA7s8KvpB-DGzSoiiD3DomfHfK9kIeTdsgpvD68vrkqwWBtdNkEEYIr5Cj1E817Czq3aPbo6W20lPqJBLBK1DU667VlLVa59-ESIUca36JCw8mZiFePy0UwM7cYZS3b0TgJ_9xW4cuXjdZRNoff_4uyK4iGCcG3ipjxsoySma2J8_AbsSwdH8cNur7SvuT825GVjuxgMPiJycP0PM8N7oWXszYomxLPieUySmZf-ixaZ6Jk9px45kZXRy6n0Wj5vWmmcX5wCCstKak8UMCc6DX5D0UOHOTcHL_I8t_l4ANKzcv_JbVp6056O_pD30w3aLCLEoOuMQr2XDZIvheNE6_ecQLCtr7nM0oIWkLxaQa4A16WBq0lHFtDxK5gSjOxBTztRLHzwIWuaiHRu6wCfKYYtkTJeJlBKKeJwDMxCXzjAmhYBcKQVzbl-4jNIza_0U-4RxNeOjFansrfGusoCxBqo8rAcLHjaotQcYqmwwsR3RNiSFhe0DaEvUXYAirkipTgkxwBvS41kQopkGanGEVe7M5Sym07yXHpOaYon-a42VrKJcdU0eOTWAqa-EavpQalHdRoaLUCrg4vFNDluNfEAr3ZWHSUZ5qqDkssNyEjWV79Zdnk_yxbHMP0r0wUf4NEsJbILYJEBxXCV6WfQGzIDQyt996vfIkgFPlZKHSEgyfdyJIqkx3KmvyY3L3yi2RbqZ35PRWKH3pL4mWxUxudS6y6cqdbPbEMRQp5pV9fN0Ki9W5sdyglpZaKq7JdXAdwf6AqbSdUyzzcp1QZwr1QgkKMij2wB-uwOpOoDbhZFzKn-spdlmI93rTy-jkWoJcik9S3yjoThLZnw50xCq2slp2J2BCkLCB-n0BMtRHlQF07yHk5kBJi7Z_sCbqg2jp8bEdPY4OIrU4fRldp64TjaTqbvlK7wgUGGwIWOw0RG0bZKnxMs9kPvOkPSrXVuhPLP76QK3ztxs8loy_wA06TH_EZjLPQp3Rx-kwjq1BZUZ6LxknQ_vdklXLfpkbHzRYdlNxxyQ-6cRBlS4gYw7iKsrmOxXgYZfPxMBYpm0bZPGXTWGUsfHugb4ydkXJG1BI7Mt7TYl7XEuOKF9o-p2yQDJIjVvgtcUMRsNeipPxL-YkozyCaXHWJxaDzAC3eZPkKgVCNgxgPuzTOpv6FjXyCa5_T3jPrPWe952HvedR7HveeJ73n19g9Tv3TLNPqciE5KmcOvSWHCHr1njae3xP7KPO__zDrpvcTvjDUH5VaZIzwacf90_L6mS_11INKg1ZMo4ojD36vmbEz9mhq8oEL9r4fdRReD8cCqI2wNhP8J8T-8hD7G4bV_-dogmu1F0YrX_D3R7NFiaR8G2XXR0ayhW2qipMM1w-hDLo9lUFRtojY6gTxcyt7KKe-1EZv_C7vYf2gN-6JG1xyxx8PNcLD-oabsv8poHeAp93Gsextuesg4P0ezV7g0yugR-BPaKwXYx7O6iBlg4zKPpZes6zvKPT3DzQKZR9pyc2TUMDSwXCQ9EGvtHbwScvGb4yo1KM9q93BzfIF2D3VFNkc3pFWZX94oau6oc3tu7DBguFwnA37EB9tb5SXlVDCOsOdNiTF2YcLYR6waAzCJ2FcwyXcY6W9o8K14rnE8hVt3SqHWyPcAdZGOwx1-esYj6JCsEIVCLnWjqCyKZT8YCGdRNl8ODo7W2ht_YqpuqEfm5NUKI96oPqdBH8JcVuicmIj0LRwBEY1fHqm9p2oaXxOOR3uz8YeteMS3h0PKRfaoCXgKSl82G2vOUlOu44h4GYjCoGqOFxY4KTyKfxy9Yq-V8JUXvaex02Go9kgZekg6yO8f4A7zUs0L0GTdMDO7W7ESYKITUM0Em_ZHG7Yr8t_pv8Y_3P87lMf62iGjx9vlwQ4Y1fZarZi8SpJk3iUjSfxdDFfxZNJOp4ms9kovb7uE1jTHo84E2oLH5eBSJIkyTRNsjhJ0lkyZVM2HCZslp5hzgsn9mHPcaeLr_DguGu8ypfC9hzvMuX51HSZ0X421fm9Y6tL--eyXNiPxPG-tUjwdF1hbvCpHW3HIB0NksGk3d36IsPHc6-yKH2m6Seax51BXkJFjk3QtbbiOQzRFtKfkC-Fd_SIrXTtIrbatdMfP_gyia1yoeBi5_Udvr1iXmMb4H24fsES8kbIctB9X2LYK7qfFAzgRlsHi_VHDxsKq_Q86D_gVliHBksI-rJnh0Kcm2I3HtJjDPN5eInYVArnJAKqUnB1Fo4txpeM_QADbu_WGXsVL8dzvFxsX52mKreFgpax-yX8sngHv6w_2h6IqbrTmRjmH-7PRoJM7cjvykTwJNH34V-TyFSY92b5kTR702PVVShh_ukD3IvC6EIrZ7SUaE7web05wV-tV_5qhGz9Cu283pzYaGG_wwaBygvQ72tlh898q1WAvgkvp1HJFRcdrTt6OY1VorZH9u9v1w80UcbiXDj4DmuE0trshDIe_j6KF-gFyvelIqxWCS94-wGWrYdZ0mE9rOklGl3hc41G-LMeGY16FYva1-6508C7T7fL2zmsHz9DmOgCrou_HlwQ4wRX14V3Tw-31k9o1gvI2AWAxEsAuLs-gzmFRQczHl4AXBIZD8-ImHGSnFyZAnP90cLNkn3-_Dm-WY4_f_7cAxa22AfG41Z2-HD7sIg_XcB0KmjNdwlja26KbtIHerkYCxZtx8749cP7WX_4U8-wYVH_1g77l_86jbpdUx1j65FeLsZC5LVjP4i8PcIpoD_12HvitmotG8OvmM-728lLTyHAY4D0AS9d5Xk6Pk3V6vzzdExLxJqKuqaK10b7movneo9nmPFxitYSLeb1_Xj46HHm98u-yzwX2mA322cq8v6CIqNbVP9cfeGXY6E2Gk5nHVG29P_XsNRPSmpeUrm1c6726yNbRWxF5WkjOQ6IwMDuIrbitYjYquD26-A3ezxr785e_mb_kCb-dGZ0qTKvxmwO1lFV2RY33vk15c-yPe2_uZ4vKUl-xW2slTwcE-Q7fHbxFtXp-lKojeHhNL_pPOjcGDTnQJstvbVaf1mqLVBKbk7VWuDsS-oXhogthmka7jmIvXSQ_XLVC9K1bqhK2hhdQRAEGhvuJhEufQHm61vQCljCsjhhcToB7iBNo2zOhrRta1PzymivqnNhtsLtmnxQaGLy5sR_J0pMQUXVptQ5QVzPlxFbrQIPnTJMm4vuRIHKNynAvObFDmM2SMLlDtWgMT4X2O8hOnk-1qhKVIXAtqq6onKUCBUV_4oQXbNotohm_vjLPont8cs0wH_AfzXCoEepD26nVTRMskGa9gAJVTyf0_qWXbxbV_a-XPD5vu5d8cSx10Yb-LO2gA--dlaIn_AXfI_ctfiPGhob7p7yRpUSS5AiLyJ2FbErqMOVHS_LiwaSu-Xqbv7LQ692Pn5ZRozFd7-7c5CCTNlNFP8qI7aITc3dLmKLP4Z9cWBKICAsHOPs2CJQIVfW3yly6--07aGSQn3FrjPg5bSLQDPHgpOewmEPl7SDOnT3jxbcTliw3QESLQsiGML3QanQ79VtfkRrkZob2mtJf90XyJPpvgpV-sYcZ3STy_PbzdsNHHQDCkODhO-c8vJuhLEOhKJhA-v54w2JbRrVM1F7bzXB51obF4C8uX5mm-eDdRgIs4hNgNab7Br-O2KrwTe760qQNs61OWYz34CxEaoMnBL_FT8cZbB-a3fkMfD2p7yJXRBZrNd9Kre_S0Wowt9hn_vTKXLmisuDE0UbO62N_RkVxW2SJL54Tvx5VZv10yyL2GI88aeDs7OhUUrpYDIaeazx6DjWpx5rFRv8V4PW74A95nSUvjJROvIpPXk5UTajbD8aTV6dyG-6YzRG-z1_Os4uiP_hAubYakj0FloVBh2GQOw1BQoF5KvHjs321rvtWCtD247vF-LH_sqIrdq2HixDG1F3j-5bD0OPkW2kowRgnTY-tIH7Wxw4rqg-RCV3aGgKI3CPvv1GOFAapFZbNFBroUL3EKepRG9CQu93_Z16-4Si_Gm4OXQdkdoiCLXnRnAi50M2R1SwF5pYKAdvyrdZOctm_A2-TceTCcvGk1nyZvd2Mx2W2Wg8GSab6aZIUjbEomTFJh3ysmDj4RvxlhbchLFhmibjLBmk5STNczbKJglm4yKNhglWXMhBVy-88R0Gb8fJbMLeSJ6jtL7tljFFVSENkvuPlm_MW8KJ82Zro2EihXX2RMUJJ32_7t3dIhot4bFrrjzrpfVtC_7sIeTNrq3Q9ttIfbtQxMZtu9Cbxsi3PygPuoAn3mqjf8OCAtgzbiO28oL9TwAAAP__Sa-P7Q">