<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/150988>150988</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            AArch64 misuse of zero register for exclusive store status (maybe latent in 20.x)
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            new issue
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          neldredge
      </td>
    </tr>
</table>

<pre>
    I found a bug in LLVM 19.x and earlier where incorrect code is generated for certain atomic operations on AArch64.  The incorrect code no longer manifests in 20.x, or at least I cannot find a test case; but this is due to an unrelated change, and I think the bug may still be latent.

Compile the following code with `clang -O -march=armv8-a -mno-outline-atomics foo.cc`:
```
void foo(volatile double *x) {
 __atomic_fetch_add(x, 42.0, __ATOMIC_RELAXED);
}
```
[Try on godbolt](https://godbolt.org/z/s4TYf45fE)

With 19.x and earlier, this expands to a compare-swap loop, where the compare-swap is implemented with an LDXR/STXR pair.  This is suboptimal but in principle would work correctly. However, the store side looks like this:
```
        stlxr   wzr, x8, [x0]
        cbnz    wzr, .LBB0_3
        b       .LBB0_1
```
The zero register `wzr` is used for the status operand of `stlxr`!  As such, `stxlr` is effectively treated as if it always succeeds (and the `cbnz` branch to retry on failure is never taken).  So if it does in fact fail, due to a concurrent store or an interrupt, this is ignored and the store never occurs at all.

If I understand the situation correctly, the whole sequence is initially treated as a [`CMP_SWAP_64` pseudo-instruction](https://github.com/llvm/llvm-project/blob/b2322772f2ab97de60db906a591353a5ef77cdfe/llvm/lib/Target/AArch64/AArch64InstrAtomics.td#L512), which has a "scratch" output corresponding to the status register.  Since this output is not used in any surrounding code, the [aarch64-dead-defs](https://github.com/llvm/llvm-project/blob/main/llvm/lib/Target/AArch64/AArch64DeadRegisterDefinitionsPass.cpp) pass identifies it as dead and assigns it `wzr`.  But the pseudo-instruction is later [expanded](https://github.com/llvm/llvm-project/blob/9c82f87aec12c77444edf04d46af4d8405becc25/llvm/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp#L235) into multiple instructions, which do write and read that register.

----

In 20.x, the implementation of floating-point atomics [changed](https://github.com/llvm/llvm-project/commit/26c3a8404f1b3327a0982faeeaee94b08d1ee481), such that on ARMv8.0-A they are now implemented with a more efficient direct LL/SC loop, which does not suffer this bug.  [Try on godbolt](https://godbolt.org/z/f7KqWcbsf).  Indeed based on [`AArch64TargetLowering::shouldExpandAtomicRMWInIR`](https://github.com/llvm/llvm-project/blob/5f20518f5b4734d01848dfe44d24aed195dc2043/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp#L28380), it looks like a compare-exchange loop would only be used in the following cases:

- `-O0` where the dead register definition pass is presumably not performed (?)
- LSE (ARMv8.1-A and later), where the `cas` instruction is available and would be used instead
- When the operation may lower to a library call?  If it expands to a call in the runtime library then presumably all is well.  But I'm not sure if that always happens?

As such, I haven't been able to reproduce the issue in 20.x.  But it doesn't seem like there have been any relevant changes to the dead register definition pass, nor to the `CMP_SWAP_64` pseudo-instruction, which makes me think this bug is still latent and could reappear if later changes invoke `CMP_SWAP_*` in other settings.  I also wonder if any other pseudo-instructions could be affected.
</pre>
<img width="1" height="1" alt="" src="http://email.email.llvm.org/o/eJykV1Fv2zrS_TXKy8CGREmW_OAHp2nwGZ-LW6TB9u5TQJEjixuK1CUpO-6vXwwlO0lbLHpvAyMyLHI4c-bMmSH3Xh0M4iYpb5Py7oaPobNuY1BLh_KAN42V580OWjsaCRya8QDKwH7_r0-QrZcvwI0E5E4rdHDq0CEoI6xzKAIIKxGUhwMadDyghNY6EOgCVwZ4sL0SYAd6p6zxYA1st050q2IJ8Nj9YMpY0NYc0EHPjWrRB0_OsHT5krAPYB3wABq5D7ADwY2xAVoV_Q7oAwjuMclvoRkDhE558k2OCMECNzAahzp6KTpuDkgmKbodrTXPEDqM4ff8DD4oraFBoA0mLJN0m6TbD7YflMa4srVa25Myh8n1kwodJKtUaG4OsPgDFj13okvyO-76Y73gsOiNXdgxaGVwMWHjobV2KUSySpOcDqAv0yfdHq0iPG3C6qPVPNDB0o6NRkjY9iVha0iq2yTdwtPTZO6pxSC6Jy5lwuqIWMGWKT2fnraPf3zafXh6-Ljf_vnxLmHrJKe9SXX33bFJefvozpSqg5WN1SEp7xJWdyEMnpxk9wm7n18trTsk7P5bwu598fjvtijbj2Q6gvWVEPmeQuRMzAy-DNxIH1MDwvYDd7jwJz6AtnagZRPZCOp3rymr_aCxR0OpjLhzA_u7Px8Sdv_l8c8HGLhykWATA_zY2CGonutIDGVgcMoINWiEkx21hJN1zzAzUZ-X8H_2hMeLswg-WIfglURy7tmDVs8Yw_gxazD_-aBfHACcvkUzLzX9T8rbl5TwfF0nGvMNXtct97e36VP-ZkEzP6c32XfHURF9Q2fB4UH5gI44SLZWKYU--rkmpzB4GP1Uj0aCbWlt9JNssQxgS2CJLrpKr170xRC2LYqgjqjPEBzGKuIeVAsqANcnfo5bBaL0kLCaDqAzqSIa842sNI4b0VHCHYaJYS1XenRRQgwBDoE_o0nYegnwxc7WpcWoAi0XIe4g9y5VDcIaMTqHJsxpIpEwoExA58YhXAlHn4OxjhyfnZs2TCdbIUbnSV-41nO571rYwWgkOh-ue1QYo5q98uVCk1NnNYLHv0Y0IgaljAqK6_eYcaJBsko_fPr89OXr9vPTqiB4Bo-jtAtlfHCjoBN-WngqdGOzFLZP2L3Wx8tjMTj7HxQhYfeNtg09WM5YVbGW8WZdSVylslmnK16us7zMeYltVQnZ4hs7ivY9cndAsjMr9eu3Hbm2nYRrGWTC8n2ZMar3WK1KdNBN8THmheOBmMTAjmEYwwSXH6yRpJnBvqXkhbyUdkXYxYzNG4kcNkxUpq5izuBH56hfXdT3koGkvOU8-rqQyOVCYut_D8WeK_PrCN0hlw9zLHfYxvRb4z9z75diGEiyB-49KIkmqFYRsQNxgryNtJzadfz5WshLgNvY0PAnJCF4qEc5Cn4SVZS_F_Na1KytK44iY6KqiqJA2aaFLFa8LWRdpGWDQrDy13H5GP36HJ0nFs1o5HuWlwSKMsFCP-oQNflNdP6VWtLCyamAESZHeIWOh1fqTCW7WCwWc_G-zg2E3LVnTMVrW2i15UGZw2KwygS4dOSkvJ3Gg3-KorB9r-gLW4mc10VatFmT56zi6bpmLUfkiOuiSWuZIRZ1NlcQKe8UE41JD5-O9TJdbMn5M3DSKXv6SeeDnjQM21YJRSIoVRyn9ntqhh_eNNMJQ5yKyY9tS3JLZdaMhyXAP2z7bfX_f30VjW8n0d4ZiSih4VSs1sxKN7Ng4sfentApcyCj-dZ31IAnfkzS8vDp687sHqgl_RaLy5alZVa3ZVNUeSHTrC5q2WJRSFZwlNm6lIKlRf439O8L6ov3VwLXeZ3OCVTh7XjwOtXgy0SomIx54rBGn2m8vKjadxMl93iZLdLtgqRg8Qd1_DdDUZSMa9eXV7WZFcbD4NCPPW_0OaZ8QNda16Ok9pzk99OctoD9l4_0y0S4bLGN9RUV5arslyOpmXMfJ4L3AsSPXGlOkyltniJ8Dc4H5DKe9bXDKdTrnSAO25pAndq5Vo3j7gyCa53k9wC7OAO8nxa51hfM3GiC6vG6L9AJbyKPSz2cUOtZRncJq_q5CGj0aKeam2eYjg8DGk_4ROzfTEQ76PiRppMqQINoIAYcx5nBWTmKCSTl_YiXW8t85jzFTHs9Yn-ZIAlZsjobNGdwqPHITZjvKP7SKf9nusk9Y91l7S8NF1dR6PkzeujxeguaNCFOzvEWNF2BYmpFTK1DQok7Am_qPRdnlTna5_cOJGw7MQYsxQseA6muJ7UArr2Fk6URi4wRANOqHx328-ENAo_DKMrljdzkcp2v-Q1usqrMK5ZmZXrTbWTdCMaqLM9WRcZZhWlbVW1WtBVjWYXZjdqwlJVpxepsxdZZuqwrRMlF3aZ8vc7bVVKk2HOllyQOJHg3MbGbrEzXdX2jeYPax0s1YwZPU9oTxuiO7TZRkprx4JMi1Yoa3tVMUEHjZpYU6JUfPVI7ej_F08yOL0KPXh2vF5BpWEpY3fPz9Wr6ekFe34xOb_62XkbPfcLu59COG_bfAAAA__9yPFx8">