<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/129748>129748</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            Missed optimizations with -fstack-protector-strong
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            new issue
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          travisdowns
      </td>
    </tr>
</table>

<pre>
    With -fstack-protector-strong, stack canary checks are even when addresses of local variables (or function parameters, etc) escape the function (or escape the fully inlined function). However, the optimization seems poor in this area: the stack checks are inserted even when the only writes to the stack on the hot path are provably-safe compiler spills.

For example, consider this case:

```
[[noreturn]] [[gnu::cold]]
void rare_function(const int& x, const int& y);

int hot_function(int x, int y) {
    if (x < y) [[unlikely]] {
 rare_function(x, y);
    }
    return x + y;
}
```

This is a reduced test case from a rich assert mechanism and this pattern is very common: we call `rare_function` very infrequently (in the case of assertions, at most once per-process invocation). `x` and `y` are passed in registers, so the hot path could simply be a comparison, jump to slow path and then return value. Instead we get:

```
hot_function(int, int):
  sub rsp, 24
  mov rax, qword ptr fs:[40]
  mov qword ptr [rsp + 16], rax
  mov dword ptr [rsp + 12], edi
  mov dword ptr [rsp + 8], esi
  cmp edi, esi
  jl .LBB0_1
  mov rax, qword ptr fs:[40]
  cmp rax, qword ptr [rsp + 16]
  jne .LBB0_5
  add esi, edi
  mov eax, esi
  add rsp, 24
  ret
.LBB0_1:
  mov rax, qword ptr fs:[40]
  cmp rax, qword ptr [rsp + 16]
  jne .LBB0_5
  lea rdi, [rsp + 12]
  lea rsi, [rsp + 8]
  call rare_function(int const&, int const&)@PLT
.LBB0_5:
  call __stack_chk_fail@PLT
```

Note that on the hot path we store the stack cookie, spill the register variables, then do the comparison with reigsters and immediately load + compare the cookie: but there are no user-controlled or dangerous writes to the stack here at all, only spills to known slots, which are statically known to be disjoint from the cookie location.

If instead `rare_function` takes its parameters by value, the whole function reduces to:

```
hot_function(int, int):
  cmp edi, esi
  jl .LBB0_2
  add esi, edi
  mov eax, esi
  ret
.LBB0_2:
 push rax
  call rare_function(int, int)@PLT
```

The hot path of the by-reference function could look the same!
</pre>
<img width="1" height="1" alt="" src="http://email.email.llvm.org/o/eJy0VtuK4zgTfhrlpkiw5TiHi1wk3TT_wPzLXgzsZaNI5VgdWfJIctLep19Kdg59GpiFhYZ2VF8dVIevJELQB4u4YeWOlY8T0cXa-U304qSDcmcbJnun-s1fOtYwrUIU8jhtvYsoo_PTEL2zB8YfIElACit8D7JGeQwgPAKe0MK5RgtCKY8hYABXgXFSGDgJr8XeYADGV85D1VkZtbPQCi8ajOgD2cYoGV8DBilahFjjDTjovZEY04O2RltUVxzj6xn8z53xhJ4sEtK1UTf6b5HsBMQmQOucB20h1jpFL1ixTdjxdrdraRvQR1R390s2renh7HXEANHdqbpBXrsIrYh1stF6dxJ700-DqBCka1pt0ENotTFhxrIty7ZPdLtX0bQGKW7pbNAK_RChFAFZsR2QbJGNf9k2FXNnncfYecvKR1Y-wnB4sB2pFFvpjBokLNuenFbghcfnW8ZW5CyCtpHxBbxe3F9PesbXrNgN3rWNdLl7dTpKWvRBYGBLQgMA6Ioq9wqseBhFKbjOGn1E018iHvDv40pGb97JHls-jl_DleEVGN9BP8aXpG8SlG1_UAapzOBRdRIVRAwx5RQq7xoSaFmDCFRpaFDWwurQgLBqSH8rYkRvycgJqetd0zhLLXNGkMIYYIvsbfCLbIBqW3n82aGNpoeUq9QeybmrRp_a2dT-IkLjQgRnJUKLnuZPYgig7clJce1vtsheyQMFyBZZn76pz8icor72eNDhMlXBvW1J6TqjIOimNT3sEURqSeF1IPsP8NI1LTV1MO48NnHKBNpL0k_CdDiDbzZEFIqycMD4aYN-bJWxUVJVt6mWoduDDy0J-DydNO4EXqTy_zw7r6CNHqpACuVung2tPMBuclbufGhTO-QLgvCHZOSCVJ8i-YhEpX-NXF2AYQDKpk1Kd0cvBmbfd7vsOf-dW5ChD7APl0n2LY4OyvRbKJV8vwsfB2OXqAj1Nr0eI8u2l1DHKvzn0RoU4IeEfSjAVR7ey1dXxzRn7xmCGCdxFeOLCwPdfq_ZPPvz-4_rXcvLXZOt5-fE2M-yPj5XQpsr-B1__OEibRwRP3D7mUjfebzfHM4ddWLwxO5JchnG2xYcN5MFNYzmbf7gTPvXoz6k8U2Tp5sGlRYRTQ_GCZXyMqjgqJ6cFlvYd5FOPCY-sA66gH4qnY3eGYMKnAcl7AG968Kn-2tQjiCMoSjTnhsWFeGO1p0tEUNMdzjXiTh9Uo6a0tqPmOiIWpQOL46Kkoj2Fmt6FlANx-33raJFm8jkMyqN4ogBdAx37wXY9wMPXdb8uXbm7sEwkD1F_a-J6Vcjzn9jAu_njY_G2y7UV3r6qrfvQvqiOX_c96OrUiL2_dRjhR5pjVzzMbC-ce44VFs0yHg-UZtCrYu1mOAmX87zPFuXeTGpN3kpysVqhbis8mq_zOVSLpfFaiUyXq5Wi2KiNzzjZVZk83xZ5GUxK3POpSzEfMVXKl9WbJ5hI7SZGXNqZs4fJjqEDjc5Xy_nq4kRezQhPUc5t3iGJGWc-GDiN6Q03XeHwOaZ0SGGm5moo8HN_3VadvePuzBMz1ev10nnzaaOsU2kxp8YfzroWHf7mXQN40_kYPxHui8oI-NPKazA-NMY92nD_wkAAP__bT2erg">