[llvm-branch-commits] [llvm] [BOLT] Gadget scanner: add less strict version of tail call checker (PR #188974)

[Anatoly Trosinenko via llvm-branch-commits]

atrosinenko wrote:

> what other cases benefit from this PR?

It might be possible to omit the fine-grained toggles "activate on return instructions" (no heuristics) vs. "activate on ret-like tail call branches" (relies on heuristics). However, the check performed by `shouldReportUnsafeTailCall` still has to be split into the lightweight and the stricter variants. This could technically be done by invoking `shouldReportUnsafeTailCall` iff `EnabledDetectors & (opts::GS_PTRAUTH_TAIL_CALLS | opts::GS_PTRAUTH_RETURN_TARGETS)` while passing `Strict = EnabledDetectors & opts::GS_PTRAUTH_TAIL_CALLS` and not modifying the command line interface exposed to the user.

Anyway, with pac-ret enabled in the compiler, we would expect any epilogue to include the authentication as long as LR was reloaded from the stack. On the other hand, inserting the full-fledged checks into tail calls that the authentication **succeeded** is a significantly less obvious trade-off. Currently, we don't have a straightforward way to ask gadget scanner to detect both
```
bad_ret:
  stp     x29, x30, [sp, #-0x10]!
  ldp     x29, x30, [sp], #0x10
  ret                             ; implicitly "ret x30"

bad_tail_call:
  stp     x29, x30, [sp, #-0x10]!
  ldp     x29, x30, [sp], #0x10
  b       callee                  ; does not use x30 itself, it is later used in the callee
```
and requiring something along these lines instead
```
good_ret:
  paciasp
  stp     x29, x30, [sp, #-0x10]!
  ldp     x29, x30, [sp], #0x10
  autiasp
  ret

good_tail_call:
  paciasp
  stp     x29, x30, [sp, #-0x10]!
  ldp     x29, x30, [sp], #0x10
  autiasp
  b       callee
```
without requiring every tail call to be done like this
```
good_tail_call:
  paciasp
  stp     x29, x30, [sp, #-0x10]!
  ldp     x29, x30, [sp], #0x10
  autiasp
  ldr     x16, [x30]              ; check
  b       callee
```
or even
```
good_tail_call:
  paciasp
  stp     x29, x30, [sp, #-0x10]!
  ldp     x29, x30, [sp], #0x10
  autiasp
  mov     x16, x30                ; check
  xpaclri                         ; check
  cmp     x16, x30                ; check
  b.eq    .Lsuccess               ; check
  brk     0xc470                  ; check
.Lsuccess:
  b       callee
```
Well, we could pass `--scanners=ptrauth-pac-ret,ptrauth-tail-calls --auth-traps-on-failure`: 1) require *safe-to-dereference* return destination for regular returns, 2) require *trusted* LR before tail calls, 3) promote *safe-to-deference* register property to *trusted* by promising that authentication failure results in an error - but that's quite hackish :)

By the way, now I wonder whether I should additionally enable `GS_PTRAUTH_SIGN_ORACLES` in the `ptrauth-pac-ret` preset to catch the code like this artificial example:
```
lr_clobbered_before_signing:
  mov x30, x0
  paciasp
  stp     x29, x30, [sp, #-0x10]!
  ldp     x29, x30, [sp], #0x10
  autiasp
  ret
```

https://github.com/llvm/llvm-project/pull/188974