[llvm-branch-commits] [llvm] [BOLT] Gadget scanner: detect signing oracles (PR #134146)

[Anatoly Trosinenko via llvm-branch-commits]

================
@@ -622,6 +628,40 @@ class MCPlusBuilder {
     return std::make_pair(getNoRegister(), getNoRegister());
   }
 
+  /// Analyzes if a pointer is checked to be valid by the end of BB.
+  ///
+  /// It is possible for pointer authentication instructions not to terminate
+  /// the program abnormally on authentication failure and return some *invalid
+  /// pointer* instead (like it is done on AArch64 when FEAT_FPAC is not
+  /// implemented). This might be enough to crash on invalid memory access
+  /// when the pointer is later used as the destination of load/store or branch
+  /// instruction. On the other hand, when the pointer is not used right away,
+  /// it may be important for the compiler to check the address explicitly not
+  /// to introduce signing or authentication oracle.
+  ///
+  /// If this function returns a (Reg, Inst) pair, then it is known that in any
+  /// successor of BB either
+  /// * Reg is trusted, provided it was safe-to-dereference before Inst, or
+  /// * the program is terminated abnormally without introducing any signing
+  ///   or authentication oracles
+  virtual std::optional<std::pair<MCPhysReg, MCInst *>>
----------------
atrosinenko wrote:

Oh, got it - it is the description of `getAuthCheckedReg` which is not as detailed as it should be. There are two overloaded methods - my current assumption is that two different kinds of "pointer checkers" can be detected:
* single instructions, such as `ldr w0, [x1]`
* a number of hardcoded instruction sequences - all these sequences are contiguous and involve branching depending on the result of validation, so they span some suffix of a basic block. This is a bit hackish but it does work for AArch64 code emitted by LLVM, aside from being unsupported when CFG is not available

I hope this is an acceptable approach, at least at first, but of course the documentation of `getAuthCheckedReg(BB)` should mention that this method does not summarize the results of `getAuthCheckedReg(Inst)` across the basic block, it detects completely different patterns.

Thus, considering your example, `getAuthCheckedReg(Inst)` should report `nullopt`, `nullopt`, `x0`, `x2` for the four instructions of `bb1`, and `getAuthCheckedReg(BB)` should report `nullopt` for `bb1`.

Initially, I planned significantly updating `bolt/docs/BinaryAnalysis.md` via a separate PR after most of the changes land. I will add your example as a test case and update the description of `getAuthCheckedReg` in this PR, of course.

https://github.com/llvm/llvm-project/pull/134146