[PATCH] D139395: Add CFI integer types normalization

[Peter Collingbourne via Phabricator via cfe-commits]

pcc added a reviewer: pcc.
pcc added a comment.

A high level question is whether we want to base the cross-language encoding on Itanium at all. Itanium has concepts such as substitutions that will complicate the implementation in other languages. Encoding pointee types can also lead to complications in cross-language encodings.

It may be better to consider developing a custom encoding. For the encoding being prototyped for the pointer authentication ABI on Android, the Rust side of the implementation is very simple:

https://github.com/pcc/rust/blob/d37ad119171635219ff21e054780d31024d24200/compiler/rustc_ty_utils/src/abi.rs#L396



================
Comment at: clang/lib/AST/ItaniumMangle.cpp:2952
+  // u<length>u<type size>
+  if (NormalizeIntegers && T->isInteger()) {
+    if (T->isSignedInteger()) {
----------------
`isInteger()` will return true for enums, but only if they are complete. This would mean that code such as
```
void (*f)(enum E *e);

void g() {
  f(0);
}
```
would use a different encoding to call `f` depending on whether the TU completes the enum `E`, if pointee types are considered part of the encoding.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D139395/new/

https://reviews.llvm.org/D139395