[clang] [llvm] [X86] Enhance kCFI type IDs with a 3-bit arity indicator. (PR #117121)

[Scott Constable via cfe-commits]

================
@@ -208,10 +209,34 @@ void llvm::setKCFIType(Module &M, Function &F, StringRef MangledType) {
   std::string Type = MangledType.str();
   if (M.getModuleFlag("cfi-normalize-integers"))
     Type += ".normalized";
+
+  uint32_t OutHash = static_cast<uint32_t>(llvm::xxHash64(Type));
+  auto T = Triple(Twine(M.getTargetTriple()));
+  if (T.isX86() && T.isArch64Bit() && T.isOSLinux()) {
+    // Estimate the function's arity (i.e., the number of arguments) at the ABI
+    // level by counting the number of parameters that are likely to be passed
+    // as registers, such as pointers and 64-bit (or smaller) integers. The
+    // Linux x86-64 ABI allows up to 6 parameters to be passed in GPRs.
+    // Additional parameters or parameters larger than 64 bits may be passed on
+    // the stack, in which case the arity is denoted as 7.
+    size_t NumParams = F.arg_size();
+    bool MayHaveStackArgs = NumParams > 6;
+
+    for (unsigned int i = 0; !MayHaveStackArgs && i < NumParams; ++i) {
+      const llvm::Type *PT = F.getArg(i)->getType();
+      if (!(PT->isPointerTy() || PT->getIntegerBitWidth() <= 64))
----------------
scottconstable wrote:

It appears that clang does not reserve stack for large arguments and instead this is done later by the LLVM X86 backend. For example:
```C
struct S {
    int *p1;
    int *p2;
    int array[8];
};

int foo(struct S s, struct S *sp) {
    return *s.p1 + *s.p2 + *sp->p1 + *sp->p2;
}
```
Then when I compile to LLVM IR I see:
```
define dso_local i32 @foo(ptr noundef byval(%struct.S) align 8 %s, ptr noundef %sp) #0 {
```
Which suggests an arity of 2. But the X86 backend transforms `foo` to pass `s` on the stack, and then `sp` becomes the sole argument and is passed in `rdi`. Hence, by the chart in the PR description, this should be treated as an arity-7 function:

| Arity Indicator | Description |
| --------------- | --------------- |
| 0 | 0 parameters |
| 1 | 1 parameter in RDI |
| 2 | 2 parameters in RDI and RSI |
| 3 | 3 parameters in RDI, RSI, and RDX |
| 4 | 4 parameters in RDI, RSI, RDX, and RCX |
| 5 | 5 parameters in RDI, RSI, RDX, RCX, and R8 |
| 6 | 6 parameters in RDI, RSI, RDX, RCX, R8, and R9 |
| 7 | At least one parameter may be passed on the stack |

This predicate:
```
      // typeof(*PT) = llvm::Type
      if (!(PT->isPointerTy() || PT->getIntegerBitWidth() <= 64))
        MayHaveStackArgs = true;
```
should prevent `s` from being counted as a register argument and correctly set the arity field to 7.

https://github.com/llvm/llvm-project/pull/117121