[clang] [clang][X86] Use a sized byval type for an incomplete interrupt frame (PR #239937)
Feng Zou via cfe-commits
cfe-commits at lists.llvm.org
Tue Nov 10 21:00:12 PST 2026
fzou1 wrote:
> Getting a byval type wrong has extremely bad consequences: in particular, certain optimizations will assume that reading or writing past the end is undefined behavior.
>
> Is the byval attribute actually doing anything useful here?
Thanks. `byval` is needed, but you're right that `i8` is the wrong type.
[`CC_X86_Intr`](https://github.com/llvm/llvm-project/blob/572743cb5f3fc51d8e85d70924aa1f0d0a5a18c4/llvm/lib/Target/X86/X86CallingConv.cpp#L297-L331) puts parameter 0 at a fixed stack location, the frame the processor pushed, and reserves five slots for it regardless of type. `byval` makes the parameter that slot's address rather than a value loaded from it ([`LowerMemArgument`](https://github.com/llvm/llvm-project/blob/572743cb5f3fc51d8e85d70924aa1f0d0a5a18c4/llvm/lib/Target/X86/X86ISelLoweringCall.cpp#L1362-L1371)). The verifier has required it since 2e0e03c6a089da39039ec3f464f7cee5df86646b, which replaced a backend special case. The type only sizes the object.
`byval(i8)` says the frame is one byte, so reading CS and the later slots is out of bounds. That is already observable for a handler reading `((unsigned long *)f)[1]`:
- `__builtin_object_size(f, 0)` is 1;
- with `-fsanitize=object-size -fsanitize-trap=object-size`, the body becomes `llvm.ubsantrap` + `unreachable`.
This isn't new with this patch: `void *` frames have always been lowered to `byval(i8)`, so they already have the problem. Normal `-O2` builds generate correct code today; the wrong size only shows up where the object size is used, such as `__builtin_object_size` and the object-size sanitizer, though future optimizations could also rely on it.
We may fix this by using the backend's frame, `[5 x i64]` on x86-64 and `[5 x i32]` on i386. The object size becomes 40 / 20, the sanitizer build keeps the plain loads, and the generated code doesn't change. GCC reads the same slots, and treats the frame as a plain pointer with no object size (`__builtin_object_size` is -1).
**Complete pointee types** smaller than the frame have the same problem:
| frame type | byval | object size | object-size sanitizer |
|---|---|---|---|
| `int *` (`attr-x86-interrupt.c`) | `i32` | 4 | trap |
| `struct { unsigned long ip; } *` | 8 bytes | 8 | trap |
| `struct { unsigned long ip, cs, flags, sp, ss; } *` | 40 bytes | 40 | ok |
So, should the byval type be `[5 x iN]`
1. only when the frame's type gives no size, or
2. always, whatever the pointee type?
Option 2 matches `CC_X86_Intr` and GCC and removes the special cases; it changes the `byval(i32)` expectations in `attr-x86-interrupt.c`/`.cpp`. Happy to do it here or in a separate PR.
https://github.com/llvm/llvm-project/pull/239937
More information about the cfe-commits
mailing list