[llvm-dev] Forcing LLVM to not spill specific variables to the stack when compiling to BPF

[Rhys Rustad-Elliott via llvm-dev]

Hi all,

I believe llvm-dev is the correct list for this, my apologies if this
would better belong somewhere else.

I'm experiencing a very niche issue with compiling C to BPF bytecode and
have exhausted every possible avenue I can on my own. Hoping someone on
here can lend a hand.

I have a rather complicated BPF probe (read: lots of variables) as C
source and compile/link it to an ELF containing BPF bytecode using
clang-12/llc-12. There are a number of different sections of the code
that will trigger this specific problem, but to give one example:

char *buf = some_function_returning_a_char_buffer();
/* Null check before store as required by BPF verifier */
if (*buf == NULL) {
    return;
}
buf[0] = '\0';

This code works fine in a simple BPF probe, buf will be null checked,
and the BPF verifier will know that by the time the final statement is
executed, buf is non-null, and thus the store is permissible. My problem
is that, as mentioned, the BPF probe I'm dealing with is very large, has
a lot of variables and puts LLVM under quite a bit of register pressure.
LLVM thus decides to spill buf to the stack.

Pre Linux 4.16, variables allocated on the BPF stack that had been
explicitly zeroed were not tracked as such by the verifier. See kernel
commit cc2b14d51053eb055c06f45e1a5cdbfcf2b79e94 for the change in 4.16.
There is a similar issue pre Linux 5.3 with the verifier not being able
to track constant scalar values that had been spilled to the BPF stack
that can be triggered in an analogous fashion (fixed by kernel commit
f7cf25b2026dc8441e0fa3a202c2aa8a56211e30). When either of these issues
are triggered, the BPF probe will fail to load entirely as the verifier
rejects it. In the case above, for instance, it doesn't recognize that
buf is guaranteed to be non-null when evaluating the final statement and
fails with something like this:

R8 invalid mem access 'inv'

My issue is that LLVM doesn't recognize that it can't spill these
variables that need to be null checked or bounded to the stack.
Unfortunately upgrading to a post-5.3 kernel isn't an option as I have
to support kernels 4.15 and up for this particular project. I'm
therefore trying to find a way to force LLVM to not spill variables of
this nature to the stack. I've managed to fudge it, if you will, in a
number of cases by playing around with inline asm. For example, doing
this:

char *buf2 = some_function_returning_a_char_buffer();
char *buf = NULL;
asm volatile ("%0 = %1" : "=r"(buf2) : "r"(buf)); /* buf = buf2 */
if (*buf == NULL) {
    return;
}
buf[0] = '\0';

actually fixed the above issue as buf was forced into a register by an
inline asm constraint (=r). Fixing things this way seems like walking on
thin ice however.

Intuitively, it seems like that bit of inline asm would force buf into a
register at least until _that single instruction_, but I'm not sure if
doing that is guaranteed to keep that variable in a register until the
final statement is executed (i.e. I'd think it would be possible for
LLVM to keep buf in a register up to the asm statement, but spill it
afterward). Unfortunately, my knowledge of LLVM doesn't go deep enough
to know if this is the case or not.

So with all that context in mind, two questions:

- Is the inline asm trick above guaranteed to force LLVM to _never_
  store the variable on the stack, even after the asm statement?

- If the answer to the above question is no, is there any way to
  guarantee that a variable isn't spilled to the stack? A really hacky
  solution and/or one that depends on an implementation detail is fine,
  I just want something that I know is going to consistently work given
  a specific version of clang/llvm.

Thanks in advance for any help. I know this one is finicky and greatly
appreciate any advice.

Rhys