[llvm-dev] RFC: callee saved register verifier

[via llvm-dev]

I’ve seen this done in the past without compiler support (for the case of handwritten or JIT’d, etc functions that need checking).

It worked something like this:

1. Pass calls to the risky functions through a macro/template or such.
2. In release mode, this turns into a no-op. In debug mode, this sends the calls through a handwritten asm wrapper that sets all the callee-save registers to random values, then calls the function.
3. When the function returns, the wrapper checks the registers are the same, and if they’re not, bails to exit() and prints the invalid registers.

This took a few dozen lines of code and worked entirely within the language, so it didn’t need compiler support.

—escha

> On May 13, 2016, at 4:10 PM, Sanjoy Das via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> 
> Hi David,
> 
> David Blaikie wrote:
> > Yeah, that's sort of what I'd be wondering too.
> >
> > It seems like if the compiler does the right thing for existing calling
> > conventions - I hope our architecture is such that changing the set of
> > callee save registers reflects on both sides (both the caller - relying
> > on them to be saved, and the caller implementing that guarantee). So I
> > wouldn't expect making weird/random calling conventions to be likely to
> > have any new/interesting bugs here.
> 
> Yes, so far we've not had any issues with LLVM upholding its side of
> the deal when it comes to calling conventions.  Humans tend to be
> more prone to making mistakes, unfortunately.  :)
> 
> >
> > But perhaps the architecture doesn't & can't work that way?
> 
> The intention is to add verification at call-sites that call into a
> hand-written stub from LLVM generated code.
> 
> -- Sanjoy
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev