[llvm-dev] RFC: callee saved register verifier

[Sanjoy Das via llvm-dev]

Hi Escha,

escha at apple.com wrote:
> I’ve seen this done in the past without compiler support (for the case of handwritten or JIT’d, etc functions that need checking).
>
> It worked something like this:
>
> 1. Pass calls to the risky functions through a macro/template or such.
> 2. In release mode, this turns into a no-op. In debug mode, this sends the calls through a handwritten asm wrapper that sets all the callee-save registers to random values, then calls the function.
> 3. When the function returns, the wrapper checks the registers are the same, and if they’re not, bails to exit() and prints the invalid registers.
>
> This took a few dozen lines of code and worked entirely within the language, so it didn’t need compiler support.

That's certainly a design point, but assert in the compiler has some
advantages:

  - It is more scalable -- I don't have to add the verification code
    for each new calling convention.  To have LLVM take advantage of a
    new calling convention, we have to codify knowledge of the calling
    convention in LLVM anyway, and it is better to re-use the
    information as much as possible.

  - Since the scheme works directly off of the calling convention as
    encoded in LLVM, it will catch bugs in the specification on the
    LLVM side (e.g. marking a function declaration with an incorrect
    calling convention should fail noisily).

The overarching point is that I'd rather have a single source of truth
(the calling convention as specified in LLVM) and assert on that.

-- Sanjoy

>
> —escha
>
>> On May 13, 2016, at 4:10 PM, Sanjoy Das via llvm-dev<llvm-dev at lists.llvm.org>  wrote:
>>
>> Hi David,
>>
>> David Blaikie wrote:
>>> Yeah, that's sort of what I'd be wondering too.
>>>
>>> It seems like if the compiler does the right thing for existing calling
>>> conventions - I hope our architecture is such that changing the set of
>>> callee save registers reflects on both sides (both the caller - relying
>>> on them to be saved, and the caller implementing that guarantee). So I
>>> wouldn't expect making weird/random calling conventions to be likely to
>>> have any new/interesting bugs here.
>> Yes, so far we've not had any issues with LLVM upholding its side of
>> the deal when it comes to calling conventions.  Humans tend to be
>> more prone to making mistakes, unfortunately.  :)
>>
>>> But perhaps the architecture doesn't&  can't work that way?
>> The intention is to add verification at call-sites that call into a
>> hand-written stub from LLVM generated code.
>>
>> -- Sanjoy
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>