[llvm-dev] [RFC] Introducing an explicit calling convention

[David Greene via llvm-dev]

Philip Reames via llvm-dev <llvm-dev at lists.llvm.org> writes:

> On the framing piece, the ABI is really a property of the callee, not
> of the arguments.

The *calling convention* might be so, but the ABI is definitely not.
For a given language, the ABI is a property of the target.  The ABI
covers a lot more ground than just the calling convention.  With more
complex argument types, the "calling convention" depends on a lot of ABI
information beyond caller-save/callee-save/argument/return register
specifications.  It has to know what the types look like in memory.

>  As such, I think this really deserves to either be a first class
>syntax for spelling a calling convention, or an attribute.  I'd suggest
>framing the description of the calling convention as "like this
>existing calling convention XXX, but w/o this callee saved register" or
>"like this existing calling convention XYZ, but with one argument
>register defined".  Possibly spellings might include:
>
> declare "C" {i64, i64} @example(i64 %a, i64 %b)
> ccoverride(noclobber={rax, rbx}, arg_reg_order={rcx, rdx},
> ret_arg_regs={rdx, rcx))
> declare "C" {i64, i64} @example(i64 %a, i64 %b)
> ccoverride="(noclobber={rax, rbx}, arg_reg_order={rcx, rdx},
> ret_arg_regs={rdx, rcx))"
>
> (The second one abuses String attributes horribly, but would be the
> easiest to implement.)

This might work for simple scalar arguments but quickly breaks down in
the presence of aggregates.  Describing, for example, the layout of C
struct types and their mapping to registers in the calling convention is
non-trivial.

> An alternate way of framing this would be to provide a clean interface
> for plugging in an externally defined calling convention w/o needing
> to rebuild LLVM.  This would require a custom driver, but would avoid
> the need to build LLVM.  This would solve our problem cleanly - and is
> probably what I'd get around to implementing someday - but I'm not
> sure how it matches your original use case.

This is an interesting idea.  Keeping in mind what Manuel Jacob[1] and
David Chisnall[2] have said, what I've really wanted in my work is
something that, given a source-level function signature, a set of
argument Values and an optional Value in which to store the returned
value, would tell me how to generate the LLVM IR to pass the arguments
and store the return value.  I've also wanted something that, given a
source type, would generate an LLVM type that correctly implements the
ABI layout.  Finally, I'd want something that, given a source type, a
Value of the correspoding LLVM type and a source field access specifier,
would generate the IR to read from or write to the field.

StructType kinda-sorta provides some of the latter two, but not really.

The above is written with a C/C++ lens and other languages may need
more/different things from an ABI library.

                         -David

[1] http://lists.llvm.org/pipermail/llvm-dev/2019-January/129184.html
[2] http://lists.llvm.org/pipermail/llvm-dev/2019-January/129207.html