[cfe-dev] RFC: Flesh out the IRGen C++ ABI class

[John McCall]

On Oct 26, 2010, at 9:28 PM, Charles Davis wrote:

> On 10/26/10 5:51 PM, Douglas Gregor wrote:
>> 
>> On Oct 26, 2010, at 4:47 PM, John McCall wrote:
>> 
>>> On Oct 26, 2010, at 4:09 PM, Charles Davis wrote:
>>>> On 10/25/10 10:28 PM, John McCall wrote:
>>>>> My inclination is to say that "canonicalizing" the function type is the wrong thing to do, and that the default CC for a function in any particular context should be determined by that context.  That means that a default-CC function type is different from a specifically-cdecl function type, but I can live with that.
>>>> So you're saying we should get rid of getCanonicalCallConv()? Then I can
>>>> dispense with the AST hook, too.
>>>> 
>>>> I don't know about this. The whole point of canonicalizing the CC was
>>>> that cdecl function types were compatible with default-CC function
>>>> types. This matters when, e.g., trying to merge two compatible
>>>> declarations, or assigning function pointers. Are you sure you can live
>>>> with it now?
>>> 
>>> There's already explicit function-merging logic that deals with CCs.
> Yeah, and it uses getCanonicalCallConv().

There will obviously still be an idea of "the default calling convention
means X in this specialized context";  that's not in question.

>>> The problem is that a function type with no explicit CC really is different in important ways from a function type with an explicit CC.  In particular, if I have
>>> typedef void callback_t(A*);
>>> then this:
>>> callback_t B::*cb = ...;  (b->*cb)(a);
>>> uses the default calling convention for instance methods, whereas this:
>>> callback_t *f; f(a);
>>> uses the default calling convention for normal functions.
> That's why, when getCanonicalCallConv() is called, it needs to know if
> it's dealing with a method instead of a normal function.

What we're saying is that that's obviously not possible in general.  The type
system needs to be able to deal with function types whose calling convention
will be determined only in context, i.e. when we're not dealing with a method
*or* a normal function.  That means that treating calling conventions as if they
can canonicalized on the type cannot be the right approach.

>>> The point is that function types exist independent of any particular declaring context, and the right rule is to fill in a sensible calling convention based on context when none was mandated.  If that means that we have to produce different template instantiations for __attribute__((cdecl)) functions than unadorned ones, I can live with that.
>> 
>> I think we have to live with the fact that we produce different template instantiations for __attribute__((cdecl)) functions than unadorned ones. I haven't looked into what GCC does, but different name-mangling for cdecl, fastcall, etc. function types makes perfect sense. 
> Last I checked, the Itanium ABI didn't mangle calling conventions at all ;).

They're non-standard, which means we can completely ignore them, right? :)

> I tried just now to gut getCanonicalCallConv(), and a bunch of tests
> started failing horribly. (By that, I mean *crash* horrible.) I don't
> think it's worth removing this function. There must be another way.

If you've never revised the type system before, I would not start with this task.

John.