[llvm-dev] Caller and Callee with different calling conventions

[Reid Kleckner via llvm-dev]

http://llvm.org/docs/FAQ.html#why-does-instcombine-simplifycfg-turn-a-call-to-a-function-with-a-mismatched-calling-convention-into-unreachable-why-not-make-the-verifier-reject-it

Basically, you can imagine code like this:

enum CC { cdecl, stdcall, fastcall };
void callit(CC cc, void *fp) {
  switch (cc) {
  case cdecl: ((void(__cdecl*)())fp)(); break;
  case stdcall: ((void(__stdcall*)())fp)(); break;
  case fastcall: ((void(__fastcall*)())fp)(); break;
  }
}

If we inline 'callit', and discover the true target of fp, those calls
become direct, and the verifier should not reject them just because we
discovered additional information.

On Fri, Jan 6, 2017 at 1:32 AM, Juan Manuel Martinez Caamaño via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Hello,
>
> If a call instruction and the function being called differ in their call
> convention, InstructionCombine will insert a "store true, undef", then
> CFGSimplify transforms this store to a llvm.trap() call, and finally,
> during execution an InvalidInstruction exception is signaled.
>
> Is there any reason not to detect the calling convention mismatch in the
> verifier ? I would like to add such check.
>
>
> Cheers,
>
> Juan
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170106/cd3c2752/attachment.html>