[llvm-dev] Vector ABI mismatch during inlining

Nikita Popov via llvm-dev llvm-dev at lists.llvm.org
Sun Mar 14 14:09:29 PDT 2021


Hi,

Consider the following code:

target triple = "x86_64-unknown-linux-gnu"

define void @test1() "target-features"="+avx" {
  call void @test2()
  ret void
}

define void @test2() {
  call void @test3(<4 x i64> <i64 0, i64 1, i64 2, i64 3>)
  ret void
}

define void @test3(<4 x i64> %arg) noinline {
  ret void
}

Inlining will inline test2 into test1, because test1 has a superset of
target features:

target triple = "x86_64-unknown-linux-gnu"

define void @test1() "target-features"="+avx" {
  call void @test3(<4 x i64> <i64 0, i64 1, i64 2, i64 3>)
  ret void
}

define void @test2() {
  call void @test3(<4 x i64> <i64 0, i64 1, i64 2, i64 3>)
  ret void
}

define void @test3(<4 x i64> %arg) noinline {
  ret void
}

Now we have a problem: X86 uses different vector ABIs depending on target
features. test3 is compiled without avx, and as such expects the argument
to be passed in two XMM registers. test2 is also compiled without avx and
performs the call correctly. test1 on the other hand is compiled with avx
and will instead pass a single YMM register.

Note that the by-value vector arguments do not necessarily have to be
present in the original code -- even if the frontend passes all vectors by
memory to avoid precisely this issue, argument promotion can convert them
to by-value arguments, as the caller/callee ABIs match at the time argument
promotion runs (between test2 and test3).

I would like to have some input on how this miscompile could be addressed.
The two general options I see are:

1. Fix call lowering to respect callee target features. That is, even if
the caller is +avx, if the callee is not then we should pass arguments via
xmm rather than ymm. I'm not sure to what degree this is possible without
our current infrastructure though.

2. Prevent inlining in this case. I don't think we can't prevent inlining
across functions with different target features entirely, as that would be
a performance disaster. But possibly we can inspect the body of the callee
to check for calls that would be problematic under the new ABI.

Regards,
Nikita
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210314/119ee7c2/attachment.html>


More information about the llvm-dev mailing list