[llvm-dev] LangRef semantics for shufflevector with undef mask is incorrect

[Nuno Lopes via llvm-dev]

Hi,

This is a follow up on a discussion around shufflevector with undef mask in 
https://reviews.llvm.org/D70641 and 
https://bugs.llvm.org/show_bug.cgi?id=43958.

The current semantics of shufflevector in 
http://llvm.org/docs/LangRef.html#shufflevector-instruction states:
"If the shuffle mask is undef, the result vector is undef. If any element of 
the mask operand is undef, that element of the result is undef."

We found this semantics to be problematic. TL;DR: instructions can't detect 
if an operand is undef.
Long story:
Undef can be replaced with any value at any time. It's valid to replace 
undef with 0 or 1 or anything else.

A possible sequence of optimizations with sufflevector could be as follows:
%v = shufflevector <2 x float> %x, <2 x float> undef, <2 x i32> <i32 undef, 
i32 0>
->
%v = shufflevector <2 x float> %x, <2 x float> undef, <2 x i32> <i32 2, i32 
0>
->
%v = <undef, %x[0]>

So this respects the semantics in LangRef: the mask is undef, so the 
resulting element is undef.

However, there's an alternative sequence of optimizations:
%v = shufflevector <2 x float> %x, <2 x float> undef, <2 x i32> <i32 undef, 
i32 0>
->
%v = shufflevector <2 x float> %x, <2 x float> undef, <2 x i32> <i32 1, i32 
0>
->
%v = <%x[1], %x[0]>

So now it depends on what the value of %x[1] is. If it's poison, we obtain:
%v = <poison, %x[0]>

This result contradicts the semantics in LangRef, even though no individual 
transformation we did above is wrong.
In summary, an instruction cannot detect undef and have special semantics 
for it.

AFAICT, the only way to fix the semantics of shufflevector is to say that if 
the mask is out-of-bounds, we yield poison. That's correct because there's 
nothing worse than poison.
Since we can replace undef with an OOB index, an undef mask can safely yield 
poison. Or it would yield one of the input elements, which is poison in the 
worst case. So we get poison in both cases.

I guess the issue to make this semantics a reality is that we would need to 
introduce a poison value (which is a good thing IMHO). Otherwise we can't 
continue doing some of the folds we have today since we don't have a poison 
constant to replace undef when folding.

Any comments/suggestions appreciated!

Thanks,
Nuno