[llvm-bugs] [Bug 44178] New: [InstCombine] Miscompile of bitcast/zext/trunc/bitcast on vectors for big-endian targets

[via llvm-bugs]

https://bugs.llvm.org/show_bug.cgi?id=44178

            Bug ID: 44178
           Summary: [InstCombine] Miscompile of bitcast/zext/trunc/bitcast
                    on vectors for big-endian targets
           Product: new-bugs
           Version: unspecified
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: enhancement
          Priority: P
         Component: new bugs
          Assignee: unassignedbugs at nondot.org
          Reporter: mikael.holmen at ericsson.com
                CC: htmldeveloper at gmail.com, llvm-bugs at lists.llvm.org

In
 llvm/test/Transforms/InstCombine/cast.ll
there is a test like this:

target datalayout = "E-p:64:64:64-p1:32:32:32-p2:64:64:64-p3:64:64:64-
a0:0:8-f32:32:32-f64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-
v64:64:64-v128:128:128-n8:16:32:64"

[...]

define <3 x i32> @test60(<4 x i32> %call4) {
; CHECK-LABEL: @test60(
; CHECK-NEXT:    [[P10:%.*]] = shufflevector <4 x i32> [[CALL4:%.*]],
<4 x i32> undef, <3 x i32> <i32 0, i32 1, i32 2>
; CHECK-NEXT:    ret <3 x i32> [[P10]]
;
  %p11 = bitcast <4 x i32> %call4 to i128
  %p9 = trunc i128 %p11 to i96
  %p10 = bitcast i96 %p9 to <3 x i32>
  ret <3 x i32> %p10

}

If we assume the input vector is e.g. <1, 2, 3, 4> then I assume %p11
would be the (hex) value 1234, %p9 would be the 234 and %p10 would then
be the vector <2, 3, 4>.

Note that the datalayout says we're using big endian.

But the CHECK-NEXT checks that the result is made up of the elements at
index 0, 1 and 2 from the input vector, which would be <1, 2, 3>.

Similarly, test61 tests bitcast/zext/bitcast, and there I think zeroes are
added at the wrong end for big-endian targets.

The transform was added in https://reviews.llvm.org/rL103354.

The problem is in optimizeVectorResize in InstCombineCasts.cpp where we do
  if (SrcTy->getNumElements() > DestTy->getNumElements()) {
    // If we're shrinking the number of elements, just shuffle in the low/high
    // elements from the input depending on endian and use undef as the
    // second shuffle input.
    V2 = UndefValue::get(SrcTy);
    for (unsigned i = 0, e = DestTy->getNumElements(); i != e; ++i)
      if (IC.getDataLayout().isBigEndian())
        ShuffleMask.push_back(i + SrcTy->getNumElements() -
DestTy->getNumElements());
      else
        ShuffleMask.push_back(i);

  } else {
    // If we're increasing the number of elements, shuffle in all of the
    // elements from InVal and fill the rest of the result elements with zeros
    // from a constant zero.
    V2 = Constant::getNullValue(SrcTy);
    unsigned SrcElts = SrcTy->getNumElements();
    for (unsigned i = 0, e = SrcElts; i != e; ++i)
      ShuffleMask.push_back(i);

    // The excess elements reference the first element of the zero input.
    for (unsigned i = 0, e = DestTy->getNumElements()-SrcElts; i != e; ++i)
      ShuffleMask.push_back(SrcElts);
  }

Endianness needs to be taken into account above.

Also see discussion on llvm-dev here:
 http://lists.llvm.org/pipermail/llvm-dev/2019-November/137297.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20191129/1efa1d65/attachment.html>