[llvm-bugs] [Bug 52164] New: computeKnownBits crashing with Assertion `isa<X>(Val) && "cast<Ty>() argument of incompatible type!"' failed.

[via llvm-bugs]

https://bugs.llvm.org/show_bug.cgi?id=52164

            Bug ID: 52164
           Summary: computeKnownBits crashing with Assertion `isa<X>(Val)
                    && "cast<Ty>() argument of incompatible type!"'
                    failed.
           Product: libraries
           Version: trunk
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: enhancement
          Priority: P
         Component: Global Analyses
          Assignee: unassignedbugs at nondot.org
          Reporter: mikael.holmen at ericsson.com
                CC: llvm-bugs at lists.llvm.org

llvm commit: d1e0f02e0ba

Add the following test in 
 llvm/unittests/Analysis/ValueTrackingTest.cpp

TEST_F(ComputeKnownBitsTest, ComputeKnownBitsCrash) {

  parseAssembly(
      "@g.a = external global i16, align 1\n"
      "define i16 @test(i16 %i) {\n"
      "entry:\n"
      "  %0 = icmp slt i16 sub (i16 0, i16 trunc (i32 udiv (i32 ptrtoint (i16*
@g.a to i32), i32 -1) to i16)), 0\n"
      "  %A = select i1 %0, i16 trunc (i32 udiv (i32 ptrtoint (i16* @g.a to
i32), i32 -1) to i16), i16 sub (i16 0, i16 trunc (i32 udiv (i32 ptrtoint (i16*
@g.a to i32), i32 -1) to i16))\n"
      "  ret i16 %A\n"
      "}\n");
  AssumptionCache AC(*F);
  KnownBits Known = computeKnownBits(
      A, M->getDataLayout(), /* Depth */ 0, &AC, F->front().getTerminator());
}

Then run
 ninja check-llvm-unit

Result:
AnalysisTests: ../include/llvm/Support/Casting.h:269: typename cast_retty<X, Y
*>::ret_type llvm::cast(Y *) [X = llvm::Instruction, Y = const llvm::Value]:
Assertion `isa<X>(Val) && "cast<Ty>() argument of incompatible type!"' failed.
 #0 0x0000000000a0da53 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int)
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0xa0da53)
 #1 0x0000000000a0bb0c llvm::sys::RunSignalHandlers()
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0xa0bb0c)
 #2 0x0000000000a0df06 SignalHandler(int)
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0xa0df06)
 #3 0x00007fe389b0a630 __restore_rt (/lib64/libpthread.so.0+0xf630)
 #4 0x00007fe38723d387 __GI_raise (/lib64/libc.so.6+0x36387)
 #5 0x00007fe38723ea78 __GI_abort (/lib64/libc.so.6+0x37a78)
 #6 0x00007fe3872361a6 __assert_fail_base (/lib64/libc.so.6+0x2f1a6)
 #7 0x00007fe387236252 (/lib64/libc.so.6+0x2f252)
 #8 0x00000000007b5b00 computeKnownBitsFromOperator(llvm::Operator const*,
llvm::APInt const&, llvm::KnownBits&, unsigned int, (anonymous
namespace)::Query const&)
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0x7b5b00)
 #9 0x000000000079e086 computeKnownBits(llvm::Value const*, llvm::APInt const&,
llvm::KnownBits&, unsigned int, (anonymous namespace)::Query const&)
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0x79e086)
#10 0x000000000079da9b computeKnownBits(llvm::Value const*, llvm::KnownBits&,
unsigned int, (anonymous namespace)::Query const&)
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0x79da9b)
#11 0x000000000079ea2a llvm::computeKnownBits(llvm::Value const*,
llvm::DataLayout const&, unsigned int, llvm::AssumptionCache*,
llvm::Instruction const*, llvm::DominatorTree const*,
llvm::OptimizationRemarkEmitter*, bool)
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0x79ea2a)
#12 0x00000000005bdbd7
ComputeKnownBitsTest_ComputeKnownBitsCrash_Test::TestBody()
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0x5bdbd7)
#13 0x0000000000a1cb4c testing::Test::Run()
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0xa1cb4c)
#14 0x0000000000a1dfd9 testing::TestInfo::Run()
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0xa1dfd9)
#15 0x0000000000a1e7a7 testing::TestSuite::Run()
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0xa1e7a7)
#16 0x0000000000a2c107 testing::internal::UnitTestImpl::RunAllTests()
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0xa2c107)
#17 0x0000000000a2b89a testing::UnitTest::Run()
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0xa2b89a)
#18 0x0000000000a1502b main
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0xa1502b)
#19 0x00007fe387229555 __libc_start_main (/lib64/libc.so.6+0x22555)
#20 0x00000000004455b2 _start
(/repo/uabelho/master-github/llvm/build-all/unittests/Analysis/./AnalysisTests+0x4455b2)


Originally I found this in opt during fuzz testing of my out-of-tree target.
For some reason I can't manage to make an opt reproducer for an in-tree target
but the problem is exposed by the unit test addition above as well.

When we hit the assertion we come from computeKnownBitsFromOperator in
ValueTracking.cpp and we do

    if (SPF == SPF_ABS) {
      // RHS from matchSelectPattern returns the negation part of abs pattern.
      // If the negate has an NSW flag we can assume the sign bit of the result
      // will be 0 because that makes abs(INT_MIN) undefined.
      if (match(RHS, m_Neg(m_Specific(LHS))) &&
          Q.IIQ.hasNoSignedWrap(cast<Instruction>(RHS)))
        Known.Zero.setSignBit();
    }

RHS is

i16 sub (i16 0, i16 trunc (i32 udiv (i32 ptrtoint (i16* @g.a to i32), i32 -1)
to i16))

which is constant and not an Instruction so we get the failed assertion.

I guess the code could be fixed with an isa or dyn_cast to check if we're
really dealing with an Instruction.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20211013/84cbfc82/attachment.html>