[PATCH] D40390: [InstCombine] Don't crash on out of bounds index in the insertelement

[Igor Laevsky via Phabricator via llvm-commits]

igor-laevsky updated this revision to Diff 124927.
igor-laevsky retitled this revision from "[InstCombine] Don't crash on unreasonable constant indexes" to "[InstCombine] Don't crash on out of bounds index in the insertelement".
igor-laevsky added a comment.

Hi Sanjay, thanks for the comments.

Updating InstSimplify is a reasonable thing to do, however it doesn't prevent instcombine from crashing. I believe we should do both changes.

I splited checks for shift operations into separate review thread: https://reviews.llvm.org/D40649 (by the way, there is InstSimplify rule for them already)

InstSimplify rule for the InsertElement is also in the separate review thread: https://reviews.llvm.org/D40650

And this review thread is now about instcombine crash specifically in the insertelement instruction with out of bounds index.


https://reviews.llvm.org/D40390

Files:
  lib/Transforms/InstCombine/InstCombineVectorOps.cpp
  test/Transforms/InstCombine/out-of-bounds-indexes.ll


Index: test/Transforms/InstCombine/out-of-bounds-indexes.ll
===================================================================
--- test/Transforms/InstCombine/out-of-bounds-indexes.ll
+++ test/Transforms/InstCombine/out-of-bounds-indexes.ll
@@ -11,3 +11,13 @@
 }
 
 declare void @llvm.assume(i1)
+
+define void @test(<4 x double> %a, <4 x double> %b) {
+entry:
+  %sub.i = fsub ninf <4 x double> %a, %b
+  %I = insertelement <4 x double> %sub.i, double 0x7FEFFFFFFFFFFFFF, i64 4294967296
+  %B = lshr i8 127, 0
+  store i8 %B, i8* undef
+  store <4 x double> %I, <4 x double>* undef
+  ret void
+}
Index: lib/Transforms/InstCombine/InstCombineVectorOps.cpp
===================================================================
--- lib/Transforms/InstCombine/InstCombineVectorOps.cpp
+++ lib/Transforms/InstCombine/InstCombineVectorOps.cpp
@@ -611,7 +611,8 @@
   // until we hit something that isn't an insert of the splatted value.
   while (CurrIE) {
     ConstantInt *Idx = dyn_cast<ConstantInt>(CurrIE->getOperand(2));
-    if (!Idx || CurrIE->getOperand(1) != SplatVal)
+    if (!Idx || Idx->getSExtValue() >= NumElements ||
+        CurrIE->getOperand(1) != SplatVal)
       return nullptr;
 
     InsertElementInst *NextIE =


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D40390.124927.patch
Type: text/x-patch
Size: 1230 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20171130/16ce2adc/attachment.bin>