[llvm] 32d9a38 - [LV] Keep Primary Induction alive when folding tail by masking

[Ayal Zaks via llvm-commits]

Author: Ayal Zaks
Date: 2020-11-24T15:12:54+02:00
New Revision: 32d9a386bf8f447dbbfaf55f6fef4fea4463205b

URL: https://github.com/llvm/llvm-project/commit/32d9a386bf8f447dbbfaf55f6fef4fea4463205b
DIFF: https://github.com/llvm/llvm-project/commit/32d9a386bf8f447dbbfaf55f6fef4fea4463205b.diff

LOG: [LV] Keep Primary Induction alive when folding tail by masking

Fix PR47390.

The primary induction should be considered alive when folding tail by masking,
because it will be used by said masking; even when it may otherwise appear
useless: feeding only its own 'bump', which is correctly considered dead, and
as the 'bump' of another induction variable, which may wrongfully want to
consider its bump = the primary induction, dead.

Differential Revision: https://reviews.llvm.org/D92017

Added: 
    

Modified: 
    llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    llvm/test/Transforms/LoopVectorize/dead_instructions.ll

Removed: 
    


################################################################################
diff  --git a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
index be5db9b4c5c4..3253678e6532 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
@@ -7132,6 +7132,12 @@ void LoopVectorizationPlanner::collectTriviallyDeadInstructions(
   for (auto &Induction : Legal->getInductionVars()) {
     PHINode *Ind = Induction.first;
     auto *IndUpdate = cast<Instruction>(Ind->getIncomingValueForBlock(Latch));
+
+    // If the tail is to be folded by masking, the primary induction variable,
+    // if exists, isn't dead: it will be used for masking. Don't kill it.
+    if (CM.foldTailByMasking() && IndUpdate == Legal->getPrimaryInduction())
+      continue;
+
     if (llvm::all_of(IndUpdate->users(), [&](User *U) -> bool {
           return U == Ind || DeadInstructions.count(cast<Instruction>(U));
         }))

diff  --git a/llvm/test/Transforms/LoopVectorize/dead_instructions.ll b/llvm/test/Transforms/LoopVectorize/dead_instructions.ll
index fb929ee4ebda..06215aac9b5c 100644
--- a/llvm/test/Transforms/LoopVectorize/dead_instructions.ll
+++ b/llvm/test/Transforms/LoopVectorize/dead_instructions.ll
@@ -40,3 +40,32 @@ for.end:
   %tmp3  = phi i64 [ %tmp2, %for.body ]
   ret i64 %tmp3
 }
+
+
+; CHECK-LABEL: @pr47390
+;
+; This test ensures that the primary induction is not considered dead when
+; acting as the 'add' of another induction, and otherwise feeding only its own
+; 'add' (recognized earlier as 'dead'), when the tail of the loop is folded by
+; masking. Such masking uses the primary induction.
+;
+; CHECK:     vector.body:
+;
+define void @pr47390(i32 *%a) {
+entry:
+  br label %loop
+
+exit:
+  ret void
+
+loop:
+  %primary = phi i32 [ 0, %entry ], [ %primary_add, %loop ]
+  %use_primary = phi i32 [ -1, %entry ], [ %primary, %loop ]
+  %secondary = phi i32 [ 1, %entry ], [ %secondary_add, %loop ]
+  %primary_add = add i32 %primary, 1
+  %secondary_add = add i32 %secondary, 1
+  %gep = getelementptr inbounds i32, i32* %a, i32 %secondary
+  %load = load i32, i32* %gep, align 8
+  %cmp = icmp eq i32 %secondary, 5
+  br i1 %cmp, label %exit, label %loop
+}