[PATCH] D92017: [LV] Keep Primary Induction alive when folding tail by masking

[Ayal Zaks via Phabricator via llvm-commits]

Ayal created this revision.
Ayal added reviewers: fhahn, gilr, SjoerdMeijer, dmgreen.
Herald added subscribers: llvm-commits, hiraditya.
Herald added a project: LLVM.
Ayal requested review of this revision.

The primary induction should be considered alive when folding tail by masking, because it will be used by said masking; even when it may otherwise appear useless: feeding only its own 'bump', which is correctly considered dead, and as the 'bump' of another induction variable, which may wrongfully want to consider its bump = the primary induction, dead.

Fixes PR47390.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D92017

Files:
  llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
  llvm/test/Transforms/LoopVectorize/dead_instructions.ll


Index: llvm/test/Transforms/LoopVectorize/dead_instructions.ll
===================================================================
--- llvm/test/Transforms/LoopVectorize/dead_instructions.ll
+++ llvm/test/Transforms/LoopVectorize/dead_instructions.ll
@@ -40,3 +40,32 @@
   %tmp3  = phi i64 [ %tmp2, %for.body ]
   ret i64 %tmp3
 }
+
+
+; CHECK-LABEL: @pr47390
+;
+; This test ensures that the primary induction is not considered dead when
+; acting as the 'add' of another induction, and otherwise feeding only its own
+; 'add' (recognized earlier as 'dead'), when the tail of the loop if folded by
+; masking. Such masking uses the primary induction.
+;
+; CHECK:     vector.body:
+;
+define void @pr47390(i32 *%a) {
+entry:
+  br label %loop
+
+exit:
+  ret void
+
+loop:
+  %primary = phi i32 [ 0, %entry ], [ %primary_add, %loop ]
+  %use_primary = phi i32 [ -1, %entry ], [ %primary, %loop ]
+  %secondary = phi i32 [ 1, %entry ], [ %secondary_add, %loop ]
+  %primary_add = add i32 %primary, 1
+  %secondary_add = add i32 %secondary, 1
+  %gep = getelementptr inbounds i32, i32* %a, i32 %secondary
+  %load = load i32, i32* %gep, align 8
+  %cmp = icmp eq i32 %secondary, 5
+  br i1 %cmp, label %exit, label %loop
+}
Index: llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
===================================================================
--- llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+++ llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
@@ -7132,6 +7132,12 @@
   for (auto &Induction : Legal->getInductionVars()) {
     PHINode *Ind = Induction.first;
     auto *IndUpdate = cast<Instruction>(Ind->getIncomingValueForBlock(Latch));
+
+    // If the tail is to be folded by masking, the primary induction variable,
+    // if exists, isn't dead: it will be used for masking. Don't kill it.
+    if (CM.foldTailByMasking() && IndUpdate == Legal->getPrimaryInduction())
+      continue;
+
     if (llvm::all_of(IndUpdate->users(), [&](User *U) -> bool {
           return U == Ind || DeadInstructions.count(cast<Instruction>(U));
         }))


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D92017.307292.patch
Type: text/x-patch
Size: 2053 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20201124/ef24bb24/attachment.bin>