[PATCH] D90055: [ARM] Don't aggressively unroll vector remainder loops

[Dave Green via Phabricator via llvm-commits]

dmgreen created this revision.
dmgreen added reviewers: samparker, SjoerdMeijer, samtebbs, simon_tatham, ostannard.
Herald added subscribers: danielkiss, zzheng, hiraditya, kristof.beyls.
Herald added a reviewer: rengolin.
Herald added a project: LLVM.
dmgreen requested review of this revision.

We already do not unroll loops with vector instructions under MVE, but that does not include the remainder loops that the vectorizer produces. These remainder loops will be rarely executed and are not worth unrolling, as the trip count is likely to be low if they get executed at all. Luckily they get llvm.loop.isvectorized to make recognizing them simpler.

We have wanted to do this for a while but hit issues with low overhead loops being reverted due to difficult registry allocation. With recent changes that seems to be less of an issue now.


https://reviews.llvm.org/D90055

Files:
  llvm/include/llvm/Transforms/Utils/LoopUtils.h
  llvm/lib/Target/ARM/ARMTargetTransformInfo.cpp
  llvm/lib/Transforms/Utils/LoopUtils.cpp
  llvm/test/Transforms/LoopUnroll/ARM/mve-nounroll.ll


Index: llvm/test/Transforms/LoopUnroll/ARM/mve-nounroll.ll
===================================================================
--- llvm/test/Transforms/LoopUnroll/ARM/mve-nounroll.ll
+++ llvm/test/Transforms/LoopUnroll/ARM/mve-nounroll.ll
@@ -70,15 +70,11 @@
 
 ; CHECK-LABEL: @remainder
 ; CHECK: vector.body:
-; CHECK:   br i1 %13, label %middle.block, label %vector.body, !llvm.loop !0
+; CHECK:   br i1 %7, label %middle.block, label %vector.body, !llvm.loop !0
 ; CHECK: middle.block:
 ; CHECK:   br i1 %cmp.n, label %for.cond.cleanup, label %for.body.preheader13
 ; CHECK: for.body:
-; CHECK:   br i1 %exitcond.3, label %for.cond.cleanup.loopexit.unr-lcssa, label %for.body, !llvm.loop !0
-; CHECK: for.body.prol.1:
-; CHECK:   br i1 %prol.iter.cmp.1, label %for.body.prol.2, label %for.body.prol.loopexit.unr-lcssa
-; CHECK: for.body.prol.2:
-; CHECK:   br label %for.body.prol.loopexit.unr-lcssa
+; CHECK:   br i1 %exitcond, label %for.cond.cleanup.loopexit, label %for.body, !llvm.loop !0
 
 define void @remainder(float* %s1, float* %s2, float* %d, i32 %n) {
 entry:
Index: llvm/lib/Transforms/Utils/LoopUtils.cpp
===================================================================
--- llvm/lib/Transforms/Utils/LoopUtils.cpp
+++ llvm/lib/Transforms/Utils/LoopUtils.cpp
@@ -297,7 +297,7 @@
   llvm_unreachable("unexpected number of options");
 }
 
-static bool getBooleanLoopAttribute(const Loop *TheLoop, StringRef Name) {
+bool llvm::getBooleanLoopAttribute(const Loop *TheLoop, StringRef Name) {
   return getOptionalBoolLoopAttribute(TheLoop, Name).getValueOr(false);
 }
 
Index: llvm/lib/Target/ARM/ARMTargetTransformInfo.cpp
===================================================================
--- llvm/lib/Target/ARM/ARMTargetTransformInfo.cpp
+++ llvm/lib/Target/ARM/ARMTargetTransformInfo.cpp
@@ -1896,6 +1896,10 @@
   if (ST->hasBranchPredictor() && L->getNumBlocks() > 4)
     return;
 
+  // Don't unroll vectorized loops, including the remainder loop
+  if (getBooleanLoopAttribute(L, "llvm.loop.isvectorized"))
+    return;
+
   // Scan the loop: don't unroll loops with calls as this could prevent
   // inlining.
   unsigned Cost = 0;
Index: llvm/include/llvm/Transforms/Utils/LoopUtils.h
===================================================================
--- llvm/include/llvm/Transforms/Utils/LoopUtils.h
+++ llvm/include/llvm/Transforms/Utils/LoopUtils.h
@@ -271,6 +271,9 @@
 void addStringMetadataToLoop(Loop *TheLoop, const char *MDString,
                              unsigned V = 0);
 
+/// Returns true if Name is applied to TheLoop and enabled.
+bool getBooleanLoopAttribute(const Loop *TheLoop, StringRef Name);
+
 /// Returns a loop's estimated trip count based on branch weight metadata.
 /// In addition if \p EstimatedLoopInvocationWeight is not null it is
 /// initialized with weight of loop's latch leading to the exit.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D90055.300316.patch
Type: text/x-patch
Size: 2864 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20201023/45fec290/attachment.bin>