[PATCH] D64972: [Loop Peeling] Do not close further unroll/peel if not full peeling happens

[Serguei Katkov via Phabricator via llvm-commits]

skatkov created this revision.
skatkov added reviewers: reames, fhahn.
Herald added subscribers: dmgreen, zzheng, hiraditya.
Herald added a project: LLVM.
skatkov added a parent revision: D64235: [Loop Peeling] Fix the handling of branch weights of peeled off branches.

Current peeling cost model can decide to peel off not all iteration
but only some of them to eliminate conditions on phi.At the same time 
if any peeling happens the door for further unroll/peel optimizations on that
loop closes because the part of the code thinks that if peeling happened
it is profile based peeling and all iterations are peeled off.

To resolve this inconsistency the patch provides the flag which states whether
the full peeling basing on profile is enabled or not and peeling cost model
is able to modify this field like it does not PeelCount.

In a separate patch I will introduce an option to allow/disallow peeling basing
on profile.


https://reviews.llvm.org/D64972

Files:
  llvm/include/llvm/Analysis/TargetTransformInfo.h
  llvm/lib/Transforms/Scalar/LoopUnrollPass.cpp
  llvm/lib/Transforms/Utils/LoopUnrollPeel.cpp


Index: llvm/lib/Transforms/Utils/LoopUnrollPeel.cpp
===================================================================
--- llvm/lib/Transforms/Utils/LoopUnrollPeel.cpp
+++ llvm/lib/Transforms/Utils/LoopUnrollPeel.cpp
@@ -275,6 +275,7 @@
     LLVM_DEBUG(dbgs() << "Force-peeling first " << UnrollForcePeelCount
                       << " iterations.\n");
     UP.PeelCount = UnrollForcePeelCount;
+    UP.FullPeeling = true;
     return;
   }
 
@@ -321,6 +322,7 @@
                         << " iteration(s) to turn"
                         << " some Phis into invariants.\n");
       UP.PeelCount = DesiredPeelCount;
+      UP.FullPeeling = false;
       return;
     }
   }
@@ -330,6 +332,9 @@
   if (TripCount)
     return;
 
+  // Do not apply profile base peeling if it is disabled.
+  if (!UP.FullPeeling)
+    return;
   // If we don't know the trip count, but have reason to believe the average
   // trip count is low, peeling should be beneficial, since we will usually
   // hit the peeled section.
Index: llvm/lib/Transforms/Scalar/LoopUnrollPass.cpp
===================================================================
--- llvm/lib/Transforms/Scalar/LoopUnrollPass.cpp
+++ llvm/lib/Transforms/Scalar/LoopUnrollPass.cpp
@@ -202,6 +202,7 @@
   UP.UpperBound = false;
   UP.AllowPeeling = true;
   UP.UnrollAndJam = false;
+  UP.FullPeeling = true;
   UP.UnrollAndJamInnerLoopThreshold = 60;
 
   // Override with any target specific settings
@@ -1139,7 +1140,7 @@
   // If the loop was peeled, we already "used up" the profile information
   // we had, so we don't want to unroll or peel again.
   if (UnrollResult != LoopUnrollResult::FullyUnrolled &&
-      (IsCountSetExplicitly || UP.PeelCount))
+      (IsCountSetExplicitly || (UP.FullPeeling && UP.PeelCount)))
     L->setLoopAlreadyUnrolled();
 
   return UnrollResult;
Index: llvm/include/llvm/Analysis/TargetTransformInfo.h
===================================================================
--- llvm/include/llvm/Analysis/TargetTransformInfo.h
+++ llvm/include/llvm/Analysis/TargetTransformInfo.h
@@ -475,6 +475,11 @@
     bool UnrollRemainder;
     /// Allow unroll and jam. Used to enable unroll and jam for the target.
     bool UnrollAndJam;
+    /// Allow full peeling. Uses to enable peeling off all iterations which
+    /// closes further unroll/peeling optimizations on the loop.
+    /// If the value is true the peeling cost model can decide to peel only
+    /// some iteration and in this case it will set this to false.
+    bool FullPeeling;
     /// Threshold for unroll and jam, for inner loop size. The 'Threshold'
     /// value above is used during unroll and jam for the outer loop size.
     /// This value is used in the same manner to limit the size of the inner


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D64972.210743.patch
Type: text/x-patch
Size: 2759 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20190719/dbd60982/attachment.bin>