[llvm] [LV][NFC] Query UnrollVectorizedLoop before executing the plan (PR #208500)

[Igor Kirillov via llvm-commits]

https://github.com/igogo-x86 updated https://github.com/llvm/llvm-project/pull/208500

>From ae10f6ce640cf8b32d3cecd9a965c50971790987 Mon Sep 17 00:00:00 2001
From: Igor Kirillov <igor.kirillov at arm.com>
Date: Thu, 9 Jul 2026 16:15:33 +0000
Subject: [PATCH 1/2] [LV][NFC] Query UnrollVectorizedLoop before executing the
 plan

LoopVectorize decides whether to attach llvm.loop.unroll.runtime.disable
metadata to the vector loop based on a single target-wide bit,
UnrollingPreferences::UnrollVectorizedLoop (AMDGPU sets it
unconditionally since D149281). To read this bit, it called
getUnrollingPreferences on the newly created vector loop and passed the
vectorizer's ScalarEvolution, so SCEV was queried on IR in the middle
of the transform.

This has a visible side effect: any SCEV query at this point fills the
SCEV caches with expressions for the new loop skeleton, and this
changes which existing values SCEVExpander reuses later when it
expands the minimum-iteration checks for the epilogue. As a result, if
a target's getUnrollingPreferences queries SCEV (ARM already does,
AArch64 will in llvm/llvm-project#205102), the emitted IR depends on
when the hook runs.

UnrollVectorizedLoop is a target constant, so query it earlier, on the
original loop - before the plan executes and before the original
loop's SCEV is forgotten. At that point the query is a cache hit and
does not add any new vector-loop expressions to the caches.
---
 .../Transforms/Vectorize/LoopVectorizationPlanner.h    |  5 +++--
 llvm/lib/Transforms/Vectorize/LoopVectorize.cpp        | 10 +++++++++-
 llvm/lib/Transforms/Vectorize/VPlan.cpp                |  6 ++----
 3 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.h b/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.h
index fa317f290022e..e7814f7485946 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.h
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.h
@@ -991,13 +991,14 @@ class LoopVectorizationPlanner {
   /// the average trip count and invocation weight of the original loop (\p
   /// OrigAverageTripCount and \p OrigLoopInvocationWeight respectively). They
   /// cannot be retrieved after the plan has been executed, as the original loop
-  /// may have been removed.
+  /// may have been removed. \p UnrollVectorizedLoop, whether the target wants
+  /// the vector loop left eligible for runtime unrolling.
   void updateLoopMetadataAndProfileInfo(
       Loop *VectorLoop, VPBasicBlock *HeaderVPBB, const VPlan &Plan,
       bool VectorizingEpilogue, MDNode *OrigLoopID,
       std::optional<unsigned> OrigAverageTripCount,
       unsigned OrigLoopInvocationWeight, unsigned EstimatedVFxUF,
-      bool DisableRuntimeUnroll);
+      bool DisableRuntimeUnroll, bool UnrollVectorizedLoop);
 
 private:
   /// Build an initial VPlan, with HCFG wrapping the original scalar loop and
diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
index 3eb6d0530da6d..3a7a4b9aecc9e 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
@@ -6073,6 +6073,14 @@ DenseMap<const SCEV *, Value *> LoopVectorizationPlanner::executePlan(
       SE.forgetLcssaPhiWithNewPredecessor(OrigLoop,
                                           &cast<VPIRPhi>(PhiR).getIRPhi());
   }
+
+  // Query whether the target wants loops it vectorizes to remain eligible for
+  // runtime unrolling. Do this here, on the original loop and before its SCEV
+  // is forgotten below.
+  TargetTransformInfo::UnrollingPreferences UP;
+  TTI.getUnrollingPreferences(OrigLoop, SE, UP, ORE);
+  bool UnrollVectorizedLoop = UP.UnrollVectorizedLoop;
+
   // Forget the original loop and block dispositions.
   SE.forgetLoop(OrigLoop);
   SE.forgetBlockAndLoopDispositions();
@@ -6111,7 +6119,7 @@ DenseMap<const SCEV *, Value *> LoopVectorizationPlanner::executePlan(
       EpilogueVecKind == EpilogueVectorizationKind::Epilogue, LID,
       OrigAverageTripCount, OrigLoopInvocationWeight,
       estimateElementCount(BestVF * BestUF, Config.getVScaleForTuning()),
-      DisableRuntimeUnroll);
+      DisableRuntimeUnroll, UnrollVectorizedLoop);
 
   // 3. Fix the vectorized code: take care of header phi's, live-outs,
   //    predication, updating analyses.
diff --git a/llvm/lib/Transforms/Vectorize/VPlan.cpp b/llvm/lib/Transforms/Vectorize/VPlan.cpp
index 37060b15a7b97..033b00cf03a8b 100644
--- a/llvm/lib/Transforms/Vectorize/VPlan.cpp
+++ b/llvm/lib/Transforms/Vectorize/VPlan.cpp
@@ -1762,7 +1762,7 @@ void LoopVectorizationPlanner::updateLoopMetadataAndProfileInfo(
     bool VectorizingEpilogue, MDNode *OrigLoopID,
     std::optional<unsigned> OrigAverageTripCount,
     unsigned OrigLoopInvocationWeight, unsigned EstimatedVFxUF,
-    bool DisableRuntimeUnroll) {
+    bool DisableRuntimeUnroll, bool UnrollVectorizedLoop) {
   // Update the metadata of the scalar loop. Skip the update when vectorizing
   // the epilogue loop to ensure it is updated only once. Also skip the update
   // when the scalar loop became unreachable.
@@ -1811,9 +1811,7 @@ void LoopVectorizationPlanner::updateLoopMetadataAndProfileInfo(
   // emit when remarks are enabled.
   if (ORE->enabled())
     VectorLoop->addIntLoopAttribute("llvm.loop.vectorize.body", 1);
-  TargetTransformInfo::UnrollingPreferences UP;
-  TTI.getUnrollingPreferences(VectorLoop, *PSE.getSE(), UP, ORE);
-  if (!UP.UnrollVectorizedLoop || VectorizingEpilogue)
+  if (!UnrollVectorizedLoop || VectorizingEpilogue)
     addRuntimeUnrollDisableMetaData(VectorLoop);
 
   // Set/update profile weights for the vector and remainder loops as original

>From d6743315169985dc6d52ddbd638008b9eeda7731 Mon Sep 17 00:00:00 2001
From: Igor Kirillov <igor.kirillov at arm.com>
Date: Wed, 5 Aug 2026 10:29:00 +0000
Subject: [PATCH 2/2] Rebase and address comment's nit

---
 llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.h b/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.h
index e7814f7485946..d488607a0c7dc 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.h
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.h
@@ -991,8 +991,8 @@ class LoopVectorizationPlanner {
   /// the average trip count and invocation weight of the original loop (\p
   /// OrigAverageTripCount and \p OrigLoopInvocationWeight respectively). They
   /// cannot be retrieved after the plan has been executed, as the original loop
-  /// may have been removed. \p UnrollVectorizedLoop, whether the target wants
-  /// the vector loop left eligible for runtime unrolling.
+  /// may have been removed. \p UnrollVectorizedLoop indicates whether the
+  /// target wants the vector loop left eligible for runtime unrolling.
   void updateLoopMetadataAndProfileInfo(
       Loop *VectorLoop, VPBasicBlock *HeaderVPBB, const VPlan &Plan,
       bool VectorizingEpilogue, MDNode *OrigLoopID,