[llvm] [LV] Don't calculate scalar costs for scalable VFs in setVectorizedCallDecision (PR #152713)

[David Sherwood via llvm-commits]

https://github.com/david-arm updated https://github.com/llvm/llvm-project/pull/152713

>From b3082f38441a57ad98e75c4ff1e8dd0bc9a00ff9 Mon Sep 17 00:00:00 2001
From: David Sherwood <david.sherwood at arm.com>
Date: Fri, 8 Aug 2025 13:28:47 +0000
Subject: [PATCH 1/2] [LV] Don't calculate scalar costs for scalable VFs in
 setVectorizedCallDecision

In setVectorizedCallDecision we attempt to calculate the scalar
costs for vectorisation calls, even for scalable VFs where we
already know the answer is Invalid. We can avoid doing unnecessary
work by skipping this completely for scalable vectors.
---
 .../Transforms/Vectorize/LoopVectorize.cpp    | 20 ++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
index 70f884016d08c..5c0dfcb6f1ab5 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
@@ -5832,14 +5832,20 @@ void LoopVectorizationCostModel::setVectorizedCallDecision(ElementCount VF) {
       // assumed to be vectors, so we need to extract individual elements from
       // there, execute VF scalar calls, and then gather the result into the
       // vector return value.
-      InstructionCost ScalarCallCost =
-          TTI.getCallInstrCost(ScalarFunc, ScalarRetTy, ScalarTys, CostKind);
-
-      // Compute costs of unpacking argument values for the scalar calls and
-      // packing the return values to a vector.
-      InstructionCost ScalarizationCost = getScalarizationOverhead(CI, VF);
+      if (VF.isFixed()) {
+        InstructionCost ScalarCallCost =
+            TTI.getCallInstrCost(ScalarFunc, ScalarRetTy, ScalarTys, CostKind);
+
+        // Compute costs of unpacking argument values for the scalar calls and
+        // packing the return values to a vector.
+        InstructionCost ScalarizationCost = getScalarizationOverhead(CI, VF);
+        ScalarCost = ScalarCallCost * VF.getKnownMinValue() + ScalarizationCost;
+      } else {
+        // There is no point attempting to calculate the scalar cost for a
+        // scalable VF as we know it will be Invalid.
+        ScalarCost = InstructionCost::getInvalid();
+      }
 
-      ScalarCost = ScalarCallCost * VF.getKnownMinValue() + ScalarizationCost;
       // Honor ForcedScalars and UniformAfterVectorization decisions.
       // TODO: For calls, it might still be more profitable to widen. Use
       // VPlan-based cost model to compare different options.

>From 566e3c14ec04d24de72d75e428b1dce58730ca69 Mon Sep 17 00:00:00 2001
From: David Sherwood <david.sherwood at arm.com>
Date: Wed, 20 Aug 2025 08:31:16 +0000
Subject: [PATCH 2/2] Address review comment

---
 llvm/lib/Transforms/Vectorize/LoopVectorize.cpp | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
index 5c0dfcb6f1ab5..2ed8b31d11ff8 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
@@ -5843,6 +5843,8 @@ void LoopVectorizationCostModel::setVectorizedCallDecision(ElementCount VF) {
       } else {
         // There is no point attempting to calculate the scalar cost for a
         // scalable VF as we know it will be Invalid.
+        assert(!getScalarizationOverhead(CI, VF).isValid() &&
+               "Unexpected valid cost for scalarizing scalable vectors");
         ScalarCost = InstructionCost::getInvalid();
       }