[llvm] [RISCV][TTI] Return valid cost when step vector from extract.last.active needs to split. (PR #192161)
Elvis Wang via llvm-commits
llvm-commits at lists.llvm.org
Thu Apr 16 21:15:08 PDT 2026
https://github.com/ElvisWang123 updated https://github.com/llvm/llvm-project/pull/192161
>From 00adcfd463a8f45172a115324b14aa634a575207 Mon Sep 17 00:00:00 2001
From: Elvis Wang <elvis.wang at sifive.com>
Date: Mon, 13 Apr 2026 23:36:23 -0700
Subject: [PATCH 1/2] [RISCV][TTI] Enable find.last.active when step vector
needs split.
After https://github.com/llvm/llvm-project/pull/190914, the
find.last.active now can be lowered when the step vector needs to split.
This patch removes the constraint in TTI that will return the invalid
cost when step vector needs to split.
---
llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp | 6 ------
.../Analysis/CostModel/RISCV/extract-last-active.ll | 10 +++++-----
2 files changed, 5 insertions(+), 11 deletions(-)
diff --git a/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp b/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
index fd762f35124b5..a6338924a8f22 100644
--- a/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
+++ b/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
@@ -1740,12 +1740,6 @@ RISCVTTIImpl::getIntrinsicInstrCost(const IntrinsicCostAttributes &ICA,
auto *StepVecTy = VectorType::get(StepTy, ValTy->getElementCount());
auto StepLT = getTypeLegalizationCost(StepVecTy);
- // Currently expandVectorFindLastActive cannot handle step vector split.
- // So return invalid when the type needs split.
- // FIXME: Remove this if expandVectorFindLastActive supports split vector.
- if (StepLT.first > 1)
- return InstructionCost::getInvalid();
-
InstructionCost Cost = 0;
unsigned Opcodes[] = {RISCV::VID_V, RISCV::VREDMAXU_VS, RISCV::VMV_X_S};
diff --git a/llvm/test/Analysis/CostModel/RISCV/extract-last-active.ll b/llvm/test/Analysis/CostModel/RISCV/extract-last-active.ll
index 84f18e3df9015..d656a7733e675 100644
--- a/llvm/test/Analysis/CostModel/RISCV/extract-last-active.ll
+++ b/llvm/test/Analysis/CostModel/RISCV/extract-last-active.ll
@@ -11,7 +11,7 @@ define void @extractions() {
; CHECK-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %v8bf16 = call bfloat @llvm.experimental.vector.extract.last.active.v8bf16(<8 x bfloat> poison, <8 x i1> poison, bfloat poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %v4f32 = call float @llvm.experimental.vector.extract.last.active.v4f32(<4 x float> poison, <4 x i1> poison, float poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %v2f64 = call double @llvm.experimental.vector.extract.last.active.v2f64(<2 x double> poison, <2 x i1> poison, double poison)
-; CHECK-NEXT: Cost Model: Invalid cost for instruction: %nxv16i8 = call i8 @llvm.experimental.vector.extract.last.active.nxv16i8(<vscale x 16 x i8> poison, <vscale x 16 x i1> poison, i8 poison)
+; CHECK-NEXT: Cost Model: Found an estimated cost of 31 for instruction: %nxv16i8 = call i8 @llvm.experimental.vector.extract.last.active.nxv16i8(<vscale x 16 x i8> poison, <vscale x 16 x i1> poison, i8 poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 18 for instruction: %nxv8i16 = call i16 @llvm.experimental.vector.extract.last.active.nxv8i16(<vscale x 8 x i16> poison, <vscale x 8 x i1> poison, i16 poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 13 for instruction: %nxv4i32 = call i32 @llvm.experimental.vector.extract.last.active.nxv4i32(<vscale x 4 x i32> poison, <vscale x 4 x i1> poison, i32 poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %nxv2i64 = call i64 @llvm.experimental.vector.extract.last.active.nxv2i64(<vscale x 2 x i64> poison, <vscale x 2 x i1> poison, i64 poison)
@@ -27,12 +27,12 @@ define void @extractions() {
; CHECK-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %v16bf16 = call bfloat @llvm.experimental.vector.extract.last.active.v16bf16(<16 x bfloat> poison, <16 x i1> poison, bfloat poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %v8f32 = call float @llvm.experimental.vector.extract.last.active.v8f32(<8 x float> poison, <8 x i1> poison, float poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %v4f64 = call double @llvm.experimental.vector.extract.last.active.v4f64(<4 x double> poison, <4 x i1> poison, double poison)
-; CHECK-NEXT: Cost Model: Invalid cost for instruction: %nxv32i8 = call i8 @llvm.experimental.vector.extract.last.active.nxv32i8(<vscale x 32 x i8> poison, <vscale x 32 x i1> poison, i8 poison)
-; CHECK-NEXT: Cost Model: Invalid cost for instruction: %nxv16i16 = call i16 @llvm.experimental.vector.extract.last.active.nxv16i16(<vscale x 16 x i16> poison, <vscale x 16 x i1> poison, i16 poison)
+; CHECK-NEXT: Cost Model: Found an estimated cost of 59 for instruction: %nxv32i8 = call i8 @llvm.experimental.vector.extract.last.active.nxv32i8(<vscale x 32 x i8> poison, <vscale x 32 x i1> poison, i8 poison)
+; CHECK-NEXT: Cost Model: Found an estimated cost of 33 for instruction: %nxv16i16 = call i16 @llvm.experimental.vector.extract.last.active.nxv16i16(<vscale x 16 x i16> poison, <vscale x 16 x i1> poison, i16 poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %nxv8i32 = call i32 @llvm.experimental.vector.extract.last.active.nxv8i32(<vscale x 8 x i32> poison, <vscale x 8 x i1> poison, i32 poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 15 for instruction: %nxv4i64 = call i64 @llvm.experimental.vector.extract.last.active.nxv4i64(<vscale x 4 x i64> poison, <vscale x 4 x i1> poison, i64 poison)
-; CHECK-NEXT: Cost Model: Invalid cost for instruction: %nxv16f16 = call half @llvm.experimental.vector.extract.last.active.nxv16f16(<vscale x 16 x half> poison, <vscale x 16 x i1> poison, half poison)
-; CHECK-NEXT: Cost Model: Invalid cost for instruction: %nxv16bf16 = call bfloat @llvm.experimental.vector.extract.last.active.nxv16bf16(<vscale x 16 x bfloat> poison, <vscale x 16 x i1> poison, bfloat poison)
+; CHECK-NEXT: Cost Model: Found an estimated cost of 33 for instruction: %nxv16f16 = call half @llvm.experimental.vector.extract.last.active.nxv16f16(<vscale x 16 x half> poison, <vscale x 16 x i1> poison, half poison)
+; CHECK-NEXT: Cost Model: Found an estimated cost of 33 for instruction: %nxv16bf16 = call bfloat @llvm.experimental.vector.extract.last.active.nxv16bf16(<vscale x 16 x bfloat> poison, <vscale x 16 x i1> poison, bfloat poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %nxv8f32 = call float @llvm.experimental.vector.extract.last.active.nxv8f32(<vscale x 8 x float> poison, <vscale x 8 x i1> poison, float poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 15 for instruction: %nxv4f64 = call double @llvm.experimental.vector.extract.last.active.nxv4f64(<vscale x 4 x double> poison, <vscale x 4 x i1> poison, double poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %v8i8 = call i8 @llvm.experimental.vector.extract.last.active.v8i8(<8 x i8> poison, <8 x i1> poison, i8 poison)
>From a02d429bdbc17c89aae273538654512a1b37e746 Mon Sep 17 00:00:00 2001
From: Elvis Wang <elvis.wang at sifive.com>
Date: Thu, 16 Apr 2026 20:45:04 -0700
Subject: [PATCH 2/2] Fixup, consider extra slide,vcpop_m and extract.
---
.../Target/RISCV/RISCVTargetTransformInfo.cpp | 37 ++++++++++++++-----
.../CostModel/RISCV/extract-last-active.ll | 10 ++---
2 files changed, 33 insertions(+), 14 deletions(-)
diff --git a/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp b/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
index a6338924a8f22..c43d1341b611a 100644
--- a/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
+++ b/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
@@ -1736,21 +1736,40 @@ RISCVTTIImpl::getIntrinsicInstrCost(const IntrinsicCostAttributes &ICA,
TLI->getVectorIdxTy(getDataLayout()), MaskTy->getElementCount(),
/*ZeroIsPoison=*/true, &VScaleRange);
EltWidth = std::max(EltWidth, MaskTy->getScalarSizeInBits());
- Type *StepTy = Type::getIntNTy(MaskTy->getContext(), EltWidth);
- auto *StepVecTy = VectorType::get(StepTy, ValTy->getElementCount());
- auto StepLT = getTypeLegalizationCost(StepVecTy);
+ Type *FindLastActiveTy = Type::getIntNTy(MaskTy->getContext(), EltWidth);
+ auto *FindLastActiveVecTy =
+ VectorType::get(FindLastActiveTy, ValTy->getElementCount());
+ auto FindLastActiveLT = getTypeLegalizationCost(FindLastActiveVecTy);
InstructionCost Cost = 0;
- unsigned Opcodes[] = {RISCV::VID_V, RISCV::VREDMAXU_VS, RISCV::VMV_X_S};
Cost += MaskLT.first *
getRISCVInstructionCost(RISCV::VCPOP_M, MaskLT.second, CostKind);
Cost += getCFInstrCost(Instruction::CondBr, CostKind, nullptr);
- Cost += StepLT.first *
- getRISCVInstructionCost(Opcodes, StepLT.second, CostKind);
- Cost += getCastInstrCost(Instruction::ZExt,
- Type::getInt64Ty(ValTy->getContext()), StepTy,
- TTI::CastContextHint::None, CostKind, nullptr);
+
+ // Cost of FindLastActive
+ unsigned FindLastActiveOpcodes[] = {RISCV::VID_V, RISCV::VREDMAXU_VS,
+ RISCV::VMV_X_S};
+ InstructionCost FindLastActiveCost = getRISCVInstructionCost(
+ FindLastActiveOpcodes, FindLastActiveLT.second, CostKind);
+ Cost += FindLastActiveCost;
+
+ // If the step vector needs to be split, it will generate multiple levels of
+ // slidedown + reduce.or + select to find the target subvector needs to
+ // be extracted.
+ unsigned SelectSubVecOpcodes[] = {RISCV::VSLIDEDOWN_VI, RISCV::VCPOP_M};
+ if (FindLastActiveLT.first > 1)
+ Cost += (FindLastActiveLT.first - 1) *
+ (getRISCVInstructionCost(SelectSubVecOpcodes, MaskLT.second,
+ CostKind) +
+ getCFInstrCost(Instruction::CondBr, CostKind, nullptr) +
+ FindLastActiveCost);
+
+ Cost += getCastInstrCost(
+ Instruction::ZExt, Type::getInt64Ty(ValTy->getContext()),
+ FindLastActiveTy, TTI::CastContextHint::None, CostKind, nullptr);
+
+ // Cost of extract last active element.
Cost += ValLT.first *
getRISCVInstructionCost({RISCV::VSLIDEDOWN_VI, RISCV::VMV_X_S},
ValLT.second, CostKind);
diff --git a/llvm/test/Analysis/CostModel/RISCV/extract-last-active.ll b/llvm/test/Analysis/CostModel/RISCV/extract-last-active.ll
index d656a7733e675..5bbbfff2e48fb 100644
--- a/llvm/test/Analysis/CostModel/RISCV/extract-last-active.ll
+++ b/llvm/test/Analysis/CostModel/RISCV/extract-last-active.ll
@@ -11,7 +11,7 @@ define void @extractions() {
; CHECK-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %v8bf16 = call bfloat @llvm.experimental.vector.extract.last.active.v8bf16(<8 x bfloat> poison, <8 x i1> poison, bfloat poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 8 for instruction: %v4f32 = call float @llvm.experimental.vector.extract.last.active.v4f32(<4 x float> poison, <4 x i1> poison, float poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 7 for instruction: %v2f64 = call double @llvm.experimental.vector.extract.last.active.v2f64(<2 x double> poison, <2 x i1> poison, double poison)
-; CHECK-NEXT: Cost Model: Found an estimated cost of 31 for instruction: %nxv16i8 = call i8 @llvm.experimental.vector.extract.last.active.nxv16i8(<vscale x 16 x i8> poison, <vscale x 16 x i1> poison, i8 poison)
+; CHECK-NEXT: Cost Model: Found an estimated cost of 34 for instruction: %nxv16i8 = call i8 @llvm.experimental.vector.extract.last.active.nxv16i8(<vscale x 16 x i8> poison, <vscale x 16 x i1> poison, i8 poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 18 for instruction: %nxv8i16 = call i16 @llvm.experimental.vector.extract.last.active.nxv8i16(<vscale x 8 x i16> poison, <vscale x 8 x i1> poison, i16 poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 13 for instruction: %nxv4i32 = call i32 @llvm.experimental.vector.extract.last.active.nxv4i32(<vscale x 4 x i32> poison, <vscale x 4 x i1> poison, i32 poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %nxv2i64 = call i64 @llvm.experimental.vector.extract.last.active.nxv2i64(<vscale x 2 x i64> poison, <vscale x 2 x i1> poison, i64 poison)
@@ -27,12 +27,12 @@ define void @extractions() {
; CHECK-NEXT: Cost Model: Found an estimated cost of 11 for instruction: %v16bf16 = call bfloat @llvm.experimental.vector.extract.last.active.v16bf16(<16 x bfloat> poison, <16 x i1> poison, bfloat poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 10 for instruction: %v8f32 = call float @llvm.experimental.vector.extract.last.active.v8f32(<8 x float> poison, <8 x i1> poison, float poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %v4f64 = call double @llvm.experimental.vector.extract.last.active.v4f64(<4 x double> poison, <4 x i1> poison, double poison)
-; CHECK-NEXT: Cost Model: Found an estimated cost of 59 for instruction: %nxv32i8 = call i8 @llvm.experimental.vector.extract.last.active.nxv32i8(<vscale x 32 x i8> poison, <vscale x 32 x i1> poison, i8 poison)
-; CHECK-NEXT: Cost Model: Found an estimated cost of 33 for instruction: %nxv16i16 = call i16 @llvm.experimental.vector.extract.last.active.nxv16i16(<vscale x 16 x i16> poison, <vscale x 16 x i1> poison, i16 poison)
+; CHECK-NEXT: Cost Model: Found an estimated cost of 74 for instruction: %nxv32i8 = call i8 @llvm.experimental.vector.extract.last.active.nxv32i8(<vscale x 32 x i8> poison, <vscale x 32 x i1> poison, i8 poison)
+; CHECK-NEXT: Cost Model: Found an estimated cost of 36 for instruction: %nxv16i16 = call i16 @llvm.experimental.vector.extract.last.active.nxv16i16(<vscale x 16 x i16> poison, <vscale x 16 x i1> poison, i16 poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %nxv8i32 = call i32 @llvm.experimental.vector.extract.last.active.nxv8i32(<vscale x 8 x i32> poison, <vscale x 8 x i1> poison, i32 poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 15 for instruction: %nxv4i64 = call i64 @llvm.experimental.vector.extract.last.active.nxv4i64(<vscale x 4 x i64> poison, <vscale x 4 x i1> poison, i64 poison)
-; CHECK-NEXT: Cost Model: Found an estimated cost of 33 for instruction: %nxv16f16 = call half @llvm.experimental.vector.extract.last.active.nxv16f16(<vscale x 16 x half> poison, <vscale x 16 x i1> poison, half poison)
-; CHECK-NEXT: Cost Model: Found an estimated cost of 33 for instruction: %nxv16bf16 = call bfloat @llvm.experimental.vector.extract.last.active.nxv16bf16(<vscale x 16 x bfloat> poison, <vscale x 16 x i1> poison, bfloat poison)
+; CHECK-NEXT: Cost Model: Found an estimated cost of 36 for instruction: %nxv16f16 = call half @llvm.experimental.vector.extract.last.active.nxv16f16(<vscale x 16 x half> poison, <vscale x 16 x i1> poison, half poison)
+; CHECK-NEXT: Cost Model: Found an estimated cost of 36 for instruction: %nxv16bf16 = call bfloat @llvm.experimental.vector.extract.last.active.nxv16bf16(<vscale x 16 x bfloat> poison, <vscale x 16 x i1> poison, bfloat poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 20 for instruction: %nxv8f32 = call float @llvm.experimental.vector.extract.last.active.nxv8f32(<vscale x 8 x float> poison, <vscale x 8 x i1> poison, float poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 15 for instruction: %nxv4f64 = call double @llvm.experimental.vector.extract.last.active.nxv4f64(<vscale x 4 x double> poison, <vscale x 4 x i1> poison, double poison)
; CHECK-NEXT: Cost Model: Found an estimated cost of 9 for instruction: %v8i8 = call i8 @llvm.experimental.vector.extract.last.active.v8i8(<8 x i8> poison, <8 x i1> poison, i8 poison)
More information about the llvm-commits
mailing list