[llvm] [RISCV][TTI] Enable masked interleave vectorization (PR #150074)
Mel Chen via llvm-commits
llvm-commits at lists.llvm.org
Wed Jul 23 01:05:25 PDT 2025
================
@@ -979,12 +979,14 @@ InstructionCost RISCVTTIImpl::getInterleavedMemoryOpCost(
Align Alignment, unsigned AddressSpace, TTI::TargetCostKind CostKind,
bool UseMaskForCond, bool UseMaskForGaps) const {
- // The interleaved memory access pass will lower interleaved memory ops (i.e
- // a load and store followed by a specific shuffle) to vlseg/vsseg
- // intrinsics.
- if (!UseMaskForCond && !UseMaskForGaps &&
+ auto *VTy = cast<VectorType>(VecTy);
+
+ // The interleaved memory access pass will lower (de)interleave ops combined
+ // with an adjacent appropriate memory to vlseg/vsseg intrinsics. We
+ // currently only support masking for the scalable path. vlseg/vsseg only
+ // support masking per-iteration (i.e. condition), not per-segment (i.e. gap).
+ if ((VTy->isScalableTy() || !UseMaskForCond) && !UseMaskForGaps &&
----------------
Mel-Chen wrote:
For fixed-length VF, shuffles are used instead of interleave intrinsics.
However, I believe getMask in the InterleavedAccessPass should already handle shuffle mask:
```
static Value *getMask(Value *WideMask, unsigned Factor,
ElementCount LeafValueEC) {
if (auto *IMI = dyn_cast<IntrinsicInst>(WideMask)) {
...
if (LeafValueEC.isFixed()) {
unsigned LeafMaskLen = LeafValueEC.getFixedValue();
SmallVector<Constant *, 8> LeafMask(LeafMaskLen, nullptr);
// If this is a fixed-length constant mask, each lane / leaf has to
// use the same mask. This is done by checking if every group with Factor
// number of elements in the interleaved mask has homogeneous values.
for (unsigned Idx = 0U; Idx < LeafMaskLen * Factor; ++Idx) {
Constant *C = ConstMask->getAggregateElement(Idx);
if (LeafMask[Idx / Factor] && LeafMask[Idx / Factor] != C)
return nullptr;
LeafMask[Idx / Factor] = C;
}
return ConstantVector::get(LeafMask);
}
}
return nullptr;
}
```
Fixed-length should be supported.
Therefore, I think doing it as shown in #149981 should be good enough, right?
```suggestion
if (!UseMaskForGaps && Factor <= TLI->getMaxSupportedInterleaveFactor())
```
https://github.com/llvm/llvm-project/pull/150074
More information about the llvm-commits
mailing list