[llvm] [LAA] Speculate the interleave factor for strided pointer inductions (PR #216036)
Ramkumar Ramachandra via llvm-commits
llvm-commits at lists.llvm.org
Sat Aug 15 01:46:45 PDT 2026
================
@@ -153,13 +152,89 @@ bool VectorizerParams::isInterleaveForced() {
return ::VectorizationInterleave.getNumOccurrences() > 0;
}
+std::optional<unsigned>
+llvm::getSpeculatedInterleaveStride(const Loop &L, ScalarEvolution &SE,
+ const SCEV *Stride) {
+ // Casts on the stride do not change the value being guessed.
+ match(Stride, m_scev_IgnoreCasts(m_SCEV(Stride)));
+
+ // Matches anything that walks the loop by Stride, which is the induction
+ // itself and every access sitting at a fixed offset off it.
+ const SCEV *Start;
+ auto StridedBy = m_scev_AffineAddRec(
+ m_SCEV(Start), m_scev_IgnoreCasts(m_scev_Specific(Stride)),
+ m_SpecificLoop(&L));
+
+ // Find the pointer induction stepping by Stride. More than one of them makes
+ // "the run off the induction" ambiguous, so don't guess.
+ const SCEV *Base = nullptr;
+ for (PHINode &Phi : L.getHeader()->phis()) {
+ if (!Phi.getType()->isPointerTy() || !match(SE.getSCEV(&Phi), StridedBy))
+ continue;
+ if (Base)
+ return std::nullopt;
+ Base = Start;
+ }
+ if (!Base)
+ return std::nullopt;
+
+ // Collect the distinct element indices accessed off that induction. All
+ // members of an interleaved group have the same type, so a mismatched or
+ // misaligned width rules the group out rather than just skipping the access.
+ const DataLayout &DL = L.getHeader()->getDataLayout();
+ SmallSet<int64_t, 8> Indices;
+ int64_t MaxIdx = 0;
+ uint64_t EltSize = 0;
+ for (BasicBlock *BB : L.blocks()) {
+ for (Instruction &I : *BB) {
+ Value *Ptr = getLoadStorePointerOperand(&I);
+ if (!Ptr || !match(SE.getSCEV(Ptr), StridedBy))
----------------
artagnon wrote:
This is wasteful. Pass the Stride pointer directly as the key of StridesMap (called PtrToStride in the caller).
https://github.com/llvm/llvm-project/pull/216036
More information about the llvm-commits
mailing list