[llvm] [VPlan] Implement interleaving as VPlan-to-VPlan transform. (PR #95842)
via llvm-commits
llvm-commits at lists.llvm.org
Sun Jun 23 03:15:26 PDT 2024
================
@@ -3665,6 +3690,29 @@ inline bool isUniformAfterVectorization(VPValue *VPV) {
return VPI->isVectorToScalar();
return false;
}
+
+/// Checks if \p C is uniform across all VFs and UFs. It is considered as such
+/// if it is either defined outside the vector region or its operand is known to
+/// be uniform across all VFs and UFs (e.g. VPDerivedIV or VPCanonicalIVPHI).
+inline bool isUniformAcrossVFsAndUFs(VPValue *V) {
+ if (auto *VPI = dyn_cast_or_null<VPInstruction>(V->getDefiningRecipe())) {
+ return VPI ==
+ VPI->getParent()->getPlan()->getCanonicalIV()->getBackedgeValue();
+ }
+ if (isa<VPCanonicalIVPHIRecipe, VPDerivedIVRecipe, VPExpandSCEVRecipe>(V))
+ return true;
+ if (isa<VPReplicateRecipe>(V) && cast<VPReplicateRecipe>(V)->isUniform() &&
+ (isa<LoadInst, StoreInst>(V->getUnderlyingValue())) &&
+ all_of(V->getDefiningRecipe()->operands(),
+ [](VPValue *Op) { return Op->isDefinedOutsideVectorRegions(); }))
+ return true;
----------------
ayalz wrote:
```suggestion
// Loads and stores that are uniform across VF lanes are handled by VPReplicateRecipe.IsUniform. They are also uniform across UF parts if all their operands are invariant:
if (isa<VPReplicateRecipe>(V) && cast<VPReplicateRecipe>(V)->isUniform() &&
(isa<LoadInst, StoreInst>(V->getUnderlyingValue())) &&
all_of(R->operands(),
[](VPValue *Op) { return Op->isDefinedOutsideVectorRegions(); }))
return true;
```
https://github.com/llvm/llvm-project/pull/95842
More information about the llvm-commits
mailing list