[llvm] [VPlan] Skip uses-scalars restriction if one of ops needs broadcast. (PR #168246)
Ramkumar Ramachandra via llvm-commits
llvm-commits at lists.llvm.org
Mon Nov 24 10:57:25 PST 2025
================
@@ -1476,32 +1476,36 @@ static void narrowToSingleScalarRecipes(VPlan &Plan) {
continue;
}
- // Skip recipes that aren't single scalars or don't have only their
- // scalar results used. In the latter case, we would introduce extra
- // broadcasts.
- if (!vputils::isSingleScalar(RepOrWidenR) ||
- !all_of(RepOrWidenR->users(), [RepOrWidenR](const VPUser *U) {
- if (auto *Store = dyn_cast<VPWidenStoreRecipe>(U)) {
- // VPWidenStore doesn't have users, and stores are always
- // profitable to widen: hence, permitting address and mask
- // operands, and single-scalar stored values is an important leaf
- // condition. The assert must hold as we checked the RepOrWidenR
- // operand against vputils::isSingleScalar.
- assert(RepOrWidenR != Store->getStoredValue() ||
- vputils::isSingleScalar(Store->getStoredValue()));
- (void)Store;
- return true;
- }
-
- if (auto *VPI = dyn_cast<VPInstruction>(U)) {
- unsigned Opcode = VPI->getOpcode();
- if (Opcode == VPInstruction::ExtractLastElement ||
- Opcode == VPInstruction::ExtractLastLanePerPart ||
- Opcode == VPInstruction::ExtractPenultimateElement)
- return true;
- }
-
- return U->usesScalars(RepOrWidenR);
+ // Skip recipes that aren't single scalars.
+ if (!vputils::isSingleScalar(RepOrWidenR))
+ continue;
+
+ // Skip recipes for which conversion to single-scalar does introduce
+ // additional broadcasts. No extra broadcasts are needed, if either only
+ // the scalars of the recipe are used, or at least one of the operands
+ // would require a broadcast. In the latter case, the single-scalar may
+ // need to be broadcasted, but another broadcast is removed.
+ if (!all_of(RepOrWidenR->users(),
+ [RepOrWidenR](const VPUser *U) {
+ if (auto *VPI = dyn_cast<VPInstruction>(U)) {
+ unsigned Opcode = VPI->getOpcode();
+ if (Opcode == VPInstruction::ExtractLastElement ||
+ Opcode == VPInstruction::ExtractLastLanePerPart ||
+ Opcode == VPInstruction::ExtractPenultimateElement)
+ return true;
+ }
+
----------------
artagnon wrote:
nit: strip newline?
https://github.com/llvm/llvm-project/pull/168246
More information about the llvm-commits
mailing list