[clang] [clang-tools-extra] [llvm] [mlir] [libcxx] [flang] [LV] Improve AnyOf reduction codegen. (PR #78304)
Florian Hahn via llvm-commits
llvm-commits at lists.llvm.org
Mon Feb 5 06:29:38 PST 2024
================
@@ -9110,6 +9111,41 @@ void LoopVectorizationPlanner::adjustRecipesForReductions(
continue;
const RecurrenceDescriptor &RdxDesc = PhiR->getRecurrenceDescriptor();
+ // Adjust AnyOf reductions; replace the reduction phi for the selected value
+ // with a boolean reduction phi node to check if the condition is true in
+ // any iteration. The final value is selected by the final
+ // ComputeReductionResult.
+ if (RecurrenceDescriptor::isAnyOfRecurrenceKind(
+ RdxDesc.getRecurrenceKind())) {
+ auto *Select = cast<VPRecipeBase>(*find_if(PhiR->users(), [](VPUser *U) {
+ return isa<VPWidenSelectRecipe>(U) ||
+ (isa<VPReplicateRecipe>(U) &&
+ cast<VPReplicateRecipe>(U)->getUnderlyingInstr()->getOpcode() ==
+ Instruction::Select);
+ }));
+ VPValue *Cmp = Select->getOperand(0);
+ // If the compare is checking the reduction PHI node, adjust it to check
+ // the start value.
----------------
fhahn wrote:
At the moment, AnyOf reduction are also formed for code like
```
define i32 @select_i32_from_icmp_same_inputs(i32 %a, i32 %b, i64 %n) {
entry:
br label %for.body
for.body: ; preds = %entry, %for.body
%0 = phi i64 [ 0, %entry ], [ %4, %for.body ]
%1 = phi i32 [ %a, %entry ], [ %3, %for.body ]
%2 = icmp eq i32 %1, 3
%3 = select i1 %2, i32 %1, i32 %b
%4 = add nuw nsw i64 %0, 1
%5 = icmp eq i64 %4, %n
br i1 %5, label %exit, label %for.body
exit: ; preds = %for.body
ret i32 %3
}
```
https://github.com/llvm/llvm-project/pull/78304
More information about the llvm-commits
mailing list