[llvm] [LV] Handle FSub Partial Reductions (PR #191186)

Benjamin Maxwell via llvm-commits llvm-commits at lists.llvm.org
Thu Apr 16 03:33:34 PDT 2026


================
@@ -9104,17 +9058,30 @@ static SmallVector<Instruction *> preparePlanForEpilogueVectorLoop(
           // Partial sub-reductions always start at 0 and account for the
           // reduction start value in a final subtraction. Update it to use the
           // resume value from the main vector loop.
-          if (PhiR->getVFScaleFactor() > 1 &&
-              PhiR->getRecurrenceKind() == RecurKind::Sub) {
+          if ((PhiR->getVFScaleFactor() > 1 &&
+               PhiR->getRecurrenceKind() == RecurKind::Sub) ||
+              PhiR->getRecurrenceKind() == RecurKind::FSub) {
             auto *Sub = cast<VPInstruction>(RdxResult->getSingleUser());
-            assert(Sub->getOpcode() == Instruction::Sub && "Unexpected opcode");
+            assert((Sub->getOpcode() == Instruction::Sub ||
+                    Sub->getOpcode() == Instruction::FSub) &&
+                   "Unexpected opcode");
             assert(isa<VPIRValue>(Sub->getOperand(0)) &&
                    "Expected operand to match the original start value of the "
                    "reduction");
-            assert(VPlanPatternMatch::match(VPI->getOperand(0),
-                                            VPlanPatternMatch::m_ZeroInt()) &&
-                   "Expected start value for partial sub-reduction to start at "
-                   "zero");
+            // For integer sub-reductions, verify start value is zero.
+            // For FP sub-reductions, verify start value is negative zero.
+            if (PhiR->getRecurrenceKind() == RecurKind::Sub) {
+              assert(VPlanPatternMatch::match(VPI->getOperand(0),
+                                              VPlanPatternMatch::m_ZeroInt()) &&
+                     "Expected start value for partial sub-reduction to start "
+                     "at zero");
+            } else if (PhiR->getRecurrenceKind() == RecurKind::FSub) {
+              assert(
+                  VPlanPatternMatch::match(VPI->getOperand(0),
+                                           VPlanPatternMatch::m_NegZeroFP()) &&
+                  "Expected start value for partial sub-reduction to start "
+                  "at negative zero");
+            }
----------------
MacDue wrote:

nit: I think you write this as:
```c++
            // For integer sub-reductions, verify start value is zero.
            // For FP sub-reductions, verify start value is negative zero.
            [[maybe_unused]] auto IsStartValueIsIdentity = [&] {
              Value *IdentityValue = getRecurrenceIdentity(
                  PhiR->getRecurrenceKind(), ResumeV->getType(), {});
              auto *StartValue = dyn_cast<VPIRValue>(VPI->getOperand(0));
              return StartValue && StartValue->getValue() == IdentityValue;
            };
            assert(IsStartValueIsIdentity() &&
                   "Expected start value for partial sub-reduction to be the "
                   "zero (or negative zero)");
```
Which would help simplify this patch as we don't need `m_NegZeroFP()` (some code paths of that matcher seem untested). 

https://github.com/llvm/llvm-project/pull/191186


More information about the llvm-commits mailing list