[llvm] [SLP] Account for fma fusion when vectorizing an ordered fadd reduction (PR #210399)

[Alexey Bataev via llvm-commits]

================
@@ -30521,9 +30217,30 @@ class HorizontalReduction {
         // Estimate cost.
         InstructionCost ReductionCost;
         if (RK == ReductionOrdering::Ordered || V.isReducedBitcastRoot() ||
-            V.isReducedCmpBitcastRoot())
+            V.isReducedCmpBitcastRoot()) {
           ReductionCost = 0;
-        else
+          // Check for potential fma fusion as vectorization would break it.
+          if (RdxKind == RecurKind::FAdd && RdxFMF.allowContract()) {
+            constexpr TTI::TargetCostKind CostKind = TTI::TCK_RecipThroughput;
+            Type *Ty = VL.front()->getType();
+            IntrinsicCostAttributes ICA(Intrinsic::fmuladd, Ty, {Ty, Ty, Ty},
+                                        RdxFMF);
+            InstructionCost FusionSaving =
+                TTI->getArithmeticInstrCost(Instruction::FMul, Ty, CostKind) +
+                TTI->getArithmeticInstrCost(Instruction::FAdd, Ty, CostKind) -
+                TTI->getIntrinsicInstrCost(ICA, CostKind);
+            if (FusionSaving.isValid() && FusionSaving > 0)
+              for (Value *RdxVal : VL) {
+                auto *FMul = dyn_cast<Instruction>(RdxVal);
+                if (FMul && FMul->getOpcode() == Instruction::FMul &&
+                    FMul->hasOneUse() &&
+                    cast<FPMathOperator>(FMul)
+                        ->getFastMathFlags()
+                        .allowContract())
+                  ReductionCost += FusionSaving;
+              }
----------------
alexey-bataev wrote:

Yea, precommit the tests first.
Regarding cost estimation, need to fix it somehow to make TTI drop the assumption that fmul-fadd is folded to fma.
The change itself is correct, just need to pass types instead of actual instructions

https://github.com/llvm/llvm-project/pull/210399