[llvm] [SLP]Fix fmul/fadd fusion costs and retry FMA seeds after all blocks (PR #226117)

[Dmitry Sidorov via llvm-commits]

================
@@ -13180,12 +13197,89 @@ bool BoUpSLP::areAllUsersVectorized(
          });
 }
 
-static InstructionCost canConvertToFMA(ArrayRef<Value *> VL,
-                                       const InstructionsState &S,
-                                       DominatorTree &DT, const DataLayout &DL,
-                                       TargetTransformInfo &TTI,
-                                       const TargetLibraryInfo &TLI,
-                                       const BoUpSLP &R);
+static InstructionCost
+canConvertToFMA(ArrayRef<Value *> VL, const InstructionsState &S,
+                DominatorTree &DT, const DataLayout &DL,
+                TargetTransformInfo &TTI, const TargetLibraryInfo &TLI,
+                const BoUpSLP &R, bool FuseEitherOperand = true);
+
+/// \returns the fmul operand of the fadd/fsub \p I that the backend fuses with
+/// \p I into an fmuladd: a contractable single-use fmul from the same block,
+/// from either operand, the first one preferred, c - a*b included. \p I must
+/// allow contraction.
+static Instruction *getFusableFMulOperand(Instruction *I) {
+  if ((I->getOpcode() != Instruction::FAdd &&
+       I->getOpcode() != Instruction::FSub) ||
+      !I->hasAllowContract())
+    return nullptr;
+  for (Value *Op : I->operands()) {
+    auto *FMul = dyn_cast<Instruction>(Op);
+    if (FMul && FMul->getOpcode() == Instruction::FMul && FMul->hasOneUse() &&
+        FMul->hasAllowContract() && FMul->getParent() == I->getParent())
----------------
MrSidims wrote:

Rather than checking, if `fmul` and `fadd/fsub` are in the same block I suggest to also ask `TTI.isProfitableToSinkOperands` whether the target sinks the `fmul` to its user.

https://github.com/llvm/llvm-project/pull/226117