[all-commits] [llvm/llvm-project] b5cae8: [SLP]Fix fmul/fadd fusion costs and retry FMA seed...

[Alexey Bataev via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: b5cae89e8463dd397960f8a37bb877a06306c0f4
      https://github.com/llvm/llvm-project/commit/b5cae89e8463dd397960f8a37bb877a06306c0f4
  Author: Alexey Bataev <a.bataev at outlook.com>
  Date:   2026-09-28 (Mon, 28 Sep 2026)

  Changed paths:
    M llvm/include/llvm/Transforms/Vectorize/SLPVectorizer.h
    M llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp
    M llvm/test/Transforms/PhaseOrdering/AArch64/reassociate-fma-pairs.ll
    M llvm/test/Transforms/SLPVectorizer/AArch64/extracts-folded-into-fmul-users-no-credit.ll
    M llvm/test/Transforms/SLPVectorizer/AArch64/fadd-with-gathered-fmul-operands.ll
    M llvm/test/Transforms/SLPVectorizer/AArch64/fma-candidates-after-store-chains.ll
    M llvm/test/Transforms/SLPVectorizer/AArch64/fma-chain-no-alt-node-reduction.ll
    M llvm/test/Transforms/SLPVectorizer/AArch64/fmul-constant-lane-fmuladd-combine.ll
    M llvm/test/Transforms/SLPVectorizer/AArch64/hor-fp-reduction-instcount.ll
    M llvm/test/Transforms/SLPVectorizer/AArch64/loop-accumulator-reduction.ll
    M llvm/test/Transforms/SLPVectorizer/AArch64/reduction-root-multi-use-fma.ll
    M llvm/test/Transforms/SLPVectorizer/AArch64/vec3-reorder-reshuffle.ll
    M llvm/test/Transforms/SLPVectorizer/AMDGPU/cross-block-fmul-fadd.ll
    M llvm/test/Transforms/SLPVectorizer/NVPTX/ordered-reduction-fma-fusion.ll
    M llvm/test/Transforms/SLPVectorizer/X86/fmul-fused-into-scalar-fadd.ll
    M llvm/test/Transforms/SLPVectorizer/X86/fsub-fmul-rhs-combine.ll
    M llvm/test/Transforms/SLPVectorizer/X86/slp-fma-loss-ordered.ll
    M llvm/test/Transforms/SLPVectorizer/consecutive-access.ll

  Log Message:
  -----------
  [SLP]Fix fmul/fadd fusion costs and retry FMA seeds after all blocks

Fix the cost for the scalar of fmul+fadd pairs: fused fmul lanes add
the full fmul cost, fadd lanes with gathered fmuls were costed as
fmuladd, and the FMulAdd combine missed fmuls in the second operand,
c - a*b and constant lanes. Cost fused fmul lanes at (fmuladd - fadd),
such fadd lanes at (fmuladd - fmul), form the combine for either
operand, and treat a multi-use reduction root as fused. Extract nodes
take no credit for extracts the target folds into their users.

Reviewers: RKSimon, bababuck, MrSidims

Pull Request: https://github.com/llvm/llvm-project/pull/226117



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications