[all-commits] [llvm/llvm-project] 4e6925: [LoopVectorize] Add cost of generating tail-foldin...

[David Sherwood via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 4e69258bf390b51ed5bc65e7cea4cbcfbb62bd44
      https://github.com/llvm/llvm-project/commit/4e69258bf390b51ed5bc65e7cea4cbcfbb62bd44
  Author: David Sherwood <david.sherwood at arm.com>
  Date:   2025-03-21 (Fri, 21 Mar 2025)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
    M llvm/test/Transforms/LoopVectorize/AArch64/conditional-branches-cost.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/induction-costs-sve.ll
    A llvm/test/Transforms/LoopVectorize/AArch64/masked-call-scalarize.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/masked-call.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/reduction-recurrence-costs-sve.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/sve-tail-folding-cost.ll
    A llvm/test/Transforms/LoopVectorize/RISCV/tail-folding-cost.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/uniform-load-store.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/vectorize-force-tail-with-evl-uniform-store.ll

  Log Message:
  -----------
  [LoopVectorize] Add cost of generating tail-folding mask to the loop (#130565)

At the moment if we decide to enable tail-folding we do not include
the cost of generating the mask per VF. This can mean we make some
poor choices of VF, which is definitely true for SVE-enabled AArch64
targets where mask generation for fixed-width vectors is more
expensive than for scalable vectors.

I've added a VPInstruction::computeCost function to return the costs
of the ActiveLaneMask and ExplicitVectorLength operations.
Unfortunately, in order to prevent asserts firing I've also had to
duplicate the same code in the legacy cost model to make sure the
chosen VFs match up. I've wrapped this up in a ifndef NDEBUG for
now. The alternative would be to disable the assert completely when
tail-folding, which I imagine is just as bad.

New tests added:

  Transforms/LoopVectorize/AArch64/sve-tail-folding-cost.ll
  Transforms/LoopVectorize/RISCV/tail-folding-cost.ll



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications