[all-commits] [llvm/llvm-project] 59ed97: [NFCI][VPlan] Split initial mem-widening into a se...

Fri Mar 6 10:11:36 PST 2026

  Branch: refs/heads/users/eas/vplan-based-stride-mv-rt-guard
  Home:   https://github.com/llvm/llvm-project
  Commit: 59ed97daf691cd9ac68780d20352f72aba1c1efd
      https://github.com/llvm/llvm-project/commit/59ed97daf691cd9ac68780d20352f72aba1c1efd
  Author: Andrei Elovikov <andrei.elovikov at sifive.com>
  Date:   2026-03-06 (Fri, 06 Mar 2026)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPRecipeBuilder.h
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.h
    M llvm/test/Transforms/LoopVectorize/AArch64/predication_costs.ll
    M llvm/test/Transforms/LoopVectorize/VPlan/vplan-print-after-all.ll

  Log Message:
  -----------
  [NFCI][VPlan] Split initial mem-widening into a separate transformation

Preparation change before implementing stride-multiversioning as a
VPlan-based transformation. Might help
https://github.com/llvm/llvm-project/pull/147297/ as well.

  Commit: c07a1db537897ea65a10a5f6830568b376c05a06
      https://github.com/llvm/llvm-project/commit/c07a1db537897ea65a10a5f6830568b376c05a06
  Author: Andrei Elovikov <andrei.elovikov at sifive.com>
  Date:   2026-03-06 (Fri, 06 Mar 2026)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

  Log Message:
  -----------
  Don't make unnecessary captures

  Commit: 016715a6f8d145520e8304cadefd93e32884ff07
      https://github.com/llvm/llvm-project/commit/016715a6f8d145520e8304cadefd93e32884ff07
  Author: Andrei Elovikov <andrei.elovikov at sifive.com>
  Date:   2026-03-06 (Fri, 06 Mar 2026)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.h

  Log Message:
  -----------
  Move to VPlanTransforms, have to pass Legal explicitly

  Commit: 050a33d5254c63571e00e518fd09f7a4481325a7
      https://github.com/llvm/llvm-project/commit/050a33d5254c63571e00e518fd09f7a4481325a7
  Author: Andrei Elovikov <andrei.elovikov at sifive.com>
  Date:   2026-03-06 (Fri, 06 Mar 2026)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp

  Log Message:
  -----------
  Braces for outer `if`

  Commit: 771dbc2a6bcd1af0cd678c9adc9b6986708f8d6a
      https://github.com/llvm/llvm-project/commit/771dbc2a6bcd1af0cd678c9adc9b6986708f8d6a
  Author: Andrei Elovikov <andrei.elovikov at sifive.com>
  Date:   2026-03-06 (Fri, 06 Mar 2026)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPRecipeBuilder.h
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp

  Log Message:
  -----------
  Fold one `Legal` use into `tryToWidenHistogram` renamed to `widenIfHistogram`

  Commit: d21e4ccb6e84f3a65baf7d538621f5454999be6e
      https://github.com/llvm/llvm-project/commit/d21e4ccb6e84f3a65baf7d538621f5454999be6e
  Author: Andrei Elovikov <andrei.elovikov at sifive.com>
  Date:   2026-03-06 (Fri, 06 Mar 2026)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPRecipeBuilder.h
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.h

  Log Message:
  -----------
  Move another `Legal` use to `VPRecipeBuilder::replaceWithFinalIfReductionStore`

  Commit: c5dff26233e78184dfb3fa0dbe732cbaa2d45c16
      https://github.com/llvm/llvm-project/commit/c5dff26233e78184dfb3fa0dbe732cbaa2d45c16
  Author: Andrei Elovikov <andrei.elovikov at sifive.com>
  Date:   2026-03-06 (Fri, 06 Mar 2026)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.h
    M llvm/test/Transforms/LoopVectorize/VPlan/vplan-print-after-all.ll

  Log Message:
  -----------
  [NFC][VPlan] Split `makeMemOpWideningDecisions` into subpasses

The idea is to have handling of strided memory operations (either from
https://github.com/llvm/llvm-project/pull/147297 or for VPlan-based
multiversioning for unit-strided accesses) done after some mandatory
processing has been performed (e.g., some types **must** be scalarized)
but before legacy CM's decision to widen (gather/scatter) or scalarize
has been committed.

And in longer term, we can uplift all other memory widening decision to
be done here directly at VPlan level. I expect this structure would also
be beneficial for that.

  Commit: 5c15036066b24f1e271ba5d7fee6a0fd8ef830a0
      https://github.com/llvm/llvm-project/commit/5c15036066b24f1e271ba5d7fee6a0fd8ef830a0
  Author: Andrei Elovikov <andrei.elovikov at sifive.com>
  Date:   2026-03-06 (Fri, 06 Mar 2026)

  Changed paths:
    A llvm/test/Transforms/LoopVectorize/VPlan/vplan-based-stride-mv.ll
    A llvm/test/Transforms/LoopVectorize/vplan-based-stride-mv.ll

  Log Message:
  -----------
  [NFC][VPlan] Add initial tests for future VPlan-based stride MV

I tried to include both the features that current
LoopAccessAnalysis-based transformation supports (e.g., trunc/sext of
stride) but also cases where the current implementation behaves poorly,
e.g., https://godbolt.org/z/h31c3zKxK; as well as some other potentially
interesting scenarios I could imagine.

The are two test files with the same content. One is for VPlan dump change of
the future transformation alone (I'll update `-vplan-print-after` in the next
PR), another is for the full vectorizer pipeline. The latter have two `RUN:`
lines:
 * No multiversioning, so the next PR diff can show the transformation itself
 * Stride multiversionin performed in LAA, so that we can compare future
   VPlan-based transformation vs old behavior.

  Commit: 69cb59f8b875ced93dd19913940720c05c44bd16
      https://github.com/llvm/llvm-project/commit/69cb59f8b875ced93dd19913940720c05c44bd16
  Author: Andrei Elovikov <andrei.elovikov at sifive.com>
  Date:   2026-03-06 (Fri, 06 Mar 2026)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.h
    M llvm/test/Transforms/LoopVectorize/VPlan/vplan-print-after-all.ll
    M llvm/test/Transforms/LoopVectorize/X86/drop-poison-generating-flags.ll
    M llvm/test/Transforms/LoopVectorize/pr37248.ll
    M llvm/test/Transforms/LoopVectorize/runtime-check-needed-but-empty.ll

  Log Message:
  -----------
  [VPlan] Scalarize to first-lane-only directly on VPlan

This is needed to enable subsequent https://github.com/llvm/llvm-project/pull/182595.

I don't think we can fully port all scalarization logic from the legacy
path to VPlan-based right now because that would require us to introduce
interleave groups much earlier in VPlan pipeline, and without that we
can't really `assert` this new decision matches the previous CM-based
one. And without those `assert`s it's really hard to ensure we properly
port all the previous logic.

As such, I decided just to implement something much simpler that would
be enough for #182595. However, we perform this transformation before
delegating to the old CM-based decision, so it **is** effective
immediately and taking precedence even for consecutive loads/stores
right away.

Depends on https://github.com/llvm/llvm-project/pull/182592 but is stacked on
top of https://github.com/llvm/llvm-project/pull/182594 to enable linear
stacking for https://github.com/llvm/llvm-project/pull/182595.

  Commit: d04ab324465bbcb739a24e166c82086b05251ccf
      https://github.com/llvm/llvm-project/commit/d04ab324465bbcb739a24e166c82086b05251ccf
  Author: Andrei Elovikov <andrei.elovikov at sifive.com>
  Date:   2026-03-06 (Fri, 06 Mar 2026)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp

  Log Message:
  -----------
  Use `reverse`/`IsaPred`

  Commit: dda7972dc45389caf9a8b8c8dba2349de6056dcf
      https://github.com/llvm/llvm-project/commit/dda7972dc45389caf9a8b8c8dba2349de6056dcf
  Author: Andrei Elovikov <andrei.elovikov at sifive.com>
  Date:   2026-03-06 (Fri, 06 Mar 2026)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.h

  Log Message:
  -----------
  Don't pass RecipeBuilder

Legacy calls `setRecipe` on all processed recipes but really queries `getRecipe`
for memory operations only, that we don't touch in the scalarization as that
happens after all memory recipes has been processed.

  Commit: 56ea5c963f27dde8d7385dad8224c9d38c962341
      https://github.com/llvm/llvm-project/commit/56ea5c963f27dde8d7385dad8224c9d38c962341
  Author: Andrei Elovikov <andrei.elovikov at sifive.com>
  Date:   2026-03-06 (Fri, 06 Mar 2026)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp

  Log Message:
  -----------
  Just use `vputils::onlyFirstLaneUsed`

  Commit: dd155ea613313dcc674ef87b4a9b2260b53fdbb5
      https://github.com/llvm/llvm-project/commit/dd155ea613313dcc674ef87b4a9b2260b53fdbb5
  Author: Andrei Elovikov <andrei.elovikov at sifive.com>
  Date:   2026-03-06 (Fri, 06 Mar 2026)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.h
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    M llvm/lib/Transforms/Vectorize/VPRecipeBuilder.h
    M llvm/lib/Transforms/Vectorize/VPlan.cpp
    M llvm/lib/Transforms/Vectorize/VPlan.h
    M llvm/lib/Transforms/Vectorize/VPlanAnalysis.cpp
    M llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
    M llvm/lib/Transforms/Vectorize/VPlanTransforms.h
    M llvm/lib/Transforms/Vectorize/VPlanUtils.cpp
    M llvm/test/Transforms/LoopVectorize/VPlan/vplan-based-stride-mv.ll
    M llvm/test/Transforms/LoopVectorize/vplan-based-stride-mv.ll

  Log Message:
  -----------
  [VPlan] Implement VPlan-based stride speculation

Compare: https://github.com/llvm/llvm-project/compare/410385599759...dd155ea61331

To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications