[all-commits] [llvm/llvm-project] 19e101: [SelectionDAG] Fix unsafe cases for loop.dependenc...

[Sam Tebbs via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 19e1011df51e3f88a02f5522073aa788e479a68b
      https://github.com/llvm/llvm-project/commit/19e1011df51e3f88a02f5522073aa788e479a68b
  Author: Sam Tebbs <samuel.tebbs at arm.com>
  Date:   2025-12-12 (Fri, 12 Dec 2025)

  Changed paths:
    M llvm/include/llvm/CodeGen/ISDOpcodes.h
    M llvm/include/llvm/Target/TargetSelectionDAG.td
    M llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp
    M llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp
    M llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
    M llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
    M llvm/lib/Target/AArch64/SVEInstrFormats.td
    M llvm/test/CodeGen/AArch64/alias_mask.ll
    M llvm/test/CodeGen/AArch64/alias_mask_nosve.ll
    M llvm/test/CodeGen/AArch64/alias_mask_scalable.ll
    M llvm/test/CodeGen/AArch64/alias_mask_scalable_nosve2.ll
    R llvm/test/CodeGen/AArch64/loop-dependence-mask-ccmp.ll

  Log Message:
  -----------
  [SelectionDAG] Fix unsafe cases for loop.dependence.{war/raw}.mask (#168565)

Both `LOOP_DEPENDENCE_WAR_MASK` and `LOOP_DEPENDENCE_RAW_MASK` are
currently hard to split correctly, and there are a number of incorrect
cases.

The difficulty comes from how the intrinsics are defined. For example,
take `LOOP_DEPENDENCE_WAR_MASK`.

It is defined as the OR of:

* `(ptrB - ptrA) <= 0`
* `elementSize * lane < (ptrB - ptrA)`

Now, if we want to split a loop dependence mask for the high half of the
mask we want to compute:

* `(ptrB - ptrA) <= 0`
* `elementSize * (lane + LoVT.getElementCount()) < (ptrB - ptrA)`

However, with the current opcode definitions, we can only modify ptrA or
ptrB, which may change the result of the first case, which should be
invariant to the lane.

This patch resolves these cases by adding a "lane offset" to the ISD
opcodes. The lane offset is always a constant. For scalable masks, it is
implicitly multiplied by vscale.

This makes splitting trivial as we increment the lane offset by
`LoVT.getElementCount()` now.

Note: In the AArch64 backend, we only support zero lane offsets (as
other cases are tricky to lower to whilewr/rw).

---------

Co-authored-by: Benjamin Maxwell <benjamin.maxwell at arm.com>



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications