[all-commits] [llvm/llvm-project] 6e30f3: [AMDGPU] always emit a soft wait even if it is tri...

[Sameer Sahasrabuddhe via All-commits]

  Branch: refs/heads/users/ssahasra/waitcnt-always-emit
  Home:   https://github.com/llvm/llvm-project
  Commit: 6e30f375f14ce77ebd75d7e65f63601925ff1481
      https://github.com/llvm/llvm-project/commit/6e30f375f14ce77ebd75d7e65f63601925ff1481
  Author: Sameer Sahasrabuddhe <sameer.sahasrabuddhe at amd.com>
  Date:   2025-07-07 (Mon, 07 Jul 2025)

  Changed paths:
    M llvm/lib/Target/AMDGPU/SIMemoryLegalizer.cpp
    M llvm/test/CodeGen/AMDGPU/GlobalISel/flat-scratch.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/memory-legalizer-atomic-fence.ll
    M llvm/test/CodeGen/AMDGPU/attributor-flatscratchinit-undefined-behavior2.ll
    M llvm/test/CodeGen/AMDGPU/call-argument-types.ll
    M llvm/test/CodeGen/AMDGPU/dynamic_stackalloc.ll
    M llvm/test/CodeGen/AMDGPU/flat-scratch.ll
    M llvm/test/CodeGen/AMDGPU/function-args.ll
    M llvm/test/CodeGen/AMDGPU/indirect-addressing-si.ll
    M llvm/test/CodeGen/AMDGPU/kernel-vgpr-spill-mubuf-with-voffset.ll
    M llvm/test/CodeGen/AMDGPU/llvm.amdgcn.update.dpp.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-fence-mmra-global.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-fence-mmra-local.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-fence.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-flat-agent.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-flat-singlethread.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-flat-system.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-flat-volatile.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-flat-wavefront.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-flat-workgroup.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-global-agent.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-global-singlethread.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-global-system.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-global-volatile.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-global-wavefront.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-global-workgroup.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-local-agent.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-local-nontemporal.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-local-singlethread.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-local-system.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-local-volatile.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-local-wavefront.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-local-workgroup.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-local.mir
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-private-volatile.ll
    M llvm/test/CodeGen/AMDGPU/memory-legalizer-region.mir
    M llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands-non-ptr-intrinsics.ll
    M llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll
    M llvm/test/CodeGen/AMDGPU/stacksave_stackrestore.ll
    M llvm/test/CodeGen/AMDGPU/trap-abis.ll
    M llvm/test/CodeGen/AMDGPU/wait-before-stores-with-scope_sys.mir

  Log Message:
  -----------
  [AMDGPU] always emit a soft wait even if it is trivially ~0

The memory legalizer is currently responsible for emitting wait instructions at
ordering operations such as acquire and release. It tries to be efficient by
emitting waits only when required. In particular, it does not emit a wait on
vmcnt at workgroup scope since that ordering is already guaranteed by the
architecture. But this is now incorrect because direct loads to LDS have an LDS
component which needs explicit ordering on vmcnt. But it is inefficient to
always emit a wait on vmcnt since majority of the programs do not use direct
loads to LDS, and this will affect all workgroup scope operations.

As a first step to that, the memory legalizer now emits a soft wait instruction
even if all counts are trivially ~0. This is a placeholder that the
SIInsertWaitcnts pass will either optimize away or strenghthen based on its
analysis of whether direct loads to LDS are pending at this point in the
program.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications