[llvm] [AMDGPU] IGLP: Fixes for VMEM load detection and unsigned int handling (PR #135090)
Robert Imschweiler via llvm-commits
llvm-commits at lists.llvm.org
Fri Apr 11 13:08:32 PDT 2025
================
@@ -2079,6 +2083,9 @@ class MFMASmallGemmSingleWaveOpt final : public IGLPStrategy {
static unsigned DSWCount = 0;
static unsigned DSWWithPermCount = 0;
static unsigned DSWWithSharedVMEMCount = 0;
+static void resetDSWCounters() {
+ DSWCount = DSWWithPermCount = DSWWithSharedVMEMCount = 0;
+}
----------------
ro-i wrote:
because the original authors actually wanted to maintain state.
There is this scheduling phase enum:
https://github.com/llvm/llvm-project/blob/a45b133d400b0e57ca1ba70d50a91fbdf11d3b93/llvm/lib/Target/AMDGPU/AMDGPUIGroupLP.h#L20
Take `MFMASmallGemmSingleWaveOpt` as an example. It is instantiated in `createIGLPStrategy`:
https://github.com/llvm/llvm-project/blob/a45b133d400b0e57ca1ba70d50a91fbdf11d3b93/llvm/lib/Target/AMDGPU/AMDGPUIGroupLP.cpp#L2329-L2343
`createIGLPStrategy` is called by an instance of `IGroupLPDAGMutation`:
https://github.com/llvm/llvm-project/blob/a45b133d400b0e57ca1ba70d50a91fbdf11d3b93/llvm/lib/Target/AMDGPU/AMDGPUIGroupLP.cpp#L2689
That's why `IGroupLPDAGMutation` is instantiated itself using a phase parameter in `createIGroupLPDAGMutation`:
https://github.com/llvm/llvm-project/blob/a45b133d400b0e57ca1ba70d50a91fbdf11d3b93/llvm/lib/Target/AMDGPU/AMDGPUIGroupLP.cpp#L2705-L2707
`createIGroupLPDAGMutation` is called, for example, (1) by `GCNSchedStage`* with phase parameters "Initial" or "PreRAReentry":
https://github.com/llvm/llvm-project/blob/a45b133d400b0e57ca1ba70d50a91fbdf11d3b93/llvm/lib/Target/AMDGPU/GCNSchedStrategy.cpp#L1192
or (2) by `GCNScheduleDAGMILive` with phase parameter "Initial":
https://github.com/llvm/llvm-project/blob/72144d119a7291f8b6b8e022a2947fbe31e66afc/llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp#L595
or (3) by `GCNPostScheduleDAGMILive` with phase parameter "PostRA":
https://github.com/llvm/llvm-project/blob/a45b133d400b0e57ca1ba70d50a91fbdf11d3b93/llvm/lib/Target/AMDGPU/GCNSchedStrategy.cpp#L2065
https://github.com/llvm/llvm-project/blob/72144d119a7291f8b6b8e022a2947fbe31e66afc/llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp#L1112
*those stages itself are objects maintained e.g. by `GCNScheduleDAGMILive`:
https://github.com/llvm/llvm-project/blob/a45b133d400b0e57ca1ba70d50a91fbdf11d3b93/llvm/lib/Target/AMDGPU/GCNSchedStrategy.cpp#L777
And the GCN schedule DAGs itself are created by `AMDGPUTargetMachine`/`GCNTargetMachine`.
Coming back to our `MFMASmallGemmSingleWaveOpt`: It wants to maintain some cached values which are computed in "initial" phase and re-used in later phases:
https://github.com/llvm/llvm-project/blob/ae0aa2dea2dee3af01326e5ff96ab436628f7e2b/llvm/lib/Target/AMDGPU/AMDGPUIGroupLP.cpp#L2092-L2094
https://github.com/llvm/llvm-project/blob/ae0aa2dea2dee3af01326e5ff96ab436628f7e2b/llvm/lib/Target/AMDGPU/AMDGPUIGroupLP.cpp#L2116-L2169
That's a rough description of the situation as far as I can see it.
https://github.com/llvm/llvm-project/pull/135090
More information about the llvm-commits
mailing list