[all-commits] [llvm/llvm-project] 490510: [OpenMP] Analyze the loop-body callback of the sta...

[Spencer Bryngelson via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 4905109b00e6916a310cf7c521bd8df19c0d4a11
      https://github.com/llvm/llvm-project/commit/4905109b00e6916a310cf7c521bd8df19c0d4a11
  Author: Spencer Bryngelson <sbryngelson at gmail.com>
  Date:   2026-08-03 (Mon, 03 Aug 2026)

  Changed paths:
    M llvm/lib/Transforms/IPO/OpenMPOpt.cpp
    A llvm/test/Transforms/OpenMP/spmdization_kernel_env_static_loop.ll

  Log Message:
  -----------
  [OpenMP] Analyze the loop-body callback of the static-loop runtime entries (#211287)

Fixes #211132. Also removes the trigger for #198621, see below.

`AAKernelInfo` treats the loop body passed to the
`__kmpc_*_static_loop_*` entries as opaque and records an unknown
parallel region for it, per the TODO at the site. Consequently
`NestedParallelism` is true for any kernel whose parallel region
contains a device workshare loop, and `MayUseNestedParallelism` is
written to the kernel environment as 1 where it should be 0.

The callback is a direct function operand at the callsite, so resolve it
and consult its `AAKernelInfo`, exactly as the `__kmpc_parallel_60`
handling already does for its parallel-region operand a few lines away.
Only record an unknown region when it does not resolve, or does reach
parallel regions. The SPMD-izability half of the TODO is left alone.

Only flang lowers device workshare loops through these entries; clang
emits `__kmpc_for_static_init_4` plus an explicit loop, which the
preceding case already handles. So flang kernels get 1 and clang kernels
get 0 on identical source. Controlled pair, flang, gfx90a:

| construct | runtime loop entry | before | after |
|---|---|---|---|
| `!$omp target parallel` | none | 0 | 0 |
| `!$omp target parallel do` | `__kmpc_for_static_loop_4u` | 1 | **0** |

What the 1 costs. It stops `config::mayUseNestedParallelism()` folding,
so the serialized branch in `__kmpc_parallel_60` stays live and carries
its own call to the microtask. After `__kmpc_parallel_60` is inlined
into the kernel there are then two calls to the outlined region rather
than one, `isSoleCallToLocalFunction` is false, and
`LastCallToStaticBonus` never applies. On AMDGPU that is 15000 x 11 =
165000; the analysis starts at -165045 with one callsite and -45 with
two. With two the region stays out of line and the kernel is no longer a
leaf.

`-Rpass-analysis=kernel-resource-usage`, VGPRs / scratch / occupancy:

| | before | after |
|---|---|---|
| reproducer from #211132, gfx90a | 212 / 48 B / 2 | **94 / 0 / 5** |
| WENO5 + HLLC NEQ=8, gfx942 | 196 / 64 B / 2 | **110 / 0 / 4** |
| NEQ=16 | 214 / 328 B / 2 | **138 / 0 / 3** |
| NEQ=24 | 196 / 456 B / 2 | **110 / 392 B / 4** |

End-to-end on gfx942 (MI325X), 1M cells, best of 50, Mcell/s, checksums
bit-identical: 1.35x, 1.47x, 1.28x at NEQ=8/16/24. Baseline run-to-run
spread across jobs is wider than the patched one, so treat the resource
numbers above, which are deterministic, as the primary evidence.

For #198621: step 2 of that root cause identifies the same
`MayUseNestedParallelism=1` as what prevents LTO folding
`omp_get_num_threads()` into a register read, which is what leaves
`DistributeFor` with `NumThreads=1` and skips a suffix of iterations.
Scalar kernels are immune there because they get 0. This refines the
field for the array-expression kernels too, so it addresses that cause
rather than the symptom.

Testing: added `spmdization_kernel_env_static_loop.ll`, which covers
both directions, a callback with no parallel region refining to 0 and a
callback that does contain one staying at 1. It fails without the patch.
`llvm/test/Transforms` (11676) and `llvm/test/CodeGen/AMDGPU` (4920),
16596 tests, 14668 passed with 39 expected failures and no regression.
The one failure, `Transforms/ThinLTOBitcodeWriter/no-type-md.ll`, is
pre-existing and fails identically with the patch reverted.

This affects performance-critical applications on large AMD GPU
supercomputers, including [MFC](https://github.com/MFlowCode/MFC).

All numbers above come from the validated reproducers attached to
#211132 and are independently reproducible; they stand on their own.

This was found and root-caused with the assistance of AI tools.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications