[llvm] [OpenMP][offload] Add enhanced cross-team reduction test (PR #195940)

[Robert Imschweiler via llvm-commits]

ro-i wrote:

Yup. If we compile the test with save-temps and then look at the definition and usages of `@__kmpc_nvptx_teams_reduce_nowait_v2` in `a.out.amdgcn.gfx90a.img.0.5.precodegen.ll`:

teams_reduce is called like this:
`call fastcc i32 @__kmpc_nvptx_teams_reduce_nowait_v2(ptr %45, ptr %.omp.reduction.red_list.ascast.i, ptr nonnull @_omp_reduction_shuffle_and_reduce_func.5, ptr nonnull @_omp_reduction_inter_warp_copy_func.6, ptr nonnull @_omp_reduction_list_to_global_copy_func.7, ptr nonnull @_omp_reduction_list_to_global_reduce_func.8, ptr nonnull @_omp_reduction_global_to_list_copy_func.9, ptr nonnull @_omp_reduction_global_to_list_reduce_func.10) #24`

or like this:
`call fastcc i32 @__kmpc_nvptx_teams_reduce_nowait_v2(ptr %45, ptr %.omp.reduction.red_list.ascast.i, ptr nonnull @_omp_reduction_shuffle_and_reduce_func.13, ptr nonnull @_omp_reduction_inter_warp_copy_func.14, ptr nonnull @_omp_reduction_list_to_global_copy_func.15, ptr nonnull @_omp_reduction_list_to_global_reduce_func.16, ptr nonnull @_omp_reduction_global_to_list_copy_func.17, ptr nonnull @_omp_reduction_global_to_list_reduce_func.18) #24`

or this:
`call fastcc i32 @__kmpc_nvptx_teams_reduce_nowait_v2(ptr %49, ptr %.omp.reduction.red_list.ascast.i, ptr nonnull @_omp_reduction_shuffle_and_reduce_func.21, ptr nonnull @_omp_reduction_inter_warp_copy_func.22, ptr nonnull @_omp_reduction_list_to_global_copy_func.23, ptr nonnull @_omp_reduction_list_to_global_reduce_func.24, ptr nonnull @_omp_reduction_global_to_list_copy_func.25, ptr nonnull @_omp_reduction_global_to_list_reduce_func.26) #24`

Note that we have different function pointers every time.

Then, looking at the definition of teams_reduce, we see stuff like this. For example, for the `@_omp_reduction_list_to_global_copy_func.X` variants:

```llvm
49:                                               ; preds = %46
  %50 = icmp eq ptr %4, @_omp_reduction_list_to_global_copy_func
  br i1 %50, label %51, label %57

51:                                               ; preds = %49
  %52 = addrspacecast ptr %1 to ptr addrspace(5)
  %53 = load ptr, ptr addrspace(5) %52, align 8
  %54 = zext nneg i32 %47 to i64
  %55 = getelementptr inbounds nuw [8 x i8], ptr %0, i64 %54
  %56 = load double, ptr %53, align 8
  store double %56, ptr %55, align 8
  br label %429

57:                                               ; preds = %49
  %58 = icmp eq ptr %4, @_omp_reduction_list_to_global_copy_func.7
  br i1 %58, label %59, label %65

59:                                               ; preds = %57
  %60 = addrspacecast ptr %1 to ptr addrspace(5)
  %61 = load ptr, ptr addrspace(5) %60, align 8
  %62 = zext nneg i32 %47 to i64
  %63 = getelementptr inbounds nuw [8 x i8], ptr %0, i64 %62
  %64 = load double, ptr %61, align 8
  store double %64, ptr %63, align 8
  br label %429

65:                                               ; preds = %57
  %66 = icmp eq ptr %4, @_omp_reduction_list_to_global_copy_func.15
  br i1 %66, label %67, label %73

... <a lot more comparisons to @_omp_reduction_list_to_global_copy_func.X are following> ...
```

Or, a different pattern, this time for the `@_omp_reduction_global_to_list_reduce_func.X` variants, where some sort of hoisting is used:

```llvm
.preheader:                                       ; preds = %623
  %626 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func
  %627 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.10
  %628 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.18
  %629 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.26
  %630 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.34
  %631 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.45
  %632 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.54
  %633 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.62
  %634 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.70
  %635 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.78
  %636 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.86
  %637 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.97
  %638 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.106
  %639 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.114
  %640 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.122
  %641 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.130
  %642 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.138
  %643 = icmp eq ptr %7, @_omp_reduction_global_to_list_reduce_func.149
  %644 = addrspacecast ptr %1 to ptr addrspace(5)
  %645 = getelementptr inbounds nuw i8, ptr %1, i64 8
  %646 = addrspacecast ptr %645 to ptr addrspace(5)
  br label %648
```

That's what I meant by "dispatch chains". And they grow by the number of reductions per translation unit. To me, this looks like the issue is that the helper functions *are* inlined, but the parent functions like teams_reduce not. Because if they were (see the other PR), then there would be a 1:1 mapping between the parent functions and the helper functions and thus no dispatch chains.

https://github.com/llvm/llvm-project/pull/195940