<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/60602>60602</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            [Openmp] Segmentation fault on goulash on amdgpu
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            openmp
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          dhruvachak
      </td>
    </tr>
</table>

<pre>
    This is reproducible with top of trunk. It seems to have started after https://reviews.llvm.org/rG16a385ba21a921a42009ff971199bce56eb2a86e and
https://reviews.llvm.org/rG3820d0eaaf4ecb557cbb260e34bf5a9eeb51e0e7 landed.

It fails with both the old and the nextgen plugin. 

Here are the steps:
Get the repo from https://github.com/LLNL/goulash
`
$ cd goulash/tests/rush_larsen/rush_larsen_gpu_omp
$ clang++  -DCOMPILERID=amd -O2 -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx90a  -DOMP -DOMP_TARGET_GPU rush_larsen_gpu_omp.cc   -o rush_larsen_gpu_omp
$ $ ./rush_larsen_gpu_omp 100000 .00000001
  0.000 (0.000s): --------------- Begin rush_larsen_gpu_omp [amd] (timer zeroed) ---------------
  0.000 (0.000s): START Rush Larsen 100000 0.00000001  cells 1  gpu_omp [amd]
  0.000 (0.000s): Version 2.0 RC1 (7/21/21)
  0.000 (0.000s): Selecting GPU 0 as default device
  0.000 (0.000s): Launching OpenMP GPU test kernel
 0.015 (0.015s): Verified OpenMP target test kernel ran on GPU
  0.015 (0.000s): Allocating and initializing kernel arrays
  0.015 (0.000s): Starting omp data map of CPU arrays to GPU
  0.015 (0.000s): Finished omp data map of CPU arrays to GPU
  0.015 (0.000s): Launching warmup iteration (not included in kernel timings)
  0.015 (0.000s): Starting kernel timings for Rush Larsen 100000 0.00000001
  0.015 (0.000s): Starting iteration 1
  0.137 (0.122s): Starting iteration  10001
  0.259 (0.122s): Starting iteration  20001
  0.380 (0.122s): Starting iteration  30001
 0.502 (0.122s): Starting iteration  40001
  0.623 (0.122s): Starting iteration  50001
  0.745 (0.122s): Starting iteration  60001
  0.867 (0.122s): Starting iteration  70001
Segmentation fault (core dumped)
`

Running under gdb shows the following stack trace:

`Thread 1 "rush_larsen_gpu" received signal SIGSEGV, Segmentation fault.
0x00007fffebe5973d in ?? () from /opt/rocm-5.4.0/lib/libhsa-runtime64.so.1
(gdb) bt
#0 0x00007fffebe5973d in ?? () from /opt/rocm-5.4.0/lib/libhsa-runtime64.so.1
#1  0x00007fffed7ad2ef in llvm::omp::target::plugin::AMDGPUQueueTy::pushKernelLaunch(llvm::omp::target::plugin::AMDGPUKernelTy const&, void*, unsigned int, unsigned long, unsigned int, llvm::omp::target::plugin::AMDGPUSignalTy*, llvm::omp::target::plugin::AMDGPUSignalTy*) ()
   from /home/dhchakra/git/aomp-trunk/llvm-project/build/lib/libomptarget.rtl.amdgpu.nextgen.so
#2 0x00007fffed7a6485 in llvm::omp::target::plugin::AMDGPUStreamTy::pushKernelLaunch(llvm::omp::target::plugin::AMDGPUKernelTy const&, void*, unsigned int, unsigned long, unsigned int, llvm::omp::target::plugin::AMDGPUMemoryManagerTy&) ()
   from /home/dhchakra/git/aomp-trunk/llvm-project/build/lib/libomptarget.rtl.amdgpu.nextgen.so
#3 0x00007fffed7a61be in llvm::omp::target::plugin::AMDGPUKernelTy::launchImpl(llvm::omp::target::plugin::GenericDeviceTy&, unsigned int, unsigned long, KernelArgsTy&, void*, llvm::omp::target::plugin::AsyncInfoWrapperTy&) const ()
   from /home/dhchakra/git/aomp-trunk/llvm-project/build/lib/libomptarget.rtl.amdgpu.nextgen.so
#4 0x00007fffed7bc046 in llvm::omp::target::plugin::GenericKernelTy::launch(llvm::omp::target::plugin::GenericDeviceTy&, void**, long*, KernelArgsTy&, llvm::omp::target::plugin::AsyncInfoWrapperTy&) const ()
   from /home/dhchakra/git/aomp-trunk/llvm-project/build/lib/libomptarget.rtl.amdgpu.nextgen.so
#5 0x00007fffed7c00c8 in llvm::omp::target::plugin::GenericDeviceTy::launchKernel(void*, void**, long*, KernelArgsTy&, __tgt_async_info*) ()
   from /home/dhchakra/git/aomp-trunk/llvm-project/build/lib/libomptarget.rtl.amdgpu.nextgen.so
#6 0x00007fffed7c39c5 in __tgt_rtl_launch_kernel ()
   from /home/dhchakra/git/aomp-trunk/llvm-project/build/lib/libomptarget.rtl.amdgpu.nextgen.so
#7 0x00007ffff7b49931 in target(ident_t*, DeviceTy&, void*, KernelArgsTy&, AsyncInfoTy&) ()
   from /home/dhchakra/git/aomp-trunk/llvm-project/build/lib/libomptarget.so.17git
#8 0x00007ffff7b42a70 in __tgt_target_kernel ()
   from /home/dhchakra/git/aomp-trunk/llvm-project/build/lib/libomptarget.so.17git
#9 0x0000555555559796 in rush_larsen_gpu_omp ()
#10 0x000055555555a252 in main ()`
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJzcWN9zozgS_mvkly5TQhgMD37wjMe51CWXXJLZuzeXQA3oBiRKEslk__orAXbsJJtNZu-2ppaqULLE1_11f_rREbdWVgpxReJPJN7MeO9qbVaiNv09L2r-bZZr8bi6q6UFacFgZ7ToC5k3CA_S1eB0B7oEZ3r1LYBzBxaxteA01PwewTpuHArgpUMDtXOdJdGasC1hW4P3Eh9s0DT3baBN5bvOwoRHaZxzFvKMhXzBKM3KMluGYZblBcYJ5oynCQJXgtANoet3GI1SRgVFzssFFnkcL4s8ZwnFaJGXMc8Q8zhEiktouBIogtHw-D53UHLZ2DHcXPuYawTdCE9haCv87ipU0DV9JVUAx_C_oUHgBocPrcOR6jB0hm7oNdhpKI1un-Wnkq7u86DQLWHbi4t_XPg-3Tfc1pOHhE4NtoBCwH6QbR1aZ33ova13DTcW1emvXdX1O912R_iGq4qwT4R9AphvPl9dXp9ffLk535Bow1sB8ysG81J3qNru0Jg7bip0dvymKtSct8L_1ZbD_N8nH73-TctNUZNoU5XfM8q956vL6_G9u1vfnH25251df4VXqAdFAQBz_drYU1j-L3g9dgipfyCg4xOOKADqe4CwdGhYwjISrWF--sAnrKR6zTmQ-BNvBYk33oaTLRr4FY1GQVj23MzbPm_v1jd3cNPbGi4GH3vO9EAaoMCmsRACvPD_tvFf0FipFbCAws3n0I8vCduycHplv8MNGyycVBV4gShwCwJL3jcOBN7LAt-GX_BeFbWHX3WoLq8HK37ewjc0CpsJTQMaxhM4jI-oy1Ki2GPHGXYMB8MVaOWtHvE4mHrisW4aXfAhDr-epZJO8kb-6jsmU9wY_mjfNnPrNzqP8QII7ji0fNgZP19_nQz4TfF36WylkrZG8UftPKX3gZu270A6NNx5vQlLlXYgVdH0An3I-0CdbKWq7Kn0bwV7ioNSm7cn6zvNPnE9QoTRckSEjL2BGJwewVicvQvGTmFRSt8Fi45gNIgpexdqceosYdG7YPEpbLmI3wVLTmFp8r5ELp9gt1i1qNw4MK5xwtJCGwTRt92wsz0_k4b3Ta-UN9srgQYqkYOt9YMdzr1SN41-8KPW8eIbOMMLPJyOB3N3tUEuwO9P7NluSxgDgwXKexTgCxnewO352e2Xs18I-wwvWU8nO_3uZ-OyLEvMMc6W0bAISLQl0dYH5vfp4UAmbKs7548PXbTzOFgElLBtI_PxXVs-N73yW3yyCKwOwv3Bk1Yi91Zyt--JKPz_3UYhHHsRSy4Ylt6Lr4d8bqO1Px6Hxv5Y9u2xdBnb68vN2fXXf_bY493jNNzb-u_DWh_3FcLSDxsc8XePUGhlHWGJl-heS0HY2jd7NZSiPifu5HejfWHy8oMPM7gdZsjd4-Twj-GzSbH9ujooV-sWCduK2pfPho-FHGFbrn0p5Otkr2Jz3847o_-DhR_Le9mIY4l1241cAuOawNdNXR9MdWZg9UFv9kzuZJHGPyT3rTPI27-W3pfYavN4yRWv0HjZkp9Ftui5bGGOPyTbPsljXzOIdd52zccEO0OFRhaboWybEvUOgUbna1PZA-ZI34_EYh9Vca5K_S_Du-5Iq2Hq_BSKLU4Vywu6SD6o2JTl1yT7X8h1SP2Y_UGj9W_I9NfSJj7VpqC0SH9Mm0NKj7QZ00dYejS3P5Lr3c5Vbsd9GndSlfonOj2SZ4mLsmI4PUbKxjW7MQW7qc7_GTgvjziXy3yRZVHoOU_KslQKVG7nJkF-a5G8KtVhrv_Jh4Uv4Zbewj7I9FmQjC_pkzAj6s8V5QXFbKIYT0-2zIYN8dULkSOKvkilz7CcxcxjWy7V_uOEzsQqElmU8RmuwmQZp1EahemsXnHBmSgXYUGLsuAhL0LGGRU8L5YUkyicyRWjLKKMpjSJ0igN0uWi4DlbxAUuYpExsqDYctkcrgln0toeVwlNKJs1PMfGDleijI13WIQxEm9mZjVkLO8rSxa0kdY93TTOnHTNcJF6NULizSv_gYBW-3s63xzn-aw3zeqNy79hI3uh1kDZErYdWP83AAD__6kYqIY">