[Openmp-dev] warning: Cuda API error detected: cuModuleGetGlobal_v2 returned (0x1f4)

[Shilei Tian via Openmp-dev]

Probably in addition to printing things out, cuda-gdb would be the best tool here.

Regards,
Shilei
________________________________
From: Itaru Kitayama <itaru.kitayama at gmail.com>
Sent: Saturday, December 11, 2021 6:52:02 PM
To: Shilei Tian <tianshilei1992 at gmail.com>
Cc: openmp-dev <openmp-dev at lists.llvm.org>
Subject: Re: [Openmp-dev] warning: Cuda API error detected: cuModuleGetGlobal_v2 returned (0x1f4)

Shilei,

I tried to debug the hang with CUDA-gdb, though  not sure if I was using it correctly. If there are other general approaches, would you let us know?

On Sun, Dec 12, 2021 at 8:38 Shilei Tian <tianshilei1992 at gmail.com<mailto:tianshilei1992 at gmail.com>> wrote:

If the function call is to get the device environment variable from the image, that could return CUDA_ERROR_NOT_FOUND (500). However, it doesn't affect the execution of the program. The hang could be something else.



--

Regards,

Shilei Tian



From: Openmp-dev <openmp-dev-bounces at lists.llvm.org<mailto:openmp-dev-bounces at lists.llvm.org>> on behalf of Itaru Kitayama via Openmp-dev <openmp-dev at lists.llvm.org<mailto:openmp-dev at lists.llvm.org>>
Date: Sunday, December 5, 2021 at 5:03 AM
To: openmp-dev <openmp-dev at lists.llvm.org<mailto:openmp-dev at lists.llvm.org>>
Subject: [Openmp-dev] warning: Cuda API error detected: cuModuleGetGlobal_v2 returned (0x1f4)

Hi,
My offloading app runs correctly on host, but gets stuck (no progress)
on the device.

In a cuda gdb session I see:

(cuda-gdb) r

The program being debugged has been started already.

Start it from the beginning? (y or n) y

Starting program:
/p/project/cjinb33/kitayama1/opt/nest-clang-offload/bin/nest
hpc_benchmark.sli

warning: Cannot parse .gnu_debugdata section; LZMA support was
disabled at compile time

[Thread debugging using libthread_db enabled]

Using host libthread_db library "/usr/lib64/libthread_db.so.1".

warning: File "/p/software/jurecadc/stages/2020/software/GCCcore/10.3.0/lib64/libstdc++.so.6.0.28-gdb.py<http://so.6.0.28-gdb.py>"
auto-loading has been declined by your `auto-load safe-path' set to
"$debugdir:$datadir/auto-load".

warning: Cannot parse .gnu_debugdata section; LZMA support was
disabled at compile time

warning: Cannot parse .gnu_debugdata section; LZMA support was
disabled at compile time

OMP: Warning #249: OMP_TARGET_OFFLOAD: syntax error, using DEFAULT.

warning: Cannot parse .gnu_debugdata section; LZMA support was
disabled at compile time

[Detaching after fork from child process 27044]

[New Thread 0x15554a92d700 (LWP 27050)]

[New Thread 0x15554a12c700 (LWP 27051)]

virtual void nest::SimulationManager::initialize() Map this pointer at
0x15552e1021a8

warning: Cuda API error detected: cuModuleGetGlobal_v2 returned (0x1f4)


warning: Cuda API error detected: cuModuleGetGlobal_v2 returned (0x1f4)


warning: Cuda API error detected: cuModuleGetGlobal_v2 returned (0x1f4)


warning: Cuda API error detected: cuModuleGetGlobal_v2 returned (0x1f4)


warning: Cuda API error detected: cuModuleGetGlobal_v2 returned (0x1f4)


warning: Cuda API error detected: cuModuleGetGlobal_v2 returned (0x1f4)


warning: Cuda API error detected: cuModuleGetGlobal_v2 returned (0x1f4)

Should I be concerned about the API error?

_______________________________________________
Openmp-dev mailing list
Openmp-dev at lists.llvm.org<mailto:Openmp-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/openmp-dev/attachments/20211212/43245d5e/attachment.html>