<font size=2 face="sans-serif">Cristobal, can you check if you have </font><tt><font size=2>LIBOMPTARGET_NVPTX_COMPUTE_CAPABILITY
set instead? In the folder where you built the compiler, there is a CMakeCache.txt
file. Can you search for LIBOMPTARGET_NVPTX_COMPUTE_CAPABILITY in there.
If you see it you should pass 70 to that.</font></tt><br><font size=2 face="sans-serif">Thanks,</font><br><br><font size=2 face="sans-serif">--Doru<br></font><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Cristobal Ortega via
Openmp-dev <openmp-dev@lists.llvm.org></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">Jonas Hahnfeld <hahnjo@hahnjo.de></font><br><font size=1 color=#5f5f5f face="sans-serif">Cc:
</font><font size=1 face="sans-serif">openmp-dev@lists.llvm.org</font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">10/05/2018 10:49 AM</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [Openmp-dev]
Target construct not offloading to GPU</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">"Openmp-dev"
<openmp-dev-bounces@lists.llvm.org></font><br><hr noshade><br><br><br><tt><font size=2>Both LLVM and clang complains about that flag:<br>CMake Warning:<br> Manually-specified variables were not used by the project:<br><br> LIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES<br><br>It seems that developers disabled this flag some time ago <br>(</font></tt><a href="https://github.com/clang-ykt/clang/issues/11"><tt><font size=2>https://github.com/clang-ykt/clang/issues/11</font></tt></a><tt><font size=2>).<br>I haven't found any other way to generate the runtime libraries, though.<br><br>Best,<br>-Cristobal<br><br>On 10/05/2018 04:42 PM, Jonas Hahnfeld wrote:<br>> Yes, now you need to pass <br>> LIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=35,60,70 (or whatever you
like <br>> to have) to get the runtime libraries.<br>><br>> On 2018-10-05 16:33, Cristobal Ortega wrote:<br>>> I compiled clang with the following line:<br>>> cmake .. -DCMAKE_C_COMPILER=${HOST_GCC}/bin/gcc<br>>> -DCMAKE_CXX_COMPILER=${HOST_GCC}/bin/g++<br>>> -DGCC_INSTALL_PREFIX=${HOST_GCC}<br>>> -DCMAKE_CXX_LINK_FLAGS="-L${HOST_GCC}/lib64<br>>> -Wl,-rpath,${HOST_GCC}/lib64"<br>>> -DCMAKE_INSTALL_PREFIX=/gpfs/projects/bsc18/bsc18833/pkg/clang/7.0.0<br>>> -DGCC_INSTALL_PREFIX=${HOST_GCC}<br>>><br>>> Indeed, output with verbose confirms that clang is trying to compile<br>>> with march=sm_35 (output is attached).<br>>> Also, trying to compile the program with<br>>> "-Xopenmp-target -march=sm_70"<br>>> fails with<br>>> "clang-7: error: nvlink command failed with exit code 255
(use -v to<br>>> see invocation)" because of several undefined references
(details in<br>>> the attached file).<br>>><br>>> So, I'm trying to re-compile clang with<br>>> CLANG_OPENMP_NVPTX_DEFAULT_ARCH but, still, clang is not generating<br>>> the library 'libomptarget-nvptx-sm_70.bc'. Therefore, compilation<br>>> doesn't complete.<br>>> Where should this library be? I have one bc file in<br>>> "clang_src/test/Driver/Inputs/libomptarget/" but it's
for sm_20<br>>> (libomptarget-nvptx-sm_20.bc).<br>><br>> That's an empty file for testing. To get Bitcode libraries, you need
<br>> to compile the OpenMP project using Clang.<br>> (I've started putting together step-by-step instructions how to build
<br>> LLVM/Clang 7.0 for OpenMP offloading, I'll send a link to the mailing
<br>> list once ready.)<br>><br>>> This is how I'm trying to compile clang:<br>>> cmake .. -DCMAKE_C_COMPILER=${HOST_GCC}/bin/gcc<br>>> -DCMAKE_CXX_COMPILER=${HOST_GCC}/bin/g++<br>>> -DGCC_INSTALL_PREFIX=${HOST_GCC} -DCLANG_OPENMP_NVPTX_DEFAULT_ARCH=70<br>><br>> Nit: Should this be -DCLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_70?<br>><br>>> Yet, in the compilation process, clang complains about the missing<br>>> library for sm_70.<br>>><br>>> Do I need to pass some flag to LLVM too?<br>>><br>>> Best,<br>>> -Cristobal<br>>><br>>><br>>><br>>> On 10/05/2018 03:22 PM, Jonas Hahnfeld wrote:<br>>>> Hi,<br>>>><br>>>> how did you build your compiler? If you didn't specify <br>>>> CLANG_OPENMP_NVPTX_DEFAULT_ARCH Clang will default to sm_35
which <br>>>> doesn't run on Volta (sm_70).<br>>>> Can you post the output of<br>>>>> clang -v -o openmp_offload openmp_offload.c -O3
-fopenmp=libomp<br>>>>> -fopenmp-targets="nvptx64-nvidia-cuda"<br>>>><br>>>> If it's indeed compiling for sm_35, can you try adding <br>>>> -Xopenmp-target -march=sm_70?<br>>>><br>>>> Regards,<br>>>> Jonas<br>>>><br>>>> On 2018-10-05 15:09, Cristobal Ortega via Openmp-dev wrote:<br>>>>> Hello,<br>>>>><br>>>>> I've been trying to compile a program (source code is
attached) that<br>>>>> offloads to a NVIDIA V-100 GPU with LLVM 7.0 and clang
7.0.<br>>>>><br>>>>> It seems that the program is successfully compiled, yet
nvprof reports<br>>>>> that "no kernels were profiled".<br>>>>> The application seems that is running on the CPU (as "top"
command<br>>>>> reports a high usage of CPUs).<br>>>>><br>>>>> Compilation line that I used:<br>>>>> clang -v -o openmp_offload openmp_offload.c -O3
-fopenmp=libomp<br>>>>> -fopenmp-targets="nvptx64-nvidia-cuda"<br>>>>><br>>>>> Output after executing the binary:<br>>>>> ==74802== NVPROF is profiling process 74802, command:
./openmp_offload<br>>>>> 10 10 10000 1<br>>>>> Number of processors: 160<br>>>>> Number of devices:
4<br>>>>> Default device:
0<br>>>>> Is initial device:
1<br>>>>> ==74802== Profiling application: ./openmp_offload 10 10
10000 1<br>>>>> ==74802== Profiling result:<br>>>>> No kernels were profiled.<br>>>>>
Type Time(%) Time
Calls Avg Min
<br>>>>> Max Name<br>>>>> API calls:
99.99% 311.50ms 1
311.50ms 311.50ms<br>>>>> 311.50ms cuCtxCreate<br>>>>>
0.00% 11.462us 4
2.8650us 1.1450us<br>>>>> 6.2010us cuDeviceGetPCIBusId<br>>>>>
0.00% 5.4850us 5
1.0970us 387ns<br>>>>> 3.7770us cuDeviceGet<br>>>>>
0.00% 4.8070us 12
400ns 232ns<br>>>>> 1.0350us cuDeviceGetAttribute<br>>>>>
0.00% 1.4360us 3
478ns 384ns<br>>>>> 640ns cuDeviceGetCount<br>>>>><br>>>>><br>>>>><br>>>>> When compiled with GCC, the application does the offloading
to the <br>>>>> GPU.<br>>>>><br>>>>> clang information:<br>>>>> $ clang -v<br>>>>> Version 6<br>>>>> Version >= 90 selected<br>>>>> libdevice.10.bc exists<br>>>>> clang version 7.0.0 (tags/RELEASE_700/final)<br>>>>> Target: powerpc64le-unknown-linux-gnu<br>>>>> Thread model: posix<br>>>>> InstalledDir: /gpfs/projects/bsc18/bsc18833/pkg/clang/7.0.0/bin<br>>>>> Found candidate GCC installation:<br>>>>> /home/user/pkg/gcc/8.2.0/lib/gcc/powerpc64le-unknown-linux-gnu/8.2.0<br>>>>> Selected GCC installation:<br>>>>> /home/user/pkg/gcc/8.2.0/lib/gcc/powerpc64le-unknown-linux-gnu/8.2.0<br>>>>> Candidate multilib: .;@m64<br>>>>> Selected multilib: .;@m64<br>>>>> Found CUDA installation: /usr/local/cuda-9.2, version
9.2<br>>>>><br>>>>><br>>>>> Hopefully somebody has an idea on what's going on here.<br>>>>> If you need any more information to find the issue, let
me know.<br>>>>> Thank you.<br>>>>><br>>>>> Best,<br>>>>> -Cristobal<br>>>>><br>>>>><br>>>>> </font></tt><a href="http://bsc.es/disclaimer"><tt><font size=2>http://bsc.es/disclaimer</font></tt></a><tt><font size=2><br>>>>> _______________________________________________<br>>>>> Openmp-dev mailing list<br>>>>> Openmp-dev@lists.llvm.org<br>>>>> </font></tt><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev"><tt><font size=2>http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</font></tt></a><tt><font size=2><br>>><br>>><br>>><br>>> </font></tt><a href="http://bsc.es/disclaimer"><tt><font size=2>http://bsc.es/disclaimer</font></tt></a><tt><font size=2><br><br><br></font></tt><a href="http://bsc.es/disclaimer"><tt><font size=2>http://bsc.es/disclaimer</font></tt></a><tt><font size=2><br>_______________________________________________<br>Openmp-dev mailing list<br>Openmp-dev@lists.llvm.org<br></font></tt><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev"><tt><font size=2>http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</font></tt></a><tt><font size=2><br><br></font></tt><br><br><BR>