<span style=" font-size:10pt;font-family:sans-serif">Excellent happy to
hear! :) </span><p style="margin-top:0px;margin-Bottom:0px"></p><p style="margin-top:0px;margin-Bottom:0px"><span style=" font-size:10pt;font-family:sans-serif">--Doru<br></span></p><br><br><br><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">From:
</span><span style=" font-size:9pt;font-family:sans-serif">Alok
Mishra <alokmishra.besu@gmail.com></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">To:
</span><span style=" font-size:9pt;font-family:sans-serif">Gheorghe-Teod
Bercea <Gheorghe-Teod.Bercea@ibm.com></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Cc:
</span><span style=" font-size:9pt;font-family:sans-serif">openmp-dev@lists.llvm.org</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Date:
</span><span style=" font-size:9pt;font-family:sans-serif">06/20/2019
02:59 PM</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Subject:
</span><span style=" font-size:9pt;font-family:sans-serif">[EXTERNAL]
Re: Re: [Openmp-dev] Using OpenMP target offloading in llvm-8.0.0</span><br><hr noshade><br><br><br><span style=" font-size:12pt">Hi Doru,</span><br><br><span style=" font-size:12pt">Thanks a lot for the help. I was stuck
on this issue for a couple of days. Setting </span><span style=" font-size:10pt;font-family:sans-serif">-Xopenmp-target
-march=sm_60 worked. :)</span><span style=" font-size:12pt"> Now
I am able to offload to GPU.</span><br><span style=" font-size:12pt">--</span><br><span style=" font-size:12pt">Thank You.</span><br><span style=" font-size:12pt">Regards,</span><br><span style=" font-size:12pt">Alok</span><div align=center><span style=" font-size:12pt;color:#5f5f5f"><i>'FOR THE
GREATER GOOD'</i></span></div><br><br><br><span style=" font-size:12pt">On Thu, Jun 20, 2019 at 11:47 AM Gheorghe-Teod
Bercea <</span><a href="mailto:Gheorghe-Teod.Bercea@ibm.com"><span style=" font-size:12pt;color:blue"><u>Gheorghe-Teod.Bercea@ibm.com</u></span></a><span style=" font-size:12pt">>
wrote:</span><br><span style=" font-size:10pt;font-family:sans-serif">HI Alok,</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>The two env vars I sent you before are just examples you should adapt them
to whatever you have locally i.e. 2 GPUs or 1 GPU and set the OMP_DEFAULT_DEVICE
accordingly 0,1, ...</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>I see you built your OpenMP for multiple compute capabilities. Which one
is the default? (there's a cmake flag for that).</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>To make sure you always compile for the one you have on your system please
use: -Xopenmp-target -march=sm_XX where XX can be 35, 60, 70 depending
on the GPU you're compiling for.</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>Thanks,</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>--Doru</span><p style="margin-top:0px;margin-Bottom:0px"><span style=" font-size:12pt"><br><br><br></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>From: </span><span style=" font-size:9pt;font-family:sans-serif">Alok
Mishra <</span><a href="mailto:alokmishra.besu@gmail.com" target="_blank"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>alokmishra.besu@gmail.com</u></span></a><span style=" font-size:9pt;font-family:sans-serif">></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>To: </span><span style=" font-size:9pt;font-family:sans-serif">Gheorghe-Teod
Bercea <</span><a href="mailto:Gheorghe-Teod.Bercea@ibm.com" target="_blank"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>Gheorghe-Teod.Bercea@ibm.com</u></span></a><span style=" font-size:9pt;font-family:sans-serif">></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Cc: </span><a href="mailto:openmp-dev@lists.llvm.org" target="_blank"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>openmp-dev@lists.llvm.org</u></span></a><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Date: </span><span style=" font-size:9pt;font-family:sans-serif">06/20/2019
02:37 PM</span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Subject: </span><span style=" font-size:9pt;font-family:sans-serif">[EXTERNAL]
Re: [Openmp-dev] Using OpenMP target offloading in llvm-8.0.0</span><span style=" font-size:12pt"><br></span><hr noshade><span style=" font-size:12pt"><br><br><br>Hi Doru,<br><br>I am running it on a linux cluster which has two NVIDIA P100-PCIE-16GB
GPUs on its node. I also tried on an AWS machine with NVIDIA Tesla V100-SXM2-16GB
GPU.<br><br>I tried setting </span><span style=" font-size:10pt;font-family:sans-serif">OMP_DEFAULT_DEVICE=1
and CUDA_VISIBLE_DEVICES=0,1,2,3, but I'm getting the same error.</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>I'm using the same OpenMP runtime which I built.</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>My LD_LIBRARY_PATH is set to $LLVM_PATH/lib and $LLVM_PATH/lib64
(where both llvm-8 and openmp-8 are installed)<br>My C_INCLUDE_PATH and CXX_INCLUDE_PATH are set to $LLVM_PATH/include</span><span style=" font-size:12pt"><br><br>--<br>Thank You.<br>Regards,<br>Alok</span></p><div align=center><span style=" font-size:12pt;color:#5f5f5f"><i>'FOR THE
GREATER GOOD'</i></span></div><p style="margin-top:0px;margin-Bottom:0px"><span style=" font-size:12pt"><br><br><br>On Thu, Jun 20, 2019 at 11:12 AM Gheorghe-Teod Bercea <</span><a href="mailto:Gheorghe-Teod.Bercea@ibm.com" target="_blank"><span style=" font-size:12pt;color:blue"><u>Gheorghe-Teod.Bercea@ibm.com</u></span></a><span style=" font-size:12pt">>
wrote:</span><span style=" font-size:10pt;font-family:sans-serif"><br>Hi Alok,<br><br>What type of machine are you running on?<br><br>Can you try to see if any of the following env variables help in any way?<br><br>export OMP_DEFAULT_DEVICE=1<br>export CUDA_VISIBLE_DEVICES=0,1,2,3<br><br>Which OpenMP runtime are you using and are you picking up the correct one?<br><br>To resolve that use -I -L in the compile line:<br><br>Use -I to point to the dir containing omp.h header and -L to point to the
OpenMP runtime library you're aiming to use. Make sure that you don't pick
them from different installations.<br><br>Make sure that your LD_LIBRARY_PATH also contains the path to the OpenMP
runtime library directory.<br><br>Thanks,<br><br>--Doru</span></p><p style="margin-top:0px;margin-Bottom:0px"><span style=" font-size:12pt"><br><br></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br><br>From: </span><span style=" font-size:9pt;font-family:sans-serif">Alok
Mishra via Openmp-dev <</span><a href="mailto:openmp-dev@lists.llvm.org" target="_blank"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>openmp-dev@lists.llvm.org</u></span></a><span style=" font-size:9pt;font-family:sans-serif">></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>To: </span><a href="mailto:openmp-dev@lists.llvm.org" target="_blank"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>openmp-dev@lists.llvm.org</u></span></a><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Date: </span><span style=" font-size:9pt;font-family:sans-serif">06/20/2019
02:01 PM</span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Subject: </span><span style=" font-size:9pt;font-family:sans-serif">[EXTERNAL]
[Openmp-dev] Using OpenMP target offloading in llvm-8.0.0</span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Sent by: </span><span style=" font-size:9pt;font-family:sans-serif">"Openmp-dev"
<</span><a href="mailto:openmp-dev-bounces@lists.llvm.org" target="_blank"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>openmp-dev-bounces@lists.llvm.org</u></span></a><span style=" font-size:9pt;font-family:sans-serif">></span></p><p style="margin-top:0px;margin-Bottom:0px"><hr noshade><span style=" font-size:12pt"><br><br><br>Hi,<br><br>When trying to use openmp target offloading with llvm I get the following
error</span><span style=" font-size:12pt;font-family:Courier New"><b><br><br>$ cat offload.cpp</b><br>#include <omp.h><br>int main() {<br>#pragma omp target teams distribute parallel for<br> for(int i=0; i<100; i++);<br> return 0;<br>}<b><br>$ clang++ -fopenmp -fopenmp-targets=nvptx64-nvidia-cuda --cuda-path=$CUDA_TOOLKIT_ROOT_DIR
offload.cpp -o offload<br>$ ./offload</b><br>Libomptarget fatal error 1: default offloading policy must switched to
mandatory or disabled<b><br>$</b></span><span style=" font-size:12pt"><br><br>I have llvm-8.0.0 installed in my system at </span><span style=" font-size:12pt;font-family:Courier New">LLVM_PATH</span><span style=" font-size:12pt">.<br>I downloaded openmp-8.0.0 source from the llvm download page. To build
openmp I used the following command:</span><span style=" font-size:12pt;font-family:Courier New"><br><br>$ mkdir build && cd build<br>$ cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=$LLVM_PATH -DCMAKE_C_COMPILER=$LLVM_PATH/bin/clang
-DCMAKE_CXX_COMPILER= $LLVM_PATH/bin/clang++ -DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=35,60,70
..<br>$ make && make install<br>$</span><span style=" font-size:12pt"><br><br>OpenMP got built without giving any error. But when trying to use target
offloading with OpenMP I get the above mentioned fatal error. I tried this
on 3 different machines, with same result.<br><br>--<br>Thank You.<br>Regards,<br>Alok</span><span style=" font-size:12pt;color:#5f5f5f"><i><br>'FOR THE GREATER GOOD'</i></span><tt><span style=" font-size:10pt">_______________________________________________<br>Openmp-dev mailing list</span></tt><span style=" font-size:12pt;color:blue"><u><br></u></span><a href="mailto:Openmp-dev@lists.llvm.org" target="_blank"><tt><span style=" font-size:10pt;color:blue"><u>Openmp-dev@lists.llvm.org</u></span></tt></a><span style=" font-size:12pt;color:blue"><u><br></u></span><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev" target="_blank"><tt><span style=" font-size:10pt;color:blue"><u>https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</u></span></tt></a><span style=" font-size:12pt"><br><br></span></p><p style="margin-top:0px;margin-Bottom:0px"></p><p style="margin-top:0px;margin-Bottom:0px"></p><p style="margin-top:0px;margin-Bottom:0px"></p><BR>