<span style=" font-size:10pt;font-family:sans-serif">HI Alok,</span><br><br><span style=" font-size:10pt;font-family:sans-serif">The two env vars
I sent you before are just examples you should adapt them to whatever you
have locally i.e. 2 GPUs or 1 GPU and set the OMP_DEFAULT_DEVICE accordingly
0,1, ...</span><br><br><span style=" font-size:10pt;font-family:sans-serif">I see you built
your OpenMP for multiple compute capabilities. Which one is the default?
(there's a cmake flag for that).</span><br><br><span style=" font-size:10pt;font-family:sans-serif">To make sure you
always compile for the one you have on your system please use: -Xopenmp-target
-march=sm_XX where XX can be 35, 60, 70 depending on the GPU you're compiling
for.</span><br><br><span style=" font-size:10pt;font-family:sans-serif">Thanks,</span><br><br><span style=" font-size:10pt;font-family:sans-serif">--Doru</span><br><p style="margin-top:0px;margin-Bottom:0px"></p><br><br><br><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">From:
</span><span style=" font-size:9pt;font-family:sans-serif">Alok
Mishra <alokmishra.besu@gmail.com></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">To:
</span><span style=" font-size:9pt;font-family:sans-serif">Gheorghe-Teod
Bercea <Gheorghe-Teod.Bercea@ibm.com></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Cc:
</span><span style=" font-size:9pt;font-family:sans-serif">openmp-dev@lists.llvm.org</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Date:
</span><span style=" font-size:9pt;font-family:sans-serif">06/20/2019
02:37 PM</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Subject:
</span><span style=" font-size:9pt;font-family:sans-serif">[EXTERNAL]
Re: [Openmp-dev] Using OpenMP target offloading in llvm-8.0.0</span><br><hr noshade><br><br><br><span style=" font-size:12pt">Hi Doru,</span><br><br><span style=" font-size:12pt">I am running it on a linux cluster which
has two NVIDIA P100-PCIE-16GB GPUs on its node. I also tried on an AWS
machine with NVIDIA Tesla V100-SXM2-16GB GPU.</span><br><br><span style=" font-size:12pt">I tried setting </span><span style=" font-size:10pt;font-family:sans-serif">OMP_DEFAULT_DEVICE=1
and CUDA_VISIBLE_DEVICES=0,1,2,3, but I'm getting the same error.</span><br><br><span style=" font-size:10pt;font-family:sans-serif">I'm using the
same OpenMP runtime which I built.</span><br><br><span style=" font-size:10pt;font-family:sans-serif">My LD_LIBRARY_PATH
is set to $LLVM_PATH/lib and $LLVM_PATH/lib64 (where both llvm-8
and openmp-8 are installed)</span><br><span style=" font-size:10pt;font-family:sans-serif">My C_INCLUDE_PATH
and CXX_INCLUDE_PATH are set to $LLVM_PATH/include</span><br><br><span style=" font-size:12pt">--</span><br><span style=" font-size:12pt">Thank You.</span><br><span style=" font-size:12pt">Regards,</span><br><span style=" font-size:12pt">Alok</span><div align=center><span style=" font-size:12pt;color:#5f5f5f"><i>'FOR THE
GREATER GOOD'</i></span></div><br><br><br><span style=" font-size:12pt">On Thu, Jun 20, 2019 at 11:12 AM Gheorghe-Teod
Bercea <</span><a href="mailto:Gheorghe-Teod.Bercea@ibm.com"><span style=" font-size:12pt;color:blue"><u>Gheorghe-Teod.Bercea@ibm.com</u></span></a><span style=" font-size:12pt">>
wrote:</span><br><span style=" font-size:10pt;font-family:sans-serif">Hi Alok,</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>What type of machine are you running on?</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>Can you try to see if any of the following env variables help in any way?</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>export OMP_DEFAULT_DEVICE=1<br>export CUDA_VISIBLE_DEVICES=0,1,2,3</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>Which OpenMP runtime are you using and are you picking up the correct one?</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>To resolve that use -I -L in the compile line:</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>Use -I to point to the dir containing omp.h header and -L to point to the
OpenMP runtime library you're aiming to use. Make sure that you don't pick
them from different installations.</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>Make sure that your LD_LIBRARY_PATH also contains the path to the OpenMP
runtime library directory.</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>Thanks,</span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>--Doru</span><p style="margin-top:0px;margin-Bottom:0px"><span style=" font-size:12pt"><br><br><br></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>From: </span><span style=" font-size:9pt;font-family:sans-serif">Alok
Mishra via Openmp-dev <</span><a href="mailto:openmp-dev@lists.llvm.org" target="_blank"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>openmp-dev@lists.llvm.org</u></span></a><span style=" font-size:9pt;font-family:sans-serif">></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>To: </span><a href="mailto:openmp-dev@lists.llvm.org" target="_blank"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>openmp-dev@lists.llvm.org</u></span></a><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Date: </span><span style=" font-size:9pt;font-family:sans-serif">06/20/2019
02:01 PM</span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Subject: </span><span style=" font-size:9pt;font-family:sans-serif">[EXTERNAL]
[Openmp-dev] Using OpenMP target offloading in llvm-8.0.0</span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Sent by: </span><span style=" font-size:9pt;font-family:sans-serif">"Openmp-dev"
<</span><a href="mailto:openmp-dev-bounces@lists.llvm.org" target="_blank"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>openmp-dev-bounces@lists.llvm.org</u></span></a><span style=" font-size:9pt;font-family:sans-serif">></span><span style=" font-size:12pt"><br></span><hr noshade><span style=" font-size:12pt"><br><br><br>Hi,<br><br>When trying to use openmp target offloading with llvm I get the following
error<br></span><span style=" font-size:12pt;font-family:Courier New"><b><br>$ cat offload.cpp</b><br>#include <omp.h><br>int main() {<br>#pragma omp target teams distribute parallel for<br> for(int i=0; i<100; i++);<br> return 0;<br>}<b><br>$ clang++ -fopenmp -fopenmp-targets=nvptx64-nvidia-cuda --cuda-path=$CUDA_TOOLKIT_ROOT_DIR
offload.cpp -o offload<br>$ ./offload</b><br>Libomptarget fatal error 1: default offloading policy must switched to
mandatory or disabled<b><br>$</b></span><span style=" font-size:12pt"><br><br>I have llvm-8.0.0 installed in my system at </span><span style=" font-size:12pt;font-family:Courier New">LLVM_PATH</span><span style=" font-size:12pt">.<br>I downloaded openmp-8.0.0 source from the llvm download page. To build
openmp I used the following command:<br></span><span style=" font-size:12pt;font-family:Courier New"><br>$ mkdir build && cd build<br>$ cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=$LLVM_PATH -DCMAKE_C_COMPILER=$LLVM_PATH/bin/clang
-DCMAKE_CXX_COMPILER= $LLVM_PATH/bin/clang++ -DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=35,60,70
..<br>$ make && make install<br>$</span><span style=" font-size:12pt"><br><br>OpenMP got built without giving any error. But when trying to use target
offloading with OpenMP I get the above mentioned fatal error. I tried this
on 3 different machines, with same result.<br><br>--<br>Thank You.<br>Regards,<br>Alok</span><span style=" font-size:12pt;color:#5f5f5f"><i><br>'FOR THE GREATER GOOD'</i></span><tt><span style=" font-size:10pt">_______________________________________________<br>Openmp-dev mailing list</span></tt><tt><span style=" font-size:10pt;color:blue"><u><br></u></span></tt><a href="mailto:Openmp-dev@lists.llvm.org" target="_blank"><tt><span style=" font-size:10pt;color:blue"><u>Openmp-dev@lists.llvm.org</u></span></tt></a><span style=" font-size:12pt;color:blue"><u><br></u></span><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev" target="_blank"><tt><span style=" font-size:10pt;color:blue"><u>https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</u></span></tt></a><span style=" font-size:12pt"><br><br><br></span></p><p style="margin-top:0px;margin-Bottom:0px"></p><p style="margin-top:0px;margin-Bottom:0px"></p><BR>