<div dir="ltr">Hi Doru,<div><br></div><div>I am running it on a linux cluster which has two NVIDIA P100-PCIE-16GB GPUs on its node. I also tried on an AWS machine with NVIDIA Tesla V100-SXM2-16GB GPU.</div><div><br></div><div>I tried setting <span style="font-family:sans-serif;font-size:10pt">OMP_DEFAULT_DEVICE=1 and </span><span style="font-family:sans-serif;font-size:10pt"> CUDA_VISIBLE_DEVICES=0,1,2,3, but I'm getting the same error.</span></div><div><span style="font-family:sans-serif;font-size:10pt"><br></span></div><div><span style="font-family:sans-serif;font-size:10pt">I'm using the same OpenMP runtime which I built.</span></div><div><span style="font-family:sans-serif;font-size:10pt"><br></span></div><div><span style="font-family:sans-serif;font-size:10pt">My LD_LIBRARY_PATH is set to $LLVM_PATH/lib and $LLVM_PATH/lib64 (where both llvm-8 and openmp-8 are installed)</span></div><div><span style="font-family:sans-serif;font-size:10pt">My C_INCLUDE_PATH and CXX_INCLUDE_PATH are set to</span><span style="font-family:sans-serif;font-size:13.3333px"> </span><span style="font-family:sans-serif;font-size:13.3333px">$LLVM_PATH/include</span></div><div><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>--<br><div>Thank You.</div><div>Regards,</div>Alok<i><br></i></div><div style="text-align:center"><i><font color="#666666">'FOR THE GREATER GOOD'</font></i></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 20, 2019 at 11:12 AM Gheorghe-Teod Bercea <<a href="mailto:Gheorghe-Teod.Bercea@ibm.com">Gheorghe-Teod.Bercea@ibm.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span style="font-size:10pt;font-family:sans-serif">Hi Alok,</span><br><br><span style="font-size:10pt;font-family:sans-serif">What type of machine
are you running on?</span><br><br><span style="font-size:10pt;font-family:sans-serif">Can you try to
see if any of the following env variables help in any way?</span><br><br><span style="font-size:10pt;font-family:sans-serif">export OMP_DEFAULT_DEVICE=1</span><br><span style="font-size:10pt;font-family:sans-serif">export CUDA_VISIBLE_DEVICES=0,1,2,3</span><br><br><span style="font-size:10pt;font-family:sans-serif">Which OpenMP runtime
are you using and are you picking up the correct one?</span><br><br><span style="font-size:10pt;font-family:sans-serif">To resolve that
use -I -L in the compile line:</span><br><br><span style="font-size:10pt;font-family:sans-serif">Use -I to point
to the dir containing omp.h header and -L to point to the OpenMP runtime
library you're aiming to use. Make sure that you don't pick them from different
installations.</span><br><br><span style="font-size:10pt;font-family:sans-serif">Make sure that
your LD_LIBRARY_PATH also contains the path to the OpenMP runtime library
directory.</span><br><br><span style="font-size:10pt;font-family:sans-serif">Thanks,</span><br><br><span style="font-size:10pt;font-family:sans-serif">--Doru</span><br><p style="margin-top:0px;margin-bottom:0px"></p><br><br><br><br><span style="font-size:9pt;color:rgb(95,95,95);font-family:sans-serif">From:
</span><span style="font-size:9pt;font-family:sans-serif">Alok
Mishra via Openmp-dev <<a href="mailto:openmp-dev@lists.llvm.org" target="_blank">openmp-dev@lists.llvm.org</a>></span><br><span style="font-size:9pt;color:rgb(95,95,95);font-family:sans-serif">To:
</span><span style="font-size:9pt;font-family:sans-serif"><a href="mailto:openmp-dev@lists.llvm.org" target="_blank">openmp-dev@lists.llvm.org</a></span><br><span style="font-size:9pt;color:rgb(95,95,95);font-family:sans-serif">Date:
</span><span style="font-size:9pt;font-family:sans-serif">06/20/2019
02:01 PM</span><br><span style="font-size:9pt;color:rgb(95,95,95);font-family:sans-serif">Subject:
</span><span style="font-size:9pt;font-family:sans-serif">[EXTERNAL]
[Openmp-dev] Using OpenMP target offloading in llvm-8.0.0</span><br><span style="font-size:9pt;color:rgb(95,95,95);font-family:sans-serif">Sent
by: </span><span style="font-size:9pt;font-family:sans-serif">"Openmp-dev"
<<a href="mailto:openmp-dev-bounces@lists.llvm.org" target="_blank">openmp-dev-bounces@lists.llvm.org</a>></span><br><hr noshade><br><br><br><span style="font-size:12pt">Hi,</span><br><br><span style="font-size:12pt">When trying to use openmp target offloading
with llvm I get the following error</span><br><br><span style="font-size:12pt;font-family:"Courier New""><b>$ cat offload.cpp</b><br>#include <omp.h><br>int main() {<br>#pragma omp target teams distribute parallel for<br> for(int i=0; i<100; i++);<br> return 0;<br>}<b><br>$ clang++ -fopenmp -fopenmp-targets=nvptx64-nvidia-cuda --cuda-path=$CUDA_TOOLKIT_ROOT_DIR
offload.cpp -o offload<br>$ ./offload</b><br>Libomptarget fatal error 1: default offloading policy must switched to
mandatory or disabled<b><br>$</b></span><br><br><span style="font-size:12pt">I have llvm-8.0.0 installed in my system
at </span><span style="font-size:12pt;font-family:"Courier New"">LLVM_PATH</span><span style="font-size:12pt">.</span><br><span style="font-size:12pt">I downloaded openmp-8.0.0 source from
the llvm download page. To build openmp I used the following command:</span><br><br><span style="font-size:12pt;font-family:"Courier New"">$ mkdir build
&& cd build<br>$ cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=$LLVM_PATH -DCMAKE_C_COMPILER=$LLVM_PATH/bin/clang
-DCMAKE_CXX_COMPILER= $LLVM_PATH/bin/clang++ -DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=35,60,70
..<br>$ make && make install<br>$</span><br><br><span style="font-size:12pt">OpenMP got built without giving any error.
But when trying to use target offloading with OpenMP I get the above mentioned
fatal error. I tried this on 3 different machines, with same result.</span><br><br><span style="font-size:12pt">--</span><br><span style="font-size:12pt">Thank You.</span><br><span style="font-size:12pt">Regards,</span><br><span style="font-size:12pt">Alok</span><br><span style="font-size:12pt;color:rgb(95,95,95)"><i>'FOR THE GREATER GOOD'</i></span><tt><span style="font-size:10pt">_______________________________________________<br>Openmp-dev mailing list<br><a href="mailto:Openmp-dev@lists.llvm.org" target="_blank">Openmp-dev@lists.llvm.org</a><br></span></tt><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev" target="_blank"><tt><span style="font-size:10pt">https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</span></tt></a><tt><span style="font-size:10pt"><br></span></tt><br><br><br>
</blockquote></div>