<span style=" font-size:10pt;font-family:sans-serif">Hi Jim,</span><br><br><span style=" font-size:10pt;font-family:sans-serif">Here is what I
get with KMP_VERSION=1.</span><br><br><span style=" font-size:10pt;font-family:sans-serif">$ LD_LIBRARY_PATH=$HOME/clang-install/lib
KMP_VERSION=1 ./a.out</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP version:
5.0.20140926</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP library
type: performance</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP link
type: dynamic</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP build
time: no_timestamp</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP build
compiler: Clang 11.0</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP alternative
compiler support: yes</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP API version:
5.0 (201611)</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP dynamic
error checking: no</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP plain
barrier branch bits: gather=2, release=2</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP forkjoin
barrier branch bits: gather=2, release=2</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP reduction
barrier branch bits: gather=1, release=1</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP plain
barrier pattern: gather=hyper, release=hyper</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP forkjoin
barrier pattern: gather=hyper, release=hyper</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP reduction
barrier pattern: gather=hyper, release=hyper</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP lock
type: run time selectable</span><br><span style=" font-size:10pt;font-family:sans-serif">LLVM OMP thread
affinity support: not used</span><br><span style=" font-size:10pt;font-family:sans-serif">0</span><br><span style=" font-size:10pt;font-family:sans-serif">1</span><br><span style=" font-size:10pt;font-family:sans-serif">3</span><br><span style=" font-size:10pt;font-family:sans-serif">2</span><br><br><span style=" font-size:10pt;font-family:sans-serif">For the mpirun
case, </span><br><br><span style=" font-size:10pt;font-family:sans-serif">$ KMP_VERSION=1
mpirun -np 1 ./a.out</span><br><span style=" font-size:10pt;font-family:sans-serif">./a.out: symbol
lookup error: /home/kli/clang-install/lib/libomp.so: undefined symbol:
ompt_start_tool</span><br><span style=" font-size:10pt;font-family:sans-serif">--------------------------------------------------------------------------</span><br><span style=" font-size:10pt;font-family:sans-serif">Primary job terminated
normally, but 1 process returned</span><br><span style=" font-size:10pt;font-family:sans-serif">a non-zero exit
code. Per user-direction, the job has been aborted.</span><br><span style=" font-size:10pt;font-family:sans-serif">--------------------------------------------------------------------------</span><br><span style=" font-size:10pt;font-family:sans-serif">--------------------------------------------------------------------------</span><br><span style=" font-size:10pt;font-family:sans-serif">mpirun detected
that one or more processes exited with non-zero status, thus causing</span><br><span style=" font-size:10pt;font-family:sans-serif">the job to be
terminated. The first process to do so was:</span><br><br><span style=" font-size:10pt;font-family:sans-serif"> Process
name: [[52030,1],0]</span><br><span style=" font-size:10pt;font-family:sans-serif"> Exit code:
127</span><br><span style=" font-size:10pt;font-family:sans-serif">--------------------------------------------------------------------------</span><br><span style=" font-size:10pt;font-family:sans-serif"><br><br>Kelvin<br></span><br><br><br><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">From:
</span><span style=" font-size:9pt;font-family:sans-serif">Jim
Cownie <jcownie@gmail.com></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">To:
</span><span style=" font-size:9pt;font-family:sans-serif">Kelvin
Li <kli@ca.ibm.com></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Cc:
</span><span style=" font-size:9pt;font-family:sans-serif">via
Openmp-dev <openmp-dev@lists.llvm.org></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Date:
</span><span style=" font-size:9pt;font-family:sans-serif">2020/10/27
11:09 AM</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Subject:
</span><span style=" font-size:9pt;font-family:sans-serif">[EXTERNAL]
Re: [Openmp-dev] : undefined symbol: ompt_start_tool</span><br><hr noshade><br><br><br><span style=" font-size:1pt;color:white">On 27 Oct 2020, at 15:00,
Kelvin Li wrote: I don't think that is the...
</span><table width=100% style="border-collapse:collapse;"><tr height=8><td width=100% bgcolor=#9ca3a7 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:16px 16px;"><table width=494 align=center style="border-collapse:collapse;"><tr valign=top height=8><td width=494 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><table width=248 align=center style="border-collapse:collapse;"><tr valign=top height=8><td width=248 bgcolor=#9ca3a7 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><table width=125 align=center style="border-collapse:collapse;"><tr height=8><td width=125 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:14pt;font-family:Helvetica"><b>This
Message Is From an External Sender</b></span><tr height=8><td width=125 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><span style=" font-size:10pt;font-family:Helvetica">This
message came from outside your organization.</span></table><p style="margin-top:0px;margin-Bottom:0px"></p></table><p style="margin-top:0px;margin-Bottom:0px"></p></table><p style="margin-top:0px;margin-Bottom:0px"></p></table><br><br><span style=" font-size:12pt">On 27 Oct 2020, at 15:00, Kelvin Li <</span><a href="mailto:kli@ca.ibm.com"><span style=" font-size:12pt;color:blue"><u>kli@ca.ibm.com</u></span></a><span style=" font-size:12pt">>
wrote:</span><br><br><span style=" font-size:10pt;font-family:sans-serif">I don't think
that is the case. There is only one task "-np 1" on one
node. Both './a.out' and 'mpirun -np 1 ./a.out' are issued on the
same node which has the same library in /home/kli/clang-install/lib. That
is puzzling me!<br></span><br><span style=" font-size:12pt">It really looks as if you’re getting
two different versions of the runtime, though, so having the runtime tell
you its properties is still likely useful.</span><br><span style=" font-size:12pt">If nothing else, it may show up that
you’re not propagating envirables as you might have hoped (if the MPI
version doesn’t print anything !)</span><br><br><span style=" font-size:12pt">-- Jim<br>James Cownie <</span><a href="mailto:jcownie@gmail.com"><span style=" font-size:12pt;color:blue"><u>jcownie@gmail.com</u></span></a><span style=" font-size:12pt">><br>Mob: +44 780 637 7146</span><br><span style=" font-size:10pt;font-family:sans-serif">Kelvin</span><span style=" font-size:12pt"><br><br><br><br></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>From: </span><span style=" font-size:9pt;font-family:sans-serif">Jim
Cownie via Openmp-dev <</span><a href="mailto:openmp-dev@lists.llvm.org"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>openmp-dev@lists.llvm.org</u></span></a><span style=" font-size:9pt;font-family:sans-serif">></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>To: </span><span style=" font-size:9pt;font-family:sans-serif">via
Openmp-dev <</span><a href="mailto:openmp-dev@lists.llvm.org"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>openmp-dev@lists.llvm.org</u></span></a><span style=" font-size:9pt;font-family:sans-serif">>,
</span><a href="mailto:openmp-dev-request@lists.llvm.org"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>openmp-dev-request@lists.llvm.org</u></span></a><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Date: </span><span style=" font-size:9pt;font-family:sans-serif">2020/10/27
04:46 AM</span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Subject: </span><span style=" font-size:9pt;font-family:sans-serif">[EXTERNAL]
Re: [Openmp-dev] : undefined symbol: ompt_start_tool</span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Sent by: </span><span style=" font-size:9pt;font-family:sans-serif">"Openmp-dev"
<</span><a href="mailto:openmp-dev-bounces@lists.llvm.org"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>openmp-dev-bounces@lists.llvm.org</u></span></a><span style=" font-size:9pt;font-family:sans-serif">></span><span style=" font-size:12pt"><br></span><hr noshade><span style=" font-size:12pt"><br><br></span><span style=" font-size:1pt;color:white"><br>Message: 1 Date: Mon, 26 Oct 2020 15:18:45 -0500 From: Kelvin Li via Openmp-dev...
</span><table width=100% style="border-collapse:collapse;"><tr height=8><td width=100% bgcolor=#9ca3a7 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:1px 1px;"><table width=494 align=center style="border-collapse:collapse;"><tr valign=top height=8><td width=492 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:1px 1px;"><table width=248 align=center style="border-collapse:collapse;"><tr valign=top height=8><td width=246 bgcolor=#9ca3a7 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:1px 1px;"><table width=125 align=center style="border-collapse:collapse;"><tr height=8><td width=123 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:1px 1px;"><span style=" font-size:14pt;font-family:Helvetica"><b>This
Message Is From an External Sender</b></span><tr height=8><td width=123 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:1px 1px;"><span style=" font-size:10pt;font-family:Helvetica">This
message came from outside your organization.</span></table><p style="margin-top:0px;margin-Bottom:0px"></p></table><p style="margin-top:0px;margin-Bottom:0px"></p></table><p style="margin-top:0px;margin-Bottom:0px"></p></table><br><span style=" font-size:12pt"><br>Message: 1<br>Date: Mon, 26 Oct 2020 15:18:45 -0500<br>From: Kelvin Li via Openmp-dev <</span><a href="mailto:openmp-dev@lists.llvm.org"><span style=" font-size:12pt;color:blue"><u>openmp-dev@lists.llvm.org</u></span></a><span style=" font-size:12pt">><br>To: </span><a href="mailto:openmp-dev@lists.llvm.org"><span style=" font-size:12pt;color:blue"><u>openmp-dev@lists.llvm.org</u></span></a><span style=" font-size:12pt"><br>Subject: [Openmp-dev] undefined symbol: ompt_start_tool<br>Message-ID:<br><</span><a href="mailto:OFF5259549.0EC65D66-ON8525860D.006EC181-8525860D.006F94A6@notes.na.collabserv.com"><span style=" font-size:12pt;color:blue"><u>OFF5259549.0EC65D66-ON8525860D.006EC181-8525860D.006F94A6@notes.na.collabserv.com</u></span></a><span style=" font-size:12pt">><br><br>Content-Type: text/plain; charset="utf-8"<br><br>Has anyone encounter the following error? I am wondering if it is
<br>something to do with how I build libomp.so.<br><br>$ LD_LIBRARY_PATH=/home/kli/clang-install/lib mpirun -np 1 ./a.out<br>a.out: symbol lookup error: /home/kli/clang-install/lib/libomp.so: <br>undefined symbol: ompt_start_tool<br>--------------------------------------------------------------------------<br>Primary job terminated normally, but 1 process returned<br>a non-zero exit code. Per user-direction, the job has been aborted.<br>--------------------------------------------------------------------------<br>--------------------------------------------------------------------------<br>mpirun detected that one or more processes exited with non-zero status,
<br>thus causing<br>the job to be terminated. The first process to do so was:<br><br>Process name: [[14546,1],0]<br>Exit code: 127<br>--------------------------------------------------------------------------<br><br>But it works without mpirun.<br><br>$ LD_LIBRARY_PATH=/home/kli/clang-install/lib ./a.out<br>0<br>1<br>2<br>3<br><br><br>Kelvin<br>Are you confident that </span><span style=" font-size:12pt;font-family:Courier New">/homie/kli/clang-install/lib</span><span style=" font-size:12pt">is
the same on all of the nodes used by the MPI program?<br>And that it contains the same version of</span><span style=" font-size:12pt;font-family:Courier New">libomp.so </span><span style=" font-size:12pt">everywhere?<br><br>Perhaps you should also set an envirable to have the OpenMP runtime print
its version, something like this </span><span style=" font-size:12pt;font-family:Courier New"><br>$ KMP_VERSION=1 ./a.out<br>LLVM OMP version: 5.0.20140926<br>LLVM OMP library type: performance<br>LLVM OMP link type: dynamic<br>LLVM OMP build time: no_timestamp<br>LLVM OMP build compiler: Clang 12.0<br>LLVM OMP alternative compiler support: yes<br>LLVM OMP API version: 5.0 (201611)<br>LLVM OMP dynamic error checking: no<br>LLVM OMP thread affinity support: no</span><span style=" font-size:12pt"><br><br>On 26 Oct 2020, at 23:44, via Openmp-dev <</span><a href="mailto:openmp-dev@lists.llvm.org"><span style=" font-size:12pt;color:blue"><u>openmp-dev@lists.llvm.org</u></span></a><span style=" font-size:12pt">>
wrote:<br><br>Send Openmp-dev mailing list submissions to</span><span style=" font-size:12pt;color:blue"><u><br></u></span><a href="mailto:openmp-dev@lists.llvm.org"><span style=" font-size:12pt;color:blue"><u>openmp-dev@lists.llvm.org</u></span></a><span style=" font-size:12pt"><br><br>To subscribe or unsubscribe via the World Wide Web, visit</span><span style=" font-size:12pt;color:blue"><u><br></u></span><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev"><span style=" font-size:12pt;color:blue"><u>https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</u></span></a><span style=" font-size:12pt"><br>or, via email, send a message with subject or body 'help' to</span><span style=" font-size:12pt;color:blue"><u><br></u></span><a href="mailto:openmp-dev-request@lists.llvm.org"><span style=" font-size:12pt;color:blue"><u>openmp-dev-request@lists.llvm.org</u></span></a><span style=" font-size:12pt"><br><br>You can reach the person managing the list at<br>openmp-dev-owner@lists.llvm.org<br><br>When replying, please edit your Subject line so it is more specific<br>than "Re: Contents of Openmp-dev digest..."<br><br><br>Today's Topics:<br><br> 1. undefined symbol: ompt_start_tool (Kelvin Li via Openmp-dev)<br> 2. Re: Declare target functions and libomptarget image<br> registration order (Manoel Römmer via Openmp-dev)<br> 3. Re: Declare target functions and libomptarget image<br> registration order (Johannes Doerfert via Openmp-dev)<br> 4. Re: Declare target functions and libomptarget image<br> registration order (Narayanaswamy, Ravi via Openmp-dev)<br><br><br>----------------------------------------------------------------------<br><br>Message: 1<br>Date: Mon, 26 Oct 2020 15:18:45 -0500<br>From: Kelvin Li via Openmp-dev <openmp-dev@lists.llvm.org><br>To: openmp-dev@lists.llvm.org<br>Subject: [Openmp-dev] undefined symbol: ompt_start_tool<br>Message-ID:<br><OFF5259549.0EC65D66-ON8525860D.006EC181-8525860D.006F94A6@notes.na.collabserv.com><br><br>Content-Type: text/plain; charset="utf-8"<br><br>Has anyone encounter the following error? I am wondering if it is
<br>something to do with how I build libomp.so.<br><br>$ LD_LIBRARY_PATH=/home/kli/clang-install/lib mpirun -np 1 ./a.out<br>a.out: symbol lookup error: /home/kli/clang-install/lib/libomp.so: <br>undefined symbol: ompt_start_tool<br>--------------------------------------------------------------------------<br>Primary job terminated normally, but 1 process returned<br>a non-zero exit code. Per user-direction, the job has been aborted.<br>--------------------------------------------------------------------------<br>--------------------------------------------------------------------------<br>mpirun detected that one or more processes exited with non-zero status,
<br>thus causing<br>the job to be terminated. The first process to do so was:<br><br>Process name: [[14546,1],0]<br>Exit code: 127<br>--------------------------------------------------------------------------<br><br>But it works without mpirun.<br><br>$ LD_LIBRARY_PATH=/home/kli/clang-install/lib ./a.out<br>0<br>1<br>2<br>3<br><br><br>Kelvin<br><br></span><br><br><span style=" font-size:12pt"><br><br></span><br><br><br><BR>