<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Lucida Console";
panose-1:2 11 6 9 4 5 4 2 2 4;}
@font-face
{font-family:"\@MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Verdana",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Static linking the OpenMP runtime is generally a bad idea.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US">As soon as you do that it becomes very easy to end up with multiple OpenMP runtimes in the same process, and have no way to avoid it.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Imagine one support library statically linked with
</span><span style="font-size:10.0pt;font-family:"Lucida Console";color:#1F497D;mso-fareast-language:EN-US">libiomp5</span><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US">, and another statically linked
with </span><span style="font-size:10.0pt;font-family:"Lucida Console";color:#1F497D;mso-fareast-language:EN-US">libgomp</span><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US">; as soon as you link against
these two libraries you have two OpenMP runtimes in the same process and there’s no way to avoid it. Even if you statically link against the same runtime you can run into problems, since you are clearly forcing two copies into the process (libomp and libiomp5
try to detect this, but it’s all pretty edgy :-)).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">-- Jim<br>
<br>
Jim Cownie <james.h.cownie@intel.com><br>
IAGS/CPDP/TCAR (Technical Computing, Analyzers, and Runtimes)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">Tel: +44 117 9071438</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><a name="_____replyseparator"></a><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Openmp-dev [mailto:openmp-dev-bounces@lists.llvm.org]
<b>On Behalf Of </b>Ye Luo via Openmp-dev<br>
<b>Sent:</b> Friday, May 3, 2019 5:20 AM<br>
<b>To:</b> Jeff Hammond <jeff.science@gmail.com><br>
<b>Cc:</b> via Openmp-dev <openmp-dev@lists.llvm.org><br>
<b>Subject:</b> Re: [Openmp-dev] libgomp.so libiomp5.so softlinks<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<p class="MsoNormal">Hi Jeff,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">No matter how good LLVM OpenMP runtime is, libgomp may be intended when I build my app with GNU.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Intel OpenMP runtime and LLVM runtime can differ in versions. In my case, they are not identical.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">When my app runs, I get tons of omp_nested deprecation warnings from libomp via MKL.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I avoid seeing the warnings with a real libiomp5 shipped with Intel release.<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The potential risk of "having multiple OpenMP libraries loaded into a single binary" is a bit convincing.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">If I build my app with clang and MKL, having two openMP runtime is clearly bad.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">At the moment, I'd like to having one app built with Intel to use Intel runtime and I don't need to see all the warnings.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">When I work on another app, I just use the llvm OpenMP runtime. Manipulating LD_LIBRARY_PATH or rpath or LD_PRELOAD is anyway painful.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I think I can live with this. It is better than the frustration seeing code crashing due two conflicting OpenMP runtime libraries.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Sadly, BG/Q is about to disappear and I mostly use dynamically linking for convenience and also a requirement for GPU.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I don't see a good solution but It seems that the current solution is the less bad one.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thank you, Jeff.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Best,<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">Ye<o:p></o:p></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">===================<br>
Ye Luo, Ph.D.<br>
Computational Science Division & Leadership Computing Facility<br>
Argonne National Laboratory<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Jeff Hammond <<a href="mailto:jeff.science@gmail.com">jeff.science@gmail.com</a>>
<span style="font-family:"Calibri",sans-serif">于</span>2019<span style="font-family:"Calibri",sans-serif">年</span>5<span style="font-family:"Calibri",sans-serif">月</span>2<span style="font-family:"Calibri",sans-serif">日周四</span>
<span style="font-family:"Calibri",sans-serif">下午</span>10:25<span style="font-family:"Calibri",sans-serif">写道：</span><o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal">The LLVM and Intel OpenMP runtimes are essentially identical.<o:p></o:p></p>
<div>
<p class="MsoNormal"><br>
The LLVM/Intel OpenMP runtime defines the GOMP symbols and works with GCC OpenMP codes, which is actually a good thing, because the GOMP runtime performs worse (<a href="https://www.nextplatform.com/2017/03/15/arm-antes-hpc-software-stack/" target="_blank">https://www.nextplatform.com/2017/03/15/arm-antes-hpc-software-stack/</a>).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">There are a number of solutions to your problem. One is to control LD_LIBRARY_PATH appropriately. You can remove LLVM folders from LD_LIBRARY_PATH and use LD_PRELOAD at runtime or -Wl,-rpath when linking to tell your LLVM OpenMP binary
where the library is.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><br>
The best option, of course, is to link statically. You are a Blue Gene user so I'm sure you understand the utility of fully static binaries :-)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Regardless of how you solve this, I object to removing the symlinks. What is happening because of this is far better than the alternative of having multiple OpenMP libraries loaded into a single binary. It is a reasonable assumption that
if you are setting LLVM OpenMP shared libraries in your global path (LD_LIBRARY_PATH), then you want that library to be used. If not, link differently.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><br>
Jeff<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Thu, May 2, 2019 at 6:41 PM Ye Luo via Openmp-dev <<a href="mailto:openmp-dev@lists.llvm.org" target="_blank">openmp-dev@lists.llvm.org</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<div>
<p class="MsoNormal">Hi,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">What is the reason of making both libgomp.so libiomp5.so softlinks to libomp.so?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I found them hijacking my application built with GNU and Intel compilers when I put llvm library folder on my LD_LIBRARY_PATH. This is very annoying. Unless there is a very good reason, I think they need be removed.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Best,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Ye<o:p></o:p></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">===================<br>
Ye Luo, Ph.D.<br>
Computational Science Division & Leadership Computing Facility<br>
Argonne National Laboratory<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
Openmp-dev mailing list<br>
<a href="mailto:Openmp-dev@lists.llvm.org" target="_blank">Openmp-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<p class="MsoNormal">Jeff Hammond<br>
<a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a><br>
<a href="http://jeffhammond.github.io/" target="_blank">http://jeffhammond.github.io/</a><o:p></o:p></p>
</div>
</blockquote>
</div>
</div>
<p>---------------------------------------------------------------------<br>
Intel Corporation (UK) Limited<br>
Registered No. 1134945 (England)<br>
Registered Office: Pipers Way, Swindon SN3 1RJ<br>
VAT No: 860 2173 47</p>
<p>This e-mail and any attachments may contain confidential material for<br>
the sole use of the intended recipient(s). Any review or distribution<br>
by others is strictly prohibited. If you are not the intended<br>
recipient, please contact the sender and delete all copies.</p></body>
</html>