<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Verdana",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><a name="_MailEndCompose"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#843C0C;mso-style-textfill-fill-color:#843C0C;mso-style-textfill-fill-alpha:100.0%">Chandler:<o:p></o:p></span></a></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#843C0C;mso-style-textfill-fill-color:#843C0C;mso-style-textfill-fill-alpha:100.0%"> 1) I completely agree with the comments some others have made about<br>
us needing to make it clear that this isn't some Intel-only thing,<br>
it’s the LLVM OpenMP runtime. Some suggestions that I think would<br>
make sense to help here:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#843C0C;mso-style-textfill-fill-color:#843C0C;mso-style-textfill-fill-alpha:100.0%"><o:p> </o:p></span></p>
<p class="MsoNormal"><i><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#843C0C;mso-style-textfill-fill-color:#843C0C;mso-style-textfill-fill-alpha:100.0%">… code owner discussion elided since Chris has endorsed Andrey…<o:p></o:p></span></i></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#843C0C;mso-style-textfill-fill-color:#843C0C;mso-style-textfill-fill-alpha:100.0%"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#843C0C;mso-style-textfill-fill-color:#843C0C;mso-style-textfill-fill-alpha:100.0%">- Clearly updating the readme and such would be appropriate.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">Certainly, no problem with that at all. As I ponted out before, people from Intel are not the right ones to be inserting compatiblity tables for the runtime when
it is running on IBM Power or ARM processors. The people who know about that should contribute…<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#843C0C;mso-style-textfill-fill-color:#843C0C;mso-style-textfill-fill-alpha:100.0%"><br>
- I suspect we should change the name of the installed library.<br>
'libiomp' is pretty clearly the Intel library. We could continue in<br>
the grand tradition of LLVM naming conventions and use 'libllomp'?<br>
Of course, we should install symlinks under the name 'libiomp' if<br>
needed for existing users to not be broken.<br>
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">Changing the file name (at least when targeting X86 processors) is a more interesting issue, and not as trivial as it seems.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">The problem here is related to inter-operation with existing (Intel and gcc) compilers, and backwards binary compatibility.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">As Hal pointed out, if you intend ever to mix code compiled by multiple, different, OpenMP compilers into the same process, even if they all use an ABI to the
runtime that it supports (remember that the LLVM runtime supports both the LLVM and libgomp interfaces), you must ensure that there is only one copy of the runtime (dynamically) linked into the process. This is because (unlike most libraries) the runtime contains
state (for instance, the thread pool), therefore if you have multiple instances even of the identical runtime linked into the process, bad things will happen. (Over-subscription and associated poor performance, critical sections not providing mutual exclusion,
…). This is why we don’t normally provide a static library; it tempts people into building their own dynamic libraries that include a static copy of the OpenMP runtime, then when their DLL is linked into an OpenMP program you instantly have two copies of the
runtime and a hard to diagnose disaster.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">The problem this causes is that unless there is an agreed name for the runtime, the dynamic linker will load it twice if it is demanded under different names.
(On Linux, you can use soname to help here, but doesn’t completely solve the problem).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">Therefore, if we do not want to make it unnecessarily hard for people who use Clang/OpenMP on X86 to link against libraries (such as Intel MKL) which use OpenMP
and were compiled with a different compiler, it is helpful to use the same file name for the runtime library no matter which compiler generated the code.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">Since there are many libraries and executables which have already been built by the Intel compiler, so already have the library name built into them (in the DT_NEEDED
ELF tag, or equivalent), changing the library name does not come for free.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">I am therefore loath to change the library name simply for political/publicity reasons when that will cause unnecessary additional complexity to the users of
one of our main target platforms. (Before anyone else says it, yes, of course X86 is the platform I care most about, but the issue isn’t about compiler competition, but about making things easy for the users of LLVM/OpenMP so that things they expect will work
do “just work” rather than adding extra complexity and more manuals to explain what has to be done to make them work. Like moving into your partner’s appartment, you’re welll advised not to move all the furniture around immediately
</span><span style="font-size:10.0pt;font-family:Wingdings;color:#1F497D">J</span><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">).</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#843C0C;mso-style-textfill-fill-color:#843C0C;mso-style-textfill-fill-alpha:100.0%"><br>
<br>
Just some bikeshed-painting: if we're expecting each compiler that<br>
uses the library to distribute a separate copy as part of that<br>
compiler's runtime, then I think the best name for clang to use for<br>
the library would probably be libclang_rt.omp-<arch> or<br>
libclang_rt.openmp-<arch> (as we do for all our other runtime<br>
libraries).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">Aside from the argument above that we shouldn’t change the name (at least on X86), note that if Hal’s efforts on Flang work out, we’ll also have a Fortran/OpenMP
implementation that will use the same runtime library. Therefore putting “clang” into the name seems short-sighted.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#843C0C;mso-style-textfill-fill-color:#843C0C;mso-style-textfill-fill-alpha:100.0%"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#843C0C;mso-style-textfill-fill-color:#843C0C;mso-style-textfill-fill-alpha:100.0%">If we expect this to be installed somewhere central on<br>
the system and shared by different compilers and different versions<br>
of the same compiler, then libllomp or similar seems reasonable to<br>
me. What's the intended distribution model here? <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">Whether OS vendors choose to install the runtime in a standard place (like /usr/lib64) is not something we can control. (Though doing so is a reasonable thing
to do).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#843C0C;mso-style-textfill-fill-color:#843C0C;mso-style-textfill-fill-alpha:100.0%">How stable is the ABI?<br>
</span><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">Stable; we add new interfaces when required by the evolution of the OpenMP language specification, but maintain backwards compatibility, so code compiled by older compilers
can be run against newer runtime libraries.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US">On the benchmarks side of things, since everyone is so keen on this
<b>not </b>only being about Intel, surely it’d be good to have some comparisons of performance vs gcc on Power or Arm as well as on X86.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">-- Jim<br>
<br>
James Cownie <james.h.cownie@intel.com><br>
SSG/DPD/TCAR (Technical Computing, Analyzers and Runtimes)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D">Tel: +44 117 9071438</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Hal Finkel [mailto:hfinkel@anl.gov]
<br>
<b>Sent:</b> Tuesday, May 5, 2015 5:39 PM<br>
<b>To:</b> Richard Smith<br>
<b>Cc:</b> Andrey Bokhanko; cfe-dev; LLVM Developers Mailing List; Douglas Gregor; C Bergström; Michael Wong; Alexey Bataev; Chandler Carruth; Cownie, James H<br>
<b>Subject:</b> Re: [LLVMdev] libiomp, not libgomp as default library linked with -fopenmp<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">
<hr size="2" width="100%" align="center" id="zwchr">
</span></div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black"><br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">From: "Richard Smith" <<a href="mailto:richard@metafoo.co.uk">richard@metafoo.co.uk</a>><br>
To: "Chandler Carruth" <<a href="mailto:chandlerc@google.com">chandlerc@google.com</a>><br>
Cc: "Andrey Bokhanko" <<a href="mailto:andreybokhanko@gmail.com">andreybokhanko@gmail.com</a>>, "cfe-dev" <<a href="mailto:cfe-dev@cs.uiuc.edu">cfe-dev@cs.uiuc.edu</a>>, "LLVM Developers Mailing List"<br>
<<a href="mailto:llvmdev@cs.uiuc.edu">llvmdev@cs.uiuc.edu</a>>, "Douglas Gregor" <<a href="mailto:dgregor@apple.com">dgregor@apple.com</a>>, "Hal Finkel" <<a href="mailto:hfinkel@anl.gov">hfinkel@anl.gov</a>>, "C Bergström"<br>
<<a href="mailto:cbergstrom@pathscale.com">cbergstrom@pathscale.com</a>>, "Michael Wong" <<a href="mailto:fraggamuffin@gmail.com">fraggamuffin@gmail.com</a>>, "Alexey Bataev" <<a href="mailto:a.bataev@gmx.com">a.bataev@gmx.com</a>><br>
Sent: Friday, May 1, 2015 1:31:06 PM<br>
Subject: Re: [LLVMdev] libiomp, not libgomp as default library linked with -fopenmp<br>
<br>
<br>
<br>
<br>
On Thu, Apr 30, 2015 at 2:51 PM, Chandler Carruth <<br>
<a href="mailto:chandlerc@google.com">chandlerc@google.com</a> > wrote:<br>
<br>
<br>
<br>
<br>
On Thu, Apr 30, 2015 at 6:52 AM Andrey Bokhanko <<br>
<a href="mailto:andreybokhanko@gmail.com">andreybokhanko@gmail.com</a> > wrote:<br>
<br>
<br>
<br>
All,<br>
<br>
<br>
I'd like to resurrect the discussion on replacing libgomp with<br>
libiomp as the default OpenMP runtime library linked with -fopenmp.<br>
<br>
<br>
Just for the record, I'm really excited to see this. =]<br>
<br>
<br>
<br>
<br>
We are very close to getting *full* OpenMP 3.1 specification<br>
supported in clang (only one (!) clause is not implemented yet, and<br>
the patch is already sent for review today:<br>
<a href="http://reviews.llvm.org/D9370">http://reviews.llvm.org/D9370</a> ). This implementation generates Intel<br>
API library calls; thus, it can't be used with libgomp and it is<br>
simply logical to link a compatible runtime (libiomp) instead.<br>
<br>
<br>
<br>
Is there no way to support libgomp here as well? I don't say this to<br>
hold up changing the defaults in any way, just curious. =]<br>
<br>
<br>
Also, for the record, I'm really excited to see the progress here as<br>
well.<br>
<br>
<br>
<br>
<br>
Chandler?<br>
<br>
<br>
<br>
Hi! ;]<br>
<br>
<br>
I totally agree, I think things are way better now. I generally<br>
support the direction. I think there are a few things I'd suggest we<br>
do as part of the process, but I think these are really small and<br>
just about "how" we switch.<br>
<br>
<br>
1) I completely agree with the comments some others have made about<br>
us needing to make it clear that this isn't some Intel-only thing,<br>
its the LLVM OpenMP runtime. Some suggestions that I think would<br>
make sense to help here:<br>
- I agree with finding some non-Intel folks to add as explicit code<br>
owners. I don't know who has been sufficiently involved, but if Hal<br>
makes sense, awesome.<br>
- Clearly updating the readme and such would be appropriate.<br>
- I suspect we should change the name of the installed library.<br>
'libiomp' is pretty clearly the Intel library. We could continue in<br>
the grand tradition of LLVM naming conventions and use 'libllomp'?<br>
Of course, we should install symlinks under the name 'libiomp' if<br>
needed for existing users to not be broken.<br>
<br>
<br>
Just some bikeshed-painting: if we're expecting each compiler that<br>
uses the library to distribute a separate copy as part of that<br>
compiler's runtime, then I think the best name for clang to use for<br>
the library would probably be libclang_rt.omp-<arch> or<br>
libclang_rt.openmp-<arch> (as we do for all our other runtime<br>
libraries). If we expect this to be installed somewhere central on<br>
the system and shared by different compilers and different versions<br>
of the same compiler, then libllomp or similar seems reasonable to<br>
me. What's the intended distribution model here? How stable is the<br>
ABI?<br>
<o:p></o:p></span></p>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black"><br>
I agree. However, for dynamic executables we need a dynamic OpenMP runtime library (there can be only one in the process because it has to keep process-wide state). The ABI is stable, and has a fairly-extensive history (on x86, specifically). cc'ing Jim so
he can also comment directly.<br>
<br>
-Hal<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black"> <br>
<br>
<br>
<br>
<br>
<br>
- Any other changes?<br>
<br>
<br>
2) I think we need to update the instructions for checking out LLVM<br>
and all the tools to include checking out the openmp project. I'm<br>
planning to try it out in a bit.<br>
<br>
<br>
3) It would be nice to get at least one boring benchmark into the<br>
test-suite that uses OpenMP just so there's more coverage that the<br>
basic stuff all works. In particular, if we could get the benchmark<br>
that Phoronix and others keep pointing at, that'd be nice.<br>
<br>
<br>
<br>
<br>
Speaking of which, have you checked the performance of some of the<br>
basic benchmarks using OpenMP with the two runtimes? Or looked at<br>
Clang vs GCC there? I'd be interested to see the numbers.<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
Yours,<br>
Andrey Bokhanko<br>
==============<br>
Software Engineer<br>
Intel Compiler Team<br>
Intel<br>
<br>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu">
http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
<br>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu">
http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
<br>
<br>
<br>
<br>
-- <o:p></o:p></span></p>
</blockquote>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black"><br>
Hal Finkel<br>
Assistant Computational Scientist<br>
Leadership Computing Facility<br>
Argonne National Laboratory<o:p></o:p></span></p>
</div>
</div>
<p>---------------------------------------------------------------------<br>
Intel Corporation (UK) Limited<br>
Registered No. 1134945 (England)<br>
Registered Office: Pipers Way, Swindon SN3 1RJ<br>
VAT No: 860 2173 47</p>
<p>This e-mail and any attachments may contain confidential material for<br>
the sole use of the intended recipient(s). Any review or distribution<br>
by others is strictly prohibited. If you are not the intended<br>
recipient, please contact the sender and delete all copies.</p></body>
</html>