[LLVMdev] libiomp, not libgomp as default library linked with -fopenmp

Wed May 6 06:39:16 PDT 2015

Chandler:
 1) I completely agree with the comments some others have made about
 us needing to make it clear that this isn't some Intel-only thing,
 it’s the LLVM OpenMP runtime. Some suggestions that I think would
 make sense to help here:

… code owner discussion elided since Chris has endorsed Andrey…

- Clearly updating the readme and such would be appropriate.
Certainly, no problem with that at all. As I ponted out before, people from Intel are not the right ones to be inserting compatiblity tables for the runtime when it is running on IBM Power or ARM processors. The people who know about that should contribute…

 - I suspect we should change the name of the installed library.
 'libiomp' is pretty clearly the Intel library. We could continue in
 the grand tradition of LLVM naming conventions and use 'libllomp'?
 Of course, we should install symlinks under the name 'libiomp' if
 needed for existing users to not be broken.

Changing the file name (at least when targeting X86 processors) is a more interesting issue, and not as trivial as it seems.
The problem here is related to inter-operation with existing (Intel and gcc) compilers, and backwards binary compatibility.
As Hal pointed out, if you intend ever to mix code compiled by multiple, different, OpenMP compilers into the same process, even if they all use an ABI to the runtime that it supports (remember that the LLVM runtime supports both the LLVM and libgomp interfaces), you must ensure that there is only one copy of the runtime (dynamically) linked into the process. This is because (unlike most libraries) the runtime contains state (for instance, the thread pool), therefore if you have multiple instances even of the identical runtime linked into the process, bad things will happen. (Over-subscription and associated poor performance, critical sections not providing mutual exclusion, …). This is why we don’t normally provide a static library; it tempts people into building their own dynamic libraries that include a static copy of the OpenMP runtime, then when their DLL is linked into an OpenMP program you instantly have two copies of the runtime and a hard to diagnose disaster.

The problem this causes is that unless there is an agreed name for the runtime, the dynamic linker will load it twice if it is demanded under different names. (On Linux, you can use soname to help here, but doesn’t completely solve the problem).

Therefore, if we do not want to make it unnecessarily hard for people who use Clang/OpenMP on X86 to link against libraries (such as Intel MKL) which use OpenMP and were compiled with a different compiler, it is helpful to use the same file name for the runtime library no matter which compiler generated the code.

Since there are many libraries and executables which have already been built by the Intel compiler, so already have the library name built into them (in the DT_NEEDED ELF tag, or equivalent), changing the library name does not come for free.

I am therefore loath to change the library name simply for political/publicity reasons when that will cause unnecessary additional complexity to the users of one of our main target platforms. (Before anyone else says it, yes, of course X86 is the platform I care most about, but the issue isn’t about compiler competition, but about making things easy for the users of LLVM/OpenMP so that things they expect will work do “just work” rather than adding extra complexity and more manuals to explain what has to be done to make them work. Like moving into your partner’s appartment, you’re welll advised not to move all the furniture around immediately ☺).

 Just some bikeshed-painting: if we're expecting each compiler that
 uses the library to distribute a separate copy as part of that
 compiler's runtime, then I think the best name for clang to use for
 the library would probably be libclang_rt.omp-<arch> or
 libclang_rt.openmp-<arch> (as we do for all our other runtime
 libraries).
Aside from the argument above that we shouldn’t change the name (at least on X86), note that if Hal’s efforts on Flang work out, we’ll also have a Fortran/OpenMP implementation that will use the same runtime library. Therefore putting “clang” into the name seems short-sighted.

If we expect this to be installed somewhere central on
 the system and shared by different compilers and different versions
 of the same compiler, then libllomp or similar seems reasonable to
 me. What's the intended distribution model here?
Whether OS vendors choose to install the runtime in a standard place (like /usr/lib64) is not something we can control. (Though doing so is a reasonable thing to do).

How stable is the ABI?
Stable; we add new interfaces when required by the evolution of the OpenMP language specification, but maintain backwards compatibility, so code compiled by older compilers can be run against newer runtime libraries.

On the benchmarks side of things, since everyone is so keen on this not only being about Intel, surely it’d be good to have some comparisons of performance vs gcc on Power or Arm as well as on X86.

-- Jim

James Cownie <james.h.cownie at intel.com>
SSG/DPD/TCAR (Technical Computing, Analyzers and Runtimes)
Tel: +44 117 9071438

From: Hal Finkel [mailto:hfinkel at anl.gov]
Sent: Tuesday, May 5, 2015 5:39 PM
To: Richard Smith
Cc: Andrey Bokhanko; cfe-dev; LLVM Developers Mailing List; Douglas Gregor; C Bergström; Michael Wong; Alexey Bataev; Chandler Carruth; Cownie, James H
Subject: Re: [LLVMdev] libiomp, not libgomp as default library linked with -fopenmp

________________________________

From: "Richard Smith" <richard at metafoo.co.uk<mailto:richard at metafoo.co.uk>>
 To: "Chandler Carruth" <chandlerc at google.com<mailto:chandlerc at google.com>>
 Cc: "Andrey Bokhanko" <andreybokhanko at gmail.com<mailto:andreybokhanko at gmail.com>>, "cfe-dev" <cfe-dev at cs.uiuc.edu<mailto:cfe-dev at cs.uiuc.edu>>, "LLVM Developers Mailing List"
 <llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu>>, "Douglas Gregor" <dgregor at apple.com<mailto:dgregor at apple.com>>, "Hal Finkel" <hfinkel at anl.gov<mailto:hfinkel at anl.gov>>, "C Bergström"
 <cbergstrom at pathscale.com<mailto:cbergstrom at pathscale.com>>, "Michael Wong" <fraggamuffin at gmail.com<mailto:fraggamuffin at gmail.com>>, "Alexey Bataev" <a.bataev at gmx.com<mailto:a.bataev at gmx.com>>
 Sent: Friday, May 1, 2015 1:31:06 PM
 Subject: Re: [LLVMdev] libiomp, not libgomp as default library linked with -fopenmp

 On Thu, Apr 30, 2015 at 2:51 PM, Chandler Carruth <
 chandlerc at google.com<mailto:chandlerc at google.com> > wrote:

 On Thu, Apr 30, 2015 at 6:52 AM Andrey Bokhanko <
 andreybokhanko at gmail.com<mailto:andreybokhanko at gmail.com> > wrote:

 All,

 I'd like to resurrect the discussion on replacing libgomp with
 libiomp as the default OpenMP runtime library linked with -fopenmp.

 Just for the record, I'm really excited to see this. =]

 We are very close to getting *full* OpenMP 3.1 specification
 supported in clang (only one (!) clause is not implemented yet, and
 the patch is already sent for review today:
 http://reviews.llvm.org/D9370 ). This implementation generates Intel
 API library calls; thus, it can't be used with libgomp and it is
 simply logical to link a compatible runtime (libiomp) instead.

 Is there no way to support libgomp here as well? I don't say this to
 hold up changing the defaults in any way, just curious. =]

 Also, for the record, I'm really excited to see the progress here as
 well.

 Chandler?

 Hi! ;]

 I totally agree, I think things are way better now. I generally
 support the direction. I think there are a few things I'd suggest we
 do as part of the process, but I think these are really small and
 just about "how" we switch.

 1) I completely agree with the comments some others have made about
 us needing to make it clear that this isn't some Intel-only thing,
 its the LLVM OpenMP runtime. Some suggestions that I think would
 make sense to help here:
 - I agree with finding some non-Intel folks to add as explicit code
 owners. I don't know who has been sufficiently involved, but if Hal
 makes sense, awesome.
 - Clearly updating the readme and such would be appropriate.
 - I suspect we should change the name of the installed library.
 'libiomp' is pretty clearly the Intel library. We could continue in
 the grand tradition of LLVM naming conventions and use 'libllomp'?
 Of course, we should install symlinks under the name 'libiomp' if
 needed for existing users to not be broken.

 Just some bikeshed-painting: if we're expecting each compiler that
 uses the library to distribute a separate copy as part of that
 compiler's runtime, then I think the best name for clang to use for
 the library would probably be libclang_rt.omp-<arch> or
 libclang_rt.openmp-<arch> (as we do for all our other runtime
 libraries). If we expect this to be installed somewhere central on
 the system and shared by different compilers and different versions
 of the same compiler, then libllomp or similar seems reasonable to
 me. What's the intended distribution model here? How stable is the
 ABI?

I agree. However, for dynamic executables we need a dynamic OpenMP runtime library (there can be only one in the process because it has to keep process-wide state). The ABI is stable, and has a fairly-extensive history (on x86, specifically). cc'ing Jim so he can also comment directly.

 -Hal

 - Any other changes?

 2) I think we need to update the instructions for checking out LLVM
 and all the tools to include checking out the openmp project. I'm
 planning to try it out in a bit.

 3) It would be nice to get at least one boring benchmark into the
 test-suite that uses OpenMP just so there's more coverage that the
 basic stuff all works. In particular, if we could get the benchmark
 that Phoronix and others keep pointing at, that'd be nice.

 Speaking of which, have you checked the performance of some of the
 basic benchmarks using OpenMP with the two runtimes? Or looked at
 Clang vs GCC there? I'd be interested to see the numbers.

 Yours,
 Andrey Bokhanko
 ==============
 Software Engineer
 Intel Compiler Team
 Intel

 _______________________________________________
 LLVM Developers mailing list
 LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu> http://llvm.cs.uiuc.edu
 http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

 _______________________________________________
 LLVM Developers mailing list
 LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu> http://llvm.cs.uiuc.edu
 http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

--

Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory
---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150506/a1cc53cc/attachment.html>