[cfe-dev] [OpenMP] offload support for static libraries
Dmitriev, Serguei N via cfe-dev
cfe-dev at lists.llvm.org
Mon Aug 20 20:17:00 PDT 2018
Hi Doru,
> By "offload link step" do you mean the linking phase that happens on the device toolchain? I assume so.
> Will the device link step just work with "-L. -labc" even if libabc.a contains fat objects?
>
> From the above I understand that what you want is to created your very own version of an NVLINK-like tool which uses the clang-offload-bundler format instead of the CUDA fatbin format.
No, I do not want to change linking for offloading devices, these steps will be done by device toolchains on device objects exactly the same way as it is done today. I suggest to change the initial step of what I called “offload linking” (i.e. linking when offload is enabled) – add partial linking step which is performed by the host toolchain linker on fat objects and libraries, followed by the unbundling of the partial linking result which is done by the existing clang-offload-bundler tool. By “offload linking” I just meant the full set of actions which are needed to link the final binary; it includes link steps for all offloading devices as well as for the host.
This diagram shows the action graph for the following command
clang++ -fopenmp -fopenmp-targets=dev1,dev2 test.cpp -L. -labc -o test
[host compilation] -> h.o host.o --------------------------------
/ \ / \
test.cpp – [dev1 compilation] -> d1.o – [bundle] -> test.o -> [ld –r test.o –L. -labc] -> [unbundle] – [dev1 link] -> d1.out – [linker script] - [host link] -> test
\ / \ /
[dev2 compilation] -> d2.o [dev2 link] -> d2.out
Here
[bundle] is [clang-offload-bundler --bundle]
[unbundle] is [clang-offload-bundler --unbundle]
Bundle operation produces fat object (it is an object that can be passed to the host toolchain). Partial linking (ld -r) is done on fat objects and libraries by the linker from the host toolchain. Unbundle operation extracts host and device parts from the fat object, it will create one host object and one or more objects for each offloading target. Device linking is done by the device toolchain linkers on the device objects extracted from the partially linked object by the unbundle operation.
BTW, I omitted dependencies between host compilation and device compilations to simply the diagram (in reality host bit code is passed to the device compilations as input. These details are not significant to the topic which we are discussing.
I hope this diagram explains my suggestion. Do you see any potential problems which can be caused by this change?
Thanks,
Serguei
From: Gheorghe-Teod Bercea [mailto:Gheorghe-Teod.Bercea at ibm.com]
Sent: Friday, August 17, 2018 3:26 PM
To: Dmitriev, Serguei N <serguei.n.dmitriev at intel.com>
Cc: 'cfe-dev at lists.llvm.org' <cfe-dev at lists.llvm.org>; Jonas Hahnfeld <hahnjo at hahnjo.de>
Subject: RE: [cfe-dev] [OpenMP] offload support for static libraries
Hi Serguei,
Thanks a lot for the explanation on the partial linking step.
> "the intent is to change offload link step to always operate with fat objects and libs"
By "offload link step" do you mean the linking phase that happens on the device toolchain? I assume so.
Will the device link step just work with "-L. -labc" even if libabc.a contains fat objects?
From the above I understand that what you want is to created your very own version of an NVLINK-like tool which uses the clang-offload-bundler format instead of the CUDA fatbin format. This to me means that the "unbundling" step could be included in the device linker step after the resolution of the "-L. -labc" options. The Clang Driver should only invoke the unbundler in the device link step (or have the device linker invoke it internally whatever works in your case). There is never a need to invoke it for the host part. The Clang Driver will be calling the "bundler" to put together the host and device objects.
If you do all this then, at least for your host and device toolchains, you don't need to call "ld -r" at all. I talk below about where to use "ld -r" though.
> "Clang-offload-bundler would need to extract each device object individually while doing unbundling operation on the partially linked object."
I think that as soon as you start unbundling you need to do a full linking step of all device parts to avoid any mess. This supports the idea that you should do the unbundling inside the device linking step where you know exactly which parts you need to link in and where to find them. You then output a single device-only object which you can then bundle with the host object file.
I think this will be equivalent to doing the following:
A. When creating object files "clang++ -fopenmp -fopenmp-targets=intel-device-triple test.cpp -c -o test.o" I need to call the clang-offload-bundler
DEVICE TC: ---[device compilation]---> dev-test.o
\-------[clang-offload-bundler --bundle]---> test.o
HOST TC: ---[host compilation]---> host-test.o /
B. when doing: "clang++ -fopenmp -fopenmp-targets=intel-device-triple test.cpp -L. -labc -o test"there will be no explicit clang-offload-bundler invocations by the clang driver. The device toolchain will look like this:
DEVICE TC: ---[device compilation]---> dev-test.o ---[MyDeviceLinker -L. -labc]--> device --> ....
The important part is that the device link stage will be able to seamlessly link the static lib, which would involve the following steps:
1. find the library using the device linker's capabilities;
2. unpack the library into object files;
3. unbundle each object file to obtain the device object;
4. create a static library with the device objects;
5. link dev-test.o against the device static library you just created.
[I have implemented steps 2->4 before in the clang-ykt compiler so it's definitely feasible once you know the path to your static library, use the linker to resolve that for you]
C. when only linking is needed: "clang++ -fopenmp -fopenmp-targets=intel-device-triple test1.o test2.o -L. -labc -o test"
Exact same happens as above only you need to also unbundle the individual objects and then invoke your usual linker. The clang-offload-bundler will be called by the device link step to "unbundle" the objects on the device.
NVLINK cannot be made to work with the clang-offload-bundler in the way that your device linker can so there will always have to be something special being done for NVPTX targets no matter what solution you choose.
HOWEVER, there is an upside!!
I suspect that the fat object produced by the clang-offload-bundler and the object produced by the OpenMP NVPTX toolchain (with my patch applied) can be successfully partially linked together by "ld -r".
Assuming we have something like this:
clang++ -fopenmp -fopenmp-targets=intel-device-triple,nvptx64-nvidia-cuda test.cpp -c -o test.o
then we can do this:
NVPTX TC: ---[nvptx compilation]---------------------> nvptx-test.o ------------------------\
INTEL TC: ---[intel compilation]---> intel-test.o \
\ ----[clang-offload-bundler]----> tmp.o ---[ld -r]---> test.o
HOST TC: ---[host compilation]---> host-test.o /
Essentially we first resolve the clang-offload-bundler bundling to obtain tmp.o for intel and other toolchains. Then we can partially link tmp.o against the nvptx-test.o object file.
Sanity checks:
The nvptx-test.o file will be treated by "ld -r" just like any other host file that doesn't contain a "bundled" device section (it was not bundled with the clang-offload-bundler but with the CUDA specific format). So if partially linking together a host only file and a bundled file works then this should work too.
test.o can be processed by the NVPTX toolchain? Yes, NVLINK will know where the device side of the object is, no unbundling required to get to the device part.
test.o can be processed by the INTEL toolchain? Yes, the object will be passed to the Device Linker where it will be unbundled and device side extracted.
static linking works for both toolchains? Yes, previous patch enables it for NVPTX, this patch would enable it for INTEL.
static linking works for both toolchains when used together? Yes, because the CUDA specific fatbinary format will not be obstructed by the clang-offload-bundling format.
Thanks,
--Doru
From: "Dmitriev, Serguei N" <serguei.n.dmitriev at intel.com<mailto:serguei.n.dmitriev at intel.com>>
To: Gheorghe-Teod Bercea <Gheorghe-Teod.Bercea at ibm.com<mailto:Gheorghe-Teod.Bercea at ibm.com>>
Cc: "'cfe-dev at lists.llvm.org'" <cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>>, Jonas Hahnfeld <hahnjo at hahnjo.de<mailto:hahnjo at hahnjo.de>>
Date: 08/16/2018 07:26 PM
Subject: RE: [cfe-dev] [OpenMP] offload support for static libraries
________________________________
Hi Doru,
Thank you for the detailed description of your changes.
> Regarding your proposal, from your slides I understand that you perform a partial linking step as the first action for all object files and/or static libraries given as input. So this clang invocation:
> clang++ -L. -labc test.cpp -o test
> would result in the same compilation steps as the current Clang version performs because the initial stage of partial linking would have no work to do (since there are no object files present to be partially linked).
No, the intent is to change offload link step to always operate with fat objects and libs, so the compilation part of the action graph for that command should produce temporary fat object which is then passed to the partial linking. I have not described that in slides, but I agree that it is probably not obvious and should have been mentioned.
> Another question I have is regarding the "ld -r" box in your slides.
> How does ld -r work with "bundled" objects? Your diagram seems to imply that ld -r does the concatenation of all device images out of the box. Is this accurate?
Actually the technique that is currently used by the clang-offload-bundler tool for creating bundled (or fat) objects is very similar to what you are going to do for bundling NVPTX and host objects. The difference is in the initial “wrapper” that is created for the device code – you are using C++ structure, while clang-offload-bundler uses LLVM bitcode file. The bundler tool creates a temporary LLVM IR which contains a global initialized array holding the device object, and this array is allocated in a section with predefined name (the name includes offload target triple). This “wrapper” bitcode file is then compiled for the host and then partially linked against the host object.
So technically fat/bundled object is just a host ELF object which has one or more additional ELF section containing device object as data (one extra section per each offloading target). Linker concatenates sections with the same name while linking multiple objects files, so the result of partial linking will have the same named ELF sections holding device code concatenated from all input (fat) objects.
Clang-offload-bundler would need to extract each device object individually while doing unbundling operation on the partially linked object. A possible way to enable this would be creating one more section holding the device object size in addition to existing section with device object in fat/bundled object. That would allow clang bundler to get sizes of device objects that were concatenated by partial linking.
Thanks,
Serguei
From:Gheorghe-Teod Bercea [mailto:Gheorghe-Teod.Bercea at ibm.com]
Sent: Thursday, August 16, 2018 6:15 AM
To: Dmitriev, Serguei N <serguei.n.dmitriev at intel.com<mailto:serguei.n.dmitriev at intel.com>>
Cc: 'cfe-dev at lists.llvm.org' <cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>>; Jonas Hahnfeld <hahnjo at hahnjo.de<mailto:hahnjo at hahnjo.de>>
Subject: RE: [cfe-dev] [OpenMP] offload support for static libraries
Hi Serguei,
Thanks a lot for the proposal.
My proposal reworks a little bit the way the OpenMP-NVPTX toolchain creates device object files: the device specific part of the object is "wrapped" in an NVLINK-friendly C++ structure that is then compiled for the host. The result is a host object file with a device part which NVLINK can detect (D.o). The D.o object file is then partially linked against the host object file H.o and thus we obtain HD.o. This is required because compilation is required to produce a single output object file (when doing "-c -o" for example). HD.o can now be passed to NVLINK directly or put in a static library and then passed to NVLINK. Either way, NVLINK will be able to detect the device part (due to the special wrapping that we did previously) without the need to "unbundle" the object file (prior to passing it to NVLINK).
The reason why the clang-offload-bundler is not involved in this is because we are using the standard object format for the object file that the OpenMP-NVPTX toolchain outputs so there's no need for a custom format in this case. The partial linking step is required to put together the host and device object files and to ensure that only one object file is produced even if we actually invoked two toolchains (one for host and one for the device).
Regarding your proposal, from your slides I understand that you perform a partial linking step as the first action for all object files and/or static libraries given as input. So this clang invocation:
clang++ -L. -labc test.cpp -o test
would result in the same compilation steps as the current Clang version performs because the initial stage of partial linking would have no work to do (since there are no object files present to be partially linked).
Another question I have is regarding the "ld -r" box in your slides.
How does ld -r work with "bundled" objects? Your diagram seems to imply that ld -r does the concatenation of all device images out of the box. Is this accurate?
Thanks a lot,
--Doru
From: "Dmitriev, Serguei N" <serguei.n.dmitriev at intel.com<mailto:serguei.n.dmitriev at intel.com>>
To: Jonas Hahnfeld <hahnjo at hahnjo.de<mailto:hahnjo at hahnjo.de>>
Cc: "'cfe-dev at lists.llvm.org'" <cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>>, Doru Bercea <gheorghe-teod.bercea at ibm.com<mailto:gheorghe-teod.bercea at ibm.com>>
Date: 08/15/2018 04:27 PM
Subject: RE: [cfe-dev] [OpenMP] offload support for static libraries
________________________________
Hi Jonas,
I guess this patch implements the proposal which Doru presented on the "OpenMP / HPC in Clang / LLVM Multi-company" meeting. As I remember he suggested to eliminate use of clang-offload-bundler tool when offload target is NTVPTX by replacing bundling operation with partial linking of host and device objects, and then relying of the NVPTX linker to perform the unbundling operation at link phase. Based on Doru's explanations NVPTX linker "knows" how to extract device parts from such objects, so the explicit unbundling operation in not required. Doru, please correct me if my understanding is not fully accurate. Doru's proposal definitely achieves the same goal for NVPTX offloading target (i.e. enables offload in static libraries), but it is NVPTX specific and cannot be extended to other offloading targets (at least that is how it looked like when Doru described it).
I propose slightly different solution which I think should work for any generic OpenMP offload target (it was also discussed on the OpenMP multi-company meeting). In general case we have to use clang-offload-bundler because we cannot assume that device object(s) can be bundled with the host object by performing partial linking of host and device objects. So bundling and unbundling operation will still be done by the clang-offload-bundler tool. The main part of my suggestion is adding partial linking of fat objects (created by offload bundler tool) and static libraries (which are composed of fat objects) and only after that do the unbundling operation on the partially linked object (followed by the appropriate link actions for all offloading devices and then for the host). This would guarantee that device parts of fat objects from static libraries will participate in the device link actions, and thus would enable offloading for static libraries.
Thanks,
Serguei
-----Original Message-----
From: Jonas Hahnfeld [mailto:hahnjo at hahnjo.de]
Sent: Tuesday, August 14, 2018 1:52 PM
To: Dmitriev, Serguei N <serguei.n.dmitriev at intel.com<mailto:serguei.n.dmitriev at intel.com>>
Cc: 'cfe-dev at lists.llvm.org' <cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>>; Doru Bercea <gheorghe-teod.bercea at ibm.com<mailto:gheorghe-teod.bercea at ibm.com>>
Subject: Re: [cfe-dev] [OpenMP] offload support for static libraries
This proposal has already been proposed for NVPTX in https://reviews.llvm.org/D47394, adding Doru.
Cheers,
Jonas
On 2018-08-14 18:43, Dmitriev, Serguei N via cfe-dev wrote:
> PROBLEM OVERVIEW
>
> OpenMP offload functionality is currently not supported in static
> libraries. Because of that an attempt to use offloading in static
> libraries ends up with a fallback execution of target regions on the
> host. This limitation clearly has significant impact on OpenMP offload
> usability.
>
> An output object file that is created by the compiler for offload
> compilation is a fat object. Such object files besides the code for
> the host architecture also contains code for the offloading targets
> which is stored as data in ELF sections with predefined names. Thus, a
> static library that is created from object files produced by offload
> compilation would be an archive of fat objects.
>
> Clang driver currently never passes fat objects directly to any
> toolchain. Instead it performs an unbundling operation for each fat
> object which extract host and device parts from the object. These
> parts are then independently processed by the corresponding target
> toolchains. However, current implementation does not assume that
> static archives may also be composed from fat objects. No unbundling
> is done for static archives (they are passed to linker as is) and thus
> device parts of objects from such archives get ignored.
>
> SUGGESTED SOLUTION
>
> It seems feasible to resolve this problem by changing the offload link
> process - adding an extra step to the link flow which will do a
> partial linking (ld -r) of fat objects and static libraries as shown
> on this diagram
>
> [Fat objects] \ / [Target1 link] \
>
> [Partial linking] - [Unbundling] - [TargetN link] -
> [Host link]
>
> [Static libs] / \--- Host part --/
>
> (You can also look at the .pdf file on this link
> https://drive.google.com/file/d/1ZTNoB-Ghin1BTaiZ312FMSRS6rISDtlr/view
> ?usp=sharing [1] for illustrations for the suggested change)
>
> Linker will pull in all necessary dependencies from static libraries
> while performing partial linking, so the result of partial linking
> would be a fat object with concatenated device parts from input fat
> objects and required dependencies from static libraries. These
> concatenated device objects will be stored in the corresponding ELF
> sections of the partially linked object.
>
> Unbundling operation on the partially linked object will create one or
> more device objects for each offloading target, and these objects will
> be linked by corresponding target toolchains the same way as it is
> done now. Offload bundler tool would require enhancements to support
> unbundling of multiple concatenated device objects for each offloading
> target.
>
> Host link action can be changed to use host part of the partially
> linked object while linking the final image.
>
> Do you see any potential problems in the proposed change?
>
> Links:
> ------
> [1]
> https://drive.google.com/file/d/1ZTNoB-Ghin1BTaiZ312FMSRS6rISDtlr/view
> ?usp=sharing _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20180821/5d3a3342/attachment.html>
More information about the cfe-dev
mailing list