[PATCH] D74262: [clang-offload-bundler] Enable handling of partially-linked fat objects

[George Rokos via Phabricator via cfe-commits]

grokos created this revision.
grokos added reviewers: hfinkel, jdoerfert, ABataev, mdtoguchi, kbobrovs, sdmitriev.
grokos added a project: clang.
Herald added a subscriber: Anastasia.

This is the bundler-side patch for enabling static library support in clang. The scheme has been discussed extensively in the past and is described in this document prepared by @sdmitriev: F11310194: offload from static libs.pdf <https://reviews.llvm.org/F11310194>.

Patch was developed in collaboration with @kbobrovs and a similar version has been merged with Intel's SYCL compiler (https://github.com/intel/llvm/tree/sycl).

When a fat object is created, for each bundle the bundler also creates a corresponding "size" section consisting of a single 64-bit integer storing the size of the bundle. When linking from static objects, the host linker will fetch all dependencies and do a partial linking on them; this action concatenates all sections with the same name across fetched dependencies into a new aggregate section, so for each target there will be an aggregate section containing the concatenated bundles and another aggregate section containing the concatenated sizes. By visiting the aggregate sizes section the unbundler can then split the aggregate bundle into separate output device objects.

The patch introduces a new type "oo" which is used when unbundling partially-linked fat objects. When "oo" is specified, the output file is not an object file itself; instead it is a text file containing the paths to the actual outputs (because we may have multiple device objects as outputs - one for each dependency that was fetched).

Invocation of the host linker (to do partial-linking) and cleanup of temporary files will be done by the Driver. Once the bundler patch lands, the Driver patch will follow.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D74262

Files:
  clang/test/Driver/clang-offload-bundler-missing-size-section.cpp
  clang/test/Driver/clang-offload-bundler-oo.cpp
  clang/tools/clang-offload-bundler/ClangOffloadBundler.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D74262.243295.patch
Type: text/x-patch
Size: 24082 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20200207/3854dac7/attachment-0001.bin>