r369019 - [Driver][Bundler] Improve bundling of object files.
Alexey Bataev via cfe-commits
cfe-commits at lists.llvm.org
Thu Aug 15 10:15:36 PDT 2019
Author: abataev
Date: Thu Aug 15 10:15:35 2019
New Revision: 369019
URL: http://llvm.org/viewvc/llvm-project?rev=369019&view=rev
Log:
[Driver][Bundler] Improve bundling of object files.
Summary:
Previously, object files were bundled using partial linking. It resulted
in the following structure of the bundled objects:
```
<host_code>
clang-offload-bundle
__CLANG_OFFLOAD_BUNDLE__<target>
<target_code>
```
But when we tried to unbundle object files, it worked correctly only for
the target objects. The host object remains bundled. It produced a lot of
junk sections in the host object files and in some cases may caused
incorrect linking.
Patch improves bundling of the object files. After this patch the
bundled object looks like this:
```
<host_code>
clang-offload-bundle
__CLANG_OFFLOAD_BUNDLE__<target>
<target_code>
__CLANG_OFFLOAD_BUNDLE__<host>
<host_code>
```
With this structure we are able to unbundle the host object files too so
that after unbundling they are the same as were before.
The host section is bundled twice. The bundled section is used to
unbundle the original host section.
Reviewers: yaxunl, tra, jlebar, hfinkel, jdoerfert
Subscribers: caomhin, kkwli0, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D65819
Modified:
cfe/trunk/test/Driver/clang-offload-bundler.c
cfe/trunk/tools/clang-offload-bundler/ClangOffloadBundler.cpp
Modified: cfe/trunk/test/Driver/clang-offload-bundler.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/clang-offload-bundler.c?rev=369019&r1=369018&r2=369019&view=diff
==============================================================================
--- cfe/trunk/test/Driver/clang-offload-bundler.c (original)
+++ cfe/trunk/test/Driver/clang-offload-bundler.c Thu Aug 15 10:15:35 2019
@@ -231,18 +231,18 @@
// RUN: clang-offload-bundler -type=o -targets=host-%itanium_abi_triple,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu -inputs=%t.o,%t.tgt1,%t.tgt2 -outputs=%t.bundle3.o -### -dump-temporary-files 2>&1 \
// RUN: | FileCheck %s --check-prefix CK-OBJ-CMD
-// CK-OBJ-CMD: private constant [1 x i8] zeroinitializer, section "__CLANG_OFFLOAD_BUNDLE__host-[[HOST:.+]]"
+// CK-OBJ-CMD: private constant [{{[0-9]+}} x i8] c"{{.+}}", section "__CLANG_OFFLOAD_BUNDLE__host-[[HOST:.+]]"
// CK-OBJ-CMD: private constant [{{[0-9]+}} x i8] c"Content of device file 1{{.+}}", section "__CLANG_OFFLOAD_BUNDLE__openmp-powerpc64le-ibm-linux-gnu"
// CK-OBJ-CMD: private constant [{{[0-9]+}} x i8] c"Content of device file 2{{.+}}", section "__CLANG_OFFLOAD_BUNDLE__openmp-x86_64-pc-linux-gnu"
// CK-OBJ-CMD: clang{{(.exe)?}}" "-r" "-target" "[[HOST]]" "-o" "{{.+}}.o" "{{.+}}.o" "{{.+}}.bc" "-nostdlib"
// RUN: clang-offload-bundler -type=o -targets=host-%itanium_abi_triple,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu -inputs=%t.o,%t.tgt1,%t.tgt2 -outputs=%t.bundle3.o
// RUN: clang-offload-bundler -type=o -targets=host-%itanium_abi_triple,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu -outputs=%t.res.o,%t.res.tgt1,%t.res.tgt2 -inputs=%t.bundle3.o -unbundle
-// RUN: diff %t.bundle3.o %t.res.o
+// RUN: diff %t.o %t.res.o
// RUN: diff %t.tgt1 %t.res.tgt1
// RUN: diff %t.tgt2 %t.res.tgt2
// RUN: clang-offload-bundler -type=o -targets=openmp-powerpc64le-ibm-linux-gnu,host-%itanium_abi_triple,openmp-x86_64-pc-linux-gnu -outputs=%t.res.tgt1,%t.res.o,%t.res.tgt2 -inputs=%t.bundle3.o -unbundle
-// RUN: diff %t.bundle3.o %t.res.o
+// RUN: diff %t.o %t.res.o
// RUN: diff %t.tgt1 %t.res.tgt1
// RUN: diff %t.tgt2 %t.res.tgt2
Modified: cfe/trunk/tools/clang-offload-bundler/ClangOffloadBundler.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-offload-bundler/ClangOffloadBundler.cpp?rev=369019&r1=369018&r2=369019&view=diff
==============================================================================
--- cfe/trunk/tools/clang-offload-bundler/ClangOffloadBundler.cpp (original)
+++ cfe/trunk/tools/clang-offload-bundler/ClangOffloadBundler.cpp Thu Aug 15 10:15:35 2019
@@ -370,13 +370,9 @@ public:
/// designated name.
///
/// In order to bundle we create an IR file with the content of each section and
-/// use incremental linking to produce the resulting object. We also add section
-/// with a single byte to state the name of the component the main object file
-/// (the one we are bundling into) refers to.
+/// use incremental linking to produce the resulting object.
///
-/// To unbundle, we use just copy the contents of the designated section. If the
-/// requested bundle refer to the main object file, we just copy it with no
-/// changes.
+/// To unbundle, we just copy the contents of the designated section.
class ObjectFileHandler final : public FileHandler {
/// The object file we are currently dealing with.
@@ -471,10 +467,7 @@ public:
return;
}
- if (Content->size() < 2)
- OS.write(Input.getBufferStart(), Input.getBufferSize());
- else
- OS.write(Content->data(), Content->size());
+ OS.write(Content->data(), Content->size());
}
void WriteHeader(raw_fd_ostream &OS,
@@ -592,22 +585,14 @@ public:
std::string SectionName = OFFLOAD_BUNDLER_MAGIC_STR;
SectionName += CurrentTriple;
- // Create the constant with the content of the section. For the input we are
- // bundling into (the host input), this is just a place-holder, so a single
- // byte is sufficient.
- assert(HostInputIndex != ~0u && "Host input index undefined??");
- Constant *Content;
- if (NumberOfProcessedInputs == HostInputIndex + 1) {
- uint8_t Byte[] = {0};
- Content = ConstantDataArray::get(VMContext, Byte);
- } else
- Content = ConstantDataArray::get(
- VMContext, ArrayRef<uint8_t>(reinterpret_cast<const uint8_t *>(
- Input.getBufferStart()),
- Input.getBufferSize()));
+ // Create the constant with the content of the section.
+ auto *Content = ConstantDataArray::get(
+ VMContext, ArrayRef<uint8_t>(reinterpret_cast<const uint8_t *>(
+ Input.getBufferStart()),
+ Input.getBufferSize()));
- // Create the global in the desired section. We don't want these globals in
- // the symbol table, so we mark them private.
+ // Create the global in the desired section. We don't want these globals
+ // in the symbol table, so we mark them private.
auto *GV = new GlobalVariable(*M, Content->getType(), /*IsConstant=*/true,
GlobalVariable::PrivateLinkage, Content);
GV->setSection(SectionName);
More information about the cfe-commits
mailing list