[llvm] 44265dc - Reland [llvm] Preliminary fat-lto-objects support
Paul Kirth via llvm-commits
llvm-commits at lists.llvm.org
Fri Jun 23 16:24:20 PDT 2023
Author: Paul Kirth
Date: 2023-06-23T23:23:58Z
New Revision: 44265dc3554ef40920b587eeb787a400663af6c7
URL: https://github.com/llvm/llvm-project/commit/44265dc3554ef40920b587eeb787a400663af6c7
DIFF: https://github.com/llvm/llvm-project/commit/44265dc3554ef40920b587eeb787a400663af6c7.diff
LOG: Reland [llvm] Preliminary fat-lto-objects support
Fat LTO objects contain both LTO compatible IR, as well as generated
object code. This allows users to defer the choice of whether to use LTO
or not to link-time. This is a feature available in GCC for some time,
and makes the existing -ffat-lto-objects flag functional in the same
way as GCC's.
Within LLVM, we add a new EmbedBitcodePass that serializes the module to
the object file, and expose a new pass pipeline for compiling fat
objects. The new pipeline initially clones the module and runs the
selected (Thin)LTOPrelink pipeline, after which it will serialize the
module into a `.llvm.lto` section of an ELF file. When compiling for
(Thin)LTO, this normally the point at which the compiler would emit a
object file containing the bitcode and metadata.
After that point we compile the original module using the
PerModuleDefaultPipeline used for non-LTO compilation. We generate
standard object files at the end of this pipeline, which contain machine
code and the new `.llvm.lto` section containing bitcode.
Since the two pipelines operate on different copies of the module, we
can be sure that the bitcode in the `.llvm.lto` section and object code
in `.text` are congruent with the existing output produced by the
default and LTO pipelines.
Original RFC: https://discourse.llvm.org/t/rfc-ffat-lto-objects-support/63977
Earlier versions of this patch were missing REQUIRES lines for llc
related tests in Transforms/EmbedBitcode. Those tests are now under
CodeGen/X86, which should avoid running the check on unsupported
platforms.
Reviewed By: tejohnson, MaskRay, nikic
Differential Revision: https://reviews.llvm.org/D146776
Added:
llvm/docs/FatLTO.rst
llvm/include/llvm/Transforms/IPO/EmbedBitcodePass.h
llvm/lib/Transforms/IPO/EmbedBitcodePass.cpp
llvm/test/CodeGen/X86/fat-lto-section.ll
llvm/test/Transforms/EmbedBitcode/embed-multiple.ll
llvm/test/Transforms/EmbedBitcode/embed-unsupported-object-format.ll
llvm/test/Transforms/EmbedBitcode/embed.ll
Modified:
llvm/docs/ReleaseNotes.rst
llvm/docs/UserGuides.rst
llvm/include/llvm/Passes/PassBuilder.h
llvm/lib/Object/ObjectFile.cpp
llvm/lib/Passes/PassBuilder.cpp
llvm/lib/Passes/PassBuilderPipelines.cpp
llvm/lib/Passes/PassRegistry.def
llvm/lib/Transforms/IPO/CMakeLists.txt
Removed:
################################################################################
diff --git a/llvm/docs/FatLTO.rst b/llvm/docs/FatLTO.rst
new file mode 100644
index 0000000000000..5773a30b7ae4a
--- /dev/null
+++ b/llvm/docs/FatLTO.rst
@@ -0,0 +1,77 @@
+===================
+FatLTO
+===================
+.. contents::
+ :local:
+ :depth: 2
+
+.. toctree::
+ :maxdepth: 1
+
+Introduction
+============
+
+FatLTO objects are a special type of `fat object file
+<https://en.wikipedia.org/wiki/Fat_binary>`_ that contain LTO compatible IR in
+addition to generated object code, instead of containing object code for
+multiple target architectures. This allows users to defer the choice of whether
+to use LTO or not to link-time, and has been a feature available in other
+compilers, like `GCC
+<https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html>`_, for some time.
+
+Under FatLTO the compiler can emit standard object files which contain both the
+machine code in the ``.text`` section and LLVM bitcode in the ``.llvm.lto``
+section.
+
+Overview
+========
+
+Within LLVM, FatLTO is supported by choosing the ``FatLTODefaultPipeline``.
+This pipeline will:
+
+#) Clone the IR module.
+#) Run the pre-link (Thin)LTO pipeline using the cloned module.
+#) Embed the pre-link bitcode in a special ``.llvm.lto`` section.
+#) Optimize the unmodified copy of the module using the normal compilation pipeline.
+#) Emit the object file, including the new ``.llvm.lto`` section.
+
+.. NOTE
+
+ At the time of writing, we conservatively run independent pipelines to
+ generate the bitcode section and the object code, which happen to be
+ identical to those used outside of FatLTO. This results in compiled
+ artifacts that are identical to those produced by the default and (Thin)LTO
+ pipelines. However, this is not a guarantee, and we reserve the right to
+ change this at any time. Explicitly, users should not rely on the produced
+ bitcode or object code to mach their non-LTO counterparts precisely. They
+ will exhibit similar performance characteristics, but may not be bit-for-bit
+ the same.
+
+Internally, the ``.llvm.lto`` section is created by running the
+``EmbedBitcodePass`` at the start of the ``PerModuleDefaultPipeline``. This
+pass is responsible for cloning and optimizing the module with the appropriate
+LTO pipeline and emitting the ``.llvm.lto`` section. Afterwards, the
+``PerModuleDefaultPipeline`` runs normally and the compiler can emit the fat
+object file.
+
+Limitations
+===========
+
+Linkers
+-------
+
+Currently, using LTO with LLVM fat lto objects is supported by LLD and by the
+GNU linkers via :doc:`GoldPlugin`. This may change in the future, but
+extending support to other linkers isn't planned for now.
+
+.. NOTE
+ For standard linking the fat object files should be usable by any
+ linker capable of using ELF objects, since the ``.llvm.lto`` section is
+ marked ``SHF_EXLUDE``.
+
+Supported File Formats
+----------------------
+
+The current implementation only supports ELF files. At time of writing, it is
+unclear if it will be useful to support other object file formats like ``COFF``
+or ``Mach-O``.
diff --git a/llvm/docs/ReleaseNotes.rst b/llvm/docs/ReleaseNotes.rst
index 6efc991ff509e..e086c235c929a 100644
--- a/llvm/docs/ReleaseNotes.rst
+++ b/llvm/docs/ReleaseNotes.rst
@@ -80,6 +80,12 @@ Changes to LLVM infrastructure
* InstructionSimplify APIs now require instructions be inserted into a
parent function.
+* A new FatLTO pipeline was added to support generating object files that have
+ both machine code and LTO compatible bitcode. See the :doc:`FatLTO`
+ documentation and the original
+ `RFC <https://discourse.llvm.org/t/rfc-ffat-lto-objects-support/63977>`_
+ for more details.
+
Changes to building LLVM
------------------------
diff --git a/llvm/docs/UserGuides.rst b/llvm/docs/UserGuides.rst
index 517a23e959206..d7335911abe1c 100644
--- a/llvm/docs/UserGuides.rst
+++ b/llvm/docs/UserGuides.rst
@@ -32,6 +32,7 @@ intermediate LLVM representation.
DebuggingJITedCode
DirectXUsage
Docker
+ FatLTO
ExtendingLLVM
GoldPlugin
HowToBuildOnARM
diff --git a/llvm/include/llvm/Passes/PassBuilder.h b/llvm/include/llvm/Passes/PassBuilder.h
index 585c335cf5728..f31c32102e694 100644
--- a/llvm/include/llvm/Passes/PassBuilder.h
+++ b/llvm/include/llvm/Passes/PassBuilder.h
@@ -234,6 +234,18 @@ class PassBuilder {
ModulePassManager buildPerModuleDefaultPipeline(OptimizationLevel Level,
bool LTOPreLink = false);
+ /// Build a fat object default optimization pipeline.
+ ///
+ /// This builds a pipeline that runs the LTO/ThinLTO pre-link pipeline, and
+ /// emits a section containing the pre-link bitcode along side the object code
+ /// generated by running the PerModuleDefaultPipeline, used when compiling
+ /// without LTO. It clones the module and runs the LTO/non-LTO pipelines
+ /// separately to avoid any inconsistencies with an ad-hoc pipeline that tries
+ /// to approximate the PerModuleDefaultPipeline from the pre-link LTO
+ /// pipelines.
+ ModulePassManager buildFatLTODefaultPipeline(OptimizationLevel Level,
+ bool ThinLTO, bool EmitSummary);
+
/// Build a pre-link, ThinLTO-targeting default optimization pipeline to
/// a pass manager.
///
diff --git a/llvm/include/llvm/Transforms/IPO/EmbedBitcodePass.h b/llvm/include/llvm/Transforms/IPO/EmbedBitcodePass.h
new file mode 100644
index 0000000000000..f323c61483fd3
--- /dev/null
+++ b/llvm/include/llvm/Transforms/IPO/EmbedBitcodePass.h
@@ -0,0 +1,58 @@
+//===-- EmbedBitcodePass.h - Embeds bitcode into global ---------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+/// \file
+///
+/// This file provides a pass which clones the current module and runs the
+/// provided pass pipeline on the clone. The optimized module is stored into a
+/// global variable in the `.llvm.lto` section. Primarily, this pass is used
+/// to support the FatLTO pipeline, but could be used to generate a bitcode
+/// section for any arbitrary pass pipeline without changing the current module.
+///
+//===----------------------------------------------------------------------===//
+//
+#ifndef LLVM_TRANSFORMS_IPO_EMBEDBITCODEPASS_H
+#define LLVM_TRANSFORMS_IPO_EMBEDBITCODEPASS_H
+
+#include "llvm/IR/PassManager.h"
+
+namespace llvm {
+class Module;
+class ModulePass;
+class Pass;
+
+struct EmbedBitcodeOptions {
+ EmbedBitcodeOptions() : EmbedBitcodeOptions(false, false) {}
+ EmbedBitcodeOptions(bool IsThinLTO, bool EmitLTOSummary)
+ : IsThinLTO(IsThinLTO), EmitLTOSummary(EmitLTOSummary) {}
+ bool IsThinLTO;
+ bool EmitLTOSummary;
+};
+
+/// Pass embeds a copy of the module optimized with the provided pass pipeline
+/// into a global variable.
+class EmbedBitcodePass : public PassInfoMixin<EmbedBitcodePass> {
+ bool IsThinLTO;
+ bool EmitLTOSummary;
+ ModulePassManager MPM;
+
+public:
+ EmbedBitcodePass(EmbedBitcodeOptions Opts)
+ : EmbedBitcodePass(Opts.IsThinLTO, Opts.EmitLTOSummary,
+ ModulePassManager()) {}
+ EmbedBitcodePass(bool IsThinLTO, bool EmitLTOSummary, ModulePassManager &&MPM)
+ : IsThinLTO(IsThinLTO), EmitLTOSummary(EmitLTOSummary),
+ MPM(std::move(MPM)) {}
+
+ PreservedAnalyses run(Module &M, ModuleAnalysisManager &);
+
+ static bool isRequired() { return true; }
+};
+
+} // end namespace llvm.
+
+#endif
diff --git a/llvm/lib/Object/ObjectFile.cpp b/llvm/lib/Object/ObjectFile.cpp
index c953a5692cbb4..0820187f32e14 100644
--- a/llvm/lib/Object/ObjectFile.cpp
+++ b/llvm/lib/Object/ObjectFile.cpp
@@ -79,7 +79,7 @@ uint32_t ObjectFile::getSymbolAlignment(DataRefImpl DRI) const { return 0; }
bool ObjectFile::isSectionBitcode(DataRefImpl Sec) const {
Expected<StringRef> NameOrErr = getSectionName(Sec);
if (NameOrErr)
- return *NameOrErr == ".llvmbc";
+ return *NameOrErr == ".llvmbc" || *NameOrErr == ".llvm.lto";
consumeError(NameOrErr.takeError());
return false;
}
diff --git a/llvm/lib/Passes/PassBuilder.cpp b/llvm/lib/Passes/PassBuilder.cpp
index a4972c8713486..d7d7db8c828e9 100644
--- a/llvm/lib/Passes/PassBuilder.cpp
+++ b/llvm/lib/Passes/PassBuilder.cpp
@@ -102,6 +102,7 @@
#include "llvm/Transforms/IPO/CrossDSOCFI.h"
#include "llvm/Transforms/IPO/DeadArgumentElimination.h"
#include "llvm/Transforms/IPO/ElimAvailExtern.h"
+#include "llvm/Transforms/IPO/EmbedBitcodePass.h"
#include "llvm/Transforms/IPO/ForceFunctionAttrs.h"
#include "llvm/Transforms/IPO/FunctionAttrs.h"
#include "llvm/Transforms/IPO/FunctionImport.h"
@@ -738,6 +739,26 @@ Expected<HWAddressSanitizerOptions> parseHWASanPassOptions(StringRef Params) {
return Result;
}
+Expected<EmbedBitcodeOptions> parseEmbedBitcodePassOptions(StringRef Params) {
+ EmbedBitcodeOptions Result;
+ while (!Params.empty()) {
+ StringRef ParamName;
+ std::tie(ParamName, Params) = Params.split(';');
+
+ if (ParamName == "thinlto") {
+ Result.IsThinLTO = true;
+ } else if (ParamName == "emit-summary") {
+ Result.EmitLTOSummary = true;
+ } else {
+ return make_error<StringError>(
+ formatv("invalid EmbedBitcode pass parameter '{0}' ", ParamName)
+ .str(),
+ inconvertibleErrorCode());
+ }
+ }
+ return Result;
+}
+
Expected<MemorySanitizerOptions> parseMSanPassOptions(StringRef Params) {
MemorySanitizerOptions Result;
while (!Params.empty()) {
diff --git a/llvm/lib/Passes/PassBuilderPipelines.cpp b/llvm/lib/Passes/PassBuilderPipelines.cpp
index 34d3b9d7467e8..515ef1bb35013 100644
--- a/llvm/lib/Passes/PassBuilderPipelines.cpp
+++ b/llvm/lib/Passes/PassBuilderPipelines.cpp
@@ -46,6 +46,7 @@
#include "llvm/Transforms/IPO/CrossDSOCFI.h"
#include "llvm/Transforms/IPO/DeadArgumentElimination.h"
#include "llvm/Transforms/IPO/ElimAvailExtern.h"
+#include "llvm/Transforms/IPO/EmbedBitcodePass.h"
#include "llvm/Transforms/IPO/ForceFunctionAttrs.h"
#include "llvm/Transforms/IPO/FunctionAttrs.h"
#include "llvm/Transforms/IPO/GlobalDCE.h"
@@ -1464,7 +1465,18 @@ PassBuilder::buildPerModuleDefaultPipeline(OptimizationLevel Level,
if (LTOPreLink)
addRequiredLTOPreLinkPasses(MPM);
+ return MPM;
+}
+ModulePassManager
+PassBuilder::buildFatLTODefaultPipeline(OptimizationLevel Level, bool ThinLTO,
+ bool EmitSummary) {
+ ModulePassManager MPM;
+ MPM.addPass(EmbedBitcodePass(ThinLTO, EmitSummary,
+ ThinLTO
+ ? buildThinLTOPreLinkDefaultPipeline(Level)
+ : buildLTOPreLinkDefaultPipeline(Level)));
+ MPM.addPass(buildPerModuleDefaultPipeline(Level));
return MPM;
}
diff --git a/llvm/lib/Passes/PassRegistry.def b/llvm/lib/Passes/PassRegistry.def
index 7a2035cf07b3d..1879dea7de165 100644
--- a/llvm/lib/Passes/PassRegistry.def
+++ b/llvm/lib/Passes/PassRegistry.def
@@ -170,6 +170,13 @@ MODULE_PASS_WITH_PARAMS("ipsccp",
},
parseIPSCCPOptions,
"no-func-spec;func-spec")
+MODULE_PASS_WITH_PARAMS("embed-bitcode",
+ "EmbedBitcodePass",
+ [](EmbedBitcodeOptions Opts) {
+ return EmbedBitcodePass(Opts);
+ },
+ parseEmbedBitcodePassOptions,
+ "thinlto;emit-summary")
#undef MODULE_PASS_WITH_PARAMS
#ifndef CGSCC_ANALYSIS
diff --git a/llvm/lib/Transforms/IPO/CMakeLists.txt b/llvm/lib/Transforms/IPO/CMakeLists.txt
index e03aff0f65d7a..034f1587ae8df 100644
--- a/llvm/lib/Transforms/IPO/CMakeLists.txt
+++ b/llvm/lib/Transforms/IPO/CMakeLists.txt
@@ -11,6 +11,7 @@ add_llvm_component_library(LLVMipo
CrossDSOCFI.cpp
DeadArgumentElimination.cpp
ElimAvailExtern.cpp
+ EmbedBitcodePass.cpp
ExtractGV.cpp
ForceFunctionAttrs.cpp
FunctionAttrs.cpp
diff --git a/llvm/lib/Transforms/IPO/EmbedBitcodePass.cpp b/llvm/lib/Transforms/IPO/EmbedBitcodePass.cpp
new file mode 100644
index 0000000000000..fa56a5b564ae6
--- /dev/null
+++ b/llvm/lib/Transforms/IPO/EmbedBitcodePass.cpp
@@ -0,0 +1,52 @@
+//===- EmbedBitcodePass.cpp - Pass that embeds the bitcode into a global---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Transforms/IPO/EmbedBitcodePass.h"
+#include "llvm/Bitcode/BitcodeWriter.h"
+#include "llvm/Bitcode/BitcodeWriterPass.h"
+#include "llvm/IR/PassManager.h"
+#include "llvm/Pass.h"
+#include "llvm/Support/ErrorHandling.h"
+#include "llvm/Support/MemoryBufferRef.h"
+#include "llvm/Support/raw_ostream.h"
+#include "llvm/TargetParser/Triple.h"
+#include "llvm/Transforms/IPO/ThinLTOBitcodeWriter.h"
+#include "llvm/Transforms/Utils/Cloning.h"
+#include "llvm/Transforms/Utils/ModuleUtils.h"
+
+#include <memory>
+#include <string>
+
+using namespace llvm;
+
+PreservedAnalyses EmbedBitcodePass::run(Module &M, ModuleAnalysisManager &AM) {
+ if (M.getGlobalVariable("llvm.embedded.module", /*AllowInternal=*/true))
+ report_fatal_error("Can only embed the module once",
+ /*gen_crash_diag=*/false);
+
+ Triple T(M.getTargetTriple());
+ if (T.getObjectFormat() != Triple::ELF)
+ report_fatal_error(
+ "EmbedBitcode pass currently only supports ELF object format",
+ /*gen_crash_diag=*/false);
+
+ std::unique_ptr<Module> NewModule = CloneModule(M);
+ MPM.run(*NewModule, AM);
+
+ std::string Data;
+ raw_string_ostream OS(Data);
+ if (IsThinLTO)
+ ThinLTOBitcodeWriterPass(OS, /*ThinLinkOS=*/nullptr).run(*NewModule, AM);
+ else
+ BitcodeWriterPass(OS, /*ShouldPreserveUseListOrder=*/false, EmitLTOSummary)
+ .run(*NewModule, AM);
+
+ embedBufferInModule(M, MemoryBufferRef(Data, "ModuleData"), ".llvm.lto");
+
+ return PreservedAnalyses::all();
+}
diff --git a/llvm/test/CodeGen/X86/fat-lto-section.ll b/llvm/test/CodeGen/X86/fat-lto-section.ll
new file mode 100644
index 0000000000000..30c56229a0e2a
--- /dev/null
+++ b/llvm/test/CodeGen/X86/fat-lto-section.ll
@@ -0,0 +1,10 @@
+;; Ensure that the .llvm.lto section has SHT_EXCLUDE set.
+; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<thinlto;emit-summary>" -S \
+; RUN: | llc --mtriple x86_64-unknown-linux-gnu -filetype=obj \
+; RUN: | llvm-readelf - --sections \
+; RUN: | FileCheck %s --check-prefix=EXCLUDE
+
+; EXCLUDE: Name Type {{.*}} ES Flg Lk Inf Al
+; EXCLUDE: .llvm.lto PROGBITS {{.*}} 00 E 0 0 1
+
+ at a = global i32 1
diff --git a/llvm/test/Transforms/EmbedBitcode/embed-multiple.ll b/llvm/test/Transforms/EmbedBitcode/embed-multiple.ll
new file mode 100644
index 0000000000000..32a7fbe2fd8ce
--- /dev/null
+++ b/llvm/test/Transforms/EmbedBitcode/embed-multiple.ll
@@ -0,0 +1,6 @@
+; RUN: not opt --mtriple x86_64-unknown-linux-gnu < %s -passes=embed-bitcode -S 2>&1 | FileCheck %s
+
+ at a = global i32 1
+ at llvm.embedded.module = private constant [4 x i8] c"BC\C0\DE"
+
+; CHECK: LLVM ERROR: Can only embed the module once
diff --git a/llvm/test/Transforms/EmbedBitcode/embed-unsupported-object-format.ll b/llvm/test/Transforms/EmbedBitcode/embed-unsupported-object-format.ll
new file mode 100644
index 0000000000000..ca04f496c8a40
--- /dev/null
+++ b/llvm/test/Transforms/EmbedBitcode/embed-unsupported-object-format.ll
@@ -0,0 +1,5 @@
+; RUN: not opt --mtriple powerpc64-unknown-aix < %s -passes=embed-bitcode -S 2>&1 | FileCheck %s
+
+ at a = global i32 1
+
+; CHECK: LLVM ERROR: EmbedBitcode pass currently only supports ELF object format
diff --git a/llvm/test/Transforms/EmbedBitcode/embed.ll b/llvm/test/Transforms/EmbedBitcode/embed.ll
new file mode 100644
index 0000000000000..dffb5cf755477
--- /dev/null
+++ b/llvm/test/Transforms/EmbedBitcode/embed.ll
@@ -0,0 +1,18 @@
+; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode" -S | FileCheck %s
+; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<thinlto>" -S | FileCheck %s
+; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<emit-summary>" -S | FileCheck %s
+; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<thinlto;emit-summary>" -S | FileCheck %s
+
+ at a = global i32 1
+
+; CHECK: @a = global i32 1
+;; Make sure the module is in the correct section.
+; CHECK: @llvm.embedded.object = private constant {{.*}}, section ".llvm.lto", align 1
+
+;; Ensure that the metadata is in llvm.compiler.used.
+; CHECK: @llvm.compiler.used = appending global [1 x ptr] [ptr @llvm.embedded.object], section "llvm.metadata"
+
+;; Make sure the metadata correlates to the .llvm.lto section.
+; CHECK: !llvm.embedded.objects = !{!1}
+; CHECK: !0 = !{}
+; CHECK: !{ptr @llvm.embedded.object, !".llvm.lto"}
More information about the llvm-commits
mailing list