[llvm] 6085eb3 - Revert "Reland [llvm] Preliminary fat-lto-objects support"
Alex Brachet via llvm-commits
llvm-commits at lists.llvm.org
Fri Jun 23 18:16:10 PDT 2023
Author: Alex Brachet
Date: 2023-06-24T01:15:50Z
New Revision: 6085eb308491aac27f67fdc2350d17dd0be71e8d
URL: https://github.com/llvm/llvm-project/commit/6085eb308491aac27f67fdc2350d17dd0be71e8d
DIFF: https://github.com/llvm/llvm-project/commit/6085eb308491aac27f67fdc2350d17dd0be71e8d.diff
LOG: Revert "Reland [llvm] Preliminary fat-lto-objects support"
This reverts commit 44265dc3554ef40920b587eeb787a400663af6c7.
Added:
Modified:
llvm/docs/ReleaseNotes.rst
llvm/docs/UserGuides.rst
llvm/include/llvm/Passes/PassBuilder.h
llvm/lib/Object/ObjectFile.cpp
llvm/lib/Passes/PassBuilder.cpp
llvm/lib/Passes/PassBuilderPipelines.cpp
llvm/lib/Passes/PassRegistry.def
llvm/lib/Transforms/IPO/CMakeLists.txt
Removed:
llvm/docs/FatLTO.rst
llvm/include/llvm/Transforms/IPO/EmbedBitcodePass.h
llvm/lib/Transforms/IPO/EmbedBitcodePass.cpp
llvm/test/CodeGen/X86/fat-lto-section.ll
llvm/test/Transforms/EmbedBitcode/embed-multiple.ll
llvm/test/Transforms/EmbedBitcode/embed-unsupported-object-format.ll
llvm/test/Transforms/EmbedBitcode/embed.ll
################################################################################
diff --git a/llvm/docs/FatLTO.rst b/llvm/docs/FatLTO.rst
deleted file mode 100644
index 5773a30b7ae4a..0000000000000
--- a/llvm/docs/FatLTO.rst
+++ /dev/null
@@ -1,77 +0,0 @@
-===================
-FatLTO
-===================
-.. contents::
- :local:
- :depth: 2
-
-.. toctree::
- :maxdepth: 1
-
-Introduction
-============
-
-FatLTO objects are a special type of `fat object file
-<https://en.wikipedia.org/wiki/Fat_binary>`_ that contain LTO compatible IR in
-addition to generated object code, instead of containing object code for
-multiple target architectures. This allows users to defer the choice of whether
-to use LTO or not to link-time, and has been a feature available in other
-compilers, like `GCC
-<https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html>`_, for some time.
-
-Under FatLTO the compiler can emit standard object files which contain both the
-machine code in the ``.text`` section and LLVM bitcode in the ``.llvm.lto``
-section.
-
-Overview
-========
-
-Within LLVM, FatLTO is supported by choosing the ``FatLTODefaultPipeline``.
-This pipeline will:
-
-#) Clone the IR module.
-#) Run the pre-link (Thin)LTO pipeline using the cloned module.
-#) Embed the pre-link bitcode in a special ``.llvm.lto`` section.
-#) Optimize the unmodified copy of the module using the normal compilation pipeline.
-#) Emit the object file, including the new ``.llvm.lto`` section.
-
-.. NOTE
-
- At the time of writing, we conservatively run independent pipelines to
- generate the bitcode section and the object code, which happen to be
- identical to those used outside of FatLTO. This results in compiled
- artifacts that are identical to those produced by the default and (Thin)LTO
- pipelines. However, this is not a guarantee, and we reserve the right to
- change this at any time. Explicitly, users should not rely on the produced
- bitcode or object code to mach their non-LTO counterparts precisely. They
- will exhibit similar performance characteristics, but may not be bit-for-bit
- the same.
-
-Internally, the ``.llvm.lto`` section is created by running the
-``EmbedBitcodePass`` at the start of the ``PerModuleDefaultPipeline``. This
-pass is responsible for cloning and optimizing the module with the appropriate
-LTO pipeline and emitting the ``.llvm.lto`` section. Afterwards, the
-``PerModuleDefaultPipeline`` runs normally and the compiler can emit the fat
-object file.
-
-Limitations
-===========
-
-Linkers
--------
-
-Currently, using LTO with LLVM fat lto objects is supported by LLD and by the
-GNU linkers via :doc:`GoldPlugin`. This may change in the future, but
-extending support to other linkers isn't planned for now.
-
-.. NOTE
- For standard linking the fat object files should be usable by any
- linker capable of using ELF objects, since the ``.llvm.lto`` section is
- marked ``SHF_EXLUDE``.
-
-Supported File Formats
-----------------------
-
-The current implementation only supports ELF files. At time of writing, it is
-unclear if it will be useful to support other object file formats like ``COFF``
-or ``Mach-O``.
diff --git a/llvm/docs/ReleaseNotes.rst b/llvm/docs/ReleaseNotes.rst
index e086c235c929a..6efc991ff509e 100644
--- a/llvm/docs/ReleaseNotes.rst
+++ b/llvm/docs/ReleaseNotes.rst
@@ -80,12 +80,6 @@ Changes to LLVM infrastructure
* InstructionSimplify APIs now require instructions be inserted into a
parent function.
-* A new FatLTO pipeline was added to support generating object files that have
- both machine code and LTO compatible bitcode. See the :doc:`FatLTO`
- documentation and the original
- `RFC <https://discourse.llvm.org/t/rfc-ffat-lto-objects-support/63977>`_
- for more details.
-
Changes to building LLVM
------------------------
diff --git a/llvm/docs/UserGuides.rst b/llvm/docs/UserGuides.rst
index d7335911abe1c..517a23e959206 100644
--- a/llvm/docs/UserGuides.rst
+++ b/llvm/docs/UserGuides.rst
@@ -32,7 +32,6 @@ intermediate LLVM representation.
DebuggingJITedCode
DirectXUsage
Docker
- FatLTO
ExtendingLLVM
GoldPlugin
HowToBuildOnARM
diff --git a/llvm/include/llvm/Passes/PassBuilder.h b/llvm/include/llvm/Passes/PassBuilder.h
index f31c32102e694..585c335cf5728 100644
--- a/llvm/include/llvm/Passes/PassBuilder.h
+++ b/llvm/include/llvm/Passes/PassBuilder.h
@@ -234,18 +234,6 @@ class PassBuilder {
ModulePassManager buildPerModuleDefaultPipeline(OptimizationLevel Level,
bool LTOPreLink = false);
- /// Build a fat object default optimization pipeline.
- ///
- /// This builds a pipeline that runs the LTO/ThinLTO pre-link pipeline, and
- /// emits a section containing the pre-link bitcode along side the object code
- /// generated by running the PerModuleDefaultPipeline, used when compiling
- /// without LTO. It clones the module and runs the LTO/non-LTO pipelines
- /// separately to avoid any inconsistencies with an ad-hoc pipeline that tries
- /// to approximate the PerModuleDefaultPipeline from the pre-link LTO
- /// pipelines.
- ModulePassManager buildFatLTODefaultPipeline(OptimizationLevel Level,
- bool ThinLTO, bool EmitSummary);
-
/// Build a pre-link, ThinLTO-targeting default optimization pipeline to
/// a pass manager.
///
diff --git a/llvm/include/llvm/Transforms/IPO/EmbedBitcodePass.h b/llvm/include/llvm/Transforms/IPO/EmbedBitcodePass.h
deleted file mode 100644
index f323c61483fd3..0000000000000
--- a/llvm/include/llvm/Transforms/IPO/EmbedBitcodePass.h
+++ /dev/null
@@ -1,58 +0,0 @@
-//===-- EmbedBitcodePass.h - Embeds bitcode into global ---------*- C++ -*-===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-/// \file
-///
-/// This file provides a pass which clones the current module and runs the
-/// provided pass pipeline on the clone. The optimized module is stored into a
-/// global variable in the `.llvm.lto` section. Primarily, this pass is used
-/// to support the FatLTO pipeline, but could be used to generate a bitcode
-/// section for any arbitrary pass pipeline without changing the current module.
-///
-//===----------------------------------------------------------------------===//
-//
-#ifndef LLVM_TRANSFORMS_IPO_EMBEDBITCODEPASS_H
-#define LLVM_TRANSFORMS_IPO_EMBEDBITCODEPASS_H
-
-#include "llvm/IR/PassManager.h"
-
-namespace llvm {
-class Module;
-class ModulePass;
-class Pass;
-
-struct EmbedBitcodeOptions {
- EmbedBitcodeOptions() : EmbedBitcodeOptions(false, false) {}
- EmbedBitcodeOptions(bool IsThinLTO, bool EmitLTOSummary)
- : IsThinLTO(IsThinLTO), EmitLTOSummary(EmitLTOSummary) {}
- bool IsThinLTO;
- bool EmitLTOSummary;
-};
-
-/// Pass embeds a copy of the module optimized with the provided pass pipeline
-/// into a global variable.
-class EmbedBitcodePass : public PassInfoMixin<EmbedBitcodePass> {
- bool IsThinLTO;
- bool EmitLTOSummary;
- ModulePassManager MPM;
-
-public:
- EmbedBitcodePass(EmbedBitcodeOptions Opts)
- : EmbedBitcodePass(Opts.IsThinLTO, Opts.EmitLTOSummary,
- ModulePassManager()) {}
- EmbedBitcodePass(bool IsThinLTO, bool EmitLTOSummary, ModulePassManager &&MPM)
- : IsThinLTO(IsThinLTO), EmitLTOSummary(EmitLTOSummary),
- MPM(std::move(MPM)) {}
-
- PreservedAnalyses run(Module &M, ModuleAnalysisManager &);
-
- static bool isRequired() { return true; }
-};
-
-} // end namespace llvm.
-
-#endif
diff --git a/llvm/lib/Object/ObjectFile.cpp b/llvm/lib/Object/ObjectFile.cpp
index 0820187f32e14..c953a5692cbb4 100644
--- a/llvm/lib/Object/ObjectFile.cpp
+++ b/llvm/lib/Object/ObjectFile.cpp
@@ -79,7 +79,7 @@ uint32_t ObjectFile::getSymbolAlignment(DataRefImpl DRI) const { return 0; }
bool ObjectFile::isSectionBitcode(DataRefImpl Sec) const {
Expected<StringRef> NameOrErr = getSectionName(Sec);
if (NameOrErr)
- return *NameOrErr == ".llvmbc" || *NameOrErr == ".llvm.lto";
+ return *NameOrErr == ".llvmbc";
consumeError(NameOrErr.takeError());
return false;
}
diff --git a/llvm/lib/Passes/PassBuilder.cpp b/llvm/lib/Passes/PassBuilder.cpp
index 25a9e7d5cde83..65d83e6ca759e 100644
--- a/llvm/lib/Passes/PassBuilder.cpp
+++ b/llvm/lib/Passes/PassBuilder.cpp
@@ -102,7 +102,6 @@
#include "llvm/Transforms/IPO/CrossDSOCFI.h"
#include "llvm/Transforms/IPO/DeadArgumentElimination.h"
#include "llvm/Transforms/IPO/ElimAvailExtern.h"
-#include "llvm/Transforms/IPO/EmbedBitcodePass.h"
#include "llvm/Transforms/IPO/ForceFunctionAttrs.h"
#include "llvm/Transforms/IPO/FunctionAttrs.h"
#include "llvm/Transforms/IPO/FunctionImport.h"
@@ -743,26 +742,6 @@ Expected<HWAddressSanitizerOptions> parseHWASanPassOptions(StringRef Params) {
return Result;
}
-Expected<EmbedBitcodeOptions> parseEmbedBitcodePassOptions(StringRef Params) {
- EmbedBitcodeOptions Result;
- while (!Params.empty()) {
- StringRef ParamName;
- std::tie(ParamName, Params) = Params.split(';');
-
- if (ParamName == "thinlto") {
- Result.IsThinLTO = true;
- } else if (ParamName == "emit-summary") {
- Result.EmitLTOSummary = true;
- } else {
- return make_error<StringError>(
- formatv("invalid EmbedBitcode pass parameter '{0}' ", ParamName)
- .str(),
- inconvertibleErrorCode());
- }
- }
- return Result;
-}
-
Expected<MemorySanitizerOptions> parseMSanPassOptions(StringRef Params) {
MemorySanitizerOptions Result;
while (!Params.empty()) {
diff --git a/llvm/lib/Passes/PassBuilderPipelines.cpp b/llvm/lib/Passes/PassBuilderPipelines.cpp
index 58a68c0852631..1bc055d376bc3 100644
--- a/llvm/lib/Passes/PassBuilderPipelines.cpp
+++ b/llvm/lib/Passes/PassBuilderPipelines.cpp
@@ -46,7 +46,6 @@
#include "llvm/Transforms/IPO/CrossDSOCFI.h"
#include "llvm/Transforms/IPO/DeadArgumentElimination.h"
#include "llvm/Transforms/IPO/ElimAvailExtern.h"
-#include "llvm/Transforms/IPO/EmbedBitcodePass.h"
#include "llvm/Transforms/IPO/ForceFunctionAttrs.h"
#include "llvm/Transforms/IPO/FunctionAttrs.h"
#include "llvm/Transforms/IPO/GlobalDCE.h"
@@ -1465,18 +1464,7 @@ PassBuilder::buildPerModuleDefaultPipeline(OptimizationLevel Level,
if (LTOPreLink)
addRequiredLTOPreLinkPasses(MPM);
- return MPM;
-}
-ModulePassManager
-PassBuilder::buildFatLTODefaultPipeline(OptimizationLevel Level, bool ThinLTO,
- bool EmitSummary) {
- ModulePassManager MPM;
- MPM.addPass(EmbedBitcodePass(ThinLTO, EmitSummary,
- ThinLTO
- ? buildThinLTOPreLinkDefaultPipeline(Level)
- : buildLTOPreLinkDefaultPipeline(Level)));
- MPM.addPass(buildPerModuleDefaultPipeline(Level));
return MPM;
}
diff --git a/llvm/lib/Passes/PassRegistry.def b/llvm/lib/Passes/PassRegistry.def
index 956e1d94efacb..11091ace5f74a 100644
--- a/llvm/lib/Passes/PassRegistry.def
+++ b/llvm/lib/Passes/PassRegistry.def
@@ -176,13 +176,6 @@ MODULE_PASS_WITH_PARAMS("ipsccp",
},
parseIPSCCPOptions,
"no-func-spec;func-spec")
-MODULE_PASS_WITH_PARAMS("embed-bitcode",
- "EmbedBitcodePass",
- [](EmbedBitcodeOptions Opts) {
- return EmbedBitcodePass(Opts);
- },
- parseEmbedBitcodePassOptions,
- "thinlto;emit-summary")
#undef MODULE_PASS_WITH_PARAMS
#ifndef CGSCC_ANALYSIS
diff --git a/llvm/lib/Transforms/IPO/CMakeLists.txt b/llvm/lib/Transforms/IPO/CMakeLists.txt
index 034f1587ae8df..e03aff0f65d7a 100644
--- a/llvm/lib/Transforms/IPO/CMakeLists.txt
+++ b/llvm/lib/Transforms/IPO/CMakeLists.txt
@@ -11,7 +11,6 @@ add_llvm_component_library(LLVMipo
CrossDSOCFI.cpp
DeadArgumentElimination.cpp
ElimAvailExtern.cpp
- EmbedBitcodePass.cpp
ExtractGV.cpp
ForceFunctionAttrs.cpp
FunctionAttrs.cpp
diff --git a/llvm/lib/Transforms/IPO/EmbedBitcodePass.cpp b/llvm/lib/Transforms/IPO/EmbedBitcodePass.cpp
deleted file mode 100644
index fa56a5b564ae6..0000000000000
--- a/llvm/lib/Transforms/IPO/EmbedBitcodePass.cpp
+++ /dev/null
@@ -1,52 +0,0 @@
-//===- EmbedBitcodePass.cpp - Pass that embeds the bitcode into a global---===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-#include "llvm/Transforms/IPO/EmbedBitcodePass.h"
-#include "llvm/Bitcode/BitcodeWriter.h"
-#include "llvm/Bitcode/BitcodeWriterPass.h"
-#include "llvm/IR/PassManager.h"
-#include "llvm/Pass.h"
-#include "llvm/Support/ErrorHandling.h"
-#include "llvm/Support/MemoryBufferRef.h"
-#include "llvm/Support/raw_ostream.h"
-#include "llvm/TargetParser/Triple.h"
-#include "llvm/Transforms/IPO/ThinLTOBitcodeWriter.h"
-#include "llvm/Transforms/Utils/Cloning.h"
-#include "llvm/Transforms/Utils/ModuleUtils.h"
-
-#include <memory>
-#include <string>
-
-using namespace llvm;
-
-PreservedAnalyses EmbedBitcodePass::run(Module &M, ModuleAnalysisManager &AM) {
- if (M.getGlobalVariable("llvm.embedded.module", /*AllowInternal=*/true))
- report_fatal_error("Can only embed the module once",
- /*gen_crash_diag=*/false);
-
- Triple T(M.getTargetTriple());
- if (T.getObjectFormat() != Triple::ELF)
- report_fatal_error(
- "EmbedBitcode pass currently only supports ELF object format",
- /*gen_crash_diag=*/false);
-
- std::unique_ptr<Module> NewModule = CloneModule(M);
- MPM.run(*NewModule, AM);
-
- std::string Data;
- raw_string_ostream OS(Data);
- if (IsThinLTO)
- ThinLTOBitcodeWriterPass(OS, /*ThinLinkOS=*/nullptr).run(*NewModule, AM);
- else
- BitcodeWriterPass(OS, /*ShouldPreserveUseListOrder=*/false, EmitLTOSummary)
- .run(*NewModule, AM);
-
- embedBufferInModule(M, MemoryBufferRef(Data, "ModuleData"), ".llvm.lto");
-
- return PreservedAnalyses::all();
-}
diff --git a/llvm/test/CodeGen/X86/fat-lto-section.ll b/llvm/test/CodeGen/X86/fat-lto-section.ll
deleted file mode 100644
index 30c56229a0e2a..0000000000000
--- a/llvm/test/CodeGen/X86/fat-lto-section.ll
+++ /dev/null
@@ -1,10 +0,0 @@
-;; Ensure that the .llvm.lto section has SHT_EXCLUDE set.
-; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<thinlto;emit-summary>" -S \
-; RUN: | llc --mtriple x86_64-unknown-linux-gnu -filetype=obj \
-; RUN: | llvm-readelf - --sections \
-; RUN: | FileCheck %s --check-prefix=EXCLUDE
-
-; EXCLUDE: Name Type {{.*}} ES Flg Lk Inf Al
-; EXCLUDE: .llvm.lto PROGBITS {{.*}} 00 E 0 0 1
-
- at a = global i32 1
diff --git a/llvm/test/Transforms/EmbedBitcode/embed-multiple.ll b/llvm/test/Transforms/EmbedBitcode/embed-multiple.ll
deleted file mode 100644
index 32a7fbe2fd8ce..0000000000000
--- a/llvm/test/Transforms/EmbedBitcode/embed-multiple.ll
+++ /dev/null
@@ -1,6 +0,0 @@
-; RUN: not opt --mtriple x86_64-unknown-linux-gnu < %s -passes=embed-bitcode -S 2>&1 | FileCheck %s
-
- at a = global i32 1
- at llvm.embedded.module = private constant [4 x i8] c"BC\C0\DE"
-
-; CHECK: LLVM ERROR: Can only embed the module once
diff --git a/llvm/test/Transforms/EmbedBitcode/embed-unsupported-object-format.ll b/llvm/test/Transforms/EmbedBitcode/embed-unsupported-object-format.ll
deleted file mode 100644
index ca04f496c8a40..0000000000000
--- a/llvm/test/Transforms/EmbedBitcode/embed-unsupported-object-format.ll
+++ /dev/null
@@ -1,5 +0,0 @@
-; RUN: not opt --mtriple powerpc64-unknown-aix < %s -passes=embed-bitcode -S 2>&1 | FileCheck %s
-
- at a = global i32 1
-
-; CHECK: LLVM ERROR: EmbedBitcode pass currently only supports ELF object format
diff --git a/llvm/test/Transforms/EmbedBitcode/embed.ll b/llvm/test/Transforms/EmbedBitcode/embed.ll
deleted file mode 100644
index dffb5cf755477..0000000000000
--- a/llvm/test/Transforms/EmbedBitcode/embed.ll
+++ /dev/null
@@ -1,18 +0,0 @@
-; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode" -S | FileCheck %s
-; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<thinlto>" -S | FileCheck %s
-; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<emit-summary>" -S | FileCheck %s
-; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<thinlto;emit-summary>" -S | FileCheck %s
-
- at a = global i32 1
-
-; CHECK: @a = global i32 1
-;; Make sure the module is in the correct section.
-; CHECK: @llvm.embedded.object = private constant {{.*}}, section ".llvm.lto", align 1
-
-;; Ensure that the metadata is in llvm.compiler.used.
-; CHECK: @llvm.compiler.used = appending global [1 x ptr] [ptr @llvm.embedded.object], section "llvm.metadata"
-
-;; Make sure the metadata correlates to the .llvm.lto section.
-; CHECK: !llvm.embedded.objects = !{!1}
-; CHECK: !0 = !{}
-; CHECK: !{ptr @llvm.embedded.object, !".llvm.lto"}
More information about the llvm-commits
mailing list