[clang] 610fc5c - [clang] Preliminary fat-lto-object support
Paul Kirth via cfe-commits
cfe-commits at lists.llvm.org
Mon Jul 17 09:26:30 PDT 2023
Author: Paul Kirth
Date: 2023-07-17T16:26:21Z
New Revision: 610fc5cbcc8b68879c562f6458608afe2473ab7f
URL: https://github.com/llvm/llvm-project/commit/610fc5cbcc8b68879c562f6458608afe2473ab7f
DIFF: https://github.com/llvm/llvm-project/commit/610fc5cbcc8b68879c562f6458608afe2473ab7f.diff
LOG: [clang] Preliminary fat-lto-object support
Fat LTO objects contain both LTO compatible IR, as well as generated
object code. This allows users to defer the choice of whether to use LTO
or not to link-time. This is a feature available in GCC for some time,
and makes the existing -ffat-lto-objects flag functional in the same
way as GCC's.
This patch adds support for that flag in the driver, as well as setting the
necessary codegen options for the backend. Largely, this means we select
the newly added pass pipeline for generating fat objects.
Users are expected to pass -ffat-lto-objects to clang in addition to one
of the -flto variants. Without the -flto flag, -ffat-lto-objects has no
effect.
// Compile and link. Use the object code from the fat object w/o LTO.
clang -fno-lto -ffat-lto-objects -fuse-ld=lld foo.c
// Compile and link. Select full LTO at link time.
clang -flto -ffat-lto-objects -fuse-ld=lld foo.c
// Compile and link. Select ThinLTO at link time.
clang -flto=thin -ffat-lto-objects -fuse-ld=lld foo.c
// Compile and link. Use ThinLTO with the UnifiedLTO pipeline.
clang -flto=thin -ffat-lto-objects -funified-lto -fuse-ld=lld foo.c
// Compile and link. Use full LTO with the UnifiedLTO pipeline.
clang -flto -ffat-lto-objects -funified-lto -fuse-ld=lld foo.c
// Link separately, using ThinLTO.
clang -c -flto=thin -ffat-lto-objects foo.c
clang -flto=thin -fuse-ld=lld foo.o -ffat-lto-objects # pass --lto=thin --fat-lto-objects to ld.lld
// Link separately, using full LTO.
clang -c -flto -ffat-lto-objects foo.c
clang -flto -fuse-ld=lld foo.o # pass --lto=full --fat-lto-objects to ld.lld
Original RFC: https://discourse.llvm.org/t/rfc-ffat-lto-objects-support/63977
Depends on D146776
Reviewed By: tejohnson, MaskRay
Differential Revision: https://reviews.llvm.org/D146777
Added:
clang/test/CodeGen/fat-lto-objects.c
clang/test/Driver/fat-lto-objects.c
Modified:
clang/docs/ReleaseNotes.rst
clang/include/clang/Basic/CodeGenOptions.def
clang/include/clang/Driver/Options.td
clang/lib/CodeGen/BackendUtil.cpp
clang/lib/Driver/Driver.cpp
clang/lib/Driver/ToolChains/Clang.cpp
clang/lib/Driver/ToolChains/CommonArgs.cpp
clang/test/Driver/clang_f_opts.c
Removed:
################################################################################
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index d334ae48cfc039..2817397a67537e 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -267,6 +267,10 @@ Modified Compiler Flags
directory (``/tmp`` on \*NIX systems, if none of the environment variables
TMPDIR, TMP, and TEMP are specified).
+- ``-ffat-lto-objects`` can now be used to emit object files with both object
+ code and LLVM bitcode. Previously this flag was ignored for GCC compatibility.
+ (`See patch <https://reviews.llvm.org/D146777>`_).
+
Removed Compiler Flags
-------------------------
- The deprecated flag `-fmodules-ts` is removed. Please use ``-std=c++20``
diff --git a/clang/include/clang/Basic/CodeGenOptions.def b/clang/include/clang/Basic/CodeGenOptions.def
index f9903b18603b58..6da2f5b039b638 100644
--- a/clang/include/clang/Basic/CodeGenOptions.def
+++ b/clang/include/clang/Basic/CodeGenOptions.def
@@ -164,6 +164,7 @@ CODEGENOPT(PrepareForThinLTO , 1, 0) ///< Set when -flto=thin is enabled on the
///< compile step.
CODEGENOPT(LTOUnit, 1, 0) ///< Emit IR to support LTO unit features (CFI, whole
///< program vtable opt).
+CODEGENOPT(FatLTO, 1, 0) ///< Set when -ffat-lto-objects is enabled.
CODEGENOPT(EnableSplitLTOUnit, 1, 0) ///< Enable LTO unit splitting to support
/// CFI and traditional whole program
/// devirtualization that require whole
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index d4e3e168b4cd01..c45eb694a1184c 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -2370,6 +2370,11 @@ def fthin_link_bitcode_EQ : Joined<["-"], "fthin-link-bitcode=">,
Flags<[CoreOption, CC1Option]>, Group<f_Group>,
HelpText<"Write minimized bitcode to <file> for the ThinLTO thin link only">,
MarshallingInfoString<CodeGenOpts<"ThinLinkBitcodeFile">>;
+defm fat_lto_objects : BoolFOption<"fat-lto-objects",
+ CodeGenOpts<"FatLTO">, DefaultFalse,
+ PosFlag<SetTrue, [CC1Option], "Enable">,
+ NegFlag<SetFalse, [CC1Option], "Disable">,
+ BothFlags<[CC1Option], " fat LTO object support">>;
def fmacro_backtrace_limit_EQ : Joined<["-"], "fmacro-backtrace-limit=">,
Group<f_Group>, Flags<[NoXarchOption, CC1Option, CoreOption]>,
HelpText<"Set the maximum number of entries to print in a macro expansion backtrace (0 = no limit)">,
@@ -5140,7 +5145,6 @@ defm caller_saves : BooleanFFlag<"caller-saves">, Group<clang_ignored_gcc_optimi
defm reorder_blocks : BooleanFFlag<"reorder-blocks">, Group<clang_ignored_gcc_optimization_f_Group>;
defm branch_count_reg : BooleanFFlag<"branch-count-reg">, Group<clang_ignored_gcc_optimization_f_Group>;
defm default_inline : BooleanFFlag<"default-inline">, Group<clang_ignored_gcc_optimization_f_Group>;
-defm fat_lto_objects : BooleanFFlag<"fat-lto-objects">, Group<clang_ignored_gcc_optimization_f_Group>;
defm float_store : BooleanFFlag<"float-store">, Group<clang_ignored_gcc_optimization_f_Group>;
defm friend_injection : BooleanFFlag<"friend-injection">, Group<clang_ignored_f_Group>;
defm function_attribute_list : BooleanFFlag<"function-attribute-list">, Group<clang_ignored_f_Group>;
diff --git a/clang/lib/CodeGen/BackendUtil.cpp b/clang/lib/CodeGen/BackendUtil.cpp
index 483f3e787a7805..cda03d69522d74 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -55,6 +55,7 @@
#include "llvm/Target/TargetOptions.h"
#include "llvm/TargetParser/SubtargetFeature.h"
#include "llvm/TargetParser/Triple.h"
+#include "llvm/Transforms/IPO/EmbedBitcodePass.h"
#include "llvm/Transforms/IPO/LowerTypeTests.h"
#include "llvm/Transforms/IPO/ThinLTOBitcodeWriter.h"
#include "llvm/Transforms/InstCombine/InstCombine.h"
@@ -1015,7 +1016,12 @@ void EmitAssemblyHelper::RunOptimizationPipeline(
});
}
- if (IsThinLTO || (IsLTO && CodeGenOpts.UnifiedLTO)) {
+ bool IsThinOrUnifiedLTO = IsThinLTO || (IsLTO && CodeGenOpts.UnifiedLTO);
+ if (CodeGenOpts.FatLTO) {
+ MPM = PB.buildFatLTODefaultPipeline(Level, IsThinOrUnifiedLTO,
+ IsThinOrUnifiedLTO ||
+ shouldEmitRegularLTOSummary());
+ } else if (IsThinOrUnifiedLTO) {
MPM = PB.buildThinLTOPreLinkDefaultPipeline(Level);
} else if (IsLTO) {
MPM = PB.buildLTOPreLinkDefaultPipeline(Level);
@@ -1071,6 +1077,21 @@ void EmitAssemblyHelper::RunOptimizationPipeline(
EmitLTOSummary));
}
}
+ if (CodeGenOpts.FatLTO) {
+ // Set module flags, like EnableSplitLTOUnit and UnifiedLTO, since FatLTO
+ // uses a
diff erent action than Backend_EmitBC or Backend_EmitLL.
+ bool IsThinOrUnifiedLTO =
+ CodeGenOpts.PrepareForThinLTO ||
+ (CodeGenOpts.PrepareForLTO && CodeGenOpts.UnifiedLTO);
+ if (!TheModule->getModuleFlag("ThinLTO"))
+ TheModule->addModuleFlag(Module::Error, "ThinLTO",
+ uint32_t(IsThinOrUnifiedLTO));
+ if (!TheModule->getModuleFlag("EnableSplitLTOUnit"))
+ TheModule->addModuleFlag(Module::Error, "EnableSplitLTOUnit",
+ uint32_t(CodeGenOpts.EnableSplitLTOUnit));
+ if (CodeGenOpts.UnifiedLTO && !TheModule->getModuleFlag("UnifiedLTO"))
+ TheModule->addModuleFlag(Module::Error, "UnifiedLTO", uint32_t(1));
+ }
// Now that we have all of the passes ready, run them.
{
diff --git a/clang/lib/Driver/Driver.cpp b/clang/lib/Driver/Driver.cpp
index be8632b0954983..b4a070587901fb 100644
--- a/clang/lib/Driver/Driver.cpp
+++ b/clang/lib/Driver/Driver.cpp
@@ -4732,8 +4732,13 @@ Action *Driver::ConstructPhaseAction(
}
case phases::Backend: {
if (isUsingLTO() && TargetDeviceOffloadKind == Action::OFK_None) {
- types::ID Output =
- Args.hasArg(options::OPT_S) ? types::TY_LTO_IR : types::TY_LTO_BC;
+ types::ID Output;
+ if (Args.hasArg(options::OPT_S))
+ Output = types::TY_LTO_IR;
+ else if (Args.hasArg(options::OPT_ffat_lto_objects))
+ Output = types::TY_PP_Asm;
+ else
+ Output = types::TY_LTO_BC;
return C.MakeAction<BackendJobAction>(Input, Output);
}
if (isUsingLTO(/* IsOffload */ true) &&
diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index ecd739e5f770bb..1b791a7ab0b0a2 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -7354,6 +7354,22 @@ void Clang::ConstructJob(Compilation &C, const JobAction &JA,
if (SplitLTOUnit)
CmdArgs.push_back("-fsplit-lto-unit");
+ if (Arg *A = Args.getLastArg(options::OPT_ffat_lto_objects,
+ options::OPT_fno_fat_lto_objects)) {
+ if (IsUsingLTO && A->getOption().matches(options::OPT_ffat_lto_objects)) {
+ assert(LTOMode == LTOK_Full || LTOMode == LTOK_Thin);
+ if (!Triple.isOSBinFormatELF()) {
+ D.Diag(diag::err_drv_unsupported_opt_for_target)
+ << A->getAsString(Args) << TC.getTripleString();
+ }
+ CmdArgs.push_back(Args.MakeArgString(
+ Twine("-flto=") + (LTOMode == LTOK_Thin ? "thin" : "full")));
+ CmdArgs.push_back("-flto-unit");
+ CmdArgs.push_back("-ffat-lto-objects");
+ A->render(Args, CmdArgs);
+ }
+ }
+
if (Arg *A = Args.getLastArg(options::OPT_fglobal_isel,
options::OPT_fno_global_isel)) {
CmdArgs.push_back("-mllvm");
diff --git a/clang/lib/Driver/ToolChains/CommonArgs.cpp b/clang/lib/Driver/ToolChains/CommonArgs.cpp
index 496d4f5e82a663..cc958d7376b626 100644
--- a/clang/lib/Driver/ToolChains/CommonArgs.cpp
+++ b/clang/lib/Driver/ToolChains/CommonArgs.cpp
@@ -617,6 +617,11 @@ void tools::addLTOOptions(const ToolChain &ToolChain, const ArgList &Args,
PluginName + Suffix,
Plugin);
CmdArgs.push_back(Args.MakeArgString(Twine(PluginPrefix) + Plugin));
+ } else {
+ // Tell LLD to find and use .llvm.lto section in regular relocatable object
+ // files
+ if (Args.hasArg(options::OPT_ffat_lto_objects))
+ CmdArgs.push_back("--fat-lto-objects");
}
const char *PluginOptPrefix = IsOSAIX ? "-bplugin_opt:" : "-plugin-opt=";
diff --git a/clang/test/CodeGen/fat-lto-objects.c b/clang/test/CodeGen/fat-lto-objects.c
new file mode 100644
index 00000000000000..2c3a4ef9c61552
--- /dev/null
+++ b/clang/test/CodeGen/fat-lto-objects.c
@@ -0,0 +1,57 @@
+// REQUIRES: x86-registered-target
+
+// RUN: %clang -cc1 -triple x86_64-unknown-linux-gnu -flto=full -ffat-lto-objects -fsplit-lto-unit -emit-llvm < %s | FileCheck %s --check-prefixes=FULL,SPLIT
+// RUN: %clang -cc1 -triple x86_64-unknown-linux-gnu -flto=full -ffat-lto-objects -emit-llvm < %s | FileCheck %s --check-prefixes=FULL,SPLIT
+
+// RUN: %clang -cc1 -triple x86_64-unknown-linux-gnu -flto=thin -fsplit-lto-unit -ffat-lto-objects -emit-llvm < %s | FileCheck %s --check-prefixes=THIN,SPLIT
+// RUN: %clang -cc1 -triple x86_64-unknown-linux-gnu -flto=thin -ffat-lto-objects -emit-llvm < %s | FileCheck %s --check-prefixes=THIN,NOSPLIT
+
+// RUN: %clang -cc1 -triple x86_64-unknown-linux-gnu -flto=full -ffat-lto-objects -fsplit-lto-unit -emit-obj < %s -o %t.full.split.o
+// RUN: llvm-readelf -S %t.full.split.o | FileCheck %s --check-prefixes=ELF
+// RUN: llvm-objcopy --dump-section=.llvm.lto=%t.full.split.bc %t.full.split.o
+// RUN: llvm-dis %t.full.split.bc -o - | FileCheck %s --check-prefixes=FULL,SPLIT,NOUNIFIED
+
+// RUN: %clang -cc1 -triple x86_64-unknown-linux-gnu -flto=full -ffat-lto-objects -emit-obj < %s -o %t.full.nosplit.o
+// RUN: llvm-readelf -S %t.full.nosplit.o | FileCheck %s --check-prefixes=ELF
+// RUN: llvm-objcopy --dump-section=.llvm.lto=%t.full.nosplit.bc %t.full.nosplit.o
+// RUN: llvm-dis %t.full.nosplit.bc -o - | FileCheck %s --check-prefixes=FULL,NOSPLIT,NOUNIFIED
+
+// RUN: %clang -cc1 -triple x86_64-unknown-linux-gnu -flto=thin -fsplit-lto-unit -ffat-lto-objects -emit-obj < %s -o %t.thin.split.o
+// RUN: llvm-readelf -S %t.thin.split.o | FileCheck %s --check-prefixes=ELF
+// RUN: llvm-objcopy --dump-section=.llvm.lto=%t.thin.split.bc %t.thin.split.o
+// RUN: llvm-dis %t.thin.split.bc -o - | FileCheck %s --check-prefixes=THIN,SPLIT,NOUNIFIED
+
+// RUN: %clang -cc1 -triple x86_64-unknown-linux-gnu -flto=thin -ffat-lto-objects -emit-obj < %s -o %t.thin.nosplit.o
+// RUN: llvm-readelf -S %t.thin.nosplit.o | FileCheck %s --check-prefixes=ELF
+// RUN: llvm-objcopy --dump-section=.llvm.lto=%t.thin.nosplit.bc %t.thin.nosplit.o
+// RUN: llvm-dis %t.thin.nosplit.bc -o - | FileCheck %s --check-prefixes=THIN,NOSPLIT,NOUNIFIED
+
+// RUN: %clang -cc1 -triple x86_64-unknown-linux-gnu -flto=thin -funified-lto -ffat-lto-objects -emit-obj < %s -o %t.unified.o
+// RUN: llvm-readelf -S %t.unified.o | FileCheck %s --check-prefixes=ELF
+// RUN: llvm-objcopy --dump-section=.llvm.lto=%t.unified.bc %t.unified.o
+// RUN: llvm-dis %t.unified.bc -o - | FileCheck %s --check-prefixes=THIN,NOSPLIT,UNIFIED
+
+// RUN: %clang -cc1 -triple x86_64-unknown-linux-gnu -flto=full -ffat-lto-objects -fsplit-lto-unit -S < %s -o - \
+// RUN: | FileCheck %s --check-prefixes=ASM
+
+/// Check that the ThinLTO metadata is only set false for full LTO.
+// FULL: ![[#]] = !{i32 1, !"ThinLTO", i32 0}
+// THIN-NOT: ![[#]] = !{i32 1, !"ThinLTO", i32 0}
+
+/// Be sure we enable split LTO units correctly under -ffat-lto-objects.
+// SPLIT: ![[#]] = !{i32 1, !"EnableSplitLTOUnit", i32 1}
+// NOSPLIT: ![[#]] = !{i32 1, !"EnableSplitLTOUnit", i32 0}
+
+// UNIFIED: ![[#]] = !{i32 1, !"UnifiedLTO", i32 1}
+// NOUNIFIED-NOT: ![[#]] = !{i32 1, !"UnifiedLTO", i32 1}
+
+// ELF: .llvm.lto
+
+// ASM: .section .llvm.lto,"e", at progbits
+// ASM-NEXT: .Lllvm.embedded.object:
+// ASM-NEXT: .asciz "BC
+// ASM-NEXT: .size .Lllvm.embedded.object
+
+int test(void) {
+ return 0xabcd;
+}
diff --git a/clang/test/Driver/clang_f_opts.c b/clang/test/Driver/clang_f_opts.c
index 65ba66fa325f55..e8bc46e23b78a3 100644
--- a/clang/test/Driver/clang_f_opts.c
+++ b/clang/test/Driver/clang_f_opts.c
@@ -424,7 +424,6 @@
// CHECK-WARNING-DAG: optimization flag '-fwhole-program' is not supported
// CHECK-WARNING-DAG: optimization flag '-fcaller-saves' is not supported
// CHECK-WARNING-DAG: optimization flag '-freorder-blocks' is not supported
-// CHECK-WARNING-DAG: optimization flag '-ffat-lto-objects' is not supported
// CHECK-WARNING-DAG: optimization flag '-fmerge-constants' is not supported
// CHECK-WARNING-DAG: optimization flag '-finline-small-functions' is not supported
// CHECK-WARNING-DAG: optimization flag '-ftree-dce' is not supported
diff --git a/clang/test/Driver/fat-lto-objects.c b/clang/test/Driver/fat-lto-objects.c
new file mode 100644
index 00000000000000..887c33fa76d71f
--- /dev/null
+++ b/clang/test/Driver/fat-lto-objects.c
@@ -0,0 +1,34 @@
+// RUN: %clang --target=x86_64-unknown-linux-gnu -flto -ffat-lto-objects -### %s -c 2>&1 | FileCheck %s -check-prefix=CHECK-CC
+// CHECK-CC: -cc1
+// CHECK-CC-SAME: -emit-obj
+// CHECK-CC-SAME: -ffat-lto-objects
+
+/// Without -flto -S will just emit normal ASM, so we don't expect -emit-{llvm,obj} or -ffat-lto-objects to be passed to cc1.
+// RUN: %clang --target=x86_64-unknown-linux-gnu -ffat-lto-objects -### %s -S 2>&1 | FileCheck %s -check-prefix=CHECK-CC-S
+// CHECK-CC-S: -cc1
+// CHECK-CC-S: -S
+// CHECK-CC-S-NOT: -emit-obj
+// CHECK-CC-S-NOT: -emit-llvm
+// CHECK-CC-S-NOT: -ffat-lto-objects
+
+/// When LTO is enabled, we expect LLVM IR output and -ffat-lto-objects to be passed to cc1.
+// RUN: %clang --target=x86_64-unknown-linux-gnu -flto -ffat-lto-objects -### %s -S 2>&1 | FileCheck %s -check-prefix=CHECK-CC-S-LTO
+// RUN: %clang --target=x86_64-unknown-linux-gnu -flto -ffat-lto-objects -### %s -S -emit-llvm 2>&1 | FileCheck %s -check-prefix=CHECK-CC-S-LTO
+// CHECK-CC-S-LTO: -cc1
+// CHECK-CC-S-LTO-SAME: -emit-llvm
+// CHECK-CC-S-LTO-SAME: -ffat-lto-objects
+
+/// Make sure we don't have a warning for -ffat-lto-objects being unused
+// RUN: %clang --target=x86_64-unknown-linux-gnu -ffat-lto-objects -fdriver-only -Werror -v %s -c 2>&1 | FileCheck %s -check-prefix=CHECK-CC-NOLTO
+// CHECK-CC-NOLTO: -cc1
+// CHECK-CC-NOLTO-SAME: -emit-obj
+// CHECK-CC-NOLTO-NOT: -ffat-lto-objects
+
+/// We need to pass an additional flag (--fat-lto-objects) to lld when linking w/ -flto -ffat-lto-objects
+/// But it should not be there when LTO is disabled w/ -fno-lto
+// RUN: %clang --target=x86_64-unknown-linux-gnu --sysroot=%S/Inputs/basic_cross_linux_tree %s \
+// RUN: -fuse-ld=lld -flto -ffat-lto-objects -### 2>&1 | FileCheck --check-prefix=LTO %s
+// RUN: %clang --target=x86_64-unknown-linux-gnu --sysroot=%S/Inputs/basic_cross_linux_tree %s \
+// RUN: -fuse-ld=lld -fno-lto -ffat-lto-objects -### 2>&1 | FileCheck --check-prefix=NOLTO %s
+// LTO: "--fat-lto-objects"
+// NOLTO-NOT: "--fat-lto-objects"
More information about the cfe-commits
mailing list