[lld] r360984 - [ELF] Implement Dependent Libraries Feature
Ben Dunbobbin via llvm-commits
llvm-commits at lists.llvm.org
Thu May 16 20:44:16 PDT 2019
Author: bd1976llvm
Date: Thu May 16 20:44:15 2019
New Revision: 360984
URL: http://llvm.org/viewvc/llvm-project?rev=360984&view=rev
Log:
[ELF] Implement Dependent Libraries Feature
This patch implements a limited form of autolinking primarily designed to allow
either the --dependent-library compiler option, or "comment lib" pragmas (
https://docs.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp?view=vs-2017) in
C/C++ e.g. #pragma comment(lib, "foo"), to cause an ELF linker to automatically
add the specified library to the link when processing the input file generated
by the compiler.
Currently this extension is unique to LLVM and LLD. However, care has been taken
to design this feature so that it could be supported by other ELF linkers.
The design goals were to provide:
- A simple linking model for developers to reason about.
- The ability to to override autolinking from the linker command line.
- Source code compatibility, where possible, with "comment lib" pragmas in other
environments (MSVC in particular).
Dependent library support is implemented differently for ELF platforms than on
the other platforms. Primarily this difference is that on ELF we pass the
dependent library specifiers directly to the linker without manipulating them.
This is in contrast to other platforms where they are mapped to a specific
linker option by the compiler. This difference is a result of the greater
variety of ELF linkers and the fact that ELF linkers tend to handle libraries in
a more complicated fashion than on other platforms. This forces us to defer
handling the specifiers to the linker.
In order to achieve a level of source code compatibility with other platforms
we have restricted this feature to work with libraries that meet the following
"reasonable" requirements:
1. There are no competing defined symbols in a given set of libraries, or
if they exist, the program owner doesn't care which is linked to their
program.
2. There may be circular dependencies between libraries.
The binary representation is a mergeable string section (SHF_MERGE,
SHF_STRINGS), called .deplibs, with custom type SHT_LLVM_DEPENDENT_LIBRARIES
(0x6fff4c04). The compiler forms this section by concatenating the arguments of
the "comment lib" pragmas and --dependent-library options in the order they are
encountered. Partial (-r, -Ur) links are handled by concatenating .deplibs
sections with the normal mergeable string section rules. As an example, #pragma
comment(lib, "foo") would result in:
.section ".deplibs","MS", at llvm_dependent_libraries,1
.asciz "foo"
For LTO, equivalent information to the contents of a the .deplibs section can be
retrieved by the LLD for bitcode input files.
LLD processes the dependent library specifiers in the following way:
1. Dependent libraries which are found from the specifiers in .deplibs sections
of relocatable object files are added when the linker decides to include that
file (which could itself be in a library) in the link. Dependent libraries
behave as if they were appended to the command line after all other options. As
a consequence the set of dependent libraries are searched last to resolve
symbols.
2. It is an error if a file cannot be found for a given specifier.
3. Any command line options in effect at the end of the command line parsing apply
to the dependent libraries, e.g. --whole-archive.
4. The linker tries to add a library or relocatable object file from each of the
strings in a .deplibs section by; first, handling the string as if it was
specified on the command line; second, by looking for the string in each of the
library search paths in turn; third, by looking for a lib<string>.a or
lib<string>.so (depending on the current mode of the linker) in each of the
library search paths.
5. A new command line option --no-dependent-libraries tells LLD to ignore the
dependent libraries.
Rationale for the above points:
1. Adding the dependent libraries last makes the process simple to understand
from a developers perspective. All linkers are able to implement this scheme.
2. Error-ing for libraries that are not found seems like better behavior than
failing the link during symbol resolution.
3. It seems useful for the user to be able to apply command line options which
will affect all of the dependent libraries. There is a potential problem of
surprise for developers, who might not realize that these options would apply
to these "invisible" input files; however, despite the potential for surprise,
this is easy for developers to reason about and gives developers the control
that they may require.
4. This algorithm takes into account all of the different ways that ELF linkers
find input files. The different search methods are tried by the linker in most
obvious to least obvious order.
5. I considered adding finer grained control over which dependent libraries were
ignored (e.g. MSVC has /nodefaultlib:<library>); however, I concluded that this
is not necessary: if finer control is required developers can fall back to using
the command line directly.
RFC thread: http://lists.llvm.org/pipermail/llvm-dev/2019-March/131004.html.
Differential Revision: https://reviews.llvm.org/D60274
Added:
lld/trunk/test/ELF/Inputs/deplibs-lib_bar.s
lld/trunk/test/ELF/Inputs/deplibs-lib_foo.s
lld/trunk/test/ELF/deplibs-colon-prefix.s
lld/trunk/test/ELF/deplibs-corrupt.s
lld/trunk/test/ELF/deplibs.s
lld/trunk/test/ELF/lto/deplibs.s
Modified:
lld/trunk/ELF/Config.h
lld/trunk/ELF/Driver.cpp
lld/trunk/ELF/Driver.h
lld/trunk/ELF/DriverUtils.cpp
lld/trunk/ELF/InputFiles.cpp
lld/trunk/ELF/Options.td
Modified: lld/trunk/ELF/Config.h
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/Config.h?rev=360984&r1=360983&r2=360984&view=diff
==============================================================================
--- lld/trunk/ELF/Config.h (original)
+++ lld/trunk/ELF/Config.h Thu May 16 20:44:15 2019
@@ -137,6 +137,7 @@ struct Configuration {
bool Cref;
bool DefineCommon;
bool Demangle = true;
+ bool DependentLibraries;
bool DisableVerify;
bool EhFrameHdr;
bool EmitLLVM;
Modified: lld/trunk/ELF/Driver.cpp
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/Driver.cpp?rev=360984&r1=360983&r2=360984&view=diff
==============================================================================
--- lld/trunk/ELF/Driver.cpp (original)
+++ lld/trunk/ELF/Driver.cpp Thu May 16 20:44:15 2019
@@ -790,6 +790,7 @@ static void readConfigs(opt::InputArgLis
Config->DefineCommon = Args.hasFlag(OPT_define_common, OPT_no_define_common,
!Args.hasArg(OPT_relocatable));
Config->Demangle = Args.hasFlag(OPT_demangle, OPT_no_demangle, true);
+ Config->DependentLibraries = Args.hasFlag(OPT_dependent_libraries, OPT_no_dependent_libraries, true);
Config->DisableVerify = Args.hasArg(OPT_disable_verify);
Config->Discard = getDiscard(Args);
Config->DwoDir = Args.getLastArgValue(OPT_plugin_opt_dwo_dir_eq);
@@ -1548,9 +1549,11 @@ template <class ELFT> void LinkerDriver:
Symtab->trace(Arg->getValue());
// Add all files to the symbol table. This will add almost all
- // symbols that we need to the symbol table.
- for (InputFile *F : Files)
- parseFile(F);
+ // symbols that we need to the symbol table. This process might
+ // add files to the link, via autolinking, these files are always
+ // appended to the Files vector.
+ for (size_t I = 0; I < Files.size(); ++I)
+ parseFile(Files[I]);
// Now that we have every file, we can decide if we will need a
// dynamic symbol table.
Modified: lld/trunk/ELF/Driver.h
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/Driver.h?rev=360984&r1=360983&r2=360984&view=diff
==============================================================================
--- lld/trunk/ELF/Driver.h (original)
+++ lld/trunk/ELF/Driver.h Thu May 16 20:44:15 2019
@@ -63,6 +63,7 @@ std::string createResponseFile(const llv
llvm::Optional<std::string> findFromSearchPaths(StringRef Path);
llvm::Optional<std::string> searchScript(StringRef Path);
+llvm::Optional<std::string> searchLibraryBaseName(StringRef Path);
llvm::Optional<std::string> searchLibrary(StringRef Path);
} // namespace elf
Modified: lld/trunk/ELF/DriverUtils.cpp
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/DriverUtils.cpp?rev=360984&r1=360983&r2=360984&view=diff
==============================================================================
--- lld/trunk/ELF/DriverUtils.cpp (original)
+++ lld/trunk/ELF/DriverUtils.cpp Thu May 16 20:44:15 2019
@@ -223,12 +223,9 @@ Optional<std::string> elf::findFromSearc
return None;
}
-// This is for -lfoo. We'll look for libfoo.so or libfoo.a from
+// This is for -l<basename>. We'll look for lib<basename>.so or lib<basename>.a from
// search paths.
-Optional<std::string> elf::searchLibrary(StringRef Name) {
- if (Name.startswith(":"))
- return findFromSearchPaths(Name.substr(1));
-
+Optional<std::string> elf::searchLibraryBaseName(StringRef Name) {
for (StringRef Dir : Config->SearchPaths) {
if (!Config->Static)
if (Optional<std::string> S = findFile(Dir, "lib" + Name + ".so"))
@@ -239,6 +236,13 @@ Optional<std::string> elf::searchLibrary
return None;
}
+// This is for -l<namespec>.
+Optional<std::string> elf::searchLibrary(StringRef Name) {
+ if (Name.startswith(":"))
+ return findFromSearchPaths(Name.substr(1));
+ return searchLibraryBaseName (Name);
+}
+
// If a linker/version script doesn't exist in the current directory, we also
// look for the script in the '-L' search paths. This matches the behaviour of
// '-T', --version-script=, and linker script INPUT() command in ld.bfd.
Modified: lld/trunk/ELF/InputFiles.cpp
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/InputFiles.cpp?rev=360984&r1=360983&r2=360984&view=diff
==============================================================================
--- lld/trunk/ELF/InputFiles.cpp (original)
+++ lld/trunk/ELF/InputFiles.cpp Thu May 16 20:44:15 2019
@@ -7,6 +7,7 @@
//===----------------------------------------------------------------------===//
#include "InputFiles.h"
+#include "Driver.h"
#include "InputSection.h"
#include "LinkerScript.h"
#include "SymbolTable.h"
@@ -499,6 +500,27 @@ template <class ELFT> void ObjFile<ELFT>
}
}
+// An ELF object file may contain a `.deplibs` section. If it exists, the
+// section contains a list of library specifiers such as `m` for libm. This
+// function resolves a given name by finding the first matching library checking
+// the various ways that a library can be specified to LLD. This ELF extension
+// is a form of autolinking and is called `dependent libraries`. It is currently
+// unique to LLVM and lld.
+static void addDependentLibrary(StringRef Specifier, const InputFile *F) {
+ if (!Config->DependentLibraries)
+ return;
+ if (fs::exists(Specifier))
+ Driver->addFile(Specifier, /*WithLOption=*/false);
+ else if (Optional<std::string> S = findFromSearchPaths(Specifier))
+ Driver->addFile(*S, /*WithLOption=*/true);
+ else if (Optional<std::string> S = searchLibraryBaseName(Specifier))
+ Driver->addFile(*S, /*WithLOption=*/true);
+ else
+ error(toString(F) +
+ ": unable to find library from dependent library specifier: " +
+ Specifier);
+}
+
template <class ELFT>
void ObjFile<ELFT>::initializeSections(
DenseSet<CachedHashStringRef> &ComdatGroups) {
@@ -740,6 +762,24 @@ InputSectionBase *ObjFile<ELFT>::createI
}
return &InputSection::Discarded;
}
+ case SHT_LLVM_DEPENDENT_LIBRARIES: {
+ if (Config->Relocatable)
+ break;
+ ArrayRef<char> Data =
+ CHECK(this->getObj().template getSectionContentsAsArray<char>(&Sec), this);
+ if (!Data.empty() && Data.back() != '\0') {
+ error(toString(this) +
+ ": corrupted dependent libraries section (unterminated string): " +
+ Name);
+ return &InputSection::Discarded;
+ }
+ for (const char *D = Data.begin(), *E = Data.end(); D < E;) {
+ StringRef S(D);
+ addDependentLibrary(S, this);
+ D += S.size() + 1;
+ }
+ return &InputSection::Discarded;
+ }
case SHT_RELA:
case SHT_REL: {
// Find a relocation target section and associate this section with that.
@@ -1302,6 +1342,9 @@ void BitcodeFile::parse(DenseSet<CachedH
for (const lto::InputFile::Symbol &ObjSym : Obj->symbols())
Symbols.push_back(createBitcodeSymbol<ELFT>(KeptComdats, ObjSym, *this));
+
+ for (auto L : Obj->getDependentLibraries())
+ addDependentLibrary(L, this);
}
static ELFKind getELFKind(MemoryBufferRef MB, StringRef ArchiveName) {
Modified: lld/trunk/ELF/Options.td
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/Options.td?rev=360984&r1=360983&r2=360984&view=diff
==============================================================================
--- lld/trunk/ELF/Options.td (original)
+++ lld/trunk/ELF/Options.td Thu May 16 20:44:15 2019
@@ -71,6 +71,10 @@ defm apply_dynamic_relocs: B<"apply-dyna
"Apply link-time values for dynamic relocations",
"Do not apply link-time values for dynamic relocations (default)">;
+defm dependent_libraries: B<"dependent-libraries",
+ "Process dependent library specifiers from input files (default)",
+ "Ignore dependent library specifiers from input files">;
+
defm as_needed: B<"as-needed",
"Only set DT_NEEDED for shared libraries if used",
"Always set DT_NEEDED for shared libraries (default)">;
Added: lld/trunk/test/ELF/Inputs/deplibs-lib_bar.s
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/ELF/Inputs/deplibs-lib_bar.s?rev=360984&view=auto
==============================================================================
--- lld/trunk/test/ELF/Inputs/deplibs-lib_bar.s (added)
+++ lld/trunk/test/ELF/Inputs/deplibs-lib_bar.s Thu May 16 20:44:15 2019
@@ -0,0 +1,2 @@
+ .global bar
+bar:
Added: lld/trunk/test/ELF/Inputs/deplibs-lib_foo.s
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/ELF/Inputs/deplibs-lib_foo.s?rev=360984&view=auto
==============================================================================
--- lld/trunk/test/ELF/Inputs/deplibs-lib_foo.s (added)
+++ lld/trunk/test/ELF/Inputs/deplibs-lib_foo.s Thu May 16 20:44:15 2019
@@ -0,0 +1,2 @@
+ .global foo
+foo:
Added: lld/trunk/test/ELF/deplibs-colon-prefix.s
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/ELF/deplibs-colon-prefix.s?rev=360984&view=auto
==============================================================================
--- lld/trunk/test/ELF/deplibs-colon-prefix.s (added)
+++ lld/trunk/test/ELF/deplibs-colon-prefix.s Thu May 16 20:44:15 2019
@@ -0,0 +1,15 @@
+# REQUIRES: x86
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t.o
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %p/Inputs/deplibs-lib_foo.s -o %tfoo.o
+# RUN: rm -rf %t.dir
+# RUN: mkdir -p %t.dir
+# RUN: llvm-ar rc %t.dir/foo.a %tfoo.o
+# RUN: not ld.lld %t.o -o /dev/null -L %t.dir 2>&1 | FileCheck %s -DOBJ=%t.o
+# CHECK: error: [[OBJ]]: unable to find library from dependent library specifier: :foo.a
+
+ .global _start
+_start:
+ call foo
+ .section ".deplibs","MS", at llvm_dependent_libraries,1
+ .asciz ":foo.a"
Added: lld/trunk/test/ELF/deplibs-corrupt.s
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/ELF/deplibs-corrupt.s?rev=360984&view=auto
==============================================================================
--- lld/trunk/test/ELF/deplibs-corrupt.s (added)
+++ lld/trunk/test/ELF/deplibs-corrupt.s Thu May 16 20:44:15 2019
@@ -0,0 +1,8 @@
+# REQUIRES: x86
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t.o
+# RUN: not ld.lld %t.o -o /dev/null -L %t.dir 2>&1 | FileCheck %s -DOBJ=%t.o
+# CHECK: error: [[OBJ]]: corrupted dependent libraries section (unterminated string): .deplibs
+
+.section ".deplibs","MS", at llvm_dependent_libraries,1
+ .ascii ":foo.a"
Added: lld/trunk/test/ELF/deplibs.s
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/ELF/deplibs.s?rev=360984&view=auto
==============================================================================
--- lld/trunk/test/ELF/deplibs.s (added)
+++ lld/trunk/test/ELF/deplibs.s Thu May 16 20:44:15 2019
@@ -0,0 +1,56 @@
+# REQUIRES: x86
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t.o
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %p/Inputs/deplibs-lib_foo.s -o %tfoo.o
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %p/Inputs/deplibs-lib_bar.s -o %tbar.o
+# RUN: rm -rf %t.dir %t.cwd
+# RUN: mkdir -p %t.dir
+
+# error if dependent libraries cannot be found
+# RUN: not ld.lld %t.o -o /dev/null 2>&1 | FileCheck %s -DOBJ=%t.o --check-prefix MISSING
+# MISSING: error: [[OBJ]]: unable to find library from dependent library specifier: foo.a
+# MISSING-NEXT: error: [[OBJ]]: unable to find library from dependent library specifier: bar
+
+# can ignore dependent libraries
+# RUN: not ld.lld %t.o -o /dev/null --no-dependent-libraries 2>&1 | FileCheck %s --check-prefix IGNORE
+# IGNORE: error: undefined symbol: foo
+# IGNORE: error: undefined symbol: bar
+
+# -r links preserve dependent libraries
+# RUN: ld.lld %t.o %t.o -r -o %t-r.o
+# RUN: not ld.lld %t-r.o -o /dev/null 2>&1 | sort | FileCheck %s -DOBJ=%t-r.o --check-prefixes MINUSR
+# MINUSR: error: [[OBJ]]: unable to find library from dependent library specifier: bar
+# MINUSR-NEXT: error: [[OBJ]]: unable to find library from dependent library specifier: foo.a
+# MINUSR-NOT: unable to find library from dependent library specifier
+
+# static archives located relative to library search paths
+# RUN: llvm-ar rc %t.dir/foo.a %tfoo.o
+# RUN: llvm-ar rc %t.dir/libbar.a %tbar.o
+# RUN: ld.lld %t.o -o /dev/null -L %t.dir
+
+# shared objects located relative to library search paths
+# RUN: rm %t.dir/libbar.a
+# RUN: ld.lld -shared -o %t.dir/libbar.so %tbar.o
+# RUN: ld.lld -Bdynamic %t.o -o /dev/null -L %t.dir
+
+# dependent libraries searched for symbols after libraries on the command line
+# RUN: mkdir -p %t.cwd
+# RUN: cd %t.cwd
+# RUN: cp %t.dir/foo.a %t.cwd/libcmdline.a
+# RUN: ld.lld %t.o libcmdline.a -o /dev/null -L %t.dir --trace 2>&1 | FileCheck %s -DOBJ=%t.o -DSO=%t.dir --check-prefix CMDLINE --implicit-check-not foo.a
+# CMDLINE: [[OBJ]]
+# CMDLINE-NEXT: {{^libcmdline\.a}}
+# CMDLINE-NEXT: [[SO]]{{[\\/]}}libbar.so
+
+# libraries can be found from specifiers as if the specifiers were listed on on the command-line.
+# RUN: cp %t.dir/foo.a %t.cwd/foo.a
+# RUN: ld.lld %t.o -o /dev/null -L %t.dir --trace 2>&1 | FileCheck %s -DOBJ=%t.o -DSO=%t.dir --check-prefix ASIFCMDLINE --implicit-check-not foo.a
+# ASIFCMDLINE: [[OBJ]]
+# ASIFCMDLINE-NEXT: {{^foo\.a}}
+# ASIFCMDLINE-NEXT: [[SO]]{{[\\/]}}libbar.so
+
+ call foo
+ call bar
+.section ".deplibs","MS", at llvm_dependent_libraries,1
+ .asciz "foo.a"
+ .asciz "bar"
Added: lld/trunk/test/ELF/lto/deplibs.s
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/ELF/lto/deplibs.s?rev=360984&view=auto
==============================================================================
--- lld/trunk/test/ELF/lto/deplibs.s (added)
+++ lld/trunk/test/ELF/lto/deplibs.s Thu May 16 20:44:15 2019
@@ -0,0 +1,15 @@
+; REQUIRES: x86
+
+; RUN: llvm-as %s -o %t.o
+; RUN: not ld.lld -shared %t.o -o /dev/null 2>&1 | FileCheck %s -DOBJ=%t.o
+
+; CHECK: error: [[OBJ]]: unable to find library from dependent library specifier: foo
+; CHECK: error: [[OBJ]]: unable to find library from dependent library specifier: bar
+
+target triple = "x86_64-unknown-linux-gnu"
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+
+!llvm.dependent-libraries = !{!0, !1}
+
+!0 = !{!"foo"}
+!1 = !{!"bar"}
More information about the llvm-commits
mailing list