[lld] r231434 - Core: Make the resolver faster.
Rui Ueyama
ruiu at google.com
Thu Mar 5 16:28:41 PST 2015
Author: ruiu
Date: Thu Mar 5 18:28:41 2015
New Revision: 231434
URL: http://llvm.org/viewvc/llvm-project?rev=231434&view=rev
Log:
Core: Make the resolver faster.
In the resolver, we maintain a list of undefined symbols, and when we
visit an archive file, we check that file if undefined symbols can be
resolved using files in the archive. The archive file class provides
find() function to lookup a symbol.
Previously, we call find() for each undefined symbols. Archive files
may be visited multiple times if they are in a --start-group and
--end-group. If we visit a file M times and if we have N undefined
symbols, find() is called M*N times. I found that that is one of the
most significant bottlenecks in LLD when linking a large executable.
find() is not a very cheap operation because it looks up a hash table
for a given string. And a string, or a symbol name, can be pretty long
if you are dealing with C++ symbols.
We can eliminate the bottleneck.
Calling find() with the same symbol multiple times is a waste. If a
result of looking up a symbol is "not found", it stays "not found"
forever because the symbol simply doesn't exist in the archive.
Thus, we should call find() only for newly-added undefined symbols.
This optimization makes O(M*N) O(N).
In this patch, all undefined symbols are added to a vector. For each
archive/shared library file, we maintain a start position P. All
symbols [0, P) are already searched. [P, end of the vector) are not
searched yet. For each file, we scan the vector only once.
This patch changes the order in which undefined symbols are looked for.
Previously, we iterated over the result of _symbolTable.undefines().
Now we iterate over the new vector. This is a benign change but caused
differences in output if remaining undefines exist. This is why some
tests are updated.
The performance improvement of this patch seems sometimes significant.
Previously, linking chrome.dll on my workstation (Xeon 2.4GHz 8 cores)
took about 70 seconds. Now it takes (only?) 30 seconds!
http://reviews.llvm.org/D8091
Modified:
lld/trunk/include/lld/Core/Resolver.h
lld/trunk/lib/Core/Resolver.cpp
lld/trunk/test/elf/Mips/rel-dynamic-01-micro.test
lld/trunk/test/elf/Mips/rel-dynamic-01.test
lld/trunk/test/elf/Mips/rel-dynamic-04-micro.test
lld/trunk/test/elf/Mips/rel-dynamic-04.test
lld/trunk/test/elf/Mips/rel-dynamic-05-micro.test
lld/trunk/test/elf/Mips/rel-dynamic-05.test
lld/trunk/test/elf/dynamic.test
lld/trunk/test/mach-o/use-simple-dylib.yaml
lld/trunk/test/pecoff/hello64.test
Modified: lld/trunk/include/lld/Core/Resolver.h
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/include/lld/Core/Resolver.h?rev=231434&r1=231433&r2=231434&view=diff
==============================================================================
--- lld/trunk/include/lld/Core/Resolver.h (original)
+++ lld/trunk/include/lld/Core/Resolver.h Thu Mar 5 18:28:41 2015
@@ -73,7 +73,7 @@ private:
bool checkUndefines();
void removeCoalescedAwayAtoms();
void checkDylibSymbolCollisions();
- void forEachUndefines(bool searchForOverrides, UndefCallback callback);
+ void forEachUndefines(File &file, bool searchForOverrides, UndefCallback callback);
void markLive(const Atom *atom);
void addAtoms(const std::vector<const DefinedAtom *>&);
@@ -102,6 +102,16 @@ private:
// Preloading
std::map<StringRef, ArchiveLibraryFile *> _archiveMap;
std::unordered_set<ArchiveLibraryFile *> _archiveSeen;
+
+ // List of undefined symbols.
+ std::vector<StringRef> _undefines;
+
+ // Start position in _undefines for each archive/shared library file.
+ // Symbols from index 0 to the start position are already searched before.
+ // Searching them again would never succeed. When we look for undefined
+ // symbols from an archive/shared library file, start from its start
+ // position to save time.
+ std::map<File *, size_t> _undefineIndex;
};
} // namespace lld
Modified: lld/trunk/lib/Core/Resolver.cpp
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/lib/Core/Resolver.cpp?rev=231434&r1=231433&r2=231434&view=diff
==============================================================================
--- lld/trunk/lib/Core/Resolver.cpp (original)
+++ lld/trunk/lib/Core/Resolver.cpp Thu Mar 5 18:28:41 2015
@@ -46,33 +46,35 @@ bool Resolver::handleFile(File &file) {
return undefAdded;
}
-void Resolver::forEachUndefines(bool searchForOverrides,
+void Resolver::forEachUndefines(File &file, bool searchForOverrides,
UndefCallback callback) {
- // Handle normal archives
- unsigned undefineGenCount = 0;
+ size_t i = _undefineIndex[&file];
do {
- undefineGenCount = _symbolTable.size();
- for (const UndefinedAtom *undefAtom : _symbolTable.undefines()) {
- StringRef undefName = undefAtom->name();
- // load for previous undefine may also have loaded this undefine
- if (!_symbolTable.isDefined(undefName))
- callback(undefName, false);
- }
-
- // search libraries for overrides of common symbols
- if (searchForOverrides) {
- for (StringRef tentDefName : _symbolTable.tentativeDefinitions()) {
- // Load for previous tentative may also have loaded
- // something that overrode this tentative, so always check.
- const Atom *curAtom = _symbolTable.findByName(tentDefName);
- assert(curAtom != nullptr);
- if (const DefinedAtom *curDefAtom = dyn_cast<DefinedAtom>(curAtom)) {
- if (curDefAtom->merge() == DefinedAtom::mergeAsTentative)
- callback(tentDefName, true);
- }
+ for (; i < _undefines.size(); ++i) {
+ StringRef undefName = _undefines[i];
+ if (undefName.empty())
+ continue;
+ if (_symbolTable.isDefined(undefName) ||
+ _symbolTable.isCoalescedAway(_symbolTable.findByName(undefName))) {
+ // The symbol was resolved by some other file. Cache the result.
+ _undefines[i] = "";
+ continue;
}
+ callback(undefName, false);
+ }
+ if (!searchForOverrides)
+ continue;
+ for (StringRef tentDefName : _symbolTable.tentativeDefinitions()) {
+ // Load for previous tentative may also have loaded
+ // something that overrode this tentative, so always check.
+ const Atom *curAtom = _symbolTable.findByName(tentDefName);
+ assert(curAtom != nullptr);
+ if (const DefinedAtom *curDefAtom = dyn_cast<DefinedAtom>(curAtom))
+ if (curDefAtom->merge() == DefinedAtom::mergeAsTentative)
+ callback(tentDefName, true);
}
- } while (undefineGenCount != _symbolTable.size());
+ } while (i < _undefines.size());
+ _undefineIndex[&file] = i;
}
bool Resolver::handleArchiveFile(File &file) {
@@ -80,7 +82,7 @@ bool Resolver::handleArchiveFile(File &f
bool searchForOverrides =
_ctx.searchArchivesToOverrideTentativeDefinitions();
bool undefAdded = false;
- forEachUndefines(searchForOverrides,
+ forEachUndefines(file, searchForOverrides,
[&](StringRef undefName, bool dataSymbolOnly) {
if (File *member = archiveFile->find(undefName, dataSymbolOnly)) {
member->setOrdinal(_ctx.getNextOrdinalAndIncrement());
@@ -98,7 +100,7 @@ void Resolver::handleSharedLibrary(File
handleFile(*sharedLibrary);
bool searchForOverrides =
_ctx.searchSharedLibrariesToOverrideTentativeDefinitions();
- forEachUndefines(searchForOverrides,
+ forEachUndefines(file, searchForOverrides,
[&](StringRef undefName, bool dataSymbolOnly) {
if (const SharedLibraryAtom *atom =
sharedLibrary->exports(undefName, dataSymbolOnly))
@@ -117,6 +119,8 @@ bool Resolver::doUndefinedAtom(const Und
// tell symbol table
bool newUndefAdded = _symbolTable.add(atom);
+ if (newUndefAdded)
+ _undefines.push_back(atom.name());
// If the undefined symbol has an alternative name, try to resolve the
// symbol with the name to give it a second chance. This feature is used
Modified: lld/trunk/test/elf/Mips/rel-dynamic-01-micro.test
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/elf/Mips/rel-dynamic-01-micro.test?rev=231434&r1=231433&r2=231434&view=diff
==============================================================================
--- lld/trunk/test/elf/Mips/rel-dynamic-01-micro.test (original)
+++ lld/trunk/test/elf/Mips/rel-dynamic-01-micro.test Thu Mar 5 18:28:41 2015
@@ -68,21 +68,21 @@
# PLT-SYM-NEXT: Section: .bss (0xD)
# PLT-SYM-NEXT: }
# PLT-SYM-NEXT: Symbol {
-# PLT-SYM-NEXT: Name: T3@ (4)
-# PLT-SYM-NEXT: Value: 0x0
+# PLT-SYM-NEXT: Name: T1@ (4)
+# PLT-SYM-NEXT: Value: 0x4001D5
# PLT-SYM-NEXT: Size: 0
# PLT-SYM-NEXT: Binding: Global (0x1)
# PLT-SYM-NEXT: Type: Function (0x2)
-# PLT-SYM-NEXT: Other: 0
+# PLT-SYM-NEXT: Other: 8
# PLT-SYM-NEXT: Section: Undefined (0x0)
# PLT-SYM-NEXT: }
# PLT-SYM-NEXT: Symbol {
-# PLT-SYM-NEXT: Name: T1@ (7)
-# PLT-SYM-NEXT: Value: 0x4001D5
+# PLT-SYM-NEXT: Name: T3@ (7)
+# PLT-SYM-NEXT: Value: 0x0
# PLT-SYM-NEXT: Size: 0
# PLT-SYM-NEXT: Binding: Global (0x1)
# PLT-SYM-NEXT: Type: Function (0x2)
-# PLT-SYM-NEXT: Other: 8
+# PLT-SYM-NEXT: Other: 0
# PLT-SYM-NEXT: Section: Undefined (0x0)
# PLT-SYM-NEXT: }
# PLT-SYM-NEXT: ]
Modified: lld/trunk/test/elf/Mips/rel-dynamic-01.test
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/elf/Mips/rel-dynamic-01.test?rev=231434&r1=231433&r2=231434&view=diff
==============================================================================
--- lld/trunk/test/elf/Mips/rel-dynamic-01.test (original)
+++ lld/trunk/test/elf/Mips/rel-dynamic-01.test Thu Mar 5 18:28:41 2015
@@ -69,25 +69,25 @@
# PLT-SYM-NEXT: Section: .bss (0xD)
# PLT-SYM-NEXT: }
# PLT-SYM-NEXT: Symbol {
-# PLT-SYM-NEXT: Name: T3@ (4)
-# PLT-SYM-NEXT: Value: 0x0
+# PLT-SYM-NEXT: Name: T1@ (4)
+# PLT-SYM-NEXT: Value: 0x400220
# PLT-SYM-NEXT: Size: 0
# PLT-SYM-NEXT: Binding: Global (0x1)
# PLT-SYM-NEXT: Type: Function (0x2)
-# PLT-SYM-NEXT: Other: 0
+# PLT-SYM-NEXT: Other: 8
# PLT-SYM-NEXT: Section: Undefined (0x0)
# PLT-SYM-NEXT: }
# PLT-SYM-NEXT: Symbol {
-# PLT-SYM-NEXT: Name: T1@ (7)
-# PLT-SYM-NEXT: Value: 0x400220
+# PLT-SYM-NEXT: Name: T3@ (10)
+# PLT-SYM-NEXT: Value: 0x0
# PLT-SYM-NEXT: Size: 0
# PLT-SYM-NEXT: Binding: Global (0x1)
# PLT-SYM-NEXT: Type: Function (0x2)
-# PLT-SYM-NEXT: Other: 8
+# PLT-SYM-NEXT: Other: 0
# PLT-SYM-NEXT: Section: Undefined (0x0)
# PLT-SYM-NEXT: }
# PLT-SYM-NEXT: Symbol {
-# PLT-SYM-NEXT: Name: T2@ (10)
+# PLT-SYM-NEXT: Name: T2@ (7)
# PLT-SYM-NEXT: Value: 0x0
# PLT-SYM-NEXT: Size: 0
# PLT-SYM-NEXT: Binding: Global (0x1)
Modified: lld/trunk/test/elf/Mips/rel-dynamic-04-micro.test
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/elf/Mips/rel-dynamic-04-micro.test?rev=231434&r1=231433&r2=231434&view=diff
==============================================================================
--- lld/trunk/test/elf/Mips/rel-dynamic-04-micro.test (original)
+++ lld/trunk/test/elf/Mips/rel-dynamic-04-micro.test Thu Mar 5 18:28:41 2015
@@ -88,8 +88,8 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
-# PLT-NEXT: Name: T3@ (1)
-# PLT-NEXT: Value: 0x4010E5
+# PLT-NEXT: Name: T2@ (4)
+# PLT-NEXT: Value: 0x4010D9
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
@@ -97,8 +97,8 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
-# PLT-NEXT: Name: T2@ (7)
-# PLT-NEXT: Value: 0x4010D9
+# PLT-NEXT: Name: T3@ (7)
+# PLT-NEXT: Value: 0x4010E5
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
@@ -106,7 +106,7 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
-# PLT-NEXT: Name: T1@ (4)
+# PLT-NEXT: Name: T1@ (1)
# PLT-NEXT: Value: 0x0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
Modified: lld/trunk/test/elf/Mips/rel-dynamic-04.test
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/elf/Mips/rel-dynamic-04.test?rev=231434&r1=231433&r2=231434&view=diff
==============================================================================
--- lld/trunk/test/elf/Mips/rel-dynamic-04.test (original)
+++ lld/trunk/test/elf/Mips/rel-dynamic-04.test Thu Mar 5 18:28:41 2015
@@ -86,8 +86,8 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
-# PLT-NEXT: Name: T3@ (1)
-# PLT-NEXT: Value: 0x4010F0
+# PLT-NEXT: Name: T2@ (4)
+# PLT-NEXT: Value: 0x4010E0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
@@ -95,8 +95,8 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
-# PLT-NEXT: Name: T2@ (7)
-# PLT-NEXT: Value: 0x4010E0
+# PLT-NEXT: Name: T3@ (7)
+# PLT-NEXT: Value: 0x4010F0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
@@ -104,7 +104,7 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
-# PLT-NEXT: Name: T1@ (4)
+# PLT-NEXT: Name: T1@ (1)
# PLT-NEXT: Value: 0x0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
Modified: lld/trunk/test/elf/Mips/rel-dynamic-05-micro.test
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/elf/Mips/rel-dynamic-05-micro.test?rev=231434&r1=231433&r2=231434&view=diff
==============================================================================
--- lld/trunk/test/elf/Mips/rel-dynamic-05-micro.test (original)
+++ lld/trunk/test/elf/Mips/rel-dynamic-05-micro.test Thu Mar 5 18:28:41 2015
@@ -70,8 +70,8 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
-# PLT-NEXT: Name: T3@ (1)
-# PLT-NEXT: Value: 0x4001E1
+# PLT-NEXT: Name: T1@ (1)
+# PLT-NEXT: Value: 0x4001C9
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
@@ -79,8 +79,8 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
-# PLT-NEXT: Name: T1@ (4)
-# PLT-NEXT: Value: 0x4001C9
+# PLT-NEXT: Name: T2@ (4)
+# PLT-NEXT: Value: 0x4001D5
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
@@ -88,8 +88,8 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
-# PLT-NEXT: Name: T2@ (7)
-# PLT-NEXT: Value: 0x4001D5
+# PLT-NEXT: Name: T3@ (7)
+# PLT-NEXT: Value: 0x4001E1
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
Modified: lld/trunk/test/elf/Mips/rel-dynamic-05.test
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/elf/Mips/rel-dynamic-05.test?rev=231434&r1=231433&r2=231434&view=diff
==============================================================================
--- lld/trunk/test/elf/Mips/rel-dynamic-05.test (original)
+++ lld/trunk/test/elf/Mips/rel-dynamic-05.test Thu Mar 5 18:28:41 2015
@@ -68,8 +68,8 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
-# PLT-NEXT: Name: T3@ (1)
-# PLT-NEXT: Value: 0x4001F0
+# PLT-NEXT: Name: T1@ (1)
+# PLT-NEXT: Value: 0x4001D0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
@@ -77,8 +77,8 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
-# PLT-NEXT: Name: T1@ (4)
-# PLT-NEXT: Value: 0x4001D0
+# PLT-NEXT: Name: T2@ (4)
+# PLT-NEXT: Value: 0x4001E0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
@@ -86,8 +86,8 @@
# PLT-NEXT: Section: Undefined (0x0)
# PLT-NEXT: }
# PLT-NEXT: Symbol {
-# PLT-NEXT: Name: T2@ (7)
-# PLT-NEXT: Value: 0x4001E0
+# PLT-NEXT: Name: T3@ (7)
+# PLT-NEXT: Value: 0x4001F0
# PLT-NEXT: Size: 0
# PLT-NEXT: Binding: Global (0x1)
# PLT-NEXT: Type: Function (0x2)
Modified: lld/trunk/test/elf/dynamic.test
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/elf/dynamic.test?rev=231434&r1=231433&r2=231434&view=diff
==============================================================================
--- lld/trunk/test/elf/dynamic.test (original)
+++ lld/trunk/test/elf/dynamic.test Thu Mar 5 18:28:41 2015
@@ -46,18 +46,18 @@ CHECK-NEXT: }
CHECK: DynamicSymbols [
CHECK: Symbol {
-CHECK: Name: i
+CHECK: Name: foo
CHECK-NEXT: Value: 0
CHECK-NEXT: Size:
CHECK-NEXT: Binding: Global
-CHECK-NEXT: Type: Object
+CHECK-NEXT: Type: Function
CHECK: }
CHECK: Symbol {
-CHECK: Name: foo
+CHECK: Name: i
CHECK-NEXT: Value: 0
CHECK-NEXT: Size:
CHECK-NEXT: Binding: Global
-CHECK-NEXT: Type: Function
+CHECK-NEXT: Type: Object
CHECK: }
CHECK: DynamicSection [ (15 entries)
Modified: lld/trunk/test/mach-o/use-simple-dylib.yaml
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/mach-o/use-simple-dylib.yaml?rev=231434&r1=231433&r2=231434&view=diff
==============================================================================
--- lld/trunk/test/mach-o/use-simple-dylib.yaml (original)
+++ lld/trunk/test/mach-o/use-simple-dylib.yaml Thu Mar 5 18:28:41 2015
@@ -119,13 +119,13 @@ install-name: libspecial.dylib
# CHECK: - name: _myStatic
# CHECK: - name: _myVariablePreviouslyKnownAsPrivateExtern
# CHECK: shared-library-atoms:
-# CHECK: - name: _myHidden
-# CHECK: load-name: libspecial.dylib
# CHECK: - name: _myGlobal
# CHECK: load-name: libspecial.dylib
-# CHECK: - name: _myHiddenWeak
-# CHECK: load-name: libspecial.dylib
# CHECK: - name: _myGlobalWeak
# CHECK: load-name: libspecial.dylib
+# CHECK: - name: _myHidden
+# CHECK: load-name: libspecial.dylib
+# CHECK: - name: _myHiddenWeak
+# CHECK: load-name: libspecial.dylib
# CHECK: - name: _myResolver
# CHECK: load-name: libspecial.dylib
Modified: lld/trunk/test/pecoff/hello64.test
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/pecoff/hello64.test?rev=231434&r1=231433&r2=231434&view=diff
==============================================================================
--- lld/trunk/test/pecoff/hello64.test (original)
+++ lld/trunk/test/pecoff/hello64.test Thu Mar 5 18:28:41 2015
@@ -11,12 +11,12 @@ CHECK: 6004: 48 c7 c1 00 00 00 00 mov
CHECK: 600b: 48 8d 15 f4 af ff ff leaq -20492(%rip), %rdx
CHECK: 6012: 4c 8d 05 e7 af ff ff leaq -20505(%rip), %r8
CHECK: 6019: 41 b9 00 00 00 00 movl $0, %r9d
-CHECK: 601f: e8 0a 00 00 00 callq 10
+CHECK: 601f: e8 12 00 00 00 callq 18
CHECK: 6024: b9 00 00 00 00 movl $0, %ecx
-CHECK: 6029: e8 08 00 00 00 callq 8
-CHECK: 602e: ff 25 d4 cf ff ff jmpq *-12332(%rip)
+CHECK: 6029: e8 00 00 00 00 callq 0
+CHECK: 602e: ff 25 cc cf ff ff jmpq *-12340(%rip)
CHECK: 6034: cc int3
CHECK: 6035: cc int3
-CHECK: 6036: ff 25 c4 cf ff ff jmpq *-12348(%rip)
+CHECK: 6036: ff 25 cc cf ff ff jmpq *-12340(%rip)
CHECK: 603c: cc int3
CHECK: 603d: cc int3
More information about the llvm-commits
mailing list