[PATCH] LLD: Core: Make the resolver faster

[Rui Ueyama]

In the resolver, we maintain a list of undefined symbols, and when we
visit an archive file, we check that file if undefined symbols can be
resolved using files in the archive. The archive file class provides
a find() function to lookup a symbol.

Previously, we call find() for each undefined symbols. Archive files
may be visited multiple times if they are in a --start-group and
--end-group. If we visit a file M times and if we have N undefined
symbols, find() is called M*N times. I found that that is one of the
most significant bottlenecks in LLD when linking a large executable.

find() is not a very cheap operation because it looks up a hash table
for a given string. And a string, or a symbol name, can be pretty long
if you are dealing with C++ symbols.

We can eliminate the bottleneck.

Calling find() with the same symbol multiple times is a waste. If a
result of looking up a symbol is "not found", it stays "not found"
forever because the symbol simply doesn't exist in the archive.
Thus, we should call find() only for newly-added undefined symbols.
This optimization makes O(M*N) O(N).

In this patch, all undefined symbols are added to a vector. For each
archive/shared library file, we maintain a start position P. All
symbols [0, P) are already searched. [P, end of the vector) are not
searched yet. For each file, we scan the vector only once.

This patch changes the order in which undefined symbols are looked for.
Previously, we iterated over the result _symbolTable.undefines().
Now we iterate over the new vector. This is a benign change but caused
differences in output if remaining undefines exist. This is why some
tests are updated.

The performance improvement of this patch seems sometimes significant.
Previously, linking chrome.dll on my workstation (Xeon 2.4GHz 8 cores)
took about 80 seconds. Now it takes 40 seconds!

http://reviews.llvm.org/D8091

Files:
  include/lld/Core/Resolver.h
  lib/Core/Resolver.cpp
  test/elf/Mips/rel-dynamic-01-micro.test
  test/elf/Mips/rel-dynamic-01.test
  test/elf/Mips/rel-dynamic-04-micro.test
  test/elf/Mips/rel-dynamic-04.test
  test/elf/Mips/rel-dynamic-05-micro.test
  test/elf/Mips/rel-dynamic-05.test
  test/elf/dynamic.test
  test/mach-o/use-simple-dylib.yaml
  test/pecoff/hello64.test

EMAIL PREFERENCES
  http://reviews.llvm.org/settings/panel/emailpreferences/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: D8091.21318.patch
Type: text/x-patch
Size: 14725 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20150305/1a0662ef/attachment.bin>