[lld] r248078 - COFF: Parallelize InputFile::parse().

[Rui Ueyama via llvm-commits]

Oops, that's a silly mistake. They should be parallel_for_each, but I
changed that to for_each to compare the performances, and accidentally
submitted that. Thank you for finding!

On Sat, Sep 19, 2015 at 1:44 PM, David Blaikie <dblaikie at gmail.com> wrote:

>
>
> On Fri, Sep 18, 2015 at 6:48 PM, Rui Ueyama via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
>
>> Author: ruiu
>> Date: Fri Sep 18 20:48:26 2015
>> New Revision: 248078
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=248078&view=rev
>> Log:
>> COFF: Parallelize InputFile::parse().
>>
>> InputFile::parse() can be called in parallel with other calls of
>> the same function. By doing that, time to self-link improves from
>> 741 ms to 654 ms or 12% faster.
>>
>> This is probably the last low hanging fruit in terms of parallelism.
>> Input file parsing and symbol table insertion takes 450 ms in total.
>> If we want to optimize further, we probably have to parallelize
>> symbol table insertion using concurrent hashmap or something.
>> That's doable, but that's not easy, especially if you want to keep
>> the exact same semantics and linking order. I'm not going to do that
>> at least soon.
>>
>> Anyway, compared to r248019 (the change before the first attempt for
>> parallelism), we achieved 36% performance improvement from 1022 ms
>> to 654 ms. MSVC linker takes 3.3 seconds to link the same program.
>> MSVC's ICF feature is very slow for some reason, but even if we
>> disable the feature, it still takes about 1.2 seconds.
>> Our number is probably good enough.
>>
>> Modified:
>>     lld/trunk/COFF/SymbolTable.cpp
>>
>> Modified: lld/trunk/COFF/SymbolTable.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/lld/trunk/COFF/SymbolTable.cpp?rev=248078&r1=248077&r2=248078&view=diff
>>
>> ==============================================================================
>> --- lld/trunk/COFF/SymbolTable.cpp (original)
>> +++ lld/trunk/COFF/SymbolTable.cpp Fri Sep 18 20:48:26 2015
>> @@ -12,6 +12,7 @@
>>  #include "Error.h"
>>  #include "SymbolTable.h"
>>  #include "Symbols.h"
>> +#include "lld/Core/Parallel.h"
>>  #include "llvm/ADT/STLExtras.h"
>>  #include "llvm/LTO/LTOCodeGenerator.h"
>>  #include "llvm/Support/Debug.h"
>> @@ -56,13 +57,15 @@ void SymbolTable::readArchives() {
>>    if (ArchiveQueue.empty())
>>      return;
>>
>> +  std::for_each(ArchiveQueue.begin(), ArchiveQueue.end(),
>> +                [](ArchiveFile *File) { File->parse(); });
>>
>
> std:for_each doesn't offer much now that we have range-based-for, is there
> a reason you've used std::for_each in some places and range-based-for in
> otehrs? I'd probably just stick to range-based-for in all cases.
>
>
>> +
>>    // Add lazy symbols to the symbol table. Lazy symbols that conflict
>>    // with existing undefined symbols are accumulated in LazySyms.
>>    std::vector<Symbol *> LazySyms;
>>    for (ArchiveFile *File : ArchiveQueue) {
>>      if (Config->Verbose)
>>        llvm::outs() << "Reading " << File->getShortName() << "\n";
>> -    File->parse();
>>      for (Lazy &Sym : File->getLazySymbols())
>>        addLazy(&Sym, &LazySyms);
>>    }
>> @@ -80,22 +83,25 @@ void SymbolTable::readObjects() {
>>
>>    // Add defined and undefined symbols to the symbol table.
>>    std::vector<StringRef> Directives;
>> -  for (size_t I = 0; I < ObjectQueue.size(); ++I) {
>> -    InputFile *File = ObjectQueue[I];
>> -    if (Config->Verbose)
>> -      llvm::outs() << "Reading " << File->getShortName() << "\n";
>> -    File->parse();
>> -    // Adding symbols may add more files to ObjectQueue
>> -    // (but not to ArchiveQueue).
>> -    for (SymbolBody *Sym : File->getSymbols())
>> -      if (Sym->isExternal())
>> -        addSymbol(Sym);
>> -    StringRef S = File->getDirectives();
>> -    if (!S.empty()) {
>> -      Directives.push_back(S);
>> +  for (size_t I = 0; I < ObjectQueue.size();) {
>> +    std::for_each(ObjectQueue.begin() + I, ObjectQueue.end(),
>> +                  [](InputFile *File) { File->parse(); });
>> +    for (size_t E = ObjectQueue.size(); I != E; ++I) {
>> +      InputFile *File = ObjectQueue[I];
>>        if (Config->Verbose)
>> -        llvm::outs() << "Directives: " << File->getShortName()
>> -                     << ": " << S << "\n";
>> +        llvm::outs() << "Reading " << File->getShortName() << "\n";
>> +      // Adding symbols may add more files to ObjectQueue
>> +      // (but not to ArchiveQueue).
>> +      for (SymbolBody *Sym : File->getSymbols())
>> +        if (Sym->isExternal())
>> +          addSymbol(Sym);
>> +      StringRef S = File->getDirectives();
>> +      if (!S.empty()) {
>> +        Directives.push_back(S);
>> +        if (Config->Verbose)
>> +          llvm::outs() << "Directives: " << File->getShortName()
>> +                       << ": " << S << "\n";
>> +      }
>>      }
>>    }
>>    ObjectQueue.clear();
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20150919/ed8bca13/attachment.html>