<div dir="ltr">Oops, that's a silly mistake. They should be parallel_for_each, but I changed that to for_each to compare the performances, and accidentally submitted that. Thank you for finding!</div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Sep 19, 2015 at 1:44 PM, David Blaikie <span dir="ltr"><<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="h5">On Fri, Sep 18, 2015 at 6:48 PM, Rui Ueyama via llvm-commits <span dir="ltr"><<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: ruiu<br>
Date: Fri Sep 18 20:48:26 2015<br>
New Revision: 248078<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=248078&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=248078&view=rev</a><br>
Log:<br>
COFF: Parallelize InputFile::parse().<br>
<br>
InputFile::parse() can be called in parallel with other calls of<br>
the same function. By doing that, time to self-link improves from<br>
741 ms to 654 ms or 12% faster.<br>
<br>
This is probably the last low hanging fruit in terms of parallelism.<br>
Input file parsing and symbol table insertion takes 450 ms in total.<br>
If we want to optimize further, we probably have to parallelize<br>
symbol table insertion using concurrent hashmap or something.<br>
That's doable, but that's not easy, especially if you want to keep<br>
the exact same semantics and linking order. I'm not going to do that<br>
at least soon.<br>
<br>
Anyway, compared to r248019 (the change before the first attempt for<br>
parallelism), we achieved 36% performance improvement from 1022 ms<br>
to 654 ms. MSVC linker takes 3.3 seconds to link the same program.<br>
MSVC's ICF feature is very slow for some reason, but even if we<br>
disable the feature, it still takes about 1.2 seconds.<br>
Our number is probably good enough.<br>
<br>
Modified:<br>
lld/trunk/COFF/SymbolTable.cpp<br>
<br>
Modified: lld/trunk/COFF/SymbolTable.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lld/trunk/COFF/SymbolTable.cpp?rev=248078&r1=248077&r2=248078&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lld/trunk/COFF/SymbolTable.cpp?rev=248078&r1=248077&r2=248078&view=diff</a><br>
==============================================================================<br>
--- lld/trunk/COFF/SymbolTable.cpp (original)<br>
+++ lld/trunk/COFF/SymbolTable.cpp Fri Sep 18 20:48:26 2015<br>
@@ -12,6 +12,7 @@<br>
#include "Error.h"<br>
#include "SymbolTable.h"<br>
#include "Symbols.h"<br>
+#include "lld/Core/Parallel.h"<br>
#include "llvm/ADT/STLExtras.h"<br>
#include "llvm/LTO/LTOCodeGenerator.h"<br>
#include "llvm/Support/Debug.h"<br>
@@ -56,13 +57,15 @@ void SymbolTable::readArchives() {<br>
if (ArchiveQueue.empty())<br>
return;<br>
<br>
+ std::for_each(ArchiveQueue.begin(), ArchiveQueue.end(),<br>
+ [](ArchiveFile *File) { File->parse(); });<br></blockquote><div><br></div></div></div><div>std:for_each doesn't offer much now that we have range-based-for, is there a reason you've used std::for_each in some places and range-based-for in otehrs? I'd probably just stick to range-based-for in all cases.</div><div><div class="h5"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
+<br>
// Add lazy symbols to the symbol table. Lazy symbols that conflict<br>
// with existing undefined symbols are accumulated in LazySyms.<br>
std::vector<Symbol *> LazySyms;<br>
for (ArchiveFile *File : ArchiveQueue) {<br>
if (Config->Verbose)<br>
llvm::outs() << "Reading " << File->getShortName() << "\n";<br>
- File->parse();<br>
for (Lazy &Sym : File->getLazySymbols())<br>
addLazy(&Sym, &LazySyms);<br>
}<br>
@@ -80,22 +83,25 @@ void SymbolTable::readObjects() {<br>
<br>
// Add defined and undefined symbols to the symbol table.<br>
std::vector<StringRef> Directives;<br>
- for (size_t I = 0; I < ObjectQueue.size(); ++I) {<br>
- InputFile *File = ObjectQueue[I];<br>
- if (Config->Verbose)<br>
- llvm::outs() << "Reading " << File->getShortName() << "\n";<br>
- File->parse();<br>
- // Adding symbols may add more files to ObjectQueue<br>
- // (but not to ArchiveQueue).<br>
- for (SymbolBody *Sym : File->getSymbols())<br>
- if (Sym->isExternal())<br>
- addSymbol(Sym);<br>
- StringRef S = File->getDirectives();<br>
- if (!S.empty()) {<br>
- Directives.push_back(S);<br>
+ for (size_t I = 0; I < ObjectQueue.size();) {<br>
+ std::for_each(ObjectQueue.begin() + I, ObjectQueue.end(),<br>
+ [](InputFile *File) { File->parse(); });<br>
+ for (size_t E = ObjectQueue.size(); I != E; ++I) {<br>
+ InputFile *File = ObjectQueue[I];<br>
if (Config->Verbose)<br>
- llvm::outs() << "Directives: " << File->getShortName()<br>
- << ": " << S << "\n";<br>
+ llvm::outs() << "Reading " << File->getShortName() << "\n";<br>
+ // Adding symbols may add more files to ObjectQueue<br>
+ // (but not to ArchiveQueue).<br>
+ for (SymbolBody *Sym : File->getSymbols())<br>
+ if (Sym->isExternal())<br>
+ addSymbol(Sym);<br>
+ StringRef S = File->getDirectives();<br>
+ if (!S.empty()) {<br>
+ Directives.push_back(S);<br>
+ if (Config->Verbose)<br>
+ llvm::outs() << "Directives: " << File->getShortName()<br>
+ << ": " << S << "\n";<br>
+ }<br>
}<br>
}<br>
ObjectQueue.clear();<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
</blockquote></div></div></div><br></div></div>
</blockquote></div><br></div>