<div dir="ltr"><div>This optimization seems very effective. I picked up a program mksnapshot.exe from Chromium and measure the time LLD spends on ICF. The program contains 27645 COMDAT sections, of which 4501 sections are redundant. The output size after ICF is 9MB.</div><div><br></div><div>Without this patch, LLD takes 93 ms to scan and group all COMDAT sections. The number of hash collisions that had to be resolved by compare section contents was 1335211.</div><div><br></div><div>With this patch, LLD takes only 33 ms to do the same thing. There was *no* false hash collision (but we still had to compare contents for positive case because the section hash value is just a CRC32 and not a cryptographic one).</div><div class="gmail_extra"><br></div><div class="gmail_extra">So this is working as we hoped, and that's probably important because ICF is on by default.</div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Sep 4, 2015 at 1:45 PM, Rui Ueyama via llvm-commits <span dir="ltr"><<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: ruiu<br>
Date: Fri Sep 4 15:45:50 2015<br>
New Revision: 246869<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=246869&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=246869&view=rev</a><br>
Log:<br>
COFF: Use section content checksum for ICF.<br>
<br>
Previously, we calculated our own hash values for section contents.<br>
Of coruse that's slow because we had to access all bytes in sections.<br>
Fortunately, COFF objects usually contain hash values for COMDAT<br>
sections. We can use that to speed up Identical COMDAT Folding.<br>
<br>
Modified:<br>
lld/trunk/COFF/Chunks.h<br>
lld/trunk/COFF/ICF.cpp<br>
lld/trunk/COFF/InputFiles.cpp<br>
<br>
Modified: lld/trunk/COFF/Chunks.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lld/trunk/COFF/Chunks.h?rev=246869&r1=246868&r2=246869&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lld/trunk/COFF/Chunks.h?rev=246869&r1=246868&r2=246869&view=diff</a><br>
==============================================================================<br>
--- lld/trunk/COFF/Chunks.h (original)<br>
+++ lld/trunk/COFF/Chunks.h Fri Sep 4 15:45:50 2015<br>
@@ -183,6 +183,10 @@ public:<br>
// and this chunk is considrered as dead.<br>
SectionChunk *Ptr;<br>
<br>
+ // The CRC of the contents as described in the COFF spec 4.5.5.<br>
+ // Auxiliary Format 5: Section Definitions. Used for ICF.<br>
+ uint32_t Checksum = 0;<br>
+<br>
private:<br>
ArrayRef<uint8_t> getContents() const;<br>
<br>
<br>
Modified: lld/trunk/COFF/ICF.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lld/trunk/COFF/ICF.cpp?rev=246869&r1=246868&r2=246869&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lld/trunk/COFF/ICF.cpp?rev=246869&r1=246868&r2=246869&view=diff</a><br>
==============================================================================<br>
--- lld/trunk/COFF/ICF.cpp (original)<br>
+++ lld/trunk/COFF/ICF.cpp Fri Sep 4 15:45:50 2015<br>
@@ -44,7 +44,7 @@ uint64_t SectionChunk::getHash() const {<br>
NumRelocs,<br>
uint32_t(Header->SizeOfRawData),<br>
std::distance(Relocs.end(), Relocs.begin()),<br>
- hash_combine_range(A.data(), A.data() + A.size()));<br>
+ Checksum);<br>
}<br>
<br>
// Returns true if this and a given chunk are identical COMDAT sections.<br>
@@ -58,6 +58,8 @@ bool SectionChunk::equals(const SectionC<br>
return false;<br>
if (NumRelocs != X->NumRelocs)<br>
return false;<br>
+ if (Checksum != X->Checksum)<br>
+ return false;<br>
<br>
// Compare data<br>
if (getContents() != X->getContents())<br>
<br>
Modified: lld/trunk/COFF/InputFiles.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lld/trunk/COFF/InputFiles.cpp?rev=246869&r1=246868&r2=246869&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lld/trunk/COFF/InputFiles.cpp?rev=246869&r1=246868&r2=246869&view=diff</a><br>
==============================================================================<br>
--- lld/trunk/COFF/InputFiles.cpp (original)<br>
+++ lld/trunk/COFF/InputFiles.cpp Fri Sep 4 15:45:50 2015<br>
@@ -225,13 +225,14 @@ Defined *ObjectFile::createDefined(COFFS<br>
if (!SC)<br>
return nullptr;<br>
<br>
- // Handle associative sections<br>
+ // Handle section definitions<br>
if (IsFirst && AuxP) {<br>
auto *Aux = reinterpret_cast<const coff_aux_section_definition *>(AuxP);<br>
if (Aux->Selection == IMAGE_COMDAT_SELECT_ASSOCIATIVE)<br>
if (auto *ParentSC = cast_or_null<SectionChunk>(<br>
SparseChunks[Aux->getNumber(Sym.isBigObj())]))<br>
ParentSC->addAssociative(SC);<br>
+ SC->Checksum = Aux->CheckSum;<br>
}<br>
<br>
auto *B = new (Alloc) DefinedRegular(this, Sym, SC);<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
</blockquote></div><br></div></div>