[lld] r246869 - COFF: Use section content checksum for ICF.

[Rui Ueyama via llvm-commits]

This optimization seems very effective. I picked up a program
mksnapshot.exe from Chromium and measure the time LLD spends on ICF. The
program contains 27645 COMDAT sections, of which 4501 sections are
redundant. The output size after ICF is 9MB.

Without this patch, LLD takes 93 ms to scan and group all COMDAT sections.
The number of hash collisions that had to be resolved by compare section
contents was 1335211.

With this patch, LLD takes only 33 ms to do the same thing. There was *no*
false hash collision (but we still had to compare contents for positive
case because the section hash value is just a CRC32 and not a cryptographic
one).

So this is working as we hoped, and that's probably important because ICF
is on by default.

On Fri, Sep 4, 2015 at 1:45 PM, Rui Ueyama via llvm-commits <
llvm-commits at lists.llvm.org> wrote:

> Author: ruiu
> Date: Fri Sep  4 15:45:50 2015
> New Revision: 246869
>
> URL: http://llvm.org/viewvc/llvm-project?rev=246869&view=rev
> Log:
> COFF: Use section content checksum for ICF.
>
> Previously, we calculated our own hash values for section contents.
> Of coruse that's slow because we had to access all bytes in sections.
> Fortunately, COFF objects usually contain hash values for COMDAT
> sections. We can use that to speed up Identical COMDAT Folding.
>
> Modified:
>     lld/trunk/COFF/Chunks.h
>     lld/trunk/COFF/ICF.cpp
>     lld/trunk/COFF/InputFiles.cpp
>
> Modified: lld/trunk/COFF/Chunks.h
> URL:
> http://llvm.org/viewvc/llvm-project/lld/trunk/COFF/Chunks.h?rev=246869&r1=246868&r2=246869&view=diff
>
> ==============================================================================
> --- lld/trunk/COFF/Chunks.h (original)
> +++ lld/trunk/COFF/Chunks.h Fri Sep  4 15:45:50 2015
> @@ -183,6 +183,10 @@ public:
>    // and this chunk is considrered as dead.
>    SectionChunk *Ptr;
>
> +  // The CRC of the contents as described in the COFF spec 4.5.5.
> +  // Auxiliary Format 5: Section Definitions. Used for ICF.
> +  uint32_t Checksum = 0;
> +
>  private:
>    ArrayRef<uint8_t> getContents() const;
>
>
> Modified: lld/trunk/COFF/ICF.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/lld/trunk/COFF/ICF.cpp?rev=246869&r1=246868&r2=246869&view=diff
>
> ==============================================================================
> --- lld/trunk/COFF/ICF.cpp (original)
> +++ lld/trunk/COFF/ICF.cpp Fri Sep  4 15:45:50 2015
> @@ -44,7 +44,7 @@ uint64_t SectionChunk::getHash() const {
>                        NumRelocs,
>                        uint32_t(Header->SizeOfRawData),
>                        std::distance(Relocs.end(), Relocs.begin()),
> -                      hash_combine_range(A.data(), A.data() + A.size()));
> +                      Checksum);
>  }
>
>  // Returns true if this and a given chunk are identical COMDAT sections.
> @@ -58,6 +58,8 @@ bool SectionChunk::equals(const SectionC
>      return false;
>    if (NumRelocs != X->NumRelocs)
>      return false;
> +  if (Checksum != X->Checksum)
> +    return false;
>
>    // Compare data
>    if (getContents() != X->getContents())
>
> Modified: lld/trunk/COFF/InputFiles.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/lld/trunk/COFF/InputFiles.cpp?rev=246869&r1=246868&r2=246869&view=diff
>
> ==============================================================================
> --- lld/trunk/COFF/InputFiles.cpp (original)
> +++ lld/trunk/COFF/InputFiles.cpp Fri Sep  4 15:45:50 2015
> @@ -225,13 +225,14 @@ Defined *ObjectFile::createDefined(COFFS
>    if (!SC)
>      return nullptr;
>
> -  // Handle associative sections
> +  // Handle section definitions
>    if (IsFirst && AuxP) {
>      auto *Aux = reinterpret_cast<const coff_aux_section_definition
> *>(AuxP);
>      if (Aux->Selection == IMAGE_COMDAT_SELECT_ASSOCIATIVE)
>        if (auto *ParentSC = cast_or_null<SectionChunk>(
>                SparseChunks[Aux->getNumber(Sym.isBigObj())]))
>          ParentSC->addAssociative(SC);
> +    SC->Checksum = Aux->CheckSum;
>    }
>
>    auto *B = new (Alloc) DefinedRegular(this, Sym, SC);
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20150916/d733b51a/attachment.html>