[lld] r349431 - [codeview] Align symbol records to save 441MB during linking clang.pdb

[Reid Kleckner via llvm-commits]

Author: rnk
Date: Mon Dec 17 17:14:05 2018
New Revision: 349431

URL: http://llvm.org/viewvc/llvm-project?rev=349431&view=rev
Log:
[codeview] Align symbol records to save 441MB during linking clang.pdb

In PDBs, symbol records must be aligned to four bytes. However, in the
object file, symbol records may not be aligned. MSVC does not pad out
symbol records to make sure they are aligned. That means the linker has
to do extra work to insert the padding. Currently, LLD calculates the
required space with alignment, and copies each record one at a time
while padding them out to the correct size. It has a fast path that
avoids this copy when the records are already aligned.

This change fixes a bug in that codepath so that the copy is actually
saved, and tweaks LLVM's symbol record emission to align symbol records.
Here's how things compare when doing a plain clang Release+PDB build:
- objs are 0.65% bigger (negligible)
- link is 3.3% faster (negligible)
- saves allocating 441MB
- new LLD high water mark is ~1.05GB

Modified:
    lld/trunk/COFF/PDB.cpp

Modified: lld/trunk/COFF/PDB.cpp
URL: http://llvm.org/viewvc/llvm-project/lld/trunk/COFF/PDB.cpp?rev=349431&r1=349430&r2=349431&view=diff
==============================================================================
--- lld/trunk/COFF/PDB.cpp (original)
+++ lld/trunk/COFF/PDB.cpp Mon Dec 17 17:14:05 2018
@@ -1014,11 +1014,13 @@ void PDBLinker::mergeSymbolRecords(ObjFi
   // Iterate every symbol to check if any need to be realigned, and if so, how
   // much space we need to allocate for them.
   bool NeedsRealignment = false;
-  unsigned RealignedSize = 0;
+  unsigned TotalRealignedSize = 0;
   auto EC = forEachCodeViewRecord<CVSymbol>(
       SymsBuffer, [&](CVSymbol Sym) -> llvm::Error {
-        RealignedSize += alignTo(Sym.length(), alignOf(CodeViewContainer::Pdb));
+        unsigned RealignedSize =
+            alignTo(Sym.length(), alignOf(CodeViewContainer::Pdb));
         NeedsRealignment |= RealignedSize != Sym.length();
+        TotalRealignedSize += RealignedSize;
         return Error::success();
       });
 
@@ -1036,9 +1038,9 @@ void PDBLinker::mergeSymbolRecords(ObjFi
   MutableArrayRef<uint8_t> AlignedSymbolMem;
   if (NeedsRealignment) {
     void *AlignedData =
-        Alloc.Allocate(RealignedSize, alignOf(CodeViewContainer::Pdb));
+        Alloc.Allocate(TotalRealignedSize, alignOf(CodeViewContainer::Pdb));
     AlignedSymbolMem = makeMutableArrayRef(
-        reinterpret_cast<uint8_t *>(AlignedData), RealignedSize);
+        reinterpret_cast<uint8_t *>(AlignedData), TotalRealignedSize);
   }
 
   // Iterate again, this time doing the real work.