[PATCH] D102888: [PDB] Enable parallel ghash type merging by default

[Reid Kleckner via Phabricator via llvm-commits]

rnk created this revision.
rnk added reviewers: akhuang, aganea.
rnk requested review of this revision.
Herald added a project: LLVM.

Ghashing is probably going to be faster in most cases, even without
precomputed ghashes in object files.

Here is my table of results linking clang.pdb:

-------------------------------

| threads | GHASH   | NOGHASH |
-------------------------------

| j1  | 51.031s | 25.141s |
| j2  | 31.079s | 22.109s |
| j4  | 18.609s | 23.156s |
| j8  | 11.938s | 21.984s |
| j28 | 8.375s  | 18.391s |
|

-------------------------------

This shows that ghashing is faster if at least four cores are available.
This may make the linker slower if most cores are busy in the middle of
a build, but in that case, the linker probably isn't on the critical
path of the build. Incremental build performance is arguably more
important than highly contended batch build link performance.

The -time output indicates that ghash computation is the dominant
factor:

    Input File Reading:             924 ms (  1.8%)
    GC:                             689 ms (  1.3%)
    ICF:                            527 ms (  1.0%)
    Code Layout:                    414 ms (  0.8%)
    Commit Output File:              24 ms (  0.0%)
    PDB Emission (Cumulative):    49938 ms ( 94.8%)
      Add Objects:                46783 ms ( 88.8%)
        Global Type Hashing:      38983 ms ( 74.0%)
        GHash Type Merging:        5640 ms ( 10.7%)
        Symbol Merging:            2154 ms (  4.1%)
      Publics Stream Layout:        188 ms (  0.4%)
      TPI Stream Layout:             18 ms (  0.0%)
      Commit to Disk:              2818 ms (  5.4%)
  --------------------------------------------------
  Total Link Time:                52669 ms (100.0%)

We can speed that up with a faster content hash (not SHA1).

Depends on D102885 <https://reviews.llvm.org/D102885>


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D102888

Files:
  lld/COFF/Driver.cpp
  lld/test/COFF/pdb-type-server-simple.test


Index: lld/test/COFF/pdb-type-server-simple.test
===================================================================
--- lld/test/COFF/pdb-type-server-simple.test
+++ lld/test/COFF/pdb-type-server-simple.test
@@ -20,7 +20,7 @@
 RUN: yaml2obj %S/Inputs/pdb-type-server-simple-a.yaml -o a.obj
 RUN: yaml2obj %S/Inputs/pdb-type-server-simple-b.yaml -o b.obj
 RUN: llvm-pdbutil yaml2pdb %S/Inputs/pdb-type-server-simple-ts.yaml -pdb ts.pdb
-RUN: lld-link a.obj b.obj -entry:main -debug -out:t.exe -pdb:t.pdb -nodefaultlib -summary | FileCheck %s -check-prefix SUMMARY
+RUN: lld-link a.obj b.obj -entry:main -debug:ghash- -out:t.exe -pdb:t.pdb -nodefaultlib -summary | FileCheck %s -check-prefix SUMMARY
 RUN: llvm-pdbutil dump -symbols -types -ids -globals %t/t.pdb | FileCheck %s
 
 Re-run with /DEBUG:GHASH
Index: lld/COFF/Driver.cpp
===================================================================
--- lld/COFF/Driver.cpp
+++ lld/COFF/Driver.cpp
@@ -693,7 +693,16 @@
   return std::string(data.str());
 }
 
-enum class DebugKind { Unknown, None, Full, FastLink, GHash, Dwarf, Symtab };
+enum class DebugKind {
+  Unknown,
+  None,
+  Full,
+  FastLink,
+  GHash,
+  NoGHash,
+  Dwarf,
+  Symtab
+};
 
 static DebugKind parseDebugKind(const opt::InputArgList &args) {
   auto *a = args.getLastArg(OPT_debug, OPT_debug_opt);
@@ -703,14 +712,15 @@
     return DebugKind::Full;
 
   DebugKind debug = StringSwitch<DebugKind>(a->getValue())
-                     .CaseLower("none", DebugKind::None)
-                     .CaseLower("full", DebugKind::Full)
-                     .CaseLower("fastlink", DebugKind::FastLink)
-                     // LLD extensions
-                     .CaseLower("ghash", DebugKind::GHash)
-                     .CaseLower("dwarf", DebugKind::Dwarf)
-                     .CaseLower("symtab", DebugKind::Symtab)
-                     .Default(DebugKind::Unknown);
+                        .CaseLower("none", DebugKind::None)
+                        .CaseLower("full", DebugKind::Full)
+                        .CaseLower("fastlink", DebugKind::FastLink)
+                        // LLD extensions
+                        .CaseLower("ghash", DebugKind::GHash)
+                        .CaseLower("ghash-", DebugKind::NoGHash)
+                        .CaseLower("dwarf", DebugKind::Dwarf)
+                        .CaseLower("symtab", DebugKind::Symtab)
+                        .Default(DebugKind::Unknown);
 
   if (debug == DebugKind::FastLink) {
     warn("/debug:fastlink unsupported; using /debug:full");
@@ -1392,7 +1402,7 @@
   // Handle /debug
   DebugKind debug = parseDebugKind(args);
   if (debug == DebugKind::Full || debug == DebugKind::Dwarf ||
-      debug == DebugKind::GHash) {
+      debug == DebugKind::GHash || debug == DebugKind::NoGHash) {
     config->debug = true;
     config->incremental = true;
   }
@@ -1415,7 +1425,8 @@
 
   // Handle /pdb
   bool shouldCreatePDB =
-      (debug == DebugKind::Full || debug == DebugKind::GHash);
+      (debug == DebugKind::Full || debug == DebugKind::GHash ||
+       debug == DebugKind::NoGHash);
   if (shouldCreatePDB) {
     if (auto *arg = args.getLastArg(OPT_pdb))
       config->pdbPath = arg->getValue();
@@ -1748,7 +1759,7 @@
   config->terminalServerAware =
       !config->dll && args.hasFlag(OPT_tsaware, OPT_tsaware_no, true);
   config->debugDwarf = debug == DebugKind::Dwarf;
-  config->debugGHashes = debug == DebugKind::GHash;
+  config->debugGHashes = debug == DebugKind::GHash || debug == DebugKind::Full;
   config->debugSymtab = debug == DebugKind::Symtab;
   config->autoImport =
       args.hasFlag(OPT_auto_import, OPT_auto_import_no, config->mingw);


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D102888.346888.patch
Type: text/x-patch
Size: 3671 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210520/c2830719/attachment.bin>