[PATCH] D55585: RFC: [LLD][COFF] Parallel GHASH generation at link-time

Mon Jan 14 14:02:57 PST 2019

aganea added a comment.

Just for reference, added timings for `/hasher:(xxhash|crc)` in the comment above.

`xxHash` is using `llvm::xxHash()`.
`crc` is a naive implementation with two concatenated crc32 values, using the SSE4.2 intrinsic `_mm_crc32_u32` - thus using only two instructions per 4 bytes. However this creates about 0.0001% hash collisions, which would not be suitable for this.

Also providing timings for a 2.2 GHz, 36-cores Intel Xeon Skylake:

Dataset 2, large EXE, the resulting PDB is 2 GB:

| (trunk) lld-link ...                             |                                      | Type Merging: 34582 ms ( 52.7%) | Total Link Time: 65572 ms (100.0%) |
| (this patch) lld-link ... /hasher:sha1 (default) | Global hashing: **6910** ms ( 14.8%) | Type Merging: 12573 ms ( 26.9%) | Total Link Time: 46768 ms          |
| (this patch) lld-link ... /hasher:md5            | Global hashing: **5785** ms ( 12.6%) | Type Merging: 12479 ms ( 27.2%) | Total Link Time: 45902 ms          |
| (this patch) lld-link ... /hasher:cityhash       | Global hashing: **5153** ms ( 11.5%) | Type Merging: 12008 ms ( 26.8%) | Total Link Time: 44728 ms          |
| (this patch) lld-link ... /hasher:xxhash         | Global hashing: **5041** ms ( 10.9%) | Type Merging: 11917 ms ( 25.9%) | Total Link Time: 46057 ms          |
| (this patch) lld-link ... /hasher:crc            | Global hashing: **4994** ms ( 11.1%) | Type Merging: 12089 ms ( 26.8%) | Total Link Time: 45076 ms          |
|

Even though the global hashing is faster than the 6-cores, the single-threaded parts drag timings down because of the slower CPU frequency.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D55585/new/

https://reviews.llvm.org/D55585