[PATCH] D22438: [llvm-profdata] Speed up merging by using a thread pool

[Vedant Kumar via llvm-commits]

vsk created this revision.
vsk added reviewers: davidxl, silvas.
vsk added a subscriber: llvm-commits.

The amount of time required to merge N profiles is a function of the number of
serial merge operations performed. Using more than one thread to perform merges
may reduce the number of serial merges needed. When this is the case, find the
optimal number of merge threads and use a thread pool to do the job [1]. When
it isn't the case, use the regular code path and avoid spawning any threads.

I tested this patch using a raw profile produced by clang (147MB). Here is the
time taken to merge 4 copies together on my laptop:

  | build | user | system | cpu | total |
  | No thread pool | 112.87s | 5.92s | 97% | 2:01.08 |
  | With 2 threads | 134.99s | 26.54s | 164% | 1:33.31 |

[1] Using T = sqrt(N) threads minimizes the number of merges M = T + N/T, since
dM/dT = 1 - N/T^2 and d^2M/dT^2 > 0.

https://reviews.llvm.org/D22438

Files:
  include/llvm/ProfileData/InstrProfWriter.h
  lib/ProfileData/InstrProfWriter.cpp
  test/tools/llvm-profdata/multiple-inputs.test
  tools/llvm-profdata/llvm-profdata.cpp
  unittests/ProfileData/InstrProfTest.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D22438.64229.patch
Type: text/x-patch
Size: 7676 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160716/62595332/attachment.bin>