[PATCH] D22438: [llvm-profdata] Speed up merging by using a thread pool

[David Li via llvm-commits]

davidxl added a comment.

There isn't any reduction of the number of merges compared with serial case. Say there are T threads, the total number of merges is

T * f(N/T) + T - 1, so f(N) is still N-1.

I suspect a much larger speed up can be obtained by unifying symbol table creation across profiles (that are generated from the same binary -- as their symtab is identical). I was planning doing that but did not find time. If you can help with that, that will be a great win.

David


================
Comment at: tools/llvm-profdata/llvm-profdata.cpp:138
@@ +137,3 @@
+  unsigned NumUsefulThreads =
+      std::max(1U, std::min(std::thread::hardware_concurrency(),
+                            unsigned(std::sqrt(Inputs.size()))));
----------------
This does not look right. Just make it a controllable option (default with 2 or 4 etc).

================
Comment at: tools/llvm-profdata/llvm-profdata.cpp:143
@@ +142,3 @@
+  // an Error object (for deferred error reporting).
+  struct WriterContext {
+    std::mutex Lock;
----------------
Move struct definition out of the function.

================
Comment at: tools/llvm-profdata/llvm-profdata.cpp:161
@@ +160,3 @@
+
+  auto mergeInput = [&ErrorLock, &WriterErrorCodes](const WeightedFile &Input,
+                                                    WriterContext *WC) {
----------------
Make this a regular function and move it out of line.


https://reviews.llvm.org/D22438