[Lldb-commits] [PATCH] D32757: Add TaskMap for iterating a function over a set of integers

[Scott Smith via Phabricator via lldb-commits]

scott.smith added a comment.

I can make more measurements on this.



================
Comment at: source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp:1994
 
-    TaskRunner<uint32_t> task_runner;
-    for (uint32_t cu_idx = 0; cu_idx < num_compile_units; ++cu_idx)
-      task_runner.AddTask(parser_fn, cu_idx);
-
-    while (true) {
-      std::future<uint32_t> f = task_runner.WaitForNextCompletedTask();
-      if (!f.valid())
-        break;
-      uint32_t cu_idx = f.get();
+    TaskMap(num_compile_units, 1, parser_fn);
 
----------------
clayborg wrote:
> Note that we lost performance here. The old code would run:
> 
> ```
> while (true) {
>     std::future<uint32_t> f = task_runner.WaitForNextCompletedTask();
>     // Do expensive work as soon as possible with any random task that completes as soon as it completes.
> ```
> 
> Your current code says "wait to do all expensive work until I complete everything and then do all of the expensive work".
> 
> 
> 
> 
The following loop is not expensive, it's simple vector concatenation of fairly simple types.  The actual slow work is Finalize(), which calls Sort().

m_function_basename_index is of type NameDIE, which has a UniqueCStringMap member, which declared collection as std::vector.

Though arguably the Append should be pushed down into the RunTasks below.



================
Comment at: source/Utility/TaskPool.cpp:100
+  for (size_t i = 0; i < est_batches; i++)
+    futures[i].wait();
+}
----------------
clayborg wrote:
> TaskRunner is smart in the way it runs things: it lets you run N things and get each item as it completes. This implementation, if read it correctly, will serialize everything. So if you have 100 items to do and item at index 0 takes 200 seconds to complete, and items 1 - 99 take 1ms to complete, you will need to wait for task 0 to complete before you can start parsing the data. This will slow down the DWARF parser if you switch over to this.
If items 1-99 complete quickly, there isn't much advantage to handling their output before handling the output of item 0, since item 0 is likely to produce more output than 1-99 combined.


Repository:
  rL LLVM

https://reviews.llvm.org/D32757