[Lldb-commits] [PATCH] D32757: Add TaskMap for iterating a function over a set of integers

Scott Smith via Phabricator via lldb-commits lldb-commits at lists.llvm.org
Tue May 2 11:44:48 PDT 2017

scott.smith added a comment.

I can make more measurements on this.

Comment at: source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp:1994
-    TaskRunner<uint32_t> task_runner;
-    for (uint32_t cu_idx = 0; cu_idx < num_compile_units; ++cu_idx)
-      task_runner.AddTask(parser_fn, cu_idx);
-    while (true) {
-      std::future<uint32_t> f = task_runner.WaitForNextCompletedTask();
-      if (!f.valid())
-        break;
-      uint32_t cu_idx = f.get();
+    TaskMap(num_compile_units, 1, parser_fn);
clayborg wrote:
> Note that we lost performance here. The old code would run:
> ```
> while (true) {
>     std::future<uint32_t> f = task_runner.WaitForNextCompletedTask();
>     // Do expensive work as soon as possible with any random task that completes as soon as it completes.
> ```
> Your current code says "wait to do all expensive work until I complete everything and then do all of the expensive work".
The following loop is not expensive, it's simple vector concatenation of fairly simple types.  The actual slow work is Finalize(), which calls Sort().

m_function_basename_index is of type NameDIE, which has a UniqueCStringMap member, which declared collection as std::vector.

Though arguably the Append should be pushed down into the RunTasks below.

Comment at: source/Utility/TaskPool.cpp:100
+  for (size_t i = 0; i < est_batches; i++)
+    futures[i].wait();
clayborg wrote:
> TaskRunner is smart in the way it runs things: it lets you run N things and get each item as it completes. This implementation, if read it correctly, will serialize everything. So if you have 100 items to do and item at index 0 takes 200 seconds to complete, and items 1 - 99 take 1ms to complete, you will need to wait for task 0 to complete before you can start parsing the data. This will slow down the DWARF parser if you switch over to this.
If items 1-99 complete quickly, there isn't much advantage to handling their output before handling the output of item 0, since item 0 is likely to produce more output than 1-99 combined.



More information about the lldb-commits mailing list