[PATCH] D114845: [llvm] [DebugInfo] DebuginfodCollection and DebuginfodServer for tracking local debuginfo. (WIP)

[Noah Shutty via Phabricator via llvm-commits]

noajshu added inline comments.


================
Comment at: llvm/lib/Debuginfod/Debuginfod.cpp:228
+  std::error_code ec;
+  for (sys::fs::recursive_directory_iterator i(Twine(Path), ec), e; i != e;
+       i.increment(ec)) {
----------------
phosek wrote:
> I expect this loop to be the performance bottleneck, so I think we should consider using a `ThreadPool` here as well to process files in parallel. That's the strategy used by elfutils' debuginfod as well.
> 
> This would likely require rethinking the API to allow sharing the thread pool between the file processing and request handling in D114846.
Good idea!
I've added this, but I find the behavior unstable when the concurrency is >1 and a large directory is scanned.
I've searched around for the bug in my code and no luck. I'm wondering if it's possible some of the supporting libraries I'm using are not thread-safe or are leaking memory. But it's more likely I'm missing something obvious. Anyways, going to upload as it's WIP and maybe there is a better task queue architecture you could recommend.
PS: the `TaskQueue` in LLVM is not appropriate as it is for serialized sequence of tasks.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D114845/new/

https://reviews.llvm.org/D114845