[PATCH] D118550: [Support] Have ThreadPool initialize a TimeTraceProfiler per thread

[Russell Gallop via Phabricator via llvm-commits]

russell.gallop added inline comments.


================
Comment at: llvm/lib/Support/ThreadPool.cpp:37
   int newThreadCount = std::min<int>(requested, MaxThreadCount);
+  TimeTraceProfiler *MainThreadProfiler = getTimeTraceProfilerInstance();
   while (static_cast<int>(Threads.size()) < newThreadCount) {
----------------
int3 wrote:
> int3 wrote:
> > mehdi_amini wrote:
> > > russell.gallop wrote:
> > > > mehdi_amini wrote:
> > > > > It seems that we'll always use the instance initialized in the thread that calls "grow".  Also, this instance has to be setup before the call to grow, and the thread can't reinitialize it for the lifetime duration of the ThreadPool if I understand correctly.
> > > > > 
> > > > > I'm not sure this makes sense in the full generality of the ThreadPool API?
> > > > > I'm not sure this makes sense in the full generality of the ThreadPool API?
> > > > 
> > > > Do we agree that time tracing all threads used by the ThreadPool is desirable and worth pursuing?
> > > This could be a useful feature to have a tracing feature for the ThreadPool. I'm not sure about:
> > > - The expected behavior in terms of threading (with respect to the thread pool creation, the growing of the pool, or the enqueuing of a task).
> > > - The current `TimeTraceProfiler` which is non-trivially coupled to some global state, making this all harder to reason about.
> > > It seems that we'll always use the instance initialized in the thread that calls "grow".
> > 
> > Actually we are just copying the value of its TimeTraceGranularity member and using that to initialize a new thread-local instance. I could have just copied the TimeTraceGranularity value itself, but I figured this was a slightly nicer abstraction -- if in the future we add more fields to TimeTraceProfilerInstance, we can keep the same initialization method signature.
> > 
> > You are right that this makes things a little less general though. In particular, there is no way to have different granularities per profiler instance -- every thread must use the same value. IDK if that's an issue... after all, the places where the TimeProfiler is getting used ATM don't take advantage of this flexibility.
> > 
> > > the thread can't reinitialize it for the lifetime duration of the ThreadPool if I understand correctly
> > 
> > I don't think there's a use case for reinitializing it...
> > 
> > That said I didn't write the TimeProfiler, so maybe @russell.gallop can confirm.
> bump -- @russell.gallop, could you chime in here?
> You are right that this makes things a little less general though. In particular, there is no way to have different granularities per profiler instance -- every thread must use the same value. IDK if that's an issue... after all, the places where the TimeProfiler is getting used ATM don't take advantage of this flexibility.

Yes, I don't imagine it is very useful to have different granularities.

> I don't think there's a use case for reinitializing it...
> That said I didn't write the TimeProfiler, so maybe @russell.gallop can confirm.

I'm not sure I follow the use of the Thread Pool where this would be required...

I think @anton-afanasyev added the time profiler, I extended for LLD tracing, but only really with the ThinLTO threading case in mind. I imagine that the profiler could be re-engineered to meet what the thread pool can do, but I'm not really aware of what the "full generality of the thread pool" is so can't really say more than that.

I don't think I'll have time to do this myself. Let me know if you don't have time and I can ask if there is someone around here who can take a look at this.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D118550/new/

https://reviews.llvm.org/D118550