[llvm-dev] Multi-Threading Compilers

[Chris Lattner via llvm-dev]

On Mar 3, 2020, at 4:12 AM, Fedor Sergeev via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> Oops.. sorry, hit send too soon..
>> I can't say anything about the GCC side, but this isn't a particularly novel aspect of the MLIR pass manager. In many ways, the pass manager is the easiest/simplest part of the multi-threading problem. The bigger problem is making sure that the rest of the compiler infrastructure is structured in a way that is thread-safe, or can be made thread-safe. This is why most of the discussion is based around how to model things like constants, global values, etc. When I made MLIR multi-threaded a year ago, a large majority of my time was spent outside of the pass manager. For a real example, I spent much more time just on multi-threaded pass timing <https://mlir.llvm.org/docs/WritingAPass/#multi-threaded-pass-timing> than making the pass manager itself multi-threaded.
> Picking on this point, perhaps a bit off-topic for the whole discussion.
> 
> I have recently realized that for the purpose of multi-threaded pass timing currently existing LLVM timers
> are hardly suitable since they measure per-process time instead of per-thread time.
> (and there seem to be no portable LLVM interfaces for per-thread time query :( )
> 
> From a first glance it seems that in your MLIR timing examples all the times are also per-process.
> How do you handle cases when half of your threads are doing something else?
> And if you handle it per-thread - can you point me to the code doing that, pls :)

Indeed, how you account for things is important.  Please take a look at "Multi-threaded Pass Timing”  on this page:
https://mlir.llvm.org/docs/WritingAPass/ <https://mlir.jackwish.net/writingapass#pass-manager>

-Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200303/f09698b8/attachment.html>