[llvm-dev] Multi-Threading Compilers

[Johannes Doerfert via llvm-dev]

On 3/26/20 6:02 PM, Chris Lattner via llvm-dev wrote:
 >
 >
 >> On Mar 26, 2020, at 8:26 AM, Doerfert, Johannes <jdoerfert at anl.gov> 
wrote:
 >>
 >> On 3/26/20 5:53 AM, Florian Hahn wrote:
 >>>> It also doesn't solve the problem of Functions themselves -- those are
 >>>> also GlobalValues…
 >>>
 >>>
 >>> I am not sure why not. Function passes should only rely on the 
information at the callsite & from the declaration IIRC. For functions, 
we coulkd add extra declarations and update the call sites. But I might 
be missing something.
 >>
 >> FWIW, I think deleting and creating functions (or globals in general)
 >> should be hidden behind an API that would allow us to synchronize 
things
 >> in parallel runs.
 >>
 >> I don't believe there are many passes that would be affected and we
 >> could "easily" make this happen. Hide the constructors and destructors
 >> so that only the friend API can access it.
 >
 > Right, this is what MLIR does.
 > However, keep in mind that this is just one piece of the puzzle.

I am aware.


 > You also have things like ValueHandles which get owned by IPA passes.

Why are they more problematic?


 > The CallGraph used by the inline persists across function pass manager
 > runs, and needs to be correctly updatable in order to run CallGraphSCC
 > passes.

Sure. We make everyone use a single `CallGraphUpdater` in which we can
synchronize appropriately. Again, the argument is the same as with
Global creation/modification/deletion, it does not happen much in most
passes.


 > Getting to a multithread clean optimizer is not one bug fix away -
 > this is a multiyear journey, and I would prefer not to see big changes
 > with unclear tradeoffs made in an effort to get a quick win.

I think I missed the suggestion of "big changes with unclear tradeoffs
made in an effort to get a quick win". So far I thought this was a
discussion of ideas, methods to gather data, and potential pitfalls.


 > Instead, let’s break down the problems and fix them (the right way!)
 > one at a time.  For example, it seems that the thread agrees that the
 > overall design of constants are a problem: let's talk about
 > (incrementally!) moving constants to the right architecture. There
 > are good suggestions about this upthread.

I'm not sure why you think anyone so far suggested anything to "fix it
all" or to do it "not-incrementally". For example, I think we are
talking about the design of constants and how it can be incrementally
improved (for different purposes). I also think there are interesting
suggestions upthread but I'm not sure sure there was any kind of
agreement on "the right architecture".

Cheers,
   Johannes

 > -Chris
 >
 > _______________________________________________
 > LLVM Developers mailing list
 > llvm-dev at lists.llvm.org
 > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev