[llvm-dev] Multi-Threading Compilers

[David Blaikie via llvm-dev]

+Chandler & Alina for new pass manager context

On Thu, Feb 27, 2020 at 9:45 PM Nicholas Krause via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

>
>
> On 2/28/20 12:19 AM, Chris Lattner wrote:
> > Hi Nicholas,
> >
> > You might want to check out MLIR: its pass manager is already
> automatically and implicitly multithreaded.
> >
> > -Chris
> Chris,
>
> I was aware that LLVM was moving to MLIR at some point due to this.


FWIW: I don't /tihnk/ that's quite an accurate representation of the
situation. MLIR is now part of the LLVM project, but so far I haven't seen
any discussions of moving the core LLVM project itself to MLIR (admittedly
I'm not super clear on the details of MLIR or what that'd look like - I
would've guessed that LLVM would itself be a lower level that a given MLIR
stack might lower to, etc). There's been some discussion of folks
interested in experimenting with using MLIR in Clang, though that's not a
project goal right now.


> I've
> curious as
> to how MLIR deals with IPO as that's the problem I was running into.
>

FWIW I believe LLVM's new pass manager (NPM) was designed with parallelism
and the ability to support this situation (that MLIR doesn't? Or doesn't to
the degree/way in which the NPM does). I'll leave it to folks (Chandler
probably has the most context here) to provide some more detail there if
they can/have time.


> Even if you have
> pipelines what guarantees do we have about IPO or are there any. For
> example if an
> analysis pass runs after a loop pass for heuristics to get good data,
> does MLIR ensure
> that?   The problem isn't getting a pipeline as much as IPO issues in
> terms of rescheduling
> in a pipeline or incorrectly splitting up into pipelines. I yet to find
> a good solution on the
> GCC side as well and it seems that there will be issues with this no
> matter what, not
> sure of what trade offs the LLVM side is willing to make.
>
> The other issues is that graph data structures to my knowledge do not
> allow insertion
> of multiple nodes at the same time or how to scale the graphs for
> callers or SSA. Not
> sure if you guys have encapsulated the operators and data structures in
> a series of
> classes as that would be the first part. The other part is figuring out
> how to link and
> interact with build systems to launch threads from make -j or other
> similar things.
>
>
> Thanks for the notice about MLIR through maybe my IPO is not really there
> but after reading parts of it seems to be a issue through a little
> smaller still
> and thanks for the  prompt response,
>
> Nick
> >> On Feb 27, 2020, at 7:05 PM, Nicholas Krause via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >>
> >> Greetings All,
> >>
> >> For the last few months since the GCC Cauldron I've been
> researching/helping out
> >> plan for multi-threading GCC. I've posted my GCC ideas here:
> >>
> https://docs.google.com/document/d/1po_RRgSCtRyYgMHjV0itW8iOzJXpTdHYIpC9gUMjOxk/edit
> >>
> >> as I've heard there is interest on the LLVM side as well. I've talked
> to some of the IBM folks from
> >> the Toronto area about it face to face due to them having more middle
> end and back end knowledge
> >> then me.
> >>
> >> Not sure if anyone is interested in my ideas or wants me to extend my
> help on the LLVM side,
> >>
> >> Nick
> >> _______________________________________________
> >> LLVM Developers mailing list
> >> llvm-dev at lists.llvm.org
> >> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200229/63f03ca7/attachment.html>