[llvm-dev] Multi-Threading Compilers

[Nicholas Krause via llvm-dev]

On 2/28/20 12:56 AM, Chris Lattner wrote:
> On Feb 27, 2020, at 9:44 PM, Nicholas Krause <xerofoify at gmail.com 
> <mailto:xerofoify at gmail.com>> wrote:
>> On 2/28/20 12:19 AM, Chris Lattner wrote:
>>> Hi Nicholas,
>>>
>>> You might want to check out MLIR: its pass manager is already 
>>> automatically and implicitly multithreaded.
>>>
>>> -Chris
>> Chris,
>>
>> I was aware that LLVM was moving to MLIR at some point due to this. 
>> I've curious as
>> to how MLIR deals with IPO as that's the problem I was running into. 
>> Even if you have
>> pipelines what guarantees do we have about IPO or are there any. For 
>> example if an
>> analysis pass runs after a loop pass for heuristics to get good 
>> data,  does MLIR ensure
>> that?   The problem isn't getting a pipeline as much as IPO issues in 
>> terms of rescheduling
>> in a pipeline or incorrectly splitting up into pipelines. I yet to 
>> find a good solution on the
>> GCC side as well and it seems that there will be issues with this no 
>> matter what, not
>> sure of what trade offs the LLVM side is willing to make.
>
> Hi Nicholas,
>
> It is very difficult to mix things like loop passes and IPO passes in 
> any system, unless one is prepared to preserve the analysis results 
> that the other depend on.  One nice thing about MLIR is that it 
> defines this away, by allowing explicit representation of loops in the 
> IR.  This means that it isn’t an analysis pass that gets “invalidated” 
> like the existing LLVM LoopInfo analysis pass.  It is just 
> structurally impossible for this to happen.  I don’t think that all of 
> the AnalysisPass’s in LLVM have been super successful.
>
>> The other issues is that graph data structures to my knowledge do not 
>> allow insertion
>> of multiple nodes at the same time or how to scale the graphs for 
>> callers or SSA. Not
>> sure if you guys have encapsulated the operators and data structures 
>> in a series of
>> classes as that would be the first part. The other part is figuring 
>> out how to link and
>> interact with build systems to launch threads from make -j or other 
>> similar things.
>
> Yeah, MLIR handles that by factoring global use-def chains on symbols 
> (e.g. functions) as being different than local use-def chains. This 
> makes it much more efficient. You can find more information on MLIR 
> symbols here <https://mlir.llvm.org/docs/SymbolsAndSymbolTables/>.
>
>> Thanks for the notice about MLIR through maybe my IPO is not really there
>> but after reading parts of it seems to be a issue through a little 
>> smaller still
>> and thanks for the  prompt response,
>
> Sure, happy to help.  It would be great to see a GIMPLE dialect in 
> MLIR someday :-)
>
> -Chris
Chris,

I asked the GCC what they want to do about MLIR and am waiting for a 
reply. Anyhow what is the status and
what parts are we planning to move to MLIR in LLVM/Clang.  I've not seen 
any discussion on that other than
starting to plan for it.  Perhaps we should start a wiki page for that 
as I don't think all parts need to be MLIR.
I don't have access to writing for the wiki so unfortunately I can't 
start writing it up unless I get access.

Regards and this is one reason you do your research before writing a 
multi-threading program a lot of the
research or work has been done at this point,
Nick
>
>>
>> Nick
>>>> On Feb 27, 2020, at 7:05 PM, Nicholas Krause via llvm-dev 
>>>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>>>
>>>> Greetings All,
>>>>
>>>> For the last few months since the GCC Cauldron I've been 
>>>> researching/helping out
>>>> plan for multi-threading GCC. I've posted my GCC ideas here:
>>>> https://docs.google.com/document/d/1po_RRgSCtRyYgMHjV0itW8iOzJXpTdHYIpC9gUMjOxk/edit
>>>>
>>>> as I've heard there is interest on the LLVM side as well. I've 
>>>> talked to some of the IBM folks from
>>>> the Toronto area about it face to face due to them having more 
>>>> middle end and back end knowledge
>>>> then me.
>>>>
>>>> Not sure if anyone is interested in my ideas or wants me to extend 
>>>> my help on the LLVM side,
>>>>
>>>> Nick
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200228/a50b3cb3/attachment.html>