[llvm-dev] Legacy PM deprecation for optimization pipeline timeline

Sun May 9 11:12:40 PDT 2021

Submitted https://reviews.llvm.org/D102136 for review. There are a few
things I'm unsure of which I've pointed out in the review.

Looking forward to hearing from you!

Mats

On Tue, May 4, 2021 at 7:46 PM Arthur Eubanks <aeubanks at google.com> wrote:

>
>
> On Tue, May 4, 2021 at 10:41 AM Mats Larsen <me at supergrecko.com> wrote:
>
>> > No, it's more of an implementation detail that users don't need to
>> worry about.
>> I see, I think we'll omit that parameter from the AfterPassFunc and
>> AfterPassInvalidatedFunc callbacks then.
>>
>> > I'm not following. Isn't it like all the other things you're passing
>> into the PassBuilder?
>> Apologies, that was pretty terribly worded. The
>> PassInstrumentationCallbacks holds all of these "  using BeforePassFunc =
>> bool(StringRef, Any);" function types, where the llvm::Any is the IR unit
>> the pass ran over. It could be a llvm::Module, llvm::Function etc...
>> Wouldn't this be "problematic" for the C interface as we wouldn't be able
>> to tell what kind of IR unit it was?
>>
> We wouldn't be exposing this to the C API right? It's just another thing
> to setup right before we create the PassBuilder.
>
>>
>> > Separating it out into a different file from PassBuilder.cpp seems
>> cleanest to me. Maybe right along side it in the same directory? We can
>> always move it around later if we find somewhere better.
>> That sounds good to me. /llvm/lib/Passes/PassBuilderBindings.cpp maybe?
>>
> Sounds good
>
>>
>> We should probably do something like heap allocate a PTO/PGOOptions and
>> provide an opaque pointer to it, and set those options, then pass it to the
>> final build/run function. Sort of like LLVMPassManagerRef, but the legacy
>> pass builder stored its options inside the pass builder itself, whereas the
>> new pass builder has a separate PTO. Then the equivalent of something like
>> LLVMPassManagerBuilderSetDisableUnrollLoops() would just modify the PTO.
>> > Yeah I think that's a nice approach. I believe that's what the MCJIT
>> bindings do with the LLVMMCJITCompilerOptions struct.
>>
>> > The legacy PM API separates building the PM and running it on some IR.
>> I'm not sure if we necessarily want to do that when we can build it and run
>> it all at once. But maybe I'm missing a use case.
>> I also thought about that... I think either approach works, and we can
>> always add an API that separates it out or vice versa further down the road
>> if need be.
>>
>> Let me know what you think. I'll start working on the API and submit a
>> patch for review soon.
>>
> SGTM, and again, thanks!
>
>>
>> Mats
>>
>> On Mon, May 3, 2021 at 7:05 PM Arthur Eubanks <aeubanks at google.com>
>> wrote:
>>
>>>
>>>
>>> On Sat, May 1, 2021 at 12:11 PM Mats Larsen <me at supergrecko.com> wrote:
>>>
>>>> Greetings, I've had the time to take a better look at the code in the
>>>> NewPMDriver.cpp source and I believe I have a good overview of the task
>>>> now. Here are some things which I'm not too sure about so I'd like your
>>>> input on this:
>>>>
>>>> - What are the PreservedAnalyses and how do we use them? Is it crucial
>>>> for the LLVM-C API?
>>>>
>>> No, it's more of an implementation detail that users don't need to worry
>>> about.
>>>
>>>> - How do we want to handle the various callbacks for the
>>>> PassInstrumentationCallbacks? They're taking the templated IRUnit which
>>>> could be different types upon each invocation.
>>>>
>>> I'm not following. Isn't it like all the other things you're passing
>>> into the PassBuilder?
>>>
>>>> - Where do we want the binding code itself? Some places have files like
>>>> ExecutionEngineBindings.cpp, while others have it at the end of files
>>>> relevant to the code being hooked into.
>>>>
>>> Separating it out into a different file from PassBuilder.cpp seems
>>> cleanest to me. Maybe right along side it in the same directory? We can
>>> always move it around later if we find somewhere better.
>>>
>>>> - I see the C++ API accepts PGO Options, should that be hooked into as
>>>> well?
>>>>
>>> Probably at some point, but that can come later. I don't see where the
>>> legacy PM's C API does that.
>>>
>>>>
>>>> With that being said, I think I have a decent plan for the interface
>>>> itself. We'll provide a PassBuilder which we can construct with the
>>>> PassInstrumentationCallbacks, PipelineTuningOptions, etc. The various
>>>> *AnalysisManagers may also be added to the PassBuilder. The
>>>> StandardInstrumentation can be added by passing a
>>>> PassInstrumentationCallbacks and optionally a FunctionAnalysisManager like
>>>> the C++ llvm::StandardInstrumentation::registerCallbacks method does.
>>>>
>>>
>>>> The PassBuilder will then be able to accept a Module and an opt-style
>>>> string with the passes to be used. A ModulePassManager will be constructed
>>>> as you described and it will run the passes over the module.
>>>>
>>> The things that make the PassBuilder customizable are the
>>> PipelineTuningOptions, PGOOptions, and DebugLogging. We should probably do
>>> something like heap allocate a PTO/PGOOptions and provide an opaque pointer
>>> to it, and set those options, then pass it to the final build/run function.
>>> Sort of like LLVMPassManagerRef, but the legacy pass builder stored its
>>> options inside the pass builder itself, whereas the new pass builder has a
>>> separate PTO. Then the equivalent of something
>>> like LLVMPassManagerBuilderSetDisableUnrollLoops() would just modify the
>>> PTO.
>>> The legacy PM API separates building the PM and running it on some IR.
>>> I'm not sure if we necessarily want to do that when we can build it and run
>>> it all at once. But maybe I'm missing a use case.
>>>
>>>>
>>>> Let me know what you think of an approach like this...
>>>>
>>>> Best regards
>>>> Mats Larsen
>>>>
>>>> On Wed, Apr 28, 2021 at 11:13 PM Mats Larsen <me at supergrecko.com>
>>>> wrote:
>>>>
>>>>> Thank you Arthur, that sounds great. Using opt's pass name parser
>>>>> sounds like a good plan and a great way to save ourselves from
>>>>> complications with the different PM types. We can then construct the
>>>>> ModulePassManager and run that over an LLVM module the user passes to us.
>>>>>
>>>>> I'll have a closer look at this over the next few days and I'll
>>>>> possibly gather a tiny list of questions I may have before we get started.
>>>>>
>>>>> Mats
>>>>>
>>>>> On Tue, Apr 27, 2021 at 6:26 PM Arthur Eubanks <aeubanks at google.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Apr 26, 2021 at 1:48 PM Mats Larsen <me at supergrecko.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello everyone
>>>>>>>
>>>>>>> I wouldn't mind looking into hooking the new PM infrastructure into
>>>>>>> the LLVM C API. However, I would like to know a little more about the task.
>>>>>>> I've had a look at the new PM infrastructure, and from taking a quick
>>>>>>> glance at the available APIs and how it's used in the tests I assume we'd
>>>>>>> look to do something similar to the existing legacy PM hooks.
>>>>>>>
>>>>>>> Things are a bit different as far as I've understood because we have
>>>>>>> Module, CGSCC, Function, and Loop passes, each of which may be adapted into
>>>>>>> "higher" kinds, eg FunctionPM -> ModulePM, all of which differs a bit from
>>>>>>> the legacy PM. From what I've gathered, we'll need to set up the PMs
>>>>>>> themselves as you mentioned and we'll have to hook a set (all?) of the new
>>>>>>> PM passes into this. Would it be possible to get a more comprehensive list
>>>>>>> of what needs to be done?
>>>>>>>
>>>>>>> I'm pretty new around here so I'll probably require some guidance
>>>>>>> further down the line.
>>>>>>>>
>>>>>>>> <https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
>>>>>>>>
>>>>>>> Awesome, I'm happy to do reviews and answer any further questions.
>>>>>>
>>>>>> I'm imagining a new PassBuilder.h in llvm/include/llvm-c where
>>>>>> PassManagerBuilder.h was.
>>>>>> Initially I was thinking we'd need wrappers around the various types
>>>>>> of pass managers as you've mentioned, as well as the adaptors
>>>>>> (e.g. createModuleToFunctionPassAdaptor()). And each of the LLVMAdd*Pass()
>>>>>> would need a corresponding new PM alternative that adds the pass to the
>>>>>> proper pass manager type.
>>>>>>
>>>>>> But now that I think about it some more, we might be able to bypass
>>>>>> all of that and instead use the new PM's pass pipeline text parsing (the
>>>>>> same thing that `opt -passes=foo` uses). This seems exactly the sort of
>>>>>> thing it would be useful for. We'll bypass the need to create wrappers for
>>>>>> the different types of pass managers since we just get back a
>>>>>> ModulePassManager from the PassBuilder.
>>>>>> The code in NewPMDriver.cpp should be a good starting place to figure
>>>>>> out what's necessary to create a PassBuilder, then how to get a
>>>>>> ModulePassManager from it and run it on some IR.
>>>>>> I imagine first you'll need some way to setup the options to pass to
>>>>>> the PassBuilder, so the C wrapper would probably own various things like
>>>>>> the various
>>>>>> *AnalysisManagers, PassInstrumentationCallbacks, StandardInstrumentations,
>>>>>> and PipelineTuningOptions. Then the user can ask for some pipeline via a
>>>>>> string, whether that's an individual pass like "instcombine" or a full
>>>>>> pipeline like "default<O2>", and you create a PassBuilder from the
>>>>>> options, then tell it to create the ModulePassManager via
>>>>>> PassBuilder::parsePassPipeline().
>>>>>>
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210509/d932a75f/attachment.html>