[llvm-dev] OptBisect implementation for new pass manager

[Fedor Sergeev via llvm-dev]

On 10/01/2018 09:20 PM, Kaylor, Andrew wrote:
>> At run-time we have a bunch of objects involved into the registration:
>>    PassBuilder
>>    OptBisect
>>    pass object to be registered
> I'm not sure I agree with the direction you're going here. Should OptBisect really be treated differently than any other pass instrumentation object? I would substitute "Pass instrumentation infrastructure" for "OptBisect" in this list.  For instance, I think it's OK for the pass builder to be aware that there is a pass instrumentation infrastructure that can change the way passes are executed, but I don't think it should be specifically aware of OptBisect.
I'm not drafting any interfaces there. Just point out exact objects that 
are involved.
OptBisect is *both* the object implementing bisect control and pass 
instrumentation object.

If we can do the necessary interaction w/o exposing specific OptBisect's 
interface - fine, I'm all for it.
I just dont want to disable any options just by definition.

regards,
   Fedor.
>
>> Are there any other known passes that present the same level of complexity wrt OptBisect interaction?
> The SelectionDAGISel pass has opt-level dependent behavior but cannot be skipped.
>
> -Andy
>
>
> -----Original Message-----
> From: Fedor Sergeev [mailto:fedor.sergeev at azul.com]
> Sent: Monday, October 01, 2018 11:02 AM
> To: David Greene <dag at cray.com>
> Cc: Kaylor, Andrew <andrew.kaylor at intel.com>; Philip Pfaffe <philip.pfaffe at gmail.com>; llvm-dev <llvm-dev at lists.llvm.org>; zhizhouy at google.com; David Blaikie <dblaikie at gmail.com>; Chandler Carruth <chandlerc at gmail.com>
> Subject: Re: [llvm-dev] OptBisect implementation for new pass manager
>
> On 10/01/2018 05:01 PM, David Greene wrote:
>> I think registration time is probably fine for IR/opt-level passes.
>> For codegen it will probably work 95% of the time but there are some
>> cases where we may not know at registration time whether the pass is
>> needed or not.  I'm making the assumption that registration happens
>> basically as it does now, where the registration logic is baked into
>> the compiler source and there is no good way to pass runtime
>> information (such as the
>> target) during pass registration.
> The registration logic is for sure baked into the sources :) And we are completely in control of how we do the PassRegistry.def registration.
>
> At run-time we have a bunch of objects involved into the registration:
>     PassBuilder
>     OptBisect
>     pass object to be registered
>
> We can make their interaction to be as complex as needed.
> Say, it is easy to extend PassInstrumentation interface to cover the registration time and have PassBuilder invoke corresponding instrumentations, leading to OptBisect being able to act upon a pass registration.
> Similarly, PassBuilder can pass control to the pass object and have it discover that OptBisect is in action.
>> I brought up scheduling before.  For most targets, scheduling is
>> purely an optimization.  If it doesn't run it won't affect
>> correctness.  But for some targets, it (theoretically) will.  I'm
>> thinking of targets like VLIW machines, targets with branch delay
>> slots, targets without hardware pipeline interlocks and so forth.
> Thanks, I got the idea.
> For now it appears that scheduler is the most complex case that we can get.
>> Now, it may be that such targets take a very conservative approach
>> during isel, so as to guarantee functional correctness even without
>> scheduling.  In that case it's fine to not run the scheduler.  But
>> that means such targets have to be aware of OptBisect at a deep level
>> -- they need to design their isel in an OptBisect-friendly way.
> Noted.
> Are there any other known passes that present the same level of complexity wrt OptBisect interaction?
>
>> This is why I said earlier that the entity that understands these
>> properties lives above passes, pass managers and OptBisect.  The thing
>> constructing the pass pipeline has all the knowledge.
> The thing constructing the pipeline is usually a PassBuilder instance.
> And it has (an indirect) reference to OptBisect, which is the entity that lives over all the pipeline life.
> And as OptBisect is also the only thing that knows the algorithmics of its control, it seems to be a perfect place to keep all the knowledge on how pipeline should function, including its most complicated parts.
>
> Say, for our Java JIT case where we do required Java->C semantics lowerings, we just need to inform OptBisect that a given pass (PassID) can not be skipped.
>
> I would implement it by tracking unskippable passes in OptBisect simply filling a list during OptBisect initialization time.
>
>>     Whatever piece of
>> code calls PassManagerBuilder methods or TargetPassConfig methods.
>> Possibly things like X86PassConfig have all the information for
>> codegen passes.  This is one of the reasons I think the codegen pass
>> pipeline stuff is a little wonky -- it works very differently from the
>> higher-level pass pipeline construction.
>>
>> We should also be aware of how this will work in the presence of
>> multiple targets (CPUs and GPUs for example).  OptBisect will
>> eventually need to understand that a pass might be fine to disable on
>> one target but not another, within the same compilation process.
> As soon as we give enough control to OptBisect it boils down to providing a proper configuration for OptBisect (which can be arbitrarily complex as needed).
>
> regards,
>     Fedor.
>
>> I'm not saying we need to solve these problems now, just that we need
>> to be aware of them.  It won't affect the vast majority of targets but
>> for those targets where it matters, it could be ugly.  If it's ugly
>> but still works, I'm totally fine with that.  Make it nice for the
>> common case and possible for the uncommon case.
>>
>>                                -David