[llvm-dev] OptBisect implementation for new pass manager

[Kaylor, Andrew via llvm-dev]

> At run-time we have a bunch of objects involved into the registration:
>   PassBuilder
>   OptBisect
>   pass object to be registered

I'm not sure I agree with the direction you're going here. Should OptBisect really be treated differently than any other pass instrumentation object? I would substitute "Pass instrumentation infrastructure" for "OptBisect" in this list.  For instance, I think it's OK for the pass builder to be aware that there is a pass instrumentation infrastructure that can change the way passes are executed, but I don't think it should be specifically aware of OptBisect.


> Are there any other known passes that present the same level of complexity wrt OptBisect interaction?

The SelectionDAGISel pass has opt-level dependent behavior but cannot be skipped.

-Andy


-----Original Message-----
From: Fedor Sergeev [mailto:fedor.sergeev at azul.com] 
Sent: Monday, October 01, 2018 11:02 AM
To: David Greene <dag at cray.com>
Cc: Kaylor, Andrew <andrew.kaylor at intel.com>; Philip Pfaffe <philip.pfaffe at gmail.com>; llvm-dev <llvm-dev at lists.llvm.org>; zhizhouy at google.com; David Blaikie <dblaikie at gmail.com>; Chandler Carruth <chandlerc at gmail.com>
Subject: Re: [llvm-dev] OptBisect implementation for new pass manager

On 10/01/2018 05:01 PM, David Greene wrote:
>
> I think registration time is probably fine for IR/opt-level passes.  
> For codegen it will probably work 95% of the time but there are some 
> cases where we may not know at registration time whether the pass is 
> needed or not.  I'm making the assumption that registration happens 
> basically as it does now, where the registration logic is baked into 
> the compiler source and there is no good way to pass runtime 
> information (such as the
> target) during pass registration.
The registration logic is for sure baked into the sources :) And we are completely in control of how we do the PassRegistry.def registration.

At run-time we have a bunch of objects involved into the registration:
   PassBuilder
   OptBisect
   pass object to be registered

We can make their interaction to be as complex as needed.
Say, it is easy to extend PassInstrumentation interface to cover the registration time and have PassBuilder invoke corresponding instrumentations, leading to OptBisect being able to act upon a pass registration.
Similarly, PassBuilder can pass control to the pass object and have it discover that OptBisect is in action.
> I brought up scheduling before.  For most targets, scheduling is 
> purely an optimization.  If it doesn't run it won't affect 
> correctness.  But for some targets, it (theoretically) will.  I'm 
> thinking of targets like VLIW machines, targets with branch delay 
> slots, targets without hardware pipeline interlocks and so forth.
Thanks, I got the idea.
For now it appears that scheduler is the most complex case that we can get.
> Now, it may be that such targets take a very conservative approach 
> during isel, so as to guarantee functional correctness even without 
> scheduling.  In that case it's fine to not run the scheduler.  But 
> that means such targets have to be aware of OptBisect at a deep level 
> -- they need to design their isel in an OptBisect-friendly way.
Noted.
Are there any other known passes that present the same level of complexity wrt OptBisect interaction?

> This is why I said earlier that the entity that understands these 
> properties lives above passes, pass managers and OptBisect.  The thing 
> constructing the pass pipeline has all the knowledge.
The thing constructing the pipeline is usually a PassBuilder instance.
And it has (an indirect) reference to OptBisect, which is the entity that lives over all the pipeline life.
And as OptBisect is also the only thing that knows the algorithmics of its control, it seems to be a perfect place to keep all the knowledge on how pipeline should function, including its most complicated parts.

Say, for our Java JIT case where we do required Java->C semantics lowerings, we just need to inform OptBisect that a given pass (PassID) can not be skipped.

I would implement it by tracking unskippable passes in OptBisect simply filling a list during OptBisect initialization time.

>    Whatever piece of
> code calls PassManagerBuilder methods or TargetPassConfig methods.
> Possibly things like X86PassConfig have all the information for 
> codegen passes.  This is one of the reasons I think the codegen pass 
> pipeline stuff is a little wonky -- it works very differently from the 
> higher-level pass pipeline construction.
>
> We should also be aware of how this will work in the presence of 
> multiple targets (CPUs and GPUs for example).  OptBisect will 
> eventually need to understand that a pass might be fine to disable on 
> one target but not another, within the same compilation process.
As soon as we give enough control to OptBisect it boils down to providing a proper configuration for OptBisect (which can be arbitrarily complex as needed).

regards,
   Fedor.

> I'm not saying we need to solve these problems now, just that we need 
> to be aware of them.  It won't affect the vast majority of targets but 
> for those targets where it matters, it could be ugly.  If it's ugly 
> but still works, I'm totally fine with that.  Make it nice for the 
> common case and possible for the uncommon case.
>
>                               -David