[llvm-dev] OptBisect implementation for new pass manager

Thu Oct 4 02:46:33 PDT 2018

On Thu, Oct 4, 2018 at 2:26 AM Philip Reames via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

>
>
> On 10/04/2018 12:58 AM, Chandler Carruth via llvm-dev wrote:
>
> Sorry I'm late to the thread (conference + vacation delayed me). I've
> tried to skim the thread, but haven't found too much real conclusions to a
> few points I'd like to make. If any of the below re-hashes stuff that was
> already covered, my apologies and feel free to just mention by whom or what
> date and I'll read more carefully.
>
>
> I feel like the design of this is made unnecessarily complex and could be
> simplified in a few ways. These all stem from a key aspect of bisection:
> this is a *development* activity. It doesn't have to hit some specific
> quality bar the way that `optnone` and -O0 (which are both exposed to
> users) need to....
>
> Some immediate simplifications:
>
> 1) I don't think we need to go out of our way to connect the IR pass
> bisection (in the new PM) with codegen's IR pass bisection. We already have
> two tools (`opt` and `llc` and can drive them separately IMO).
>
> Definitely agreed.
>
>
> 2) I also don't think we need the subtle *correctness* guarantees of
> `optnone` which really and truly IMO require *passes* to make the decision
> rather than some abstract pass pipeline system.
>
> 3) I think we really do want high *resolution* of bisection even if it
> isn't 100% functional. Let's imagine that this skips a "necessary" pass for
> some behavior. The IR will still be valid, and this step of bisection can
> still be very useful for reducing crashes and assert failures.
>
> Chandler, I think you've missed a really key use case here.  The primary
> use case many of us have for opt-bisect is specifically reducing
> miscompiles, not crashes.  Compiler crashes are generally fairly easy to
> reduce with bugpoint.  Where we need opt-bisect is when trying to isolate
> something which can't be effectively reduced with bugpoint.
>

FWIW, both miscompiles and crashes need reduction both in the IR axis and
the pass axis...

>   Continuing to produce runnable code during reduction is a key feature
> since it's the only way we can judge whether a given transform was correct
> or not.
>

Sure, I'm not trying to ignore reductions that require producing runnable
code as a use case.

>
> My typical workflow for opt-bisect looks something like the following:
> given a determinism java program whose result differs from golden output
> identify which method is miscompiled by delta reducing over compilation
> decisions (out of scope for opt-bisect)
> while (still producing wrong answer)  { reduce opt-bisect limit }
>
> My point here is that I really don't think we can disregard the notion of
> required passes.  They're key to the use case.
>

See below for details, but I'm not just trying to disergard them.

>
>
> Regarding #2 which is I think the most surprising thing... Keep in mind
> that *after* we finish bisection, we can still run some minimal second set
> of passes in order to generate "correct" code. Also, I believe debug
> counters has the ability to disable only for a range of counts and then
> re-enable. Well designed bisection test scripts should be able to preserve
> and/or synthesize the necessary bits to keep IR "working" for whatever
> constraints a particular reduction has.
>
> Running a bunch of separate post processing scripts is incompatible with
> my use case.  It could be made to work, but only with a lot of currently
> unnecessary complexity.
>

I don't really understand yet why this is the case.

Already, you will very often have to take the IR and hand it to codegen,
and will have to link an executable of some form, then run it and then
check that the result either is "good" or "bad" (for whatever bisection
test criteria you have).

All I'm suggesting is that part of "hand it to codegen" may involve "run a
minimal set of required passes and hand it to codegen".

In fact, `llc -O0` already does exactly this.

It is perhaps also important to look at what constitute "required passes"
-- there are *extremely* few of these outside of the code generation
pipeline. Typically there is one or two at the beginning (something like
internalize) and maaaaybe something at the end when you don't have a
traditional codegeneration step such as the last pass in the ThinLTO case
that is "required". So the set of things that could be necessary here seems
exceedingly small and likely already covered by existing tools.

Does this make sense?

Most of the complexity with the `optnone` implementation of bisect comes
not from the passes themselves being "required" but from dealing with
subtle cases where one function continues to be optimized but the other
function does not... The only other really interesting case I recall is
always_inline, which can again be very easily handled by just running `opt
-passes=always-inline` over the IR either before or after bisection. And
that only really needs to be done when you have a test case where
`always_inline` is semantically necessary to reproduce a specific
miscompilation you are bisecting, which in my experience is exceedingly
rare.

>
>
> Hope all of this makes some sense.
> -Chandler
>
> On Wed, Sep 26, 2018 at 9:54 AM Fedor Sergeev via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Greetings!
>>
>> As the generic Pass Instrumentation framework for new pass manager is
>> finally *in*,
>> I'm glad to start the discussion on implementation of -opt-bisect
>> through that framework.
>>
>> As it has already been discovered while porting other features (namely,
>> -time-passes)
>> blindly copying the currently existing legacy implementation is most
>> likely not a perfect
>> way forward. Now is a chance to take a fresh look at the overall
>> approach and perhaps
>> do better, without the restrictions that legacy pass manager framework
>> imposed on
>> the implementation.
>>
>> Kind of a summary of what we have now:
>>    - There is a single OptBisect object, requested through LLVMContext
>>      (managed as ManagedStatic).
>>
>>    - OptBisect is defined in lib/IR, but does use analyses,
>>      which is a known layering issue
>>
>>    - Pass hierarchy provides skipModule etc helper functions
>>
>>    - Individual passes opt-in to OptBisect activities by manually
>> calling skip* helper functions
>>      whenever appropriate
>>
>> With current state of new-pm PassInstrumentation potential OptBisect
>> implementation
>> will have the following properties/issues:
>>    - OptBisect object that exists per compilation pipeline, managed
>> similar to PassBuilder/PassManagers
>>      (which makes it more suitable for use in parallel compilations)
>>
>>    - no more layering issues imposed by implementation since
>> instrumentations by design
>>      can live anywhere - lib/Analysis, lib/Passes etc
>>
>>    - since Codegen is still legacy-only we will have to make a joint
>> implementation that
>>      provides a sequential passes numbering through both new-PM IR and
>> legacy Codegen pipelines
>>
>>    - as of right now there is no mechanism for opt-in/opt-out, so it
>> needs to be designed/implemented
>>      Here I would like to ask:
>>          - what would be preferable - opt-in or opt-out?
>>
>>          - with legacy implementation passes opt-in both for bisect and
>> attribute-optnone support at once.
>>            Do we need to follow that in new-pm implementation?
>>
>> Also, I would like to ask whether people see current user interface for
>> opt-bisect limiting?
>> Do we need better controls for more sophisticated bisection?
>> Basically I'm looking for any ideas on improving opt-bisect user
>> experience that might
>> affect design approaches we take on the initial implementation.
>>
>> regards,
>>    Fedor.
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
>
> _______________________________________________
> LLVM Developers mailing listllvm-dev at lists.llvm.orghttp://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181004/d7c63d22/attachment-0001.html>