[llvm-dev] OptBisect implementation for new pass manager

[Chandler Carruth via llvm-dev]

On Thu, Oct 4, 2018 at 4:11 PM Kaylor, Andrew <andrew.kaylor at intel.com>
wrote:

> We (Intel) use opt-bisect most often in conjunction with clang and
> frequently with applications that have non-trivial compilation mechanisms.
> If there isn’t a way to get something hooked up through code generation, it
> doesn’t really meet the needs of the way we are using it.  As an interim
> solution it would be tolerable to have two separate bisect-like options,
> one that works its way through the pre-codegen passes and a different one
> that bisects codegen.
>

To an extent, I think that will be a very likely necessary step due to the
fact that all of codegen is a separate pass manager at the moment.

However, given that the goal is bisection, I would imagine a somewhat
simpler approach of the first step of bisection be to determine whether the
regression comes before codegen or in codegen.

That said, I think it is interesting that you are using this directly with
Clang. I'm honestly somewhat surprised that this works, but hey, nice!


>
>
> Skipping “necessary” passes is a somewhat different matter, because you
> can’t skip something like register allocation on your way to executable
> code.  Most of the passes that are really significant in this way are
> codegen passes, so we might still be able to work something out. I believe
> there are a few pre-codegen passes that are required but they do trivial
> things like removing intrinsics that don’t get lowered.
>

FWIW, even the "IR" passes like this are actually run as part of codegen
using the legacy PM in all cases right now. So to an extent, I think this
will "just work" out of the box once you can select between the bisection
occuring over the main optimization pipeline (w/ the new PM) or the code
generation pipeline (still w/ legacy PM).

Perhaps we could have some mechanism that builds a set of clean up passes
> that get run under some very limited circumstances. Is that what you were
> suggesting in your comment about running “some minimal second set of
> passes” after the bisection?
>

This is one approach that seems like it might be fruitful. It does make it
hard to use bisection techniques to isolate issues caused by passes
specifically when intermingled with those "necessary" passes like register
allocation. I'm not sure how important bisection is for those passes
currently though. To be honest, while I've used pass bisection a *lot* for
a lot of different things, I've never really found it useful once I'm in
MI-land because so few of the passes can realistically be disabled. So I
was somewhat focused on solving the problem in a more simple way focused on
the more flexible passes.

There is another approach that I think might be more interesting long-term:
rather than merely bisection to isolate a single point which introduces a
bug, I'd like us to generalize the debug counter pattern. Right now, you
can skip N times and then run M times. What I would like is to support an
arbitrary number of skip and run counts. If you combine this with a test
harness that marks compilations that fail, or runs that fail differently
from the bug in question, you can use this to *detect* and preserve the
necessary passes because they will fail the "sanity" or "baseline" test
prior to being "interesting".

This is a pretty common pattern in delta reduction (verifying that the
reduction remains "interesting" includes verifying that it doesn't fail for
unrelated reasons), and I think could be used to avoid building in complex
logic to handle "required" passes.


If none of this works, and we *need* to share the logic between `optnone`
and bisection because we have no other way of expressing "required" passes,
my inclination would be to actually try to use "optnone" directly rather
than trying to build a unified "skip" routine that somehow coordinates with
the pass manager. But I'm not sure how to do that really effectively
outside of function passes. Or maybe there is some other way of easily
communicating "required"-ness here, but currently I struggle to see it. So
I'm somewhat hoping that in practice, one of the above will be enough.


>
> The current opt-bisect mechanism really doesn’t do much more than
> increment a counter, but it does produce output that indicates what was run
> and what wasn’t.  This is very useful for finding the proper owner for a
> bug once the failure has been diagnosed with opt-bisect.  I don’t really
> care how the mechanism is implemented, but I’d really like to know what the
> last optimization was that ran in a failing case and which IR unit it was
> run on.  Knowing the highest pass count that was used is also helpful to
> establish a starting point.
>

Having clear output would definitely be useful. Fedor was already thinking
of replacing the pass manager debug output with the instrumentation, and it
would seem very easy to similarly wire together helpful output from the
bisection instrumentation.


Thanks for all the thoughts on different ways to use bisection!
-Chandler

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181005/29b3d8bd/attachment-0001.html>