[llvm-dev] OptBisect implementation for new pass manager

[David Greene via llvm-dev]

Chandler Carruth via llvm-dev <llvm-dev at lists.llvm.org> writes:

> There is another approach that I think might be more interesting
> long-term: rather than merely bisection to isolate a single point
> which introduces a bug, I'd like us to generalize the debug counter
> pattern. Right now, you can skip N times and then run M times. What I
> would like is to support an arbitrary number of skip and run counts.
> If you combine this with a test harness that marks compilations that
> fail, or runs that fail differently from the bug in question, you can
> use this to *detect* and preserve the necessary passes because they
> will fail the "sanity" or "baseline" test prior to being
> "interesting".

Yes, multiple skip/count pairs would be very useful.

I've mentioned before that we developed something we call PassMax:

opt -pass-max=licm=50 -O2 ...

This tells opt to run 50 iterations of LICM and then not run any more.
It's a convenient way to bisect "within a pass" once the pass is
identified as being problematic.

Like DebugCounters, PassMax requires instrumentation of the pass with a
variable to track the number of times it has been run:

bool LICM::runOnFunction(Function &F) {
  bool Change = false;
  do {
    Change = OneRoundOfLICM();
    IncrementPassMax(this);
    if (AtPassMax(this)) {
      break;
    }
  } while (Change);

  return Change;
}

The PassMax value is persistent across invocations of the pass' run
method.

So DebugCounters are operationally a superset of PassMax.

At some future time I had hoped to integrate PassMax with something like
bugpoint to do a heirarchical search:

DoBugPoint:
  PassNames = BisectToPasses()
  for (Pass : PassNames)
    BisectInPass(Pass)

[ Not really correct, but you get the idea. ]

If something like PassMax were pervasive, bugpoint (or another tool)
could be taught how to use it to further refine the search.

It does seem like OptBisect should build upon DebugCounters as you
suggest.  It would be great if every Pass automatically got a
DebugCounter that controls how many times it does a transformation.  By
default passes would just ignore this counter as they wouldn't even know
about it.  Someone would have to go in and instrument the pass to enable
the limiting behavior.

The advantage of automatic availability is that external interfaces to
such counters could be standardized and tools could be taught to use
them without knowing intricate details of how they're implemented.  For
example, if every pass automatically got such a DebugCounter, their
naming could be intuitive:

opt -debug-counter=<pass name>-limit-skip=<x>,<pass-name>-limit-count=<y> ...

This is how the PassMax stuff works.  Counters register themselves using
the pass name declared by the Pass initialization mechanism.  That
makes things scriptable as the name of a pass returned by some tool can
be input to another tool to enable the pass' counter.

                               -David