<div dir="ltr">Hi Andrew,<br><div class="gmail_quote"><div dir="ltr"><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">We absolutely need to be able to generate executable programs using opt-bisect, so some mechanism for not skipping required passes is needed. It might be nice to have a mode where no passes are skipped and the IR/MIR is dumped when the bisect limit is reached, but I don't see that as a requirement.<br></blockquote><div>At this point it makes no sense to worry about the code generation pipeline. As long as there is no new-PM design for that, this point is just moot. Designing opt-bisect against code generation without actually understanding what we're designing against is guaranteed to produce a bad architecture. So lets figure out the optimizer bisect first, and incrementally upgrade that once we've ironed out codegen.</div><div> </div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Regarding opt-in versus opt-out, I think we want to make this as easy and transparent to pass developers as possible. It would be nice to have the mechanism be opt-out so that passes that were added with no awareness of opt-bisect would be automatically included. However, there is a small wrinkle to this. I can't defend this as a reasonable design choice, but the SelectionDAGISel pass has a sort of hybrid behavior. It can't actually be skipped, but it OptBisect says it should be skipped it drops the optimization level to OptNone. That's a machine function pass, so it doesn't matter so much right now. It's just something to think about.<br>
<br>
One of the reasons that we combined the optnone handling and the opt-bisect handling is that we specifically wanted these two behaviors to be linked. The exact rule we would like to use for opt bisect is that no pass which runs at O0 is skipped by opt-bisect. There's a test that verifies this. Conversely, if a pass is able to respect the optnone attribute then it should also be skippable by opt-bisect. Of course, I would be open to considering a use case where this reasoning isn't desirable.<br></blockquote><div><br></div><div>Mixing OptNone and bisect is a software engineering bug: it's mixing different layers of abstraction. Bisect is something that's at pass manager scope: run passes until whatever. OptNone in turn doesn't belong in the pass manager layer. It only concerns function passes, and crumbles quickly considering other IRUnits or even codegen. I don't have a clear vision what OptNone handling should look like, but at this point I consider it entirely orthogonal to bisect handling.</div><div><br></div><div>From a software architecture perspective I don't see a reason why passes should even _know_ about something like bisect happening. That is simply not their domain. If a pass shouldn't be skipped for whatever reason, that's not something the pass should worry about, that's the bisect driver's problem! My proposal here would be make it an opt-out design, but let the driver control that. E.g., for skipping, let the user provide a list of passes they don't want skipped.<br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
With regard to there being one OptBisect object per compilation pipeline, I have some concerns. Specifically, the behavior of opt-bisect depends on the sequence of passes run before the limit is reached being consistent and repeatable. My inclination would be to not allow parallel compilation when opt-bisect is enabled.</blockquote><div>I don't have a strong opinion here, but just as a data point: my mental model here is to expect bisect to expect a deterministic outcome _per module_. That model isn't threatened by parallel execution.</div><div><br></div><div><br></div><div>Cheers,</div><div>Philip</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> I can imagine cases where you might specifically want to debug something that only happens in a parallel build, but it's more difficult to imagine something that only happens in a parallel build and doesn't depend on interactions between threads. In such a case, would we be able to guarantee that the sequence of passes and any interaction between pipelines was repeatable. Basically, here I feel like I'm exploring a hypothetical idea where other people have specific use cases. If so, please explain the use case to me.<br>
<br>
-Andy<br>
<br>
-----Original Message-----<br>
From: Fedor Sergeev [mailto:<a href="mailto:fedor.sergeev@azul.com" target="_blank">fedor.sergeev@azul.com</a>] <br>
Sent: Wednesday, September 26, 2018 9:54 AM<br>
To: llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>>; Zhizhou Yang <<a href="mailto:zhizhouy@google.com" target="_blank">zhizhouy@google.com</a>>; David Greene <<a href="mailto:dag@cray.com" target="_blank">dag@cray.com</a>>; David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>>; Kaylor, Andrew <<a href="mailto:andrew.kaylor@intel.com" target="_blank">andrew.kaylor@intel.com</a>>; Chandler Carruth <<a href="mailto:chandlerc@gmail.com" target="_blank">chandlerc@gmail.com</a>><br>
Subject: OptBisect implementation for new pass manager<br>
<br>
Greetings!<br>
<br>
As the generic Pass Instrumentation framework for new pass manager is finally *in*, I'm glad to start the discussion on implementation of -opt-bisect through that framework.<br>
<br>
As it has already been discovered while porting other features (namely,<br>
-time-passes)<br>
blindly copying the currently existing legacy implementation is most likely not a perfect way forward. Now is a chance to take a fresh look at the overall approach and perhaps do better, without the restrictions that legacy pass manager framework imposed on the implementation.<br>
<br>
Kind of a summary of what we have now:<br>
- There is a single OptBisect object, requested through LLVMContext<br>
(managed as ManagedStatic).<br>
<br>
- OptBisect is defined in lib/IR, but does use analyses,<br>
which is a known layering issue<br>
<br>
- Pass hierarchy provides skipModule etc helper functions<br>
<br>
- Individual passes opt-in to OptBisect activities by manually calling skip* helper functions<br>
whenever appropriate<br>
<br>
With current state of new-pm PassInstrumentation potential OptBisect implementation will have the following properties/issues:<br>
- OptBisect object that exists per compilation pipeline, managed similar to PassBuilder/PassManagers<br>
(which makes it more suitable for use in parallel compilations)<br>
<br>
- no more layering issues imposed by implementation since instrumentations by design<br>
can live anywhere - lib/Analysis, lib/Passes etc<br>
<br>
- since Codegen is still legacy-only we will have to make a joint implementation that<br>
provides a sequential passes numbering through both new-PM IR and legacy Codegen pipelines<br>
<br>
- as of right now there is no mechanism for opt-in/opt-out, so it needs to be designed/implemented<br>
Here I would like to ask:<br>
- what would be preferable - opt-in or opt-out?<br>
<br>
- with legacy implementation passes opt-in both for bisect and attribute-optnone support at once.<br>
Do we need to follow that in new-pm implementation?<br>
<br>
Also, I would like to ask whether people see current user interface for opt-bisect limiting?<br>
Do we need better controls for more sophisticated bisection?<br>
Basically I'm looking for any ideas on improving opt-bisect user experience that might affect design approaches we take on the initial implementation.<br>
<br>
regards,<br>
Fedor.<br>
<br>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div></div>