[PATCH][PM] Add pass run listeners to the pass manager.

Tue Apr 29 14:16:29 PDT 2014

On Apr 29, 2014, at 1:00 PM, Chandler Carruth <chandlerc at google.com> wrote:

> On Tue, Apr 29, 2014 at 11:55 AM, Andrew Trick <atrick at apple.com> wrote:
> 
> On Apr 29, 2014, at 11:39 AM, Duncan P. N. Exon Smith <dexonsmith at apple.com> wrote:
> 
> >
> > On 2014-Apr-28, at 16:08, Chandler Carruth <chandlerc at google.com> wrote:
> >
> >> First and foremost, I'm extremely concerned about baking the Pass type into the C API. The on-going work to change how the pass manager works at a fundamental level is going to run completely afoul of this.
> >
> > This is clearly important.
> 
> I don’t understand the issue at all.
> 
> So, for starters, no one working on the pass manager was even involved in the design discussion. As a consequence, the design of the C API doesn't really match the design of the pass manager. See below for details.
>  
> 
> >> I would really like to keep this out of the C API which we have to be backwards compatible with until the pass manager work settles down. If we don't, the entire thing is going to be really challenging to make progress on.
> >
> > If I understand correctly, all that's been exposed in the C API here is
> > that there is a type for a Pass, and that there's a way to get a name
> > from a Pass.
> >
> > In the new API, are we not going to have a Pass base class?
> 
> Correct, there is no longer a base class.
> 
> If you want to expose a type-erased generic interface to all passes in the C API we'll actually need to define that new interface. And I'm somewhat concerned that it will be defined in a way that is quite awkward to implement with the new system. Generally, I've had more success designing the core set of interfaces used in the C++ side, and only after they have stabilized, wrapping them in a C interface.
>  
> > Or am I
> > missing something else?  (Sorry if this is obvious; I haven't spent the
> > time to grok the new design yet.)
> 
> Exactly. The C API should be totally independent of PassManager implementation. If we need to to hold up a useful C API because of PassManager changes, then something is wrong with the API—but I’m not sure what that is.
> 
> If you want the C API to be totally independent of the implementation, you've not succeeded -- you're defining a single opaque base class for a pass (not a pass manager) which is an artifact of the current implementation. It's close though -- you could define a wrapper class instead and then it would work just fine.
> 
> However, I'm more worried about the assumptions this is baking into the *interface*. While I'm glad to see that the units of IR are incorporated now, they don't even really match the way we use the current pass manager. Here are my concerns, I'm numbering them to help keep both my thoughts and responses attached to the right topics.
> 
> 1) You expose the set of IR units which passes can operate on in the number and types of arguments to the C API. This completely breaks extension in the future.

We could make the IR argument opaque and typeless. An additional argument could specify the type of the IR argument. This way existing uses of the C API won’t break with the introduction of an new IR type and
the IR types could be extended.
> 
> 2) The list of IR units is simply wrong.
That could be fixed with what I mentioned above.
> 2a) We don't want to expose basic block passes. We only have four of them, and most are used in the standard compilation model.
Why artificially put a limit here? The listener interface can also be used by the standard compilation model and would include it in the C interface for completeness.
> 2b) You don't expose the CGSCC passes. The CGSCC pass manager is a fundamental component of the LLVM optimizer. I don't know what use this serves if we can't interact with those.
Can be added when addressing 1.
> 2c) You also don't expose the Loop passes. Maybe we shouldn't have them, maybe we should, but today we definitely have them and they are critical.
Can be added when addressing 1.
> 
> 3) I share Duncan's concerns that it is extremely unclear exactly at what event these callbacks are triggered. The documentation is very sparse there.

Using better function names a Duncan suggested should make this clearer.

> 
> 4) A fundamental aspect of the new design is that analysis passes are cached dynamically so that we can access function analyses from non-function passes (and other benefits, such as not invalidating analyses when passes fail to change anything). I don't understand how these will interact with that model. Perhaps this is just a specific example of #3?
> 
> 5) How does this work in a world where we parallelize the pass manager? This is particularly concerning as the discussion seems to indicate that there are thread safety guarantees made by the pass manager in its invocation of these callbacks, and I don't know how to parallelize the pass manager and preserve them.
> 
> 
> If email isn't the right forum to have this discussion, I'm happy to use another. But I think these are really important things to figure out before we start baking this stuff into the C API for MCJIT. I would even be interested to see us iterate on the C++ callback API and make progress on implementing it with both pass managers in the tree for some time. Wrapping it in a stable C API seems best after it (along with perhaps the pass manager stuff itself) has stabilized.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140429/44ce9bff/attachment.html>