[PATCH][PM] Add pass run listeners to the pass manager.

[Chandler Carruth]

On Tue, Apr 29, 2014 at 11:55 AM, Andrew Trick <atrick at apple.com> wrote:

>
> On Apr 29, 2014, at 11:39 AM, Duncan P. N. Exon Smith <
> dexonsmith at apple.com> wrote:
>
> >
> > On 2014-Apr-28, at 16:08, Chandler Carruth <chandlerc at google.com> wrote:
> >
> >> First and foremost, I'm extremely concerned about baking the Pass type
> into the C API. The on-going work to change how the pass manager works at a
> fundamental level is going to run completely afoul of this.
> >
> > This is clearly important.
>
> I don’t understand the issue at all.
>

So, for starters, no one working on the pass manager was even involved in
the design discussion. As a consequence, the design of the C API doesn't
really match the design of the pass manager. See below for details.


>
> >> I would really like to keep this out of the C API which we have to be
> backwards compatible with until the pass manager work settles down. If we
> don't, the entire thing is going to be really challenging to make progress
> on.
> >
> > If I understand correctly, all that's been exposed in the C API here is
> > that there is a type for a Pass, and that there's a way to get a name
> > from a Pass.
> >
> > In the new API, are we not going to have a Pass base class?
>

Correct, there is no longer a base class.

If you want to expose a type-erased generic interface to all passes in the
C API we'll actually need to define that new interface. And I'm somewhat
concerned that it will be defined in a way that is quite awkward to
implement with the new system. Generally, I've had more success designing
the core set of interfaces used in the C++ side, and only after they have
stabilized, wrapping them in a C interface.


> > Or am I
> > missing something else?  (Sorry if this is obvious; I haven't spent the
> > time to grok the new design yet.)
>
> Exactly. The C API should be totally independent of PassManager
> implementation. If we need to to hold up a useful C API because of
> PassManager changes, then something is wrong with the API—but I’m not sure
> what that is.
>

If you want the C API to be totally independent of the implementation,
you've not succeeded -- you're defining a single opaque base class for a
pass (not a pass manager) which is an artifact of the current
implementation. It's close though -- you could define a wrapper class
instead and then it would work just fine.

However, I'm more worried about the assumptions this is baking into the
*interface*. While I'm glad to see that the units of IR are incorporated
now, they don't even really match the way we use the current pass manager.
Here are my concerns, I'm numbering them to help keep both my thoughts and
responses attached to the right topics.

1) You expose the set of IR units which passes can operate on in the number
and types of arguments to the C API. This completely breaks extension in
the future.

2) The list of IR units is simply wrong.
2a) We don't want to expose basic block passes. We only have four of them,
and most are used in the standard compilation model.
2b) You don't expose the CGSCC passes. The CGSCC pass manager is a
fundamental component of the LLVM optimizer. I don't know what use this
serves if we can't interact with those.
2c) You also don't expose the Loop passes. Maybe we shouldn't have them,
maybe we should, but today we definitely have them and they are critical.

3) I share Duncan's concerns that it is extremely unclear exactly at what
event these callbacks are triggered. The documentation is very sparse there.

4) A fundamental aspect of the new design is that analysis passes are
cached dynamically so that we can access function analyses from
non-function passes (and other benefits, such as not invalidating analyses
when passes fail to change anything). I don't understand how these will
interact with that model. Perhaps this is just a specific example of #3?

5) How does this work in a world where we parallelize the pass manager?
This is particularly concerning as the discussion seems to indicate that
there are thread safety guarantees made by the pass manager in its
invocation of these callbacks, and I don't know how to parallelize the pass
manager and preserve them.


If email isn't the right forum to have this discussion, I'm happy to use
another. But I think these are really important things to figure out before
we start baking this stuff into the C API for MCJIT. I would even be
interested to see us iterate on the C++ callback API and make progress on
implementing it with both pass managers in the tree for some time. Wrapping
it in a stable C API seems best after it (along with perhaps the pass
manager stuff itself) has stabilized.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140429/c32118b8/attachment.html>