[PATCH][PM] Add pass run listeners to the pass manager.

Thu May 1 15:07:12 PDT 2014

Hi @ll,

I incorporated your suggestions into this patch. It extends the pass listener to also work with the new pass manager. There is some additional glue that can be removed
once we get rid of the legacy pass manager. The C API has been reworked to accommodate future changes and additions to the pass listener.

The callback function signature has changed to incorporate an opaque user defined value and abstracts the current pass and IR behind a pass invocation reference.
The invocation type specifies when this callback has been called. Currently this supports pre- and post-pass notification callbacks.

void (*LLVMPassListenerCallback)(void *Opaque, LLVMContextRef C, LLVMPassInvocationType T, LLVMPassInvocationRef P);

The pass invocation reference currently only supports one API - the pass name. This could be extended in the future to obtain the pass related IR, more pass related
information, etc.

Cheers,
Juergen

On Apr 29, 2014, at 2:16 PM, Juergen Ributzka <juergen at apple.com> wrote:

> 
> On Apr 29, 2014, at 1:00 PM, Chandler Carruth <chandlerc at google.com> wrote:
> 
>> On Tue, Apr 29, 2014 at 11:55 AM, Andrew Trick <atrick at apple.com> wrote:
>> 
>> On Apr 29, 2014, at 11:39 AM, Duncan P. N. Exon Smith <dexonsmith at apple.com> wrote:
>> 
>> >
>> > On 2014-Apr-28, at 16:08, Chandler Carruth <chandlerc at google.com> wrote:
>> >
>> >> First and foremost, I'm extremely concerned about baking the Pass type into the C API. The on-going work to change how the pass manager works at a fundamental level is going to run completely afoul of this.
>> >
>> > This is clearly important.
>> 
>> I don’t understand the issue at all.
>> 
>> So, for starters, no one working on the pass manager was even involved in the design discussion. As a consequence, the design of the C API doesn't really match the design of the pass manager. See below for details.
>>  
>> 
>> >> I would really like to keep this out of the C API which we have to be backwards compatible with until the pass manager work settles down. If we don't, the entire thing is going to be really challenging to make progress on.
>> >
>> > If I understand correctly, all that's been exposed in the C API here is
>> > that there is a type for a Pass, and that there's a way to get a name
>> > from a Pass.
>> >
>> > In the new API, are we not going to have a Pass base class?
>> 
>> Correct, there is no longer a base class.
>> 
>> If you want to expose a type-erased generic interface to all passes in the C API we'll actually need to define that new interface. And I'm somewhat concerned that it will be defined in a way that is quite awkward to implement with the new system. Generally, I've had more success designing the core set of interfaces used in the C++ side, and only after they have stabilized, wrapping them in a C interface.
>>  
>> > Or am I
>> > missing something else?  (Sorry if this is obvious; I haven't spent the
>> > time to grok the new design yet.)
>> 
>> Exactly. The C API should be totally independent of PassManager implementation. If we need to to hold up a useful C API because of PassManager changes, then something is wrong with the API—but I’m not sure what that is.
>> 
>> If you want the C API to be totally independent of the implementation, you've not succeeded -- you're defining a single opaque base class for a pass (not a pass manager) which is an artifact of the current implementation. It's close though -- you could define a wrapper class instead and then it would work just fine.
>> 
>> However, I'm more worried about the assumptions this is baking into the *interface*. While I'm glad to see that the units of IR are incorporated now, they don't even really match the way we use the current pass manager. Here are my concerns, I'm numbering them to help keep both my thoughts and responses attached to the right topics.
>> 
>> 1) You expose the set of IR units which passes can operate on in the number and types of arguments to the C API. This completely breaks extension in the future.
> 
> We could make the IR argument opaque and typeless. An additional argument could specify the type of the IR argument. This way existing uses of the C API won’t break with the introduction of an new IR type and
> the IR types could be extended.
>> 
>> 2) The list of IR units is simply wrong.
> That could be fixed with what I mentioned above.
>> 2a) We don't want to expose basic block passes. We only have four of them, and most are used in the standard compilation model.
> Why artificially put a limit here? The listener interface can also be used by the standard compilation model and would include it in the C interface for completeness.
>> 2b) You don't expose the CGSCC passes. The CGSCC pass manager is a fundamental component of the LLVM optimizer. I don't know what use this serves if we can't interact with those.
> Can be added when addressing 1.
>> 2c) You also don't expose the Loop passes. Maybe we shouldn't have them, maybe we should, but today we definitely have them and they are critical.
> Can be added when addressing 1.
>> 
>> 3) I share Duncan's concerns that it is extremely unclear exactly at what event these callbacks are triggered. The documentation is very sparse there.
> 
> Using better function names a Duncan suggested should make this clearer.
> 
>> 
>> 4) A fundamental aspect of the new design is that analysis passes are cached dynamically so that we can access function analyses from non-function passes (and other benefits, such as not invalidating analyses when passes fail to change anything). I don't understand how these will interact with that model. Perhaps this is just a specific example of #3?
>> 
>> 5) How does this work in a world where we parallelize the pass manager? This is particularly concerning as the discussion seems to indicate that there are thread safety guarantees made by the pass manager in its invocation of these callbacks, and I don't know how to parallelize the pass manager and preserve them.
>> 
>> 
>> If email isn't the right forum to have this discussion, I'm happy to use another. But I think these are really important things to figure out before we start baking this stuff into the C API for MCJIT. I would even be interested to see us iterate on the C++ callback API and make progress on implementing it with both pass managers in the tree for some time. Wrapping it in a stable C API seems best after it (along with perhaps the pass manager stuff itself) has stabilized.
> 
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140501/4a667486/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-PM-Extend-pass-listener-to-also-work-with-the-new-pa.patch
Type: application/octet-stream
Size: 30188 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140501/4a667486/attachment.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140501/4a667486/attachment-0001.html>