[PATCH][PM] Add pass run listeners to the pass manager.

[Juergen Ributzka]

On May 9, 2014, at 6:01 PM, Chandler Carruth <chandlerc at google.com> wrote:

> On Thu, May 8, 2014 at 3:12 PM, Juergen Ributzka <juergen at apple.com> wrote:
> Hi Chandler,
> 
> I have a quick question for you. Are any of the remaining issues related to the C API?
> If not, I would like to lock down the C API rather sooner than later. We can fix the
> implementation details of the stuff that happens behind the curtains in followup commits.
> 
> Sadly, yes, I think the core of the design shows up in the C API.
> 
> Quickly, a few naming issues. First, I much prefer the nomenclature of Callback to Listener, mostly because Callback is already reasonably widely used in the C APIs.
> 
> Second, while the design makes perfect sense, the term "Invocation" isn't sitting well with me:
> 
> +/** @see llvm::PassInvocation */
> +typedef struct LLVMOpaquePassInvocation *LLVMPassInvocationRef;
> +
> +typedef enum {
> +  LLVMPassPreRun,
> +  LLVMPassPostRun
> +} LLVMPassInvocationType;
> 
> Here we have a bunch of things oddly conflated into the term "Invocation". I would probably call the struct LLVMPassRunInfo, because its really a limited set of information pertaining to a single run of a pass over some chunk of IR. I'd like to avoid inventing a new name for that (invocation). It also seems like the enum isn't really pertaining to the Invocation (or RunInfo or whatever) but more to the callback -- ie, at what phase was the callback delivered. Maybe call the function LLVMAddPassRunCallback and the enum LLVMPassRunCallbackType? That would then suggest the result is an LLVMPassRunCallbackRef, and the function pointer argument type would be a LLVMPassRunCallback I think, but I may have swirled the names around too many times.
> 
> 

Sure. These can be changed easily.

> 
> Ok, after the naming bits, there is only one high level thing I don't really understand. Not trying to say its wrong, I just find it surprising. I would have tried to design it the other way, but there may be some reason that doesn't work.
> 
> +/**
> + * Create and add a new pass listener to the context.
> + *
> + * This transfers the owenership of the pass listener to the context. When the
> + * context is disposed, then all the listeners are disposed too.
> + */
> +LLVMPassListenerRef LLVMAddPassListener(LLVMContextRef, void *,
> +                                        LLVMPassListenerCallback);
> 
> This ties the the callback to the Context in a way I find surprising. Naively (and especially with the new pass manager, but not exclusively) I would have imagined that you would attach the callback to the LLVMPassManagerRef returned by one of the LLVMCreate.*PassManager functions. These "roots" of the current implicitly-built pass manager tree seem like reasonably places to hook into in both the current and future setup... but maybe that doesn't work for a reason I don't yet understand.

I originally had this hooked-up to the pass manager, but that would only catch the optimizer passes that were created and added by the client. All the codegen passes are hidden in a separate temporary pass manger in the MCJIT. This would have require adding additional C API, that would attach the callback to a JIT engine and then the engine would attach it to the pass manager every time it creates a new one. Adding it to the context made this callback automatically visible to all pass managers running in the given context regardless of when and where they are created.

> 
> From chatting briefly with Andy, it seems like the use case really isn't to find out how far along a particular pass pipeline has made it on a module (or function, or whatever), but instead to simply observe the progress of any and all compilation activity taking place within an LLVM context? In particular, the problem is that you have a need to understand where within a JIT's compilation it is safe to interrupt things without leaving mutexs, resources, etc held. Is that the right understanding of the use case?
> 
> There is a different use case for JIT clients to essentially be able to keep track of the progress of your JIT's compilation. However, I think trying to solve these two use cases with the same technique is risky. For example, if we ever end up with some parallelism *within* the compilation process (which I *really* want to see happen in the not-too-distant future), you might reasonably want the reporting of progress to happen asynchronously, which makes it no longer serve the goal of knowing where safe interruption/quiescent points are in the JIT.
> 
> Still thinking about this underlying design issue to try to understand what the best path forward is, but at least for me, I hadn't really internalized what the real use case was, and so I'm now having to retrace my thinking on the design.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140510/f42676aa/attachment.html>