[llvm-dev] RFC: Pass Execution Instrumentation interface

[Fedor Sergeev via llvm-dev]

TL;DR
====

This RFC suggests an API to enable customizable instrumentation of pass 
execution.
The intent is to have a common machinery to implement all the 
pass-execution-debugging
features mentioned here.

Prime target of the interface is the new pass manager.
The overall approach and most of the implementation details should be 
equially applicable
to the legacy one though.


Background
==========

There are quite a few important debugging facilities in LLVM that affect
pass execution sequence:

    -print-after/-print-before[-all]
        execute IR-print pass before or after a particularly 
insteresting pass
        (or all passes)

    -verify-each
        execute verifier pass after each

    -opt-bisect-limit
        execute passes only up to a selected "pass-counter"

    -time-passes
        track execution time for each pass

There are also quite a few similar ideas floating around, i.e:
    -git-commit-after-all
    -debugify-each

All these facilities essentially require instrumentation of pass 
execution points
in the pass manager, each being implemented in a legacy pass manager 
through their
own custom way, e.g:
   * -time-passes has a bunch of dedicated code in each of the pass managers

   * -print-before/after/verify-each insert additional passes before/after
     the passes in the pipeline

And there is no implementation of any of these features for the new pass 
manager,
which obviously is a problem for new pass manager transition.

Proposal
========

Main idea:
   - introduce an API that allows to instrument points of pass execution
   - access through LLVM Context (allows to control life-time and scope 
in multi-context execution)
   - wrap it into an analysis for easier access from pass managers


Details:
   1. introduce llvm::PassInstrumentation

      This is the main interface that handles the customization and 
provides instrumentation calls

      - resides in IR
      - is accessible through LLVMContext::getPassInstrumentation()
        (with context owning this object).

   2. every single point of Pass execution in the (new) PassManager(s) 
will query
      this analysis and run instrumentation call specific to a 
particular point.

      Instrumentation points:

         bool BeforePass (PassID, PassExecutionCounter);
         void AfterPass (PassID, PassExecutionCounter);

          Run before/after a particular pass execution
              BeforePass instrumentation call returns true if this 
execution is allowed to run.

             'PassID'
                  certain unique identifier for a pass (pass name?).

             'PassExecutionCounter'
                  a number that uniquely identifies this particular pass 
execution
                  in current pipeline, as tracked by Pass Manager.

         void StartPipeline()
         void EndPipeline()

          Run at the start/end of a pass pipeline execution.
          (useful for initialization/finalization purposes)


   3. custom callbacks are registered with 
PassInstrumentation::register* interfaces

      A sequence of registered callbacks is called at each 
instrumentation point as appropriate.

   4. introduce llvm::ExecutionCounter to track execution of passes

      (akin to DebugCounter, yet enabled in Release mode as well?)

      Note: it is somewhat nontrivial to uniquely track pass executions 
with counters in new pass
      manager as its pipeline schedule can be dynamic. Ideas are welcome 
on how to efficiently
      implement unique execution tracking that does not break in 
presence of fixed-point iteration
      passes like RepeatedPass/DevirtSCCRepeatedPass

      Also, the intent is for execution counters to be able provide 
thread-safety in multi-threaded
      pipeline execution (though no work planned for it yet).

   5. introduce a new analysis llvm::PassInstrumentationAnalysis

      This is a convenience wrapper to provide an access to 
PassInstrumentation via analysis framework.
      If using analysis is not convenient (?legacy) then 
PassInstrumentation can be queried
      directly from LLVMContext.


Additional goals
================

   - layering problem
     Currently OptBisect/OptPassGate has layering issue - interface 
dependencies on all the "IR units",
     even those that are analyses - Loop, CallGraphSCC.

     Generic PassInstrumentation facilitiy allows to inject arbitrary 
call-backs in run-time,
     removing any compile-time interface dependencies on internals of 
those callbacks,
     effectively solving this layering issue.

   - life-time/scope control for multi-context execution

     Currently there are issues with multi-context execution of, say, 
-time-passes which store
     their data in global maps.

     With LLVMContext owning PassInstrumentation there should be no 
problem with multi-context execution
     (callbacks can be made owning the instrumentation data).

Open Questions
==============

   - whats the best way to handle ownership of PassInstrumentation

     Any problems with owning by LLVMContext?
     Something similar to TargetLibraryInfo (owned by 
TargetLibraryAnalysis/TargetLibraryInfoWrapperPass)?

   - using PassInstrumentationAnalysis or directly querying LLVMContext

     PassInstrumentationAnalysis appeared to be a nice idea, only until 
I tried querying it
     in new pass manager framework, and amount of hooplas to jump over 
makes me shiver a bit...

     Querying LLVMContext is plain and straightforward, but we do not 
have a generic way to access LLVMContext
     from a PassManager template (need to introduce generic 
IRUnit::getContext?)

Implementation
==============

PassInstrumentationAnalysis proof-of-concept unfinished prototype 
implementation:
(Heavily under construction, do not enter without wearing a hard hat...)

    https://reviews.llvm.org/D47858