[LLVMdev] design question on inlining through statepoints and patchpoints

[Sanjoy Das]

I've been looking at inlining invokes / calls done through statepoints
and I want to have a design discussion before I sink too much time
into something I'll have to throw away.  I'm not actively working on
adding inlining support to patchpoints, but I suspect these issues are
applicable towards teaching LLVM to inline through patchpoints as
well.


There are two distinct problems to solve before LLVM can inline
through statepoints:

# Managing data flow for the extra metadata args.

LLVM needs some logic to "transfer" the extra live values attached to
a statepoint/patchpoint into the body of the inlinee somehow.  How
this is handled depends on the semantics of the live values (something
the frontend knows).  There needs to be a clean way for the frontend
to communicate this information to LLVM, or we need to devise a
convention that is sensible for the kinds of frontends we wish to
support.  Initially I plan to sidestep this problem by only inlining
through statepoints have *no* extra live values / gc pointers.


# Managing the call graph

This is the problem we need to solve first.  Currently LLVM views the
a statepoint or patchpoint call as

  1. A call to an intrisic.  This does not add an edge to the call
     graph (not even to the dedicated external node).

  2. An escaping use of the callee.

IIUC, (2) is (conservatively) imprecise and (1) is incorrect.  (1)
makes LLVM believe that a function that calls @f via a statepoint does
not call @f at all.  (2) makes LLVM believe that @f is visible
externally, even if it has internal linkage.

Given this starting point, I can think of three ways to model
statepoint's (and patchpoint's) control flow semantics within a call
graph:

  1. Model calls to statepoint, patchpoint and stackmap intrinsics as
     calling the external node.  Teach the inliner pass to
     "devirtualize" calls through statepoints when posssible, except
     that the "devirtualization" is only a facade (i.e. we don't
     mutate the IR to change the statepoint to a direct call).  We add
     some abstraction to the inlining utility functions to inline
     through something more general than a CallSite.

  2. Introduce a new abstraction InlineSite (bikeshedding on the name
     is welcome).  InlineSite sits on top of a CallSite and knows how
     to extract the semantic call out of a statepoint or a patchpoint
     (similar to the llvm::Statepoint class).  The inliner and the
     call graph analysis works on top of this InlineSite abstraction
     instead of the CallSite abstraction.

  3. Change all the places that matter (CallGraph, CallGraphSCCPass
     etc.) from

       if (CallSite CS = ...)

     to

       if (Statepoint SP = ...)
          ...
       else if (CallSite CS = ...)

     or something equivalent to this.

Personally, I'd prefer going with (1) if it is viable, and (2) if not.

What do you think?

-- Sanjoy