[llvm-dev] RFC: XRay -- A Function Call Tracing System

[Sanjoy Das via llvm-dev]

Hi Dean, Eric,

This is great!  I'll wait for others from the community to chime in,
but I'm happy to see this land upstream.

 > details are contained in the whitepaper at:
 > https://storage.googleapis.com/xray-downloads/whitepaper/XRayAFunctionCallTracingSystem.pdf

In the code that you patch in, have you considered excising the "r10d
= <function id>" bit and just having a 5 byte call to
__xray_FunctionEntryStub?  If "<function id>" is a function of the PC
being instrumented then you should be able to reconstruct it by
peeking into the return PC on the stack.  This means your return
sequence will have to call @__xray_FunctionExitStub (instead of
branching), so that the return PC on the stack is correct.

Using a 5 byte call will let you keep one 5 byte nop instead of the 10/11
byte nop sequence you have today (will really only help for the prologue).

 > *LLVM:* Add new LLVM pseudo instructions for indicating where the
 > instrumentation sleds should be placed, and what kind of sleds these
 > are. For entry sleds, we can use the `PATCHABLE_OP` instruction, and a
 > new instruction for patching exits (`PATCHABLE_RET`?) that get handled
 > by the assembly printer per platform. Currently only implemented for
 > x86_64.

Nitpicky comment -- we can discuss this in more detail during code
review:

I'd prefer to add more functionality to `PATCHABLE_OP` instead of
introducing yet one more pseudo instruction.  `PATCHABLE_OP` can
already wrap an arbitrary machine instruction, teaching it a few
different flavors of nop-insertion should not be a big deal.

-- Sanjoy