[llvm-dev] [XRay] Build instrumented Clang, some analysis results

[C Bergström via llvm-dev]

On Wed, Jul 20, 2016 at 6:26 PM, Dean Michael Berris
<dean.berris at gmail.com> wrote:
>
>> On 20 Jul 2016, at 20:02, C Bergström <cbergstrom at pathscale.com> wrote:
>>
>> Some general questions about X-Ray
>> -------------
>> Is there a plan to make a separate mailing list or project around
>> this? Do you have a list of planned features?
>
> Interesting question -- so far we haven't decided yet whether XRay will live as another project. I'm certainly open to this possibility. No concrete plans yet. It's an open question in the original RFC too (http://lists.llvm.org/pipermail/llvm-dev/2016-April/098901.html).
>
> There's a white paper that details what we plan to implement out in the open (http://research.google.com/pubs/pub45287.html). We're still working our way to getting to a full version as described in that white paper (basically blocked by my lack of familiarity with the LLVM codebase, and other n00b-y things :D).
>
> There's not a concrete list of features, and we're certainly open to contributions from the community to add features that make sense. :)
>
>>
>> Graphics tools for analysis? AMD open sourced their CodeAnalyst - What
>> about some integration with that?
>>
>
> Thanks for the pointer! Yes, I'd love to have more integration with existing visualisation tools that read a particular well-documented format. Others have mentioned Jumpshot which might be a little dated, but still something that some people use for similar things.
>
>> Linux + Perf support (planned/exists)?
>>
>
> There are no plans to support the perf counter-lookups yet. Although I certainly think that's a nice source of data to be logging XRay-style.
>
> FWIW, the API for XRay allows us to decouple the things being logged at function entry/exit. Getting performance counters at those points is a nice idea, it should be doable.
>
>> How much is this tied to something specific about Linux or it could be
>> easily ported to another platform?
>
> Currently, the only Linux-specific part I can remember is getting the cpu frequency (looking at sysfs files). That can be implemented on a platform-agnostic (or at least pluggable and portable) manner.
>
> There are x86'isms and I'm working on understanding how to do this in Aarch64 or ARM.

Ack - actually x86 probably makes some of this a lot easier. I'm
recently (frequently) annoyed (as hell) with how AArch64 isn't
exposing a bunch of basic things that I *want* (demand!) to know.

For example:
clock frequency on AArch64 + Linux == forget it. I had to use a
benchmark in order to basically brute force calculate some processors.
(They don't hard code it in /proc/cpuinfo or sys as you'd want) and
I'm really uncertain about what happens if there's stepping involved
(current AArch64 processors that I'm aware of don't have this feature
though) /* Maybe Google can help kick the linux devs into accepting
these patches */

For FBSD and iOS - I don't know how/if they expose this information..
(Is FBSD ported to AArch64 yet.. ?)

>
>>
>> What's the benefit of this vs a stable and production ready tool like Dtrace?
>>
>
> I think I've pointed out the differences in a separate mail (some mail filters may have squashed that response, so apologies if that was missed): http://lists.llvm.org/pipermail/llvm-dev/2016-July/101922.html -- the short version is:
>
> - Dtrace requires kernel-side support.
> - XRay is completely in-process and controllable by the process through an API (not sure if dtrace is the same).
> - XRay is selective and configurable by the application developer.
> - XRay's cost is borne by the application only, and does not require stopping the application.

Just as you're instrumenting around functions - DTrace can similarly
inject "probes" (basically the same thing) - The other more common way
for DTrace to be used is for the application to not be changed and
it's just profiled. (Ok you must leave SP otherwise it won't work.. so
for the purist I guess you're relying on applications not to be
/fully/ optimized.. I forget if DWARF or CFW is required, but I don't
think so )

Dtrace also doesn't require stopping the application fwiw and you can
control probably a lot more of what's probed/instrumented. (There's a
full scripting langauge in order to control what you instrument
actually)

I'm not trying to take away from X-Ray, I think profiling is extremely
important, but I'm just wondering how much (if any) evaluation of
existing solutions was done. Maybe the DTrace licensing, CTF dep or
linux support was a dealbreaker, I just hate to see NIH when there's
good tools available that cover a significant amount of the needs or
more.

At the end of the day it's probably quite complementary, like all the
work you do for X-Ray, someone could likely leverage to automatically
inject DTrace probes and get a lot of the same stuff.

In your response you mentioned "O(100) cycles" - is that 100
instructions of "skid" between point of measurement? (Seems really
high for instrumenting, but maybe I'm mistaken..)

Lastly and again just side comments - in terms of data formats - JSON
has pretty good support, streams and compresses nicely, high
performance parses exist with liberal licensing as well as I think
there's a binary version of it. This could be handy *if* your app is
on Node B and you'd like the logs to be sent to Node A.

Anywho - cool work..