[llvm-dev] RFC: Comprehensive Static Instrumentation

Fri Jun 17 13:22:02 PDT 2016

> On Jun 17, 2016, at 11:56 AM, Reid Kleckner <rnk at google.com> wrote:
> 
> Mehdi, I think TB is saying that LTO is actually *more* incremental than standard compilation for people writing instrumentation tools.
> 
> It's far simpler to rebuild your CSI instrumentation hooks and re-run LTO on Apache than it is to re-run someone else's complicated build system.

I *highly* doubt about that. I’d even say that I’m heavily convinced of the opposite: any build system has a facility to add a compiler flag. What could be easier than adding `-csi=path_to_my_tool` to the CFLAGS?

On the other hand, LTO is a whole other beast. First you also need to modify the CFLAGS to enable LTO, so you don’t avoid the “complicated build system” interaction.
But then you’ll need to make sure the rest of the toolchain is compatible (Never had an issue with static archives? I did a lot of time!). Also what if the projects involves subprojects with dylib? You need to make sure every link is correctly performed with the instrumented runtime.

Nothing “impossible” here, but I don’t buy at all the “it is simpler” argument.

> In fact, you save all the time that clang would spend parsing C and C++ source code, since you're starting with several blobs of semi-optimized LLVM IR.

LTO alone on the clang binary is longer than a full clean build (and if you build all of the llvm binaries, we one or two order of magnitudes away), so it won't buy you anything. And even if it would (apparently on Apache httpd it would), I doubt it would be significant enough to make it an argument to *tie* the design to LTO.

> It's worth exploring what you mentioned up-thread, where you inline the instrumentation into the code during normal compliation instead of waiting until link time, but I think it makes more sense to make it easy to build tools before we try to make those new tools compile faster.

So a few line earlier your argument for relying on LTO was to save “parsing C and C++” (so basically compile time), and now you’re saying that “making those new tools compile faster” should not drive the design?

I agree with “making it easy to build tools” part, I just don’t see any reason why forcing to use LTO supports this goal.

> It seems like a nice-to-have feature for instrumentation tools that manage to graduate from research idea to production tool.

I believe this last sentence isn’t related to the LTO aspect, CSI looks like a great idea, let’s just do it in a way that it does not require LTO.

— 
Mehdi

> 
> On Fri, Jun 17, 2016 at 11:42 AM, Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> This reduction in the number of compile operations needed, and in the number intermediate object/bitcode files produced, is indeed an advantage of the CSI approach. 
> 
> It is a very artificial advantage, what are you saving? Temporary Disk Space?
> 
> 
>> As an aside, we've been experimenting with linking CSI-instrumented bitcodes against the "null tool," which implements every instrumentation hook as a nop, and comparing the performance of those binaries against production binaries.  Our preliminary tests have shown some promising results.  For generating main executables, using LTO to link CSI-instrumented bitcodes with the null tool produces executables that are as fast as the production executables.  For generating dynamic libraries, however, using LTO to link the CSI-instrumented bitcode of a dynamic library with the null tool seems to produce a binary that is slower than production.  (The Apache HTTP server benchmark we've tried runs roughly 30% slower when using such null-tool-instrumented dynamic libraries.)  These results suggest that using LTO to link CSI-instrumented bitcodes with the null tool is almost, but not quite, able to produce binaries with production performance
> 
> These results suggests that “adding instrumentation has a cost” nothing more, and is unrelated to LTO at all.
> You would provide the runtime to the compiler directly during the compile phase and you would get the same results.
> 
> 
>> , which would allow tool users to only compile their sources once.
> 
> LTO means basically “compiles during the link”. You won’t save much.
> 
> I haven’t seen a single compelling argument to *tie* CSI to LTO in this thread until now.
> 
> 
> — 
> Mehdi
> 
> 
> 
> 
> 
>>  
>> 
>> It's possible that the math doesn't really work out in practice if the cost of the LTO-link dwarfs the compile times.
>> -- 
>> Employee of Qualcomm Innovation Center, Inc.
>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
>> 
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160617/4c07dbbd/attachment.html>