<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Jun 22, 2016, at 10:36 PM, TB Schardl via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div style="font-size: 12.8px;" class=""><div style="font-size: 12.8px;" class="">Hey David and Evgenii,</div><div style="font-size: 12.8px;" class=""><br class=""></div><div style="font-size: 12.8px;" class="">Thank you for your responses; they clarify a lot.</div><div style="font-size: 12.8px;" class=""><br class=""></div><div style="font-size: 12.8px;" class="">We like the idea of allowing tool users to optimize instrumentation without needing LTO, and based on our understanding of your model, we think that the existing CSI system can support this usage model with minimal changes. </div></div></div></div></blockquote><div><br class=""></div><div>That was my point from the beginning.</div><div><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div style="font-size: 12.8px;" class=""><div style="font-size: 12.8px;" class=""><span style="font-size: 12.8px;" class="">To describe the modification, I've attached a diagram of the compilation flow for CSI's existing design, assuming the tool-user enables LTO. (Gray rectangles delineate the concerns of the tool-user, the tool-writer, and the CSI-provided libraries. The tan shapes denote source, bitcode, or executable units for the program under test. The blue shapes denote units for the tool. Components of the CSI system itself are colored orange.)</span></div><div style="font-size: 12.8px;" class=""><br class=""></div><div style="font-size: 12.8px;" class="">Conceptually, supporting the usage model of optimizing of instrumentation at compile time, a model we'll call CSI:CTO, involves a simple change. T<span style="font-size: 12.8px;" class="">he simplest option seems to be to have the tool-user link in the tool bitcode (specifically, box O in the diagram) when compiling each source unit, just as Evgenii described, and then to run optimizations after the CSI pass. One problem with this approach is the introduction of multiple copies of the tool-writer's defined instrumentation hooks, which creates errors at static link time. Based on some experiments with -mlink-bitcode-file and Evgenii's suggestion, it seems that we can resolve these duplicate-symbol issues by setting the linkage of the CSI tool symbols to "linkonce_odr."</span></div><div style="font-size: 12.8px;" class=""><br class=""></div><div style="font-size: 12.8px;" class="">Here is a sketch of the modifications to the existing compilation flow in order to support CSI:CTO:</div></div><span style="font-size: 12.8px;" class=""><div style="font-size: 12.8px;" class=""><br class=""></div><blockquote style="margin: 0px 0px 0px 40px; border: none; padding: 0px;" class=""><div class=""><div style="font-size: 12.8px;" class="">1) The tool-user passes the null-default tool bitcode to the CSI pass, e.g., by issuing "clang -fcsi=<tool>.bc ..." to compile each source unit. (In the diagram, this flow corresponds to new edges from box O to boxes C and G.)</div></div><div class=""><div style="font-size: 12.8px;" class="">2) In addition to inserting instrumentation hooks throughout the IR of the translation-unit being compiled, the CSI pass also ensures that the symbols of <tool>.bc have "linkonce_odr" linkage.</div></div><div class=""><div style="font-size: 12.8px;" class="">3) Optimization passes run after the CSI pass to optimize the instrumentation, which includes eliding null hooks.</div></div></blockquote><blockquote style="margin: 0px 0px 0px 40px; border: none; padding: 0px;" class=""><div style="font-size: 12.8px;" class="">4) The tool bitcode no longer has to be linked in at link time.</div></blockquote><div class=""><div style="font-size: 12.8px;" class=""><br class=""></div><div style="font-size: 12.8px;" class="">We believe that CSI:CTO achieves the effects compile-time-optimization effect you describe and is a simple addition to the existing CSI system. Is there any functionality in the scheme you propose that CSI:CTO seems to miss? Of course, we might have overlooked some implementation issues in this design sketch, but <span style="font-size: 12.8px;" class="">we plan to explore this scheme further and add support for CSI:CTO on top of the existing system.</span></div><div style="font-size: 12.8px;" class=""><br class=""></div><div style="font-size: 12.8px;" class="">One issue with CSI:CTO is the risk that the tool-user incorporates the wrong tool by mistake when compiling some unit of his program. </div></div></span></div></div></blockquote><div><br class=""></div><div>Yes, but just as he may not enable LTO by mistake on some units of his program. Having consistent CFLAGS is an issue in both cases. </div><div>I’d try make sure we can easily diagnose these situations: for instance a script could use the linker map to find all the input objects and inspect them to figure how they were instrumented (we’d need to make it possible to figure from an object file how it was instrumented of course).</div><div><br class=""></div><div>— </div><div>Mehdi</div><div><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-size: 12.8px;" class=""><div class=""><div style="font-size: 12.8px;" class=""> One advantage of using LTO/ThinLTO is that it avoids many potential instances of this issue by incorporating the tool bitcode at only one point in the compilation flow of the program-under-test. Nevertheless, the option of incorporating the tool and optimizing the instrumentation at compile time seems appealing. We can investigate techniques to avoid the inclusion of different tools in different units down the road.</div><div style="font-size: 12.8px;" class=""><br class=""></div><div style="font-size: 12.8px;" class="">Cheers,</div><div style="font-size: 12.8px;" class="">TB</div></div></span></div><div class="gmail_extra" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class=""><div class="gmail_quote">On Mon, Jun 20, 2016 at 7:27 PM, Evgenii Stepanov<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:eugenis@google.com" target="_blank" class="">eugenis@google.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;">I like this non-LTO use model a lot. In my opinion, it does not<br class="">conflict with the LTO model - the only difference is when the<br class="">implementation of CSI hooks is injected into the build - compile time<br class="">vs link time. I don't see any problem with injecting it both at<br class="">compile and link time - this would take care of CSI hooks in third<br class="">party libraries.<br class=""><br class="">I also don't understand the need of the export list. As I see it, the<br class="">IR library of the tool is loaded at compilation time, internalized and<br class="">merged with the IR for the module being compiled. If any CSI hooks<br class="">remain undefined at this point, they are replaced with empty<br class="">functions. The IR tool library should not have any global state, of<br class="">course - it is an equivalent of the C++ header file. Exactly the same<br class="">happens at LTO link time, in case the link brought in new CSI hooks<br class="">unresolved at compilation time.<br class=""><br class="">ThinLTO uses a different build model - it's not compile+link, but<br class="">compile+merge+compile+link or something like that AFAIK. This requires<br class="">build system changes, and I imagine the extra work would not be worth<br class="">the benefits for many smaller projects. I'd love if CSI worked without<br class="">any kind of LTO with reasonable performance (btw, do you have the<br class="">numbers for the slowdown with an empty tool without LTO?).<br class=""><br class="">On Mon, Jun 20, 2016 at 4:00 PM, TB Schardl via llvm-dev<br class=""><div class="HOEnZb"><div class="h5"><<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class="">> Hey David,<br class="">><br class="">> Thank you for your feedback. I'm trying to understand the model you<br class="">> sketched in order to compare it to CSI's current approach, but the details<br class="">> of your proposal are still fuzzy to me. In particular, although the model<br class="">> you described would avoid using LTO to elide unused hooks, it seems more<br class="">> complicated for both tool writers and tool users to use. Please clarify<br class="">> your model and shed some light on the questions below.<br class="">><br class="">> 1) What does the "CSI dev tool" you describe look like? In particular, how<br class="">> does the tool writer produce and maintain an export list of non-stub hook<br class="">> definitions for her tool? Maintaining an export list manually would seem to<br class="">> be an error-prone hassle. One could imagine generating the export list<br class="">> automatically with a custom compile-time pass that the tool-writer runs, but<br class="">> maintaining consistency between a tool and its export list remains a concern<br class="">> for both the tool writer and the tool user.<br class="">><br class="">> 2) To clarify, in your scheme, the tool writer produces an export list as<br class="">> well as an object/bitcode for the tool. The tool user compiles the<br class="">> program-under-test using the export list, and then incorporates the tool<br class="">> object/bitcode at link time. Is this what you have in mind?<br class="">><br class="">> 3) Do I understand correctly that your model only gets rid of the need for<br class="">> LTO to elide unused instrumentation hooks? In particular, are other<br class="">> optimizations on the instrumentation still contingent on LTO (or ThinLTO)?<br class="">> One nice feature of the current design is that tool writers can use<br class="">> properties passed to hooks to elide instrumentation conditionally based on<br class="">> common static analysis. It looks like optimizations based on properties are<br class="">> still possible in the model you propose, but as with the current design,<br class="">> they would still rely on LTO or ThinLTO.<br class="">><br class="">> 4) In your model, if the tool user wants to analyze his program with several<br class="">> different tools, then he must recompile his program from source once for<br class="">> each tool. Is this correct? The argument in this thread seems to be that<br class="">> recompiling the program from source is no worse than using LTO because LTO<br class="">> incurs high overhead. (From the results I've found online<br class="">><span class="Apple-converted-space"> </span><a href="http://llvm.org/devmtg/2015-04/slides/ThinLTO_EuroLLVM2015.pdf" rel="noreferrer" target="_blank" class="">http://llvm.org/devmtg/2015-04/slides/ThinLTO_EuroLLVM2015.pdf</a>, however,<br class="">> ThinLTO seems to be much lower overhead than LTO. Are these results still<br class="">> accurate?)<br class="">><br class="">> 5) What happens when the program-under-test is built from multiple source<br class="">> files? It seems that all source files must be compiled using the same<br class="">> export list, which is a burden on the tool user. Although this complexity<br class="">> could be managed by the program's build system, managing this complexity<br class="">> through the build system or by some other means still seems like a burden on<br class="">> the tool user.<br class="">><br class="">> 6) What happens when the program-under-test uses third-party libraries? If<br class="">> the tool user does not have access to the sources of those libraries, then I<br class="">> don't see how they can properly elide hooks using an export list. At best,<br class="">> it would seem that the library writer would distribute a object/bitcode for<br class="">> the library with all hooks in place already, which is exactly what happens<br class="">> with CSI's the existing design.<br class="">><br class="">> Thanks again for your feedback.<br class="">><br class="">> Cheers,<br class="">> TB<br class="">><br class="">> On Sun, Jun 19, 2016 at 6:45 PM, Xinliang David Li <<a href="mailto:xinliangli@gmail.com" class="">xinliangli@gmail.com</a>><br class="">> wrote:<br class="">>><br class="">>> It is great that CSI does not depend on lto for correctness, but I think<br class="">>> it should remove the dependency completely. Being able to work with lto<br class="">>> should be something that is supported, but lto is not required.<br class="">>><br class="">>> Such a design will greatly increase CSI usability not only for tool<br class="">>> users(app developers), but also for CSI tool developers.<br class="">>><br class="">>> My understanding is that it is not far to get there. Here is the CSI use<br class="">>> model I am thinking:<br class="">>><br class="">>> 1) a CSI dev tool can be provided to tool developers to produce a) hook<br class="">>> library in native object format b) hook lib in bitcode format c) export list<br class="">>> of non stub hook defs<br class="">>><br class="">>> 2) the compiler implementation of CSI lowering will check the export list<br class="">>> and decide whether to lower a hook call to noop or not<br class="">>><br class="">>> 3) in thinlto mode, the bitcode can be readin for inlining. This works for<br class="">>> lto too<br class="">>><br class="">>> 4) compiler driver can hide all the above from the users<br class="">>><br class="">>><br class="">>> In short, making it work in default mode makes it easier to deploy and can<br class="">>> get you a long way<br class="">>><br class="">>> Thanks,<br class="">>><br class="">>> David<br class="">>><br class="">>><br class="">>> On Sunday, June 19, 2016, TB Schardl <<a href="mailto:neboat@mit.edu" class="">neboat@mit.edu</a>> wrote:<br class="">>>><br class="">>>> Hey Peter and David,<br class="">>>><br class="">>>> Thank you for your comments.<br class="">>>><br class="">>>> As mentioned elsewhere, the current design of CSI does not rely on LTO<br class="">>>> for correctness. The tool-instrumented executable will run correctly even<br class="">>>> if the linker performs no optimization. In particular, unused<br class="">>>> instrumentation hooks are implemented by default as nop functions, which<br class="">>>> just return immediately. CSI is a system that can use LTO to improve tool<br class="">>>> performance; it does not require LTO to function.<br class="">>>><br class="">>>> One of our considerations when developing CSI version 1 was design<br class="">>>> simplicity. As such, CSI version 1 essentially consists of three<br class="">>>> components:<br class="">>>> 1) A compile-time pass that the tool user runs to insert instrumentation<br class="">>>> hooks.<br class="">>>> 2) A null-tool library that provides default nop implementations for each<br class="">>>> instrumentation hook. When a tool writer implements a tool using the CSI<br class="">>>> API, the tool writer's implemented hooks override the corresponding default<br class="">>>> implementations.<br class="">>>> 3) A runtime library that implements certain powerful features of CSI,<br class="">>>> including contiguous sets of ID's for hooks.<br class="">>>><br class="">>>> We've been thinking about how CSI might work with -mlink-bitcode-file.<br class="">>>> From our admittedly limited understanding of that feature, it seems that a<br class="">>>> design that uses -mlink-bitcode-file would still require something like the<br class="">>>> first and third components of the existing design. Additional complexity<br class="">>>> might be needed to get CSI to work with -mlink-bitcode-file, but these two<br class="">>>> components seem to be core to CSI, regardless of whether -mlink-bitcode-file<br class="">>>> is used. (Eliminating the null-tool library amounts to eliminating a pretty<br class="">>>> simple 39-line C file, which at first blush doesn't look like a big win in<br class="">>>> design complexity).<br class="">>>><br class="">>>> CSI focuses on making it easy for tool writers to create many<br class="">>>> dynamic-analysis tools. CSI can leverage standard compiler optimizations to<br class="">>>> improve tool performance, if the tool user employs mechanisms such as LTO or<br class="">>>> thinLTO, but LTO itself is not mandatory. It might be worthwhile to explore<br class="">>>> other approaches with different trade-offs, such as -mlink-bitcode-file, but<br class="">>>> the existing design doesn't preclude these approaches down the road, and<br class="">>>> they will be able to share the same infrastructure. Unless the other<br class="">>>> approaches are dramatically simpler, the existing design seems like a good<br class="">>>> place to start.<br class="">>>><br class="">>>> Cheers,<br class="">>>> TB<br class="">>>><br class="">>>> On Fri, Jun 17, 2016 at 4:25 PM, Peter Collingbourne via llvm-dev<br class="">>>> <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class="">>>>><br class="">>>>><br class="">>>>><br class="">>>>> On Thu, Jun 16, 2016 at 10:16 PM, Xinliang David Li via llvm-dev<br class="">>>>> <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class="">>>>>><br class="">>>>>><br class="">>>>>><br class="">>>>>> On Thu, Jun 16, 2016 at 3:27 PM, Mehdi Amini via llvm-dev<br class="">>>>>> <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class="">>>>>>><br class="">>>>>>> Hi TB,<br class="">>>>>>><br class="">>>>>>> Thanks for you answer.<br class="">>>>>>><br class="">>>>>>> On Jun 16, 2016, at 2:50 PM, TB Schardl <<a href="mailto:neboat@mit.edu" class="">neboat@mit.edu</a>> wrote:<br class="">>>>>>><br class="">>>>>>> Hey Mehdi,<br class="">>>>>>><br class="">>>>>>> Thank you for your comments. I've CC'd the CSI mailing list with your<br class="">>>>>>> comments and put my responses inline. Please let me know any other<br class="">>>>>>> questions you have.<br class="">>>>>>><br class="">>>>>>> Cheers,<br class="">>>>>>> TB<br class="">>>>>>><br class="">>>>>>> On Thu, Jun 16, 2016 at 3:48 PM, Mehdi Amini <<a href="mailto:mehdi.amini@apple.com" class="">mehdi.amini@apple.com</a>><br class="">>>>>>> wrote:<br class="">>>>>>>><br class="">>>>>>>><br class="">>>>>>>> On Jun 16, 2016, at 9:01 AM, TB Schardl via llvm-dev<br class="">>>>>>>> <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class="">>>>>>>><br class="">>>>>>>> Hey LLVM-dev,<br class="">>>>>>>><br class="">>>>>>>> We propose to build the CSI framework to provide a comprehensive<br class="">>>>>>>> suite of compiler-inserted instrumentation hooks that dynamic-analysis tools<br class="">>>>>>>> can use to observe and investigate program runtime behavior. Traditionally,<br class="">>>>>>>> tools based on compiler instrumentation would each separately modify the<br class="">>>>>>>> compiler to insert their own instrumentation. In contrast, CSI inserts a<br class="">>>>>>>> standard collection of instrumentation hooks into the program-under-test.<br class="">>>>>>>> Each CSI-tool is implemented as a library that defines relevant hooks, and<br class="">>>>>>>> the remaining hooks are "nulled" out and elided during link-time<br class="">>>>>>>> optimization (LTO), resulting in instrumented runtimes on par with custom<br class="">>>>>>>> instrumentation. CSI allows many compiler-based tools to be written as<br class="">>>>>>>> simple libraries without modifying the compiler, greatly lowering the bar<br class="">>>>>>>> for<br class="">>>>>>>> developing dynamic-analysis tools.<br class="">>>>>>>><br class="">>>>>>>> ================<br class="">>>>>>>> Motivation<br class="">>>>>>>> ================<br class="">>>>>>>><br class="">>>>>>>> Key to understanding and improving the behavior of any system is<br class="">>>>>>>> visibility -- the ability to know what is going on inside the system.<br class="">>>>>>>> Various dynamic-analysis tools, such as race detectors, memory checkers,<br class="">>>>>>>> cache simulators, call-graph generators, code-coverage analyzers, and<br class="">>>>>>>> performance profilers, rely on compiler instrumentation to gain visibility<br class="">>>>>>>> into the program behaviors during execution. With this approach, the tool<br class="">>>>>>>> writer modifies the compiler to insert instrumentation code into the<br class="">>>>>>>> program-under-test so that it can execute behind the scene while the<br class="">>>>>>>> program-under-test runs. This approach, however, means that the development<br class="">>>>>>>> of new tools requires compiler work, which many potential tool writers are<br class="">>>>>>>> ill equipped to do, and thus raises the bar for building new and innovative<br class="">>>>>>>> tools.<br class="">>>>>>>><br class="">>>>>>>> The goal of the CSI framework is to provide comprehensive static<br class="">>>>>>>> instrumentation through the compiler, in order to simplify the task of<br class="">>>>>>>> building efficient and effective platform-independent tools. The CSI<br class="">>>>>>>> framework allows the tool writer to easily develop analysis tools that<br class="">>>>>>>> require<br class="">>>>>>>> compiler instrumentation without needing to understand the compiler<br class="">>>>>>>> internals or modifying the compiler, which greatly lowers the bar for<br class="">>>>>>>> developing dynamic-analysis tools.<br class="">>>>>>>><br class="">>>>>>>> ================<br class="">>>>>>>> Approach<br class="">>>>>>>> ================<br class="">>>>>>>><br class="">>>>>>>> The CSI framework inserts instrumentation hooks at salient locations<br class="">>>>>>>> throughout the compiled code of a program-under-test, such as function entry<br class="">>>>>>>> and exit points, basic-block entry and exit points, before and after each<br class="">>>>>>>> memory operation, etc. Tool writers can instrument a program-under-test<br class="">>>>>>>> simply by first writing a library that defines the semantics of relevant<br class="">>>>>>>> hooks<br class="">>>>>>>> and then statically linking their compiled library with the<br class="">>>>>>>> program-under-test.<br class="">>>>>>>><br class="">>>>>>>> At first glance, this brute-force method of inserting hooks at every<br class="">>>>>>>> salient location in the program-under-test seems to be replete with<br class="">>>>>>>> overheads. CSI overcomes these overheads through the use of<br class="">>>>>>>> link-time-optimization (LTO), which is now readily available in most major<br class="">>>>>>>> compilers, including GCC and LLVM. Using LTO, instrumentation hooks that<br class="">>>>>>>> are not used by a particular tool can be elided, allowing the overheads of<br class="">>>>>>>> these hooks to be avoided when the<br class="">>>>>>>><br class="">>>>>>>><br class="">>>>>>>> I don't understand this flow: the front-end emits all the possible<br class="">>>>>>>> instrumentation but the useless calls to the runtime will be removed during<br class="">>>>>>>> the link?<br class="">>>>>>>> It means that the final binary is specialized for a given tool right?<br class="">>>>>>>> What is the advantage of generating this useless instrumentation in the<br class="">>>>>>>> first place then? I'm missing a piece here...<br class="">>>>>>><br class="">>>>>>><br class="">>>>>>> Here's the idea. When a tool user, who has a program they want to<br class="">>>>>>> instrument, compiles their program source into an object/bitcode, he can<br class="">>>>>>> turn on the CSI compile-time pass to insert instrumentation hooks (function<br class="">>>>>>> calls to instrumentation routines) throughout the IR of the program.<br class="">>>>>>> Separately, a tool writer implements a particular tool by writing a library<br class="">>>>>>> that defines the subset of instrumentation hooks she cares about. At link<br class="">>>>>>> time, the object/bitcode of the program source is linked with the object<br class="">>>>>>> file/bitcode of the tool, resulting in a tool-instrumented executable. When<br class="">>>>>>> LTO is used at link time, unused instrumentation is elided, and additional<br class="">>>>>>> optimizations can run on the instrumented program. (I'm happy to send you a<br class="">>>>>>> nice picture that we have of this flow, if the mailing list doesn't mind.)<br class="">>>>>>><br class="">>>>>>><br class="">>>>>>> Ok this is roughly what I had in mind.<br class="">>>>>>><br class="">>>>>>> I still believe it is not great to rely on LTO, and better, it is not<br class="">>>>>>> needed to achieve this result.<br class="">>>>>>><br class="">>>>>>> For instance, I don't see why the "library" that defines the subset of<br class="">>>>>>> instrumentation hooks used by this tool can't be fed during a regular<br class="">>>>>>> compile, and the useless hook be eliminated at this point.<br class="">>>>>>> Implementation detail, but in practice, instead of feeding the library<br class="">>>>>>> itself, the "framework" that allows to generate the library for the tool<br class="">>>>>>> writer can output a "configuration file" along side the library, and this<br class="">>>>>>> configuration file is what is fed to the compiler and tells the<br class="">>>>>>> instrumentation pass which of the hooks to generate. It sounds more<br class="">>>>>>> efficient to me, and remove the dependency on LTO.<br class="">>>>>>> I imagine there is a possible drawback that I'm missing right now...<br class="">>>>>><br class="">>>>>><br class="">>>>>><br class="">>>>>> I agree that the tool does not need to depend on full LTO. What is<br class="">>>>>> needed is essentially an option or configuration such that the compiler can<br class="">>>>>> find the bit code file(s) for the hooks during compilation time. It is<br class="">>>>>> pretty much similar to how math function inlining can be done ...<br class="">>>>><br class="">>>>><br class="">>>>> I agree, and I would strongly prefer that the design worked like this<br class="">>>>> rather than relying on LTO.<br class="">>>>><br class="">>>>> The flag for loading bitcode already exists, and is called<br class="">>>>> -mlink-bitcode-file. Projects such as libclc already use it, I believe.<br class="">>>>><br class="">>>>> What might be useful is if CSI improved the infrastructure around<br class="">>>>> -mlink-bitcode-file to make it more convenient to produce compatible bitcode<br class="">>>>> files. libclc for example relies on a post-processing pass to change symbol<br class="">>>>> linkage, and I think that can be avoided by changing symbol linkages as they<br class="">>>>> are imported from the bitcode file.<br class="">>>>><br class="">>>>> Peter<br class="">>>>><br class="">>>>>> David<br class="">>>>>><br class="">>>>>><br class="">>>>>><br class="">>>>>>><br class="">>>>>>><br class="">>>>>>><br class="">>>>>>><br class="">>>>>>> The final binary is specialized to a given tool. One advantage of<br class="">>>>>>> CSI, however, is that a single set of instrumentation covers the needs of a<br class="">>>>>>> wide variety of tools, since different tools provide different<br class="">>>>>>> implementations of the same hooks. The specialization of a binary to a<br class="">>>>>>> given tool happens at link time.<br class="">>>>>>><br class="">>>>>>>><br class="">>>>>>>><br class="">>>>>>>> instrumented program-under-test is run. Furthermore, LTO can<br class="">>>>>>>> optimize a tool's instrumentation within a program using traditional<br class="">>>>>>>> compiler optimizations. Our initial study indicates that the use of LTO<br class="">>>>>>>> does not unduly slow down the build time<br class="">>>>>>>><br class="">>>>>>>><br class="">>>>>>>> This is a false claim: LTO has a very large overhead, and especially<br class="">>>>>>>> is not parallel, so the more core you have the more the difference will be.<br class="">>>>>>>> We frequently observes builds that are 3 times slower. Moreover, LTO is not<br class="">>>>>>>> incremental friendly and during debug (which is very relevant with<br class="">>>>>>>> sanitizer) rebuilding involves waiting for the full link to occur again.<br class="">>>>>>>><br class="">>>>>>><br class="">>>>>>> Can you please point us towards some projects where LTO incurs a 3x<br class="">>>>>>> slowdown? We're interested in the overhead of LTO on build times, and<br class="">>>>>>> although we've found LTO to incur more overhead on parallel build times than<br class="">>>>>>> serial build times, as you mentioned, the overheads we've measured on serial<br class="">>>>>>> or parallel builds have been less than 40% (which we saw when building the<br class="">>>>>>> Apache HTTP server).<br class="">>>>>>><br class="">>>>>>><br class="">>>>>>> I expect this to be reproducible on most non-trivial C/C++ programs.<br class="">>>>>>> But taking clang as an example, just running `ninja clang` on OS X a<br class="">>>>>>> not-so-recent 12-cores machine takes 970s with LTO and 252s without (and I<br class="">>>>>>> believe this is without debug info...).<br class="">>>>>>> Running just `ninja` to build all of llvm/clang here would take *a<br class="">>>>>>> lot* longer with LTO, and not so much without.<br class="">>>>>>><br class="">>>>>>> The LTO builds without assert<br class="">>>>>>><br class="">>>>>>> Best,<br class="">>>>>>><br class="">>>>>>> --<br class="">>>>>>> Mehdi<br class="">>>>>>><br class="">>>>>>><br class="">>>>>>><br class="">>>>>>><br class="">>>>>>><br class="">>>>>>> We've also designed CSI such that it does not depend on LTO for<br class="">>>>>>> correctness; the program and tool will work correctly with ordinary ld. Of<br class="">>>>>>> course, the downside of not using LTO is that instrumentation is not<br class="">>>>>>> optimized, and in particular, unused instrumentation will incur overhead.<br class="">>>>>>><br class="">>>>>>>><br class="">>>>>>>><br class="">>>>>>>> --<br class="">>>>>>>> Mehdi<br class="">>>>>>>><br class="">>>>>>>> , and the LTO can indeed optimize away unused hooks. One of our<br class="">>>>>>>> experiments with Apache HTTP server shows that, compiling with CSI and<br class="">>>>>>>> linking with the "null" CSI-tool (which consists solely of empty hooks)<br class="">>>>>>>> slows down the build time of the Apache HTTP server by less than 40%, and<br class="">>>>>>>> the resulting tool-instrumented executable is as fast as the original<br class="">>>>>>>> uninstrumented code.<br class="">>>>>>>><br class="">>>>>>>><br class="">>>>>>>> ================<br class="">>>>>>>> CSI version 1<br class="">>>>>>>> ================<br class="">>>>>>>><br class="">>>>>>>> The initial version of CSI supports a basic set of hooks that covers<br class="">>>>>>>> the following categories of program objects: functions, function exits<br class="">>>>>>>> (specifically, returns), basic blocks, call sites, loads, and stores. We<br class="">>>>>>>> prioritized instrumenting these IR objects based on the need of seven<br class="">>>>>>>> example CSI tools, including a race detector, a cache-reuse analyzer, and a<br class="">>>>>>>> code-coverage analyzer. We plan to evolve the CSI API over time to be more<br class="">>>>>>>> comprehensive, and we have designed the CSI API to be extensible, allowing<br class="">>>>>>>> new instrumentation to be added as needs grow. We chose to initially<br class="">>>>>>>> implement a minimal "core" set of hooks, because we felt it was best to add<br class="">>>>>>>> new instrumentation on an as-needed basis in order to keep the interface<br class="">>>>>>>> simple.<br class="">>>>>>>><br class="">>>>>>>> There are three salient features about the design of CSI. First, CSI<br class="">>>>>>>> assigns each instrumented program object a unique integer identifier within<br class="">>>>>>>> one of the (currently) six program-object categories. Within each category,<br class="">>>>>>>> the ID's are consecutively numbered from 0 up to the number of such objects<br class="">>>>>>>> minus 1. The contiguous assignment of the ID's allows the tool writer to<br class="">>>>>>>> easily keep track of IR objects in the program and iterate through all<br class="">>>>>>>> objects in a category (whether the object is encountered during execution or<br class="">>>>>>>> not). Second, CSI provides a platform-independent means to relate a given<br class="">>>>>>>> program object to locations in the source code. Specifically, CSI provides<br class="">>>>>>>> "front-end-data (FED)" tables, which provide file name and source lines for<br class="">>>>>>>> each program object given the object's ID. Third, each CSI hook takes in as<br class="">>>>>>>> a parameter a "property": a 64-bit unsigned integer that CSI uses to export<br class="">>>>>>>> the results of compiler analyses and other information known at compile<br class="">>>>>>>> time. The use of properties allow the tool to rely on compiler analyses to<br class="">>>>>>>> optimize instrumentation and decrease overhead. In particular, since the<br class="">>>>>>>> value of a property is known at compile time, LTO can constant-fold the<br class="">>>>>>>> conditional test around a property to elide unnecessary instrumentation.<br class="">>>>>>>><br class="">>>>>>>> ================<br class="">>>>>>>> Future plan<br class="">>>>>>>> ================<br class="">>>>>>>><br class="">>>>>>>> We plan to expand CSI in future versions by instrumenting additional<br class="">>>>>>>> program objects, such as atomic instructions, floating-point instructions,<br class="">>>>>>>> and exceptions. We are also planning to provide additional static<br class="">>>>>>>> information to tool writers, both through information encoded in the<br class="">>>>>>>> properties passed to hooks and by other means. In particular, we are also<br class="">>>>>>>> looking at mechanisms to present tool writers with more complex static<br class="">>>>>>>> information, such as how different program objects relate to each other,<br class="">>>>>>>> e.g., which basic blocks belong to a given function.<br class="">>>>>>>><br class="">>>>>>>> _______________________________________________<br class="">>>>>>>> LLVM Developers mailing list<br class="">>>>>>>><span class="Apple-converted-space"> </span><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class="">>>>>>>><span class="Apple-converted-space"> </span><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class="">>>>>>><br class="">>>>>>><br class="">>>>>>><br class="">>>>>>> _______________________________________________<br class="">>>>>>> LLVM Developers mailing list<br class="">>>>>>><span class="Apple-converted-space"> </span><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class="">>>>>>><span class="Apple-converted-space"> </span><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class="">>>>>>><br class="">>>>>><br class="">>>>>><br class="">>>>>> _______________________________________________<br class="">>>>>> LLVM Developers mailing list<br class="">>>>>><span class="Apple-converted-space"> </span><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class="">>>>>><span class="Apple-converted-space"> </span><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class="">>>>>><br class="">>>>><br class="">>>>><br class="">>>>><br class="">>>>> --<br class="">>>>> --<br class="">>>>> Peter<br class="">>>>><br class="">>>>> _______________________________________________<br class="">>>>> LLVM Developers mailing list<br class="">>>>><span class="Apple-converted-space"> </span><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class="">>>>><span class="Apple-converted-space"> </span><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class="">>>>><br class="">>>><br class="">><br class="">><br class="">> _______________________________________________<br class="">> LLVM Developers mailing list<br class="">><span class="Apple-converted-space"> </span><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class="">><span class="Apple-converted-space"> </span><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class="">><br class=""></div></div></blockquote></div><br class=""></div><span id="cid:BD5AD105-07BA-43C8-9FD4-D9E8A39EFCCB"><CSI-arch.pdf></span><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">LLVM Developers mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="mailto:llvm-dev@lists.llvm.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">llvm-dev@lists.llvm.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></div></blockquote></div><br class=""></body></html>