[llvm-dev] [LTO] -time-passes and libLTO

[Mehdi AMINI via llvm-dev]

2017-05-02 8:42 GMT-07:00 Duncan P. N. Exon Smith <dexonsmith at apple.com>:

> +Teresa, Mehdi
>
> On May 2, 2017, at 08:31, James Henderson <jh7370.2008 at my.bristol.ac.uk>
> wrote:
>
> Hi,
>
> We have been investigating an issue when running LTO with our proprietary
> linker, which links against libLTO dynamically. The issue is that when we
> pass -time-passes via the lto_codegen_debug_options function in the LTO C
> API, no time information is produced during compilation. The reason for
> this is that time information is stored in state owned by a ManagedStatic
> instance, and is only printed when the state is destroyed. This in turn
> only happens when ManagedStatics are cleaned up, via the llvm_shutdown
> function. As we do not link against LLVM (except libLTO dynamically), we
> have no access to llvm_shutdown, which in turn means we are not able to
> clean up ManagedStatic instances and thus no timing information is produced.
>
> We have considered a few options and have come up with the following
> suggestions, and would appreciate some feedback:
>
> 1)      Add llvm_shutdown (or rather likely some wrapper function that
> does the same job) to the C interface of libLTO. This should be called when
> we are done with the library.
>
> This seems pretty reasonable to me.  I'm not sure what others think.
>
> 2)      Add a “full-shutdown” command-line option to LLVM - that can be
> passed via lto_codegen_debug_options - which causes ManagedStatic-owned
> state to be destroyed on shutdown. This could even be more widely useful
> outside the LTO case.
>
> 3)      Call llvm_shutdown() immediately after compilation as part of the
> compile function.
>
> 4)      None of the above, because there’s a better way that I am unaware
> of to clean up this state.
>
>
In the short term, adding a way to print and reset the counters and call it
at the end of the LTO process seems like a good immediate tradeoff to me.
We're kind of doing it with statistics already:
https://llvm.org/svn/llvm-project/llvm/trunk/lib/LTO/ThinLTOCodeGenerator.cpp
(look at the very end of the file)


> Perhaps this timing stuff should live in the LLVMContext instead?  And
> then you get timing-per-LLVMContext?
>


This last suggestion is my favorite answer by far!
I've been talking with Matthias about doing this for statistic as well:
having a dedicated StatisticManager and TimerGroupManager (straw man names)
that could live in the LLVMContext and have to be set explicitly by the
client of the LLVMContext is what I'd aim for.
This is needed to solve the issue we have with ThinLTO and threading in
general.

It is possible (likely...) that some places are using statistics without
having a LLVMContext, but we should be able to thread through a
StatisticManager somehow.


-- 
Mehdi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170502/2782df4a/attachment.html>