[llvm-dev] [LTO] -time-passes and libLTO

[James Henderson via llvm-dev]

Ok, I'm happy to make a couple of changes based on the above comments.
Here's my proposal:

1) Modify the timing information classes to provide a method to print and
reset the counters other than at destruction time.
2) Add to the end of LTOCodeGenerator::compileOptimized and
ThinLTOCodeGenerator::run a call to the new method from 1).

I do think we've got the bandwidth to implement the Manager classes at this
time, but that does seem like a good idea in the longer term to me.

James

On 3 May 2017 at 06:00, Mehdi AMINI <joker.eph at gmail.com> wrote:

>
>
> 2017-05-02 8:42 GMT-07:00 Duncan P. N. Exon Smith <dexonsmith at apple.com>:
>
>> +Teresa, Mehdi
>>
>> On May 2, 2017, at 08:31, James Henderson <jh7370.2008 at my.bristol.ac.uk>
>> wrote:
>>
>> Hi,
>>
>> We have been investigating an issue when running LTO with our proprietary
>> linker, which links against libLTO dynamically. The issue is that when we
>> pass -time-passes via the lto_codegen_debug_options function in the LTO C
>> API, no time information is produced during compilation. The reason for
>> this is that time information is stored in state owned by a ManagedStatic
>> instance, and is only printed when the state is destroyed. This in turn
>> only happens when ManagedStatics are cleaned up, via the llvm_shutdown
>> function. As we do not link against LLVM (except libLTO dynamically), we
>> have no access to llvm_shutdown, which in turn means we are not able to
>> clean up ManagedStatic instances and thus no timing information is produced.
>>
>> We have considered a few options and have come up with the following
>> suggestions, and would appreciate some feedback:
>>
>> 1)      Add llvm_shutdown (or rather likely some wrapper function that
>> does the same job) to the C interface of libLTO. This should be called when
>> we are done with the library.
>>
>> This seems pretty reasonable to me.  I'm not sure what others think.
>>
>> 2)      Add a “full-shutdown” command-line option to LLVM - that can be
>> passed via lto_codegen_debug_options - which causes ManagedStatic-owned
>> state to be destroyed on shutdown. This could even be more widely useful
>> outside the LTO case.
>>
>> 3)      Call llvm_shutdown() immediately after compilation as part of
>> the compile function.
>>
>> 4)      None of the above, because there’s a better way that I am
>> unaware of to clean up this state.
>>
>>
> In the short term, adding a way to print and reset the counters and call
> it at the end of the LTO process seems like a good immediate tradeoff to
> me. We're kind of doing it with statistics already: https://llvm.org/svn/
> llvm-project/llvm/trunk/lib/LTO/ThinLTOCodeGenerator.cpp (look at the
> very end of the file)
>
>
>> Perhaps this timing stuff should live in the LLVMContext instead?  And
>> then you get timing-per-LLVMContext?
>>
>
>
> This last suggestion is my favorite answer by far!
> I've been talking with Matthias about doing this for statistic as well:
> having a dedicated StatisticManager and TimerGroupManager (straw man names)
> that could live in the LLVMContext and have to be set explicitly by the
> client of the LLVMContext is what I'd aim for.
> This is needed to solve the issue we have with ThinLTO and threading in
> general.
>
> It is possible (likely...) that some places are using statistics without
> having a LLVMContext, but we should be able to thread through a
> StatisticManager somehow.
>
>
> --
> Mehdi
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170503/e8bdcca1/attachment.html>