[llvm-dev] [RFC] Adding time-trace to LLD?

[Rui Ueyama via llvm-dev]

It is indeed true that the features are overlapping. One thing I saw as an
advantage of the proposed scheme over the existing /TIME is that, as
Russell mentioned, the timer implementation is shared between the compiler
and the linker. That enables us to use the same set of analysis tools for
both. In fact, when you are doing LTO, the linker does compile bitcode into
native code, in order to get unified timing info, we need to use the same
time-trace framework.

JSON is not as easy to read as /TIME output as David pointed out. In
particular, when an output is long and not intended, it is almost
impossible to read. I wonder how much is that improved if we indent results
(inserting a lot of newlines and space characters makes things slower, so
we probably should do this only when the output filename is `-`). If it is
still hard to read, we should consider adding a new feature to the
time-trace framework to print out results in a more human-readable format.

On Thu, Oct 17, 2019 at 11:42 PM Russell Gallop via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Hi David,
>
> Thanks for the feedback.
>
> > The COFF flavor of lld has an option to show basic timing information on
> stdout: https://llvm.org/r322736. I enabled it on my project's CI and
> find it quite useful in logs. I'd be happy to see it ported to ELF.
>
> It's interesting to compare. They're quite similar in implementation:
> using scoped timers. I wonder whether it would be possible to combine them
> with options to control level of tracing (fine or coarse), and how the
> output is produced (stdout or JSON). I think that would require more than
> just a time granularity control, maybe an indication of "level" on each
> trace analogous to logging levels.
>
> One thing I like about the approach I propose here is that it is shared
> between the compiler and linker. It shares the timer implementation and
> makes the output more consistent for someone analysing a whole build.
>
> > From looking at the screenshot on your review, it looks like your
> proposal provides much richer information, but the extra friction of
> opening a json file (and pulling it off our builders) means I'd be less
> inclined to reach for this feature unless I was debugging a specific issue.
> > I guess it depends on whether you're going for day-to-day health
> monitoring or targeted problem-solving.
>
> I think you're suggesting /TIME is better for day-to-day health monitoring
> and -time-trace for targeted problem-solving. I agree they have different
> strengths. On the compiler side I find that the JSON format is helpful for
> monitoring as well, as it's easier to automate checks and analysis from a
> machine readable format. Whatever approach does need to be low enough
> overhead to be useful in that way.
>
> Regards
> Russ
>
> On Wed, 16 Oct 2019 at 18:15, David Major <dmajor at mozilla.com> wrote:
>
>> The COFF flavor of lld has an option to show basic timing information on
>> stdout: https://llvm.org/r322736. I enabled it on my project's CI and
>> find it quite useful in logs. I'd be happy to see it ported to ELF.
>>
>> From looking at the screenshot on your review, it looks like your
>> proposal provides much richer information, but the extra friction of
>> opening a json file (and pulling it off our builders) means I'd be less
>> inclined to reach for this feature unless I was debugging a specific issue.
>> I guess it depends on whether you're going for day-to-day health monitoring
>> or targeted problem-solving.
>>
>>
>>
>> On Wed, Oct 16, 2019 at 11:34 AM Russell Gallop via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> Hi all,
>>>
>>> I've been using -ftime-trace on the compiler and find it very useful for
>>> analysing compile times and whole build times. I've made a first attempt at
>>> adding this to LLD (ELF) so link times can be traced in the same way:
>>> https://reviews.llvm.org/D69043
>>>
>>> This works for LTO and ThinLTO, but needs some more work to make sure it
>>> scales well across multiple threads.
>>>
>>> Please let me know whether you think this is worth pursuing?
>>>
>>> Thanks
>>> Russ
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191018/5a4e5e3b/attachment.html>