[llvm-dev] bugpoint, LLVM tools (opt, llc, etc), and symbolizing costs

[David Blaikie via llvm-dev]

So - fixed part of this in D33804/r305056 (at least not symbolizing crashes
of the 'opt' tool when run from bugpoint - I still need to generalize this
over the other tools/way bugpoint runs programs & fix those too)

But realized there's another place where we're paying a lot for symbolizing
and this one isn't saved by bugpoint's memory limit*: GUnit death tests.

For example ADTTests takes about 20 seconds without symbolizing, and about
1m35 with symbolizing. (though doesn't seem to have a significant impact on
total runtime of "check-llvm" on my machine - probably some other longer
poles propping it up)

Any idea how we can/should disable symbolizing for these tests? Wondering
if we could have a scoped device to disable symbolizing around
known/intended crashes/assert failures? Though would be annoying to have to
have another trick to be needed in any death test - people could easily
forget. :/

* After doing more investigation I found out why Split DWARF debug info was
so much slower here than internal - bugpoint places a memory limit,
including an address space limit, of 400MB on the processes it launches.
llvm-symbolizer inherits this limit when symbolizing a crash. Memory
mapping in a binary with non-split DWARF exceeds the limit and fails - so
it's fast because it doesn't symbolize.

On Tue, May 23, 2017 at 6:38 PM David Blaikie <dblaikie at gmail.com> wrote:

> On Tue, May 23, 2017 at 2:40 PM Reid Kleckner <rnk at google.com> wrote:
>
>> We have the LLVM_DISABLE_CRASH_REPORT environment variable, but that's
>> more about whether we should do crash reporting or not.
>>
>> It would be pretty reasonable to have another one to disable all this
>> stuff. It would also be reasonable to have a cmake option that compiles
>> this stuff away, since it basically never works on user machines that don't
>> have debug info.
>>
>
> Fair - any thoughts of having a dynamic option (rather than a build time
> option) so that things like bugpoint could disable it for runs of tools,
> but the tools itself could still have the functionality for once you finish
> running bugpoint and then run the same command manually?
>
>
>>
>> On Tue, May 23, 2017 at 2:02 PM, David Blaikie via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> test/BugPoint/metadata.ll
>>> Takes about 6 seconds to run if llvm-symbolizer isn't present
>>> 13 seconds with debug info
>>> 1m30s with split DWARF/Fission
>>>
>>> So, couple of things:
>>> 1) llvm-symbolizer is /really slow/ on split-dwarf files...
>>> 2) Why are stack traces even attempted when using bugpoint? Even if it's
>>> just the 6->13s slowdown, that's still halving the speed of bugpoint in the
>>> presence of llvm-symbolizer+debug info (I didn't test in the presence of
>>> llvm-symbolizer but no debug info)?
>>>
>>> It doesn't look like the LLVM tools have any way to disable
>>> symbolization/stack trace/crash handling. Is that correct? Is there a flag
>>> somewhere that I missed?
>>>
>>> Would it be worth adding such a flag and passing it from bugpoint to the
>>> various tools it invokes?
>>>
>>> (anyone interested in looking at exactly what makes llvm-symbolizer /so/
>>> much slower on split DWARF? I guess that's probably something for me to
>>> look at, really (though I checked that it's at least not due to my recent
>>> changes - but there have been some other refactorings here recently too) -
>>> but figured I'd check)
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170611/7249186b/attachment.html>