[lldb-dev] [PATCH] Don't calculate whole file crc for ELF core file.

[Todd Fiala]

Hey Piotr,

Thanks for taking a look at that.  I'll have a run through it today or
tomorrow.  I had noticed some really slow backtrace times on relatively
simple Linux x86_64 cores but just hadn't had the cycles to look at it yet.

I'll get back on this once I have a chance to look at it.  (If anybody else
has a chance to look at it before me, feel free).


On Sun, Mar 23, 2014 at 12:07 PM, Piotr Rak <piotr.rak at gmail.com> wrote:

> Hi,
>
> Here is rebased version of same patch - so it applies cleanly on top of:
>
> https://llvm.org/svn/llvm-project/lldb/trunk@204545
>
> as it also touches ObjectFileELF.cpp and I somehow missed it yesterday...
>
>
> 2014-03-23 19:28 GMT+01:00 Piotr Rak <piotr.rak at gmail.com>:
>
> Hi,
>>
>> Working on something else I've noticed that ProcessElfCore plugin can
>> take considerable amount of time when once given big core file.
>>
>> The file in question was 4.1 GiB core file generated by linux kernel, of
>> process that crashed.
>>
>> I took the small detour to investigate and it turned out to be spending
>> ~72s
>> calculating crc32 for whole file from:
>>
>> ProcessElfCore::CanDebug
>>   ModuleList::GetShareModule
>>
>> That in the result blocks ObjectFileELF::GetModuleSpecification.
>>
>> That time was measured with Debug+Assert build variant, on puny i5 laptop
>> with traditional HDD, thus in reality it would be bit better, but still
>> meaningful.
>>
>> If I understand correctly the main purpose of UUID's for ObjectFileELF is
>> aiding gnu_debuglink which is used to locate debug info. The other one is
>> related to ModuleSpec and ModuleList where it is used for identification.
>>
>> First in case of modules doesn't make much sense, it won't be correct, or
>> have even possibility to work, while second turns out to be useful, anyway.
>>
>> Simple workaround or just inventing UUID for ModuleSpec before passing it
>> to ModuleList::GetShareModule would probably work too for this particular
>> case. ELF core files have no universal information reassembling
>> gnu.debuglink or .note.gnu.build-id, thus we will be falling back to quite
>> costly crc computation that is not strictly required.
>>
>> I have not noticed also easy way to make this computation in background,
>> so I decided limit it to PT_NOTE segments, which still have quite bit of
>> unique info, like: combinations pids/tids/uids/register contexts, and
>> sometimes much more.
>>
>> That brings down time to somewhat more appealing value:
>> 0.397 ms
>>
>> Please note that while it was bit optimistic case for that solution,
>> since it was single threaded process, and its PT_NOTE segment had only 4680
>> bytes.
>>
>> For other core binary that had PT_NOTE segment of size ~48MiB it still
>> takes:
>> 1.206 s
>>
>> in same setup.
>> That was pretty artificial case of *fork bomb* with almost 30k (29357)
>> threads.
>>
>> While using only note segment makes  hitting collision it bit more
>> likely, I think it is pretty remote, and not meaningful in practice.
>>
>> Also if anyone has ideas about other/better solution to this problem I
>> will gladly look into it, as performance of core file handling is important
>> for my use-cases.
>>
>> If this solution is fine for trunk, please commit, I don't have commit
>> access.
>> Both raw diff and git path of same change are attached.
>>
>> Cheers,
>> /Piotr
>>
>
>
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>
>


-- 
Todd Fiala | Software Engineer | tfiala at google.com | 650-943-3180
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20140324/5b887094/attachment.html>