[lldb-dev] [PATCH] Don't calculate whole file crc for ELF core file.

[Greg Clayton]

Looks ok, but I _would_ cache the calculated UUID in ObjectFile::m_uuid within the ObjectFileELF::GetUUID() function so they never have to be done again. If other linux folks are happy with this ELF patch, it looks ok to me.

Greg


On Mar 23, 2014, at 12:07 PM, Piotr Rak <piotr.rak at gmail.com> wrote:

> Hi,
> 
> Here is rebased version of same patch - so it applies cleanly on top of:
> 
> https://llvm.org/svn/llvm-project/lldb/trunk@204545
> 
> as it also touches ObjectFileELF.cpp and I somehow missed it yesterday...
> 
> 
> 2014-03-23 19:28 GMT+01:00 Piotr Rak <piotr.rak at gmail.com>:
> Hi,
> 
> Working on something else I've noticed that ProcessElfCore plugin can take considerable amount of time when once given big core file.
> 
> The file in question was 4.1 GiB core file generated by linux kernel, of process that crashed.
> 
> I took the small detour to investigate and it turned out to be spending ~72s
> calculating crc32 for whole file from:
> 
> ProcessElfCore::CanDebug
>   ModuleList::GetShareModule
> 
> That in the result blocks ObjectFileELF::GetModuleSpecification.
> 
> That time was measured with Debug+Assert build variant, on puny i5 laptop with traditional HDD, thus in reality it would be bit better, but still meaningful.
> 
> If I understand correctly the main purpose of UUID's for ObjectFileELF is aiding gnu_debuglink which is used to locate debug info. The other one is related to ModuleSpec and ModuleList where it is used for identification.
> 
> First in case of modules doesn't make much sense, it won't be correct, or have even possibility to work, while second turns out to be useful, anyway. 
> 
> Simple workaround or just inventing UUID for ModuleSpec before passing it to ModuleList::GetShareModule would probably work too for this particular case. ELF core files have no universal information reassembling gnu.debuglink or .note.gnu.build-id, thus we will be falling back to quite costly crc computation that is not strictly required.
> 
> I have not noticed also easy way to make this computation in background, so I decided limit it to PT_NOTE segments, which still have quite bit of unique info, like: combinations pids/tids/uids/register contexts, and sometimes much more.
> 
> That brings down time to somewhat more appealing value:
> 0.397 ms
> 
> Please note that while it was bit optimistic case for that solution, since it was single threaded process, and its PT_NOTE segment had only 4680 bytes.  
> 
> For other core binary that had PT_NOTE segment of size ~48MiB it still takes:
> 1.206 s 
> 
> in same setup.
> That was pretty artificial case of *fork bomb* with almost 30k (29357) threads.
> 
> While using only note segment makes  hitting collision it bit more likely, I think it is pretty remote, and not meaningful in practice.
> 
> Also if anyone has ideas about other/better solution to this problem I will gladly look into it, as performance of core file handling is important for my use-cases.
> 
> If this solution is fine for trunk, please commit, I don't have commit access.
> Both raw diff and git path of same change are attached.
> 
> Cheers,
> /Piotr
> 
> <Don-t-calculate-whole-file-crc-for-ELF-core-file_v2.diff><0001-Don-t-calculate-whole-file-crc-for-ELF-core-file_v2.patch>_______________________________________________
> lldb-dev mailing list
> lldb-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev