<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D'>Try “image search-paths add” as a replacement for “set solib-search-path”<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D'>--<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D'>Qualcomm Innovation Center, Inc.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D'>The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project<o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>From:</span></b><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'> lldb-dev [mailto:lldb-dev-bounces@lists.llvm.org] <b>On Behalf Of </b>Eugene Birukov via lldb-dev<br><b>Sent:</b> Friday, January 20, 2017 12:35 PM<br><b>To:</b> Pavel Labath <labath@google.com><br><b>Cc:</b> rdorr@microsoft.com; LLDB <lldb-dev@lists.llvm.org><br><b>Subject:</b> Re: [lldb-dev] LLDB 3.9 on Linux crashes when loading core dump<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><div id=divtagdefaultwrapper><p><span style='font-family:"Calibri",sans-serif;color:black'>Hello Pavel,<o:p></o:p></span></p><p><span style='font-family:"Calibri",sans-serif;color:black'><o:p> </o:p></span></p><p><span style='font-family:"Calibri",sans-serif;color:black'>Thanks for the reply. Unfortunately I cannot share the core dump with you.<o:p></o:p></span></p><p><span style='font-family:"Calibri",sans-serif;color:black'><o:p> </o:p></span></p><p><span style='font-family:"Calibri",sans-serif;color:black'>Yes, Rob has figured that LLDB does not find this shared library and that causes the problem. To understand what is going on here, I need to add one more detail that was missing from my original post: this is a cross-machine investigation. I.e. the core dump collected on one machine (CentOs) was sent to another<o:p></o:p></span></p><p><span style='font-family:"Calibri",sans-serif;color:black'>(Ubuntu) where I tried to open it.<o:p></o:p></span></p><p><span style='font-family:"Calibri",sans-serif;color:black'><o:p> </o:p></span></p><p><span style='font-family:"Calibri",sans-serif;color:black'>LLDB opens the executable without paying attention that there is a core dump attached and tries to locate shared libraries. Apparently the ones that exist on my machine are not good, so it then looks in the directory with the executable itself. There is no way to "set solib-search-path" as we do on GDB and force it to look somewhere else. After we dumped all the shared libraries in the folder with the executable LLDB was able to open the dump. This is a bit inconvenient, but works as a workaround for now.<o:p></o:p></span></p><p><span style='font-family:"Calibri",sans-serif;color:black'><o:p> </o:p></span></p><div id=Signature><p><span style='font-family:"Calibri",sans-serif;color:black'>Sent from <a href="http://aka.ms/weboutlook" id=LPNoLP>Outlook</a><o:p></o:p></span></p></div><p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-family:"Calibri",sans-serif;color:black'><o:p> </o:p></span></p><div><div><div class=MsoNormal align=center style='text-align:center'><span style='font-family:"Calibri",sans-serif;color:black'><hr size=2 width="98%" align=center></span></div><div id="x_divRplyFwdMsg"><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;color:black'>From:</span></b><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;color:black'> Pavel Labath <<a href="mailto:labath@google.com">labath@google.com</a>><br><b>Sent:</b> Friday, January 20, 2017 6:55 AM<br><b>To:</b> Eugene Birukov<br><b>Cc:</b> LLDB; <a href="mailto:rdorr@microsoft.com">rdorr@microsoft.com</a><br><b>Subject:</b> Re: [lldb-dev] LLDB 3.9 on Linux crashes when loading core dump</span><span style='font-family:"Calibri",sans-serif;color:black'> <o:p></o:p></span></p><div><p class=MsoNormal><span style='font-family:"Calibri",sans-serif;color:black'> <o:p></o:p></span></p></div></div></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Calibri",sans-serif;color:black'>Hello Eugene,<br><br>I have been aware of this problem for a while, but I haven't found a<br>really good solution so far, partially due to lack of a good repro<br>case -- I think your analysis has helped me with this, and I am<br>finally starting to piece together the sequence of events leading to<br>the crash. If you have a repro case you can send me, it would be even<br>better.<br><br>I don't really have an answer to your quesiton, but here are a couple<br>of observations (the details might be a bit sketchy - it's been a long<br>time since I looked at this):<br>- reading the section headers from memory should be a fallback.<br>Normally we try first to locate the file on disk and read data from<br>there. This was mainly added for the vdso "module", as that is not<br>really present on disk. One of the ways of fixing this crash could be<br>to figure out why we are not finding the c++abi binary on disk.<br><br>- we trust far too much the data we read from inferior memory. We<br>should be much more careful when doing things based on "untrusted"<br>data. Checking the memory maps as you suggest could be one idea.<br>Another option I am considering is teaching ReadMemory to allocate<br>data in (reasonably sized, say a couple of MB) chunks. Right now it<br>allocates the full buffer without even trying to read the memory. If<br>it instead tried to read data in smaller chunks it would error out due<br>to failure to read inferior memory long before it gets a chance to<br>exhaust own address space. (With a sufficiently large chunk, this<br>should not affect performance of normal reads).<br><br>hope that helps,<br>pl<br><br><br><br>On 19 January 2017 at 19:41, Eugene Birukov via lldb-dev<br><<a href="mailto:lldb-dev@lists.llvm.org">lldb-dev@lists.llvm.org</a>> wrote:<br>> Hi,<br>><br>><br>> I have a Linux core dump that causes LLDB 3.9 on Linux crash. I would<br>> greatly appreciate any advise how to deal with the problem or what else I<br>> should look at.<br>><br>><br>> The core dump was produced by GDB and GDB itself opens it without problems.<br>><br>><br>> So, during loading the core we call<br>> DynamicLoaderPOSIXDYLD::LoadAllCurrentModules() which enumerates all the<br>> modules and does some processing. In the course of actions, it calls the<br>> ObjectFileELF::GetSectionHeaderInfo() for each module. This guy tries to<br>> load section headers and read string table. Well, it gets some garbage in<br>> the section header struct and tries to allocate 1.5TB memory which causes<br>> operator new throw.<br>><br>><br>> So, why we get garbage?<br>><br>><br>> The module in question is libc++abi.so.1:<br>><br>><br>> 521 ModuleSP module_sp = LoadModuleAtAddress(I->file_spec,<br>> I->link_addr, I->base_addr, true);<br>><br>> (gdb) p I->file_spec<br>><br>> $95 = {<br>><br>> m_directory = {<br>><br>> m_string = 0x829a58 "... redacted ..."<br>><br>> },<br>><br>> m_filename = {<br>><br>> m_string = 0x7cc9e8 "libc++abi.so.1"<br>><br>> },<br>><br>> m_is_resolved = false,<br>><br>> m_syntax = lldb_private::FileSpec::ePathSyntaxPosix<br>><br>> }<br>><br>><br>> The module header lives at address 0x7f699a270000 and looks OK. The section<br>> headers are supposed to be at offset 2495600 == 0x261470<br>><br>><br>> $96 = (const elf::ELFHeader &) @0x953a78: {<br>><br>> e_ident = "\177ELF\002\001\001\000\000\000\000\000\000\000\000",<br>><br>> e_entry = 33392,<br>><br>> e_phoff = 64,<br>><br>> e_shoff = 2495600,<br>><br>> e_flags = 0,<br>><br>> e_version = 1,<br>><br>> e_type = 3,<br>><br>> e_machine = 62,<br>><br>> e_ehsize = 64,<br>><br>> e_phentsize = 56,<br>><br>> e_phnum = 7,<br>><br>> e_shentsize = 64,<br>><br>> e_shnum = 38,<br>><br>> e_shstrndx = 35<br>><br>> }<br>><br>><br>> LLDB tries to read the section headers from that address 0x7f699a270000 +<br>> 0x261470 == 0x7F699A4D1470 without a second thought, but this number is a<br>> lie. The /proc/<pid>/maps file shows it as belonging to something else:<br>><br>><br>> 7f699a270000-7f699a2ba000 r-xp 00000000 fd:02 537796791<br>> .../libc++abi.so.1<br>> 7f699a2ba000-7f699a4b9000 ---p 0004a000 fd:02 537796791<br>> .../libc++abi.so.1<br>> 7f699a4b9000-7f699a4bb000 r--p 00049000 fd:02 537796791<br>> .../libc++abi.so.1<br>> 7f699a4bb000-7f699a4bc000 rw-p 0004b000 fd:02 537796791<br>> .../libc++abi.so.1<br>> 7f699a4bc000-7f699a520000 r-xp 00000000 fd:00 202587414<br>> /usr/lib64/libssl.so.1.0.1e<br>> 7f699a520000-7f699a71f000 ---p 00064000 fd:00 202587414<br>> /usr/lib64/libssl.so.1.0.1e<br>> 7f699a71f000-7f699a723000 r--p 00063000 fd:00 202587414<br>> /usr/lib64/libssl.so.1.0.1e<br>> 7f699a723000-7f699a72a000 rw-p 00067000 fd:00 202587414<br>> /usr/lib64/libssl.so.1.0.1e<br>><br>> I.e. LLDB should verify the module boundaries and fall back to some other<br>> plan if the memory is not there.<br>><br>><br>> Now the question is - where would be the right place to do the fix?<br>><br>><br>> Thanks,<br>><br>> Eugene<br>><br>><br>><br>><br>><br>><br>><br>><br>> Sent from Outlook<br>><br>><br>> _______________________________________________<br>> lldb-dev mailing list<br>> <a href="mailto:lldb-dev@lists.llvm.org">lldb-dev@lists.llvm.org</a><br>> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev</a><br>><o:p></o:p></span></p></div></div></div></div></div></body></html>