[lldb-dev] Loading debug symbols (C++, Ubuntu 14.04)

Wed May 11 04:30:05 PDT 2016

Hi,

thank you for investigating this.

So, what you are describing should normally work. Generally
obj_file->GetDebugSymbolFilePaths() (SymbolVendorELF.cpp:99) should
return the correct name to look for the debug file. E.g., in my case
for libpthread it returns "libpthread-2.19.so", and indeed the debug
file is found at
"/usr/lib/debug/lib/x86_64-linux-gnu/libpthread-2.19.so". This
information should come from the .gnu_debuglink section on the main
file. Could you check what does your file specify (e.g., something
like readelf -p .gnu_debuglink /lib/x86_64-linux-gnu/libpthread.so.0).
If the correct file name is listed there, then we need to figure out
why it's not picked up. If it has an incorrect file, then we need to
find another way to locate the file)

pl

On 9 May 2016 at 13:29, Stefan Kratochwil <kratochwil at teco.edu> wrote:
> Hi Pavel,
>
> I stepped through SymbolVendorELF::CreateInstance(), and it seems that the
> problem lies within Symbols::LocateExecutableSymbolFile
> (Host/common/Symbols.cpp). This method looks at the correct locations:
>
> (gdb) print debug_file_search_paths
> $35 = {m_files = std::vector of length 3, capacity 4 = {
>   {m_directory = {m_string = 0x7994f8 "/usr/lib"},
>    m_filename = {m_string = 0x7fffe400ba88 "x86_64-linux-gnu"},
>    m_is_resolved = true,
>    m_syntax = lldb_private::FileSpec::ePathSyntaxPosix},
>
>   {m_directory = {m_string = 0x7fffe400bae0
> "/home/stefan/git/dsu-ld/libdsu-project"},
>    m_filename = {m_string = 0x7fffe400bac8 "."},
>    m_is_resolved = true,
>    m_syntax = lldb_private::FileSpec::ePathSyntaxPosix},
>
>   {m_directory = {m_string = 0x7994f8 "/usr/lib"},
>    m_filename = {m_string = 0x7fffe400bb18 "debug"},
>    m_is_resolved = true,
>    m_syntax = lldb_private::FileSpec::ePathSyntaxPosix}}}
>
>
> But it looks for the wrong file names. Under Ubuntu, the contents of a *-dbg
> packge are stored under /usr/lib/debug, using the exact same file names as
> their regular executable counterparts. In the case of apache2-dbg, the files
> under /usr/lib/debug are (for example):
>
> /usr/lib/debug/usr/bin/rotatelogs
> /usr/lib/debug/usr/sbin/apache2
> /usr/lib/debug/usr/lib/apache2/modules/mod_ssl.so
>
> and so on.
>
>
>
> If i now look at the file names lldb looks for, I find for
> /usr/lib/x86_64-linux-gnu:
> (gdb) print files
> $36 = std::vector of length 4, capacity 4 = {
>   "/usr/lib/x86_64-linux-gnu/9d72ee0b01e9b2947c7f95e7b4ebbbf05a8aa5.debug",
>
> "/usr/lib/x86_64-linux-gnu/.debug/9d72ee0b01e9b2947c7f95e7b4ebbbf05a8aa5.debug",
>
> "/usr/lib/x86_64-linux-gnu/.build-id/B5/9D72EE0B01E9B2947C7F95E7B4EBBBF05A8AA5.debug",
>
> "/usr/lib/x86_64-linux-gnu/usr/lib/x86_64-linux-gnu/9d72ee0b01e9b2947c7f95e7b4ebbbf05a8aa5.debug"}
>
> For my /home/stefan/git/dsu-ld/libdsu-project/:
> (gdb) print files
> $38 = std::vector of length 4, capacity 4 = {
>
> "/home/stefan/git/dsu-ld/libdsu-project/./9d72ee0b01e9b2947c7f95e7b4ebbbf05a8aa5.debug",
>
> "/home/stefan/git/dsu-ld/libdsu-project/./.debug/9d72ee0b01e9b2947c7f95e7b4ebbbf05a8aa5.debug",
>
> "/home/stefan/git/dsu-ld/libdsu-project/./.build-id/B5/9D72EE0B01E9B2947C7F95E7B4EBBBF05A8AA5.debug",
>
> "/home/stefan/git/dsu-ld/libdsu-project/./usr/lib/x86_64-linux-gnu/9d72ee0b01e9b2947c7f95e7b4ebbbf05a8aa5.debug"}
>
>
> And finally for the /usr/lib/debug directory:
> (gdb) print files
> $39 = std::vector of length 4, capacity 4 = {
>   "/usr/lib/debug/9d72ee0b01e9b2947c7f95e7b4ebbbf05a8aa5.debug",
>   "/usr/lib/debug/.debug/9d72ee0b01e9b2947c7f95e7b4ebbbf05a8aa5.debug",
>
> "/usr/lib/debug/.build-id/B5/9D72EE0B01E9B2947C7F95E7B4EBBBF05A8AA5.debug",
>
> "/usr/lib/debug/usr/lib/x86_64-linux-gnu/9d72ee0b01e9b2947c7f95e7b4ebbbf05a8aa5.debug"}
>
>
> So, lldb expects that the uuid/build of the Module it is about to load is
> also contained in the symbol file name.
>
>
> Imho, we also have to take the FileSpec contents of the ModuleSpec we are
> currently looking at, and add its path to the files vector. But I am not
> that familiar with the whole lldb backend to tell for sure what would happen
> then.
>
>
> I also checked the current HEAD of the lldb source tree to see if there are
> any changes, compared with my older revision. There is only a slight
> difference in how Symbols::LoadExecutableSymbolFile uses the /usr/lib/debug
> directory:
>
> // Some debug files may stored in the module directory like this:
> //   /usr/lib/debug/usr/lib/library.so.debug
> if (!file_dir.IsEmpty())
>     files.push_back (dirname + file_dir.AsCString() + "/" ++
> symbol_filename);
>
>
>
> This would be in fact the solution to my problem, if it wouldn't use the
> symbol_filename (which contains a uuid, something the *-dbg packages don't
> have).
>
>
> Well, what do you think?
>
>
> Cheers,
> Stefan
>
>
>
>
>
>
> On 05/09/2016 11:23 AM, Pavel Labath wrote:
>>
>> You can also use SBCommandInterpreter::HandleCommand, which gives you
>> a bit more control over what happens.
>>
>> good luck with your thesis.
>>
>> pl
>>
>> On 9 May 2016 at 10:00, Stefan Kratochwil <kratochwil at teco.edu> wrote:
>>>
>>> Hi Pavel,
>>>
>>> thanks for your quick reply. I am currently using a release build, so it
>>> may
>>> take a moment until I can step through there. I'll report back as soon as
>>> possible (my masters thesis somehow depends on that ;) ).
>>>
>>> I was just about to implement an addon for the API tonight, when I found
>>> SBDebugger::HandleCommand("target symbols add ..."). Yeah, it's a little
>>> weird way of using an API, and it's a one way command with stdout as
>>> reply
>>> channel, but it works ^^
>>>
>>> Cheers,
>>> Stefan
>>>
>>>
>>>
>>> On 05/09/2016 10:22 AM, Pavel Labath wrote:
>>>>
>>>>
>>>> Hi Stefan,
>>>>
>>>> what you are describing should work out of the box, so the fact that
>>>> you are having to add the symbols manually is a bug. I haven't tried
>>>> it with the apache binary specifically, but lldb can certainly find
>>>> external debug symbols for libc (definitely on ubuntu 14.04, as that's
>>>> what I use). We need to figure out what is different in your case. The
>>>> most important function here is SymbolVendorELF::CreateInstance. Could
>>>> you step through that function, and see why it fails to find the file?
>>>>
>>>> On 7 May 2016 at 19:45, Stefan Kratochwil via lldb-dev
>>>> <lldb-dev at lists.llvm.org> wrote:
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> I need to gather information about variables on the stack frames of a
>>>>> given
>>>>> program, lets say apache2 (httpd), using the C++ flavour of the lldb
>>>>> scripting bridge.
>>>>>
>>>>> Now, if I attach to the corresponding process id, lldb does not find
>>>>> any
>>>>> debug symbols, although I have installed the debug symbols package for
>>>>> apache2 (apache2-dbg).
>>>>>
>>>>> If I use the lldb cli, I have to use
>>>>> 'target symbols load /usr/lib/debug/usr/sbin/apache2'
>>>>> (or any other path under /usr/lib/debug, depending on the stack frame /
>>>>> module I am currently looking at) to load the corresponding debug
>>>>> symbols.
>>>>>
>>>>> However, if I try the same with gdb, then it automagically uses the
>>>>> binaries
>>>>> under /usr/lib/debug (the non-stripped binaries from any *-dbg package
>>>>> are
>>>>> located here) for symbolification.
>>>>>
>>>>>
>>>>> Now, there are two questions for me:
>>>>>
>>>>> 1.: Is there a way to let lldb automagically load debug symbols from
>>>>> /usr/lib/debug, if there are any?
>>>>>
>>>>> 2.: If not, how can I do this manually using the C++ API?
>>>>> The SBTarget class does not have suitable methods for this - the
>>>>> closest
>>>>> thing is SBTarget#AddModule(), but this gives me a _new_ module,
>>>>> instead
>>>>> of
>>>>> associating a symbolfile with an existing one. The code path of
>>>>> CommandObjectTarget.cpp#AddModuleSymbols() seems not to be reflected in
>>>>> the
>>>>> scripting bridge, or at least I haven't found it yet.
>>>>
>>>>
>>>>
>>>> You can use SBDebugger::HandleCommand to execute any command which
>>>> does have an API equivalent. It's not ideal, but it works. :)
>>>>
>>>> cheers,
>>>> pl
>>>>
>>>
>