[Lldb-commits] [PATCH] D75750: [lldb] integrate debuginfod

Mon Mar 30 13:37:14 PDT 2020

clayborg added a comment.

In D75750#1949527 <https://reviews.llvm.org/D75750#1949527>, @labath wrote:

> In D75750#1948273 <https://reviews.llvm.org/D75750#1948273>, @clayborg wrote:
>
> > Currently we have a solution for macOS to locate symbol files in the "lldb/source/Symbol/LocateSymbolFile.cpp" file in the Symbols::LocateExecutableSymbolFile(...) function:
> >
> >   static FileSpec Symbols::LocateExecutableSymbolFile(const ModuleSpec &module_spec, const FileSpecList &default_search_paths);
> >
> >
> > This will locate any files that are already on the system and return the symbol file. When you don't have a symbol file, we can call:
> >
> >   static bool Symbols::DownloadObjectAndSymbolFile(ModuleSpec &module_spec, bool force_lookup = true);
> >
> >
> > This might ping a build server and download the symbols.
> >
> > As for source file remappings, as Jim stated, on mac, each dSYM has path remappings already inside of it that are applied on the Module (not a target wide setting) itself and no modifications need to be done to the SourceManager.
> >
> > So my question is: can be use debuginfod to find the symbol file for a given build ID via Symbols::LocateExecutableSymbolFile(...), and when/if a symbol file is fetched from debuginfod, apply all path remappings to the module itself that we hand out? Then no changes would be needed in the SourceManager, we would just ask for a symbol file and get one back with all the remappings that are needed.
>
>
> I've been thinking about that a lot too. The thing that's not clear to me is, does `DownloadObjectAndSymbolFile` download source files too? If so, how?

We should probably make a new SymbolServer plug-in and convert the Apple version to compile and be installed only for Apple. The API for this should be something like:

  class SymbolServer {
    /// Get a cached symbol file that is already present on this machine.
    /// This gets called by all LLDB during normal debugging to fetch 
    /// and cached symbol files. 
    virtual ModuleSP GetSymbolFile(ModuleSpec module_spec);

    /// Download a symbol file when requested by the user. 
    /// This only gets run in response to the use requesting the symbols, 
    /// not as part of the normal debug work flow
    virtual FileSpec DownloadSymbolFile(ModuleSpec module_spec);

    /// New function that allows individual access to source files when 
    /// they aren't available on disk.
    virtual FileSpec GetSourceFile(FileSpec file, ....)
  };

Then debuginfod would fit right in there. The one thing that this interace doesn't cover is adding source remappings to modules, but it would be great if we can do this somehow with this new interface. Maybe SymbolServer::GetSymbolFile() can take a module_sp of the existing module so it can modify the source remappings if it has any?

> I am expecting that this feature will hook in very near to `DownloadObjectAndSymbolFile` for downloading the debug info, but it's not clear to me how would the source files fit in. Currently, debuginfod only provides an api to retrieve a single source file, so this code would have to parse all of the debug info, pry out the source files, and download them one by one -- a complicated and slow process.

This would be taken care of by the SymbolServer plug-in described above. For the Apple version, it would download the symbol file and remap the paths as it already does and the SymbolServer::GetSourceFile(FileSpec file) would just return the FileSpec that was passed in since it already is an NFS mount path. For debuginfod it can grab the source file one by one.

One possibility is to apply a module level source remapping that is unique to the symbol file that is returned from SymbolServer::GetSymbolFile(). Maybe we prepend the UUID of the module to all paths in the debug info. Something like mapping:

  "/..." to "/<UUID/..."

Then when we run into this kind of path, we know we need to call into the SymbolServer to resolve it (by possibly downloading it and caching it first).

> Now if debuginfod provided an api to download all source files in a single request (*), that might be workable. However, in principle, I see nothing wrong with being able to download the files on demand, if we have the option to do that. (debuginfod's long term goal seems to be to provide an api to download the debug info in chunks too -- that would be very interesting, though I also expect it to be very complicated.)

Agreed, lazy is good. I don't see the need to download sources in chunks though.

So using SymbolServer with unique path mappings ("/..." to "/<UUID/...") would allow us to accomplish lazy file access. Another option would be to mark and FileSpec objects that are handed out by any symbol files retrieved from the symbol server as needing resolution in the SymbolServer. Later code could do something like:

if file_spec.PathNeedsSymbolServerResolution():

  file_spec.ResolveWithSymbolServer(symbol_server);

And the path would update itself in the SymbolFile before it makes it out to any users. So stack frames that do lookups would end up resolving these paths before handing the information out to the user.

> (*) Though it seems very wasteful to download all files when we are going to need only a handful of them, it may not really be that way -- we're going to be downloading all of debug info anyway, and this is going to be much larger that all of source code put together.

I think we should be able to figure out how to make this lazy with a new plug-in interface

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75750/new/

https://reviews.llvm.org/D75750