[lldb-dev] Directories to search for separate debugging information

[Greg Clayton]

On Nov 16, 2012, at 11:11 AM, Sara Burke <Sara.Burke at mathworks.com> wrote:

> Hi Greg,
> 
>> It uses /System/Library/PrivateFrameworks/DebugSymbols.framework which internally uses spotlight, shell commands, explicit paths (like you want to specify), implicit paths, and file mapped paths where you can specify a directory that contains directories that are the UUID chopped up into 4 byte sequences. 
>> 
>> You can write a shell command that will be given a UUID value and is expected to return a plist with certain keys that specify where the binary is. Let me know if you want to know more about this.
> 
> Yes, I would like to know more about this :)

You can enable the shell script using:

% defaults write com.apple.DebugSymbols DBGShellCommands /path/to/shellscript

Your shell script will be invoked with a UUID string value like "23516BE4-29BE-350C-91C9-F36E7999F0F1". The shell script can respond with a plist in the following format:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>23516BE4-29BE-350C-91C9-F36E7999F0F1</key>
	<dict>
		<key>DBGArchitecture</key>
		<string>i386</string>
		<key>DBGBuildSourcePath</key>
		<string>/path/to/build/sources</string>
		<key>DBGSourcePath</key>
		<string>/path/to/actual/sources</string>
		<key>DBGDSYMPath</key>
		<string>/path/to/foo.dSYM/Contents/Resources/DWARF/foo</string>
		<key>DBGSymbolRichExecutable</key>
		<string>/path/to/unstripped/exectuable</string>
	</dict>
	<key>A40597AA-5529-3337-8C09-D8A014EB1578</key>
	<dict>
		<key>DBGArchitecture</key>
		<string>x86_64</string>
		.....
	</dict>
</dict>
</plist>

Note that this file contained two architectures so it has two UUID key/value pairs.

Since dSYM files are bundles, you can also place UUID info plists files inside your dSYM bundles:

/path/to/foo.dSYM/Contents/Resources/23516BE4-29BE-350C-91C9-F36E7999F0F1.plist
/path/to/foo.dSYM/Contents/Resources/A40597AA-5529-3337-8C09-D8A014EB1578.plist


% cat /path/to/foo.dSYM/Contents/Resources/23516BE4-29BE-350C-91C9-F36E7999F0F1.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>DBGArchitecture</key>
    <string>i386</string>
    <key>DBGBuildSourcePath</key>
    <string>/path/to/build/sources</string>
    <key>DBGSourcePath</key>
    <string>/path/to/actual/sources</string>
    <key>DBGDSYMPath</key>
    <string>/path/to/foo.dSYM/Contents/Resources/DWARF/foo</string>
    <key>DBGSymbolRichExecutable</key>
    <string>/path/to/unstripped/exectuable</string>
</dict>
</plist>

These have the exact same content as the above shell script output, so it makes making the results of your shell script very easy to create by combining two plists into a single one where you take the UUID and use it a string key, and the value is the contents of the plist.

Then LLDB will always know that your debug info contains source paths like "/path/to/build/sources" and it will remap them to "/path/to/actual/sources" by parsing these plist files in the dSYM bundle. This is something your build system can take care of.

We have a shell script at Apple and it goes a step further and caches these files locally in a known location so the next query for the dSYM file will be able to use the cached version. To do this, we "file map" the UUID values by chopping up the UUID into 4 character chunks and making the last value a symlink to the locally cached dSYM file:

~/Library/SymbolCache/dsyms/uuids/2351/6BE4/29BE/350C/91C9/F36E7999F0F1
~/Library/SymbolCache/dsyms/uuids/A405/97AA/5529/3337/8C09/D8A014EB1578

The last entries in these file mapped directories are symlinks to the actual dsym mach file in the dsym bundle:

ls -lAF ~/Library/SymbolCache/dsyms/uuids/2351/6BE4/29BE/350C/91C9/F36E7999F0F1

~/Library/SymbolCache/dsyms/uuids/2351/6BE4/29BE/350C/91C9/F36E7999F0F1@ -> ../../../../../../dsyms/foo.dSYM/Contents/Resources/DWARF/foo

Then you can also tell DebugSymbols to check this "file mapped" cache first using:

% defaults write com.apple.DebugSymbols DBGFileMappedPaths ~/Library/SymbolCache/dsyms/uuids

So if you enable both:

% defaults write com.apple.DebugSymbols DBGShellCommands /path/to/shellscript
% defaults write com.apple.DebugSymbols DBGFileMappedPaths ~/Library/SymbolCache/dsyms/uuids

We will only call your "/path/to/shellscript" if we didn't find an already cached symbol file in a symlink in "~/Library/SymbolCache/dsyms/uuids". This allows efficient caching and debugging...

This allows any company to implement a smart symbol finding and caching solution that matches their build systems.

Greg Clayton