[lldb-dev] Problem with large sym files
Somorjai, Akos
ASomorjai at graphisoft.com
Mon Sep 26 08:15:17 PDT 2011
On 9/24/11 12:07 AM, "Greg Clayton" <gclayton at apple.com> wrote:
>
>On Sep 23, 2011, at 10:59 AM, Somorjai, Akos wrote:
>
>> We only have an x64 architecture; we invoke dsymutil several times for
>>the
>> frameworks and the plugin bundles, but only once for the main
>>executable,
>> like this
>>
>> dsymutil
>> /Volumes/Work/D-084/Bin.Mactel/Programs_GCC4_2_64_LLVM.dev/ArchiCAD\
>> 16.app/Contents/MacOS/ArchiCAD -o
>>
>>/Volumes/Work/D-084/Bin.Mactel/Programs_GCC4_2_64_LLVM.dev/Support/MAPs/A
>>rc
>> hiCAD 16.dsym
>
>I take it you have quotes around your second path with the space in it?
>You have a backslash before the space for the app, but not for the dSYM
>above. The propper case for the extension is "dSYM" in case you want it
>to look like the what Xcode produces, though this likely won't matter on
>case insensitive file systems, though it would be good to change just in
>case.
Yeah, you're right, I do have quotes around the paths, I was just typing
the path directly instead of copying it from the Terminal; sorry.
>
>>
>> Is it a problem that the .app has a different name than the executable?
>>Or
>> the dSYM file is created in a separate folder?
>
>No, as long as spotlight can see the folder, you should easily be able to
>locate your dSYM file. If you want to make sure spotlight can see it, do
>this:
>
>% mdls
>"/Volumes/Work/D-084/Bin.Mactel/Programs_GCC4_2_64_LLVM.dev/Support/MAPs/A
>rchiCAD 16.dsym"
>
>You should see some UUID key/value pairs:
>
>% mdls ~/Documents/src/attach/a.out.dSYM
>com_apple_xcode_dsym_paths = (
> "Contents/Resources/DWARF/a.out"
>)
>com_apple_xcode_dsym_uuids = (
> "FDB4DE40-DC70-3E53-907A-442A6E379283"
>)
>
>If you see these, the debuggers will be able to find your dSYM file no
>matter what the name.
I see this kind of output.
>
>>
>>
>> If I do a clean debug build, then the dsym file is created fine. If I do
>> an incremental build, then the dsym file is created only partially, and
>>I
>> cannot see the source when debugging the main executable's code.
>
>Sounds like you have a stripping issue. Don't strip your executable
>except on the install phase of your build. Why? Because in a build like:
>
>- build all .o files
>- link executable
>- make dSYM
>- strip executable
>
>you now have an executable that is newer than your dSYM file. If you use
>a Makefile, it will notice that your dSYM file depends on your
>executable, and it will try to rebuild your dSYM file which will now use
>a stripped executable which won't have any of the needed debug map
>entries that dsymutil uses to link the dSYM files and all debug info will
>be lost.
Good point! A stripping step was included after creating the dsym file;
the reason is that we used this mechanism only for the final release
builds. I had to turn it on to be able to debug with LLDB, but I missed
the stripping.
>
>> Breakpoints in the other frameworks work fine; I can step through the
>> source.
>> I also tried to create a flat dSYM file, to no avail.
>> With verbose logging turned on, I got a bit more than 11 000 lines like
>> this: "address information attributes will be removed because there is
>>no
>> relocation address"
>
>That is fine. We dead strip the DWARF and remove functions, types and
>other info that didn't make it into the final linked executable. If you
>indeed are stripping your executable, this will happen for just about
>everything which you don't want. So try to not do any stripping as part
>of your normal compile/link/debug/fix cycles.
>
>Does that make sense and/or help?
Thanks, yes.
The problem is that we're back at square number one, I still get an error
message from dsymutil while extracting the symbols from the main
executable:
error: unable to restore file position to 0x00000c58 for section
__DWARF.__debug_info
Any other ideas?
Thanks a lot,
Akos
>
>>
>> Thanks,
>>
>> Ákos Somorjai
>> Developer Support Manager
>>
>> GRAPHISOFT | Graphisoft Park 1. Budapest 1031 Hungary | +36 1 437-3000 |
>> asomorjai at graphisoft.com
>>
>>
>>
>>
>>
>>
>> On 9/23/11 7:20 PM, "Greg Clayton" <gclayton at apple.com> wrote:
>>
>>> Looks like dsymutil bailed when it couldn't write the mach-o dSYM file.
>>>
>>> Are you invoking dsymutil yourself multiple times? If you have a fat
>>> file, you probably want to wait until you have your universal final
>>> binary, and run dsymutil on it.
>>>
>>> Let me know what your invocation of dsymutil looks like and how/where
>>>it
>>> is used in your builds (on each individual arch slice, or on the
>>> universal result, or only on one arch)?
>>>
>>> On Sep 23, 2011, at 7:49 AM, Somorjai, Akos wrote:
>>>
>>>> Hello greg,
>>>>
>>>> Further info after running dwarfdump on the partially written dSYM
>>>>file:
>>>>
>>>> dwarfdump
>>>>
>>>>/Volumes/Work/D-084/Bin.Mactel/Programs_GCC4_2_64_LLVM.dev/Support/MAPs
>>>>/A
>>>> rchiCAD\ 16.dsym
>>>> ----------------------------------------------------------------------
>>>> File:
>>>>
>>>>/Volumes/Work/D-084/Bin.Mactel/Programs_GCC4_2_64_LLVM.dev/Support/MAPs
>>>>/A
>>>> rchiCAD 16.dsym/Contents/Resources/DWARF/ArchiCAD.x86_64 (x86_64)
>>>> ----------------------------------------------------------------------
>>>> warning: (x86_64) __DWARF.__debug_info section has offset (0x02544a80)
>>>> that is beyond the end (0x02544a80) of architecture slice in
>>>>
>>>>'/Volumes/Work/D-084/Bin.Mactel/Programs_GCC4_2_64_LLVM.dev/Support/MAP
>>>>s/
>>>> ArchiCAD 16.dsym/Contents/Resources/DWARF/ArchiCAD.x86_64'
>>>> .debug_info contents:
>>>> < EMPTY >
>>>>
>>>> Do you have any clues?
>>>>
>>>> Thanks,
>>>> Ákos Somorjai
>>>> Developer Support Manager
>>>>
>>>> GRAPHISOFT | Graphisoft Park 1. Budapest 1031 Hungary | +36 1 437-3000
>>>> | asomorjai at graphisoft.com
>>>>
>>>>
>>>>
>>>> From: Ákos Somorjai <asomorjai at graphisoft.com>
>>>> Date: Fri, 23 Sep 2011 15:28:03 +0200
>>>> To: Greg Clayton <gclayton at apple.com>
>>>> Cc: "lldb-dev at cs.uiuc.edu" <lldb-dev at cs.uiuc.edu>
>>>> Subject: Re: [lldb-dev] Problem with large sym files
>>>>
>>>> Hello Greg,
>>>>
>>>> I'm running into another problem with dsymutil; it can't extract the
>>>> dsym information from our main executable (> 200 MB in size) when
>>>> building the debug version repeatedly. I get the following error
>>>>message:
>>>>
>>>> error: unable to restore file position to 0x00000c58 for section
>>>> __DWARF.__debug_info
>>>>
>>>> Any ideas?
>>>>
>>>> Thanks,
>>>> Ákos Somorjai
>>>> Developer Support Manager
>>>>
>>>> GRAPHISOFT | Graphisoft Park 1. Budapest 1031 Hungary | +36 1 437-3000
>>>> | asomorjai at graphisoft.com
>>>>
>>>>
>>>>
>>>> Date: Tue, 20 Sep 2011 17:24:15 +0200
>>>> To: Greg Clayton <gclayton at apple.com>
>>>> Cc: "lldb-dev at cs.uiuc.edu" <lldb-dev at cs.uiuc.edu>
>>>> Subject: Re: [lldb-dev] Problem with large sym files
>>>>
>>>> Hello Greg,
>>>>
>>>> Thanks for the useful information!
>>>>
>>>> Moving to dSYM files really helped, now I can debug ArchiCAD on my
>>>> machine. The dSYM option was already set for the distribution build,
>>>>so
>>>> it was an easy change. Is there any other way besides using dsymutil
>>>>to
>>>> produce those files?
>>>>
>>>> I'm also looking forward to the large memory improvements you
>>>> mentioned; shall I check out the trunk and compile lldb from source?
>>>>
>>>> We are using an external make file system (Perforce's Jam based), so
>>>> building in Xcode is not an issue. Though it seems that even in this
>>>> case Xcode holds onto the loaded symbols files between debugging
>>>> sessions.
>>>>
>>>> Another question: are the planned improvements making their way into
>>>> the Xcode 4.2 GM?
>>>>
>>>> I'm very pleased that you are addressing the large application issues.
>>>>
>>>> Best, Akos
>>>>
>>>> On Sep 20, 2011, at 6:28 AM, Greg Clayton wrote:
>>>>
>>>>> I made some large memory improvements a few days ago which should
>>>>>help
>>>>> with this issues, so the issue is actively being worked on.
>>>>>
>>>>> One thing you can do that might improve things for debugging with
>>>>>LLDB
>>>>> is to create dSYM files for any binaries that you debug. With the
>>>>>DWARF
>>>>> in the .o files, we end up with a ton of debug info and a large
>>>>>memory
>>>>> footprint, which again, I just submitted a fix for this a few days
>>>>>ago
>>>>> and this hasn't made it into a build yet.
>>>>>
>>>>> So try making dSYM files and see if this helps. There is also a
>>>>> compounding issue going on in the Xcode builds where Xcode holds onto
>>>>> references to our symbol files between builds (we have a radar for
>>>>> this) so each time you modify and rebuild and debug again, we still
>>>>> have a copy of the old stuff in memory, so you end up with two copies
>>>>> of the LLVM/Clang executables and debug info. This can only be cured
>>>>>by
>>>>> quitting and restarting Xcode and we do have a fix on the way in the
>>>>> next release.
>>>>>
>>>>> So for now try:
>>>>> - making dSYM files for all executables you want to debug
>>>>> - quit Xcode if you end up rebuilding and linking after a few builds
>>>>>
>>>>> We also have new accelerator tables we are building into the
>>>>> compilers and into the dSYM files that will greatly help with this
>>>>> issue (speed and memory).
>>>>>
>>>>> Greg Clayton
>>>>>
>>>>>
>>>>> On Sep 19, 2011, at 6:16 AM, Somorjai, Akos wrote:
>>>>>
>>>>>> Hello everyone,
>>>>>>
>>>>>> I've just recently started to convert our codebase to LLVM 3.0, and
>>>>>> also gave a try to debugging with LLDB. And that's where I ran into
>>>>>>a
>>>>>> serious wall: lldb is unable to load the sym files for our main
>>>>>> executable (not to mention the numerous shared libraries it uses).
>>>>>>I'm
>>>>>> using LLDB-75 on Mac OS X 10.6.8 on a Core i7 MacBook Pro, have 8 GB
>>>>>> memory, the sym format is DWARF-2.
>>>>>>
>>>>>> Well, it loads the sym files after 15 mins or so, but then setting a
>>>>>> breakpoint also took 10 mins, and step one line during debugging is
>>>>>> another 15 mins.
>>>>>>
>>>>>> At the end of the process Xcode used 35 GBs of virtual and 6.5 GBs
>>>>>>of
>>>>>> real memory (according to the Activity Monitor).
>>>>>>
>>>>>> Is there anything I can do about it?
>>>>>>
>>>>>> GDB handles this situation quite gracefullyŠ
>>>>>> Thanks,
>>>>>>
>>>>>> Ákos Somorjai
>>>>>> Developer Support Manager
>>>>>>
>>>>>> GRAPHISOFT | Graphisoft Park 1. Budapest 1031 Hungary | +36 1
>>>>>> 437-3000 | asomorjai at graphisoft.com
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> lldb-dev mailing list
>>>>>> lldb-dev at cs.uiuc.edu
>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>>>>>
>>>>
>>>
>>
>
More information about the lldb-dev
mailing list