[lldb-dev] Problem with large sym files

[Greg Clayton]

Looks like dsymutil bailed when it couldn't write the mach-o dSYM file.

Are you invoking dsymutil yourself multiple times? If you have a fat file, you probably want to wait until you have your universal final binary, and run dsymutil on it.

Let me know what your invocation of dsymutil looks like and how/where it is used in your builds (on each individual arch slice, or on the universal result, or only on one arch)?

On Sep 23, 2011, at 7:49 AM, Somorjai, Akos wrote:

> Hello greg,
> 
> Further info after running dwarfdump on the partially written dSYM file:
> 
> dwarfdump /Volumes/Work/D-084/Bin.Mactel/Programs_GCC4_2_64_LLVM.dev/Support/MAPs/ArchiCAD\ 16.dsym 
> ----------------------------------------------------------------------
>  File: /Volumes/Work/D-084/Bin.Mactel/Programs_GCC4_2_64_LLVM.dev/Support/MAPs/ArchiCAD 16.dsym/Contents/Resources/DWARF/ArchiCAD.x86_64 (x86_64)
> ----------------------------------------------------------------------
> warning: (x86_64) __DWARF.__debug_info section has offset (0x02544a80) that is beyond the end (0x02544a80) of architecture slice in '/Volumes/Work/D-084/Bin.Mactel/Programs_GCC4_2_64_LLVM.dev/Support/MAPs/ArchiCAD 16.dsym/Contents/Resources/DWARF/ArchiCAD.x86_64'
> .debug_info contents:
> < EMPTY >
> 
> Do you have any clues?
> 
> Thanks,
> Ákos Somorjai
> Developer Support Manager
> 
> GRAPHISOFT | Graphisoft Park 1. Budapest 1031 Hungary | +36 1 437-3000 | asomorjai at graphisoft.com
> 
> 
> 
> From: Ákos Somorjai <asomorjai at graphisoft.com>
> Date: Fri, 23 Sep 2011 15:28:03 +0200
> To: Greg Clayton <gclayton at apple.com>
> Cc: "lldb-dev at cs.uiuc.edu" <lldb-dev at cs.uiuc.edu>
> Subject: Re: [lldb-dev] Problem with large sym files
> 
> Hello Greg,
> 
> I'm running into another problem with dsymutil; it can't extract the dsym information from our main executable (> 200 MB in size) when building the debug version repeatedly. I get the following error message:
> 
> error: unable to restore file position to 0x00000c58 for section __DWARF.__debug_info
> 
> Any ideas?
> 
> Thanks,
> Ákos Somorjai
> Developer Support Manager
> 
> GRAPHISOFT | Graphisoft Park 1. Budapest 1031 Hungary | +36 1 437-3000 | asomorjai at graphisoft.com
> 
> 
> 
> Date: Tue, 20 Sep 2011 17:24:15 +0200
> To: Greg Clayton <gclayton at apple.com>
> Cc: "lldb-dev at cs.uiuc.edu" <lldb-dev at cs.uiuc.edu>
> Subject: Re: [lldb-dev] Problem with large sym files
> 
> Hello Greg,
> 
> Thanks for the useful information!
> 
> Moving to dSYM files really helped, now I can debug ArchiCAD on my machine. The dSYM option was already set for the distribution build, so it was an easy change. Is there any other way besides using dsymutil to produce those files?
> 
> I'm also looking forward to the large memory improvements you mentioned; shall I check out the trunk and compile lldb from source?
> 
> We are using an external make file system (Perforce's Jam based), so building in Xcode is not an issue. Though it seems that even in this case Xcode holds onto the loaded symbols files between debugging sessions.
> 
> Another question: are the planned improvements making their way into the Xcode 4.2 GM? 
> 
> I'm very pleased that you are addressing the large application issues.
> 
> Best, Akos
> 
> On Sep 20, 2011, at 6:28 AM, Greg Clayton wrote:
> 
>> I made some large memory improvements a few days ago which should help with this issues, so the issue is actively being worked on.
>> 
>> One thing you can do that might improve things for debugging with LLDB is to create dSYM files for any binaries that you debug. With the DWARF in the .o files, we end up with a ton of debug info and a large memory footprint, which again, I just submitted a fix for this a few days ago and this hasn't made it into a build yet.
>> 
>> So try making dSYM files and see if this helps. There is also a compounding issue going on in the Xcode builds where Xcode holds onto references to our symbol files between builds (we have a radar for this) so each time you modify and rebuild and debug again, we still have a copy of the old stuff in memory, so you end up with two copies of the LLVM/Clang executables and debug info. This can only be cured by quitting and restarting Xcode and we do have a fix on the way in the next release.
>> 
>> So for now try:
>> - making dSYM files for all executables you want to debug
>> - quit Xcode if you end up rebuilding and linking after a few builds
>> 
>> We also have new accelerator tables  we are building into the compilers and into the dSYM files that will greatly help with this issue (speed and memory).
>> 
>> Greg Clayton
>> 
>> 
>> On Sep 19, 2011, at 6:16 AM, Somorjai, Akos wrote:
>> 
>>> Hello everyone,
>>> 
>>> I've just recently started to convert our codebase to LLVM 3.0, and also gave a try to debugging with LLDB. And that's where I ran into a serious wall: lldb is unable to load the sym files for our main executable (not to mention the numerous shared libraries it uses). I'm using LLDB-75 on Mac OS X 10.6.8 on a Core i7 MacBook Pro, have 8 GB memory, the sym format is DWARF-2.
>>> 
>>> Well, it loads the sym files after 15 mins or so, but then setting a breakpoint also took 10 mins, and step one line during debugging is another 15 mins.
>>> 
>>> At the end of the process Xcode used 35 GBs of virtual and 6.5 GBs of real memory (according to the Activity Monitor).
>>> 
>>> Is there anything I can do about it?
>>> 
>>> GDB handles this situation quite gracefully…
>>> Thanks,
>>> 
>>> Ákos Somorjai
>>> Developer Support Manager
>>> 
>>> GRAPHISOFT | Graphisoft Park 1. Budapest 1031 Hungary | +36 1 437-3000 | asomorjai at graphisoft.com
>>> 
>>> 
>>> _______________________________________________
>>> lldb-dev mailing list
>>> lldb-dev at cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>> 
>