<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none"><!-- p { margin-top: 0px; margin-bottom: 0px; }--></style>
</head>
<body dir="ltr" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p></p>
<div>>*nod* That's been the historic ELF+DWARF approach, but both MacOS (with dsyms+DWARF) and Windows </div>
<div>>(COFF+CodeView+PDB) don't do it that way, and instead involve the linker to a degree.</div>
<div>>Mostly I'm wondering if it'd be reasonable to (and if anyone would be interested in doing it) do</div>
<div>>something more like the PDB support - fully debug-aware linking.</div>
<div> </div>
<div>Honestly saying I only know how ELF linker works and may be my thoughts below are silly for some reason or duplicating</div>
<div>some already existent approach. Looking at what .dwp do, looks there are two main things reducing size debug data:</div>
<div>1) "It must allow for the removal of duplicate type units".</div>
<div>2) "It must allow for the removal of duplicate strings".</div>
<div><br>
</div>
<div>Linker already deduplicates strings by itself, though it can delegate it to some API for debug sections. </div>
<div>And what it could probably do is call some library API. Linker could give it a some set (or all of) </div>
<div>.debug_* sections so this library would rebuild and optimize the dwarf data, eliminate duplicates, and</div>
<div>return optimized debug sections back to linker. Then linker would perform relocations and emit the result to output.</div>
<div><br>
</div>
<div>That way library can be used for stand alone post proccessing tool probably</div>
<div>and linker should be able to work with data on a sections level only and be not DWARF aware.</div>
<div><br>
</div>
<div>>Sure - but it works/is supported/is implemented. If someone wants to implement the newer thing, that's cool, but I don't have any </div>
<div>>personal motivation to do so for example. (& honestly we've been throwing around some ideas about how to further generalize the </div>
<div>>debug_info contributions to reduce some of the overhead of isolating types - so maybe if we're lazy enough, we might leapfrog </div>
<div>>this particular state and just implement that future better thing)</div>
<div> </div>
<div>I see. Basing on all comments in this thread I am inclined to agree that implementing newer thing does not make much sence atm.<br>
</div>
<div>For now I prepared patch to error out when LLD faces objects with multiple .debug_* sections for cases when we do not support it.<br>
</div>
<div>(D40950). (In LLD we are supporting deduplicating COMDATs, so generally such object is not a problem as already supported, </div>
<div>but for error reporting purposes and for <span style="font-size: 12pt;">--gdb-index we assume debug sections are unique in object, </span></div>
<div><span style="font-size: 12pt;">so in that case we looks want to error out).</span></div>
<div><br>
</div>
<div>Have last thoughts/question about this though :)</div>
<div><br>
</div>
<div>Currently clang -gdwarf-5 -fdebug-types-section works. And so linker can deduplicate types. Though that probably violates</div>
<div>specification saying there is no more .debug_type sections. But behavior is convinent for users of -fdebug-types-section.</div>
<div>I do not know how transition from v4 to v5 will happen/happens (or how transition between dwarf standarts usually happens). </div>
<div>I suppose one day clang just will start to produce v5 debug data by default.</div>
<div>And at the same time multiple .debug_info sections mentioned in DWARF5 spec as an optimization, so it should not be a mandatory</div>
<div>thing to implement. If so it just seems that either we will need to implement this optimization before switching to v5 by default or allow</div>
<div>-gdwarf-5 -fdebug-types-section to support existent use case. And since it is already works and already allowed in releases it probably means it is</div>
<div>acceptable to keep (and use) this behavior ? (If so, attempt to leapfrog can be nice strategy IMO).</div>
<div><br>
</div>
<div>>>>>I think Paul covered some of the reasons type units might not be a reasonable default.</div>
<div>>>></div>
<div>>>>One additional reason is that if you use Split DWARF (another great way to massively reduce the amount of debug info going to the linker)</div>
<div>>>>type units are mostly /just/ overhead in the .dwo files: since the debug info is not linked, there's no opportunity to remove the</div>
<div>>>>duplication anyway (unless you're making a DWP - like a >dsym file)</div>
<div>>></div>
<div>>>Yeah. Looks -gsplit-dwarf and -fdebug-types-section are harmfull together. Probably it worth to restrict using of them together or</div>
<div>>>emit a warning (both clang and gcc silently allows the combination and output has size penalty you describing).</div>
<div>></div>
<div>>Nah, only if you're not producing a DWP at the end ( https://gcc.gnu.org/wiki/DebugFissionDWP ).</div>
<div><br>
</div>
<div>Sure DWP do great job here it seems, but even for DWP use case flow it does not look make sence to force compiler to do excessive job</div>
<div>to produce types sections, because DWP producing tools probably should have no benefit from larger .dwo files with .debug_types at all I think.</div>
<div><br>
</div>
<div>I can only imagine now that somebody could use -gsplit-dwarf and -fdebug-types-section together so that can parse .debug_types.dwo</div>
<div>instead of parsing .debug_info.dwo to look for types in a bit more convinent way, but that looks too synthetic case.</div>
<div><br>
</div>
<div>>In short, I probably wouldn't change any of LLVM's defaults. But there are certainly flags people can use to reduce their debug info size.</div>
<div>></div>
<div>>You mentioned starting with this because LLVM's defaults mean the DWARF is too large to link with DWARF 32 bit? How does gold cope with this? </div>
<div>>I haven't seen failures/error messages/etc from either gold or lld related to this? (though I mostly use Split DWARF myself)</div>
<div><br>
</div>
<div>I posted some results earlier here: https://bugs.llvm.org//show_bug.cgi?id=31109#c3,</div>
<div>in short: gold 2.26.1 silently ignored this (probably produced broken output), and </div>
<div>newer versions of gold are able to report and catch the same error.</div>
<div><br>
</div>
<div>I think it is simply still not common to have such a large debug sections, we had only single bug about this so far. And hopefully</div>
<div>DWARF64 can be a solution, though it can just hide the issue, looks would be nice to reduce amount of debug data we produce still.</div>
<div><br>
</div>
<div>Best regards,</div>
<div>George | Developer | Access Softek, Inc<br>
</div>
<p><br>
</p>
</body>
</html>