[llvm-dev] Pardon the newbie question

Fri Apr 16 13:32:01 PDT 2021

On Fri, Apr 16, 2021 at 5:41 AM pawel k. <pawel.kunio at gmail.com> wrote:
>
> Hello,
> Splitdwarf worked nicely for me. Big thaks to all contributing and biy there were many useful hints.
>
> I think that would be useful. I can try to collect them all and brief each and then somebody could apply the styling if need be or if just pure text paragraph would suffice I can prepare some file like that. I especially liked lineinfo+funcaddrs only option. Very useful for stacktraces and asserts if not planning to use gdb.

I think most of this is written up here:
https://llvm.org/docs/GettingStarted.html#common-problems

Though looks like it could have a bullet point for -g1-for-backtraces too.

>
> Hope to help somehow.
>
> Big thanks and best regards,
> Pk
>
>
> pt., 16.04.2021, 14:18 użytkownik kuter dinel <kuterdinel at gmail.com> napisał:
>>
>> Thanks everyone for great advice, maybe we should mention some of this on the https://llvm.org/docs/GettingStarted.html page.
>> Maybe under a subtitle like "Development on low-end devices" or something. This would be helpful for newcomers who are working with low-end devices.
>>
>> Also since it was not mentioned, ccache can help with build times during development.
>>
>> pawel k. via llvm-dev <llvm-dev at lists.llvm.org>, 16 Nis 2021 Cum, 03:10 tarihinde şunu yazdı:
>>>
>>> Hello,
>>> Not sure i have anything useful to say on discussion on ctorhoming but should it be so ill try to chime in on discussion. On bugs i cant talk. Didnt heavily test it or use it much at all but i could say how it may seem when trying to compete with vc and how it may look from john does sw devel side possibly not a superstar one.
>>>
>>> Best regards,
>>> Pawel Kunio
>>>
>>> pt., 16.04.2021, 02:03 użytkownik David Blaikie <dblaikie at gmail.com> napisał:
>>>>
>>>> On Thu, Apr 15, 2021 at 4:57 PM pawel k. <pawel.kunio at gmail.com> wrote:
>>>> >
>>>> > Is constructor homing this featurette where class di is emitted only for compunits with reference to constructor?
>>>>
>>>> Yep - both Clang and GCC already do something equivalent for any type
>>>> with virtual functions - the debug info for that type is emitted only
>>>> where the vtable is emitted (so called "vtable homing") so it's not
>>>> without precedent. Yeah, it means potentially missing any type that
>>>> isn't constructed (types used only to hold a bunch of static members,
>>>> for instance) - but perhaps such a type isn't needed anyway. This
>>>> isn't done for types with trivial constructors - since they don't
>>>> generate code, etc.
>>>>
>>>> >  Id speculate therell be problems with it as i understand the idea by now.
>>>>
>>>> Certainly open to any critiques/bugs/etc. It's currently accessible
>>>> under -fuse-ctor-homing
>>>> https://www.mail-archive.com/cfe-commits@lists.llvm.org/msg218269.html
>>>>
>>>> > Shared type table feels bit more complex on impl side fir merging etc but feels stronger.
>>>>
>>>> Tradeoffs to be sure - anything that's shared during compile time's
>>>> not really usable in the build environment (massively distributed) I
>>>> support, but can work for others (like MSVC does). C++ modules (both
>>>> Clang Header Modules and C++20 standard modules) offer another way to
>>>> "home" debug info - since modules are then a first class build product
>>>> they can be used to home debug info quite reliably (template
>>>> instantiations may be duplicated - since they may be instantiated in
>>>> multiple unrelated places, but even then any module depending on a
>>>> module with an insntantiation need not re-do that instantiation).
>>>>
>>>> - Dave
>>>>
>>>> >
>>>> > Best regards,
>>>> > Pawel Kunio
>>>> >
>>>> > pt., 16.04.2021, 01:40 użytkownik David Blaikie <dblaikie at gmail.com> napisał:
>>>> >>
>>>> >> On Thu, Apr 15, 2021 at 4:22 PM pawel k. <pawel.kunio at gmail.com> wrote:
>>>> >> >
>>>> >> > Ok for gdb, load time was major pain.
>>>> >>
>>>> >> Ah, for large binaries you probably want to compile with
>>>> >> -ggnu-pubnames -Wl,--gdb-index (you must compile with this for Split
>>>> >> DWARF - it relies on the index) that takes, for instance, gdb startup
>>>> >> time debugging clang from 3 minutes to about 3 seconds. (lldb does a
>>>> >> better job lazy loading DWARF - so faster startup time even without an
>>>> >> index (& it can't use the gdb index anyway) trading off some
>>>> >> operations later will be slower as things are lazy loaded later on)
>>>> >>
>>>> >> > Type casting was patchy some types missing etc,
>>>> >>
>>>> >> Hard to say what's at work there - happy to speculate/help if you have
>>>> >> a specific/isolated example.
>>>> >>
>>>> >> > some var tracking especially under optimizations was tricky for example having to cast registers because info this var is stored in this register was missing
>>>> >>
>>>> >> Yeah, that's less about the workflow/technology, but more as we like
>>>> >> to say, the "quality of implementation" - keeping track of variables
>>>> >> under optimization is tricky, and different compilers do it
>>>> >> well/poorly - DWARF certainly offers a good variety of ways to express
>>>> >> these things (and gaining more as DWARF improves - it's certainly not
>>>> >> "complete", but I don't think it ever can be - there'll always be more
>>>> >> cunning optimizations and cunning ways to try to recover the value of
>>>> >> user variables under those optimizations).
>>>> >>
>>>> >> > hmm also one mire thing that could minimize debug info size could be shared types section with removing local copies per compilation unit.
>>>> >>
>>>> >> Yep - that can already be done with DWARF type units
>>>> >> (-fdebug-types-section), but also various strategies to reduce
>>>> >> emitting the types in the first place (hopefully we'll be enabling the
>>>> >> constructor homing strategy by default soon which should reduce
>>>> >> duplicate type information significantly)
>>>> >>
>>>> >> > Also global vars etc could be stored in shared segment. Also we could think of somehow indexing the db.
>>>> >>
>>>> >> Yeah, as mentioned above, gdb (& lld and gold) support the gdb_index
>>>> >> format and that can be generated at link time with -Wl,--gdb-index
>>>> >> (objects need to be compiled with -ggnu-pubnames (that comes by
>>>> >> default if you specify -gsplit-dwarf I think... ) for this to work).
>>>> >> And there is a DWARFv5 debug_names section, but it's not fully
>>>> >> supported yet (we don't have any linker that supports merging
>>>> >> .debug_names from object files into a unified .debug_names in the
>>>> >> linked binary - nor any debuggers that can consume the .debug_names
>>>> >> index).
>>>> >>
>>>> >> > My other idea is indeed distributing stripped binaries and devels keeping debug info dbs snapshots locally or via debug servers. If we dont wanna debug servers, stack traces etc should contain stack dumps maybe or some in the middle solution with lineinfo only possibly hashed etc.
>>>> >> >
>>>> >> > Best regards,
>>>> >> > Pawel Kunio
>>>> >> >
>>>> >> > pt., 16.04.2021, 01:06 użytkownik David Blaikie <dblaikie at gmail.com> napisał:
>>>> >> >>
>>>> >> >> On Thu, Apr 15, 2021 at 2:43 PM pawel k. via llvm-dev
>>>> >> >> <llvm-dev at lists.llvm.org> wrote:
>>>> >> >> >
>>>> >> >> > Hiya,
>>>> >> >> >
>>>> >> >> > Enormous thanks for all suggests and extensive too. Testing splitdwarf asap.
>>>> >> >> >
>>>> >> >> > Makes me think what i thought previously. We need dwarf go pdbish way.
>>>> >> >>
>>>> >> >> Split DWARF provides something like this, though with some different
>>>> >> >> tradeoffs - for instance Split DWARF is easier to use in a distributed
>>>> >> >> build (since the compiler doesn't have to communicate with a separate
>>>> >> >> server while it's compiling).
>>>> >> >>
>>>> >> >> > Separate dblike possibly adressable by url.
>>>> >> >>
>>>> >> >> Symbol servers are another problem - even using classic DWARF you can
>>>> >> >> then strip the main binary, keep the unstripped binary and put that
>>>> >> >> unstripped binary on some kind of symbol server system (I'm not sure
>>>> >> >> if gdb supports something like that natively, but could be done
>>>> >> >> without DWARF changes).
>>>> >> >>
>>>> >> >> > I was hacking pdb techno before it got sexy. I know a bit how it is organized and could try to help with architecting or designing such solution. Loved dbinfo on vstudio. Gdbish not so much.
>>>> >> >>
>>>> >> >> What sort of problems have you had with gdb or the DWARF debugging
>>>> >> >> workflow more generally?
>>>> >> >>
>>>> >> >> > And i was slaving on my corpo cotton plantation passively using gdb for about a decade.
>>>> >> >>
>>>> >> >> (analogies to slavery aren't suitable for this community)
>>>> >> >>
>>>> >> >> - Dave
>>>> >> >>
>>>> >> >> >
>>>> >> >> > If catches anyones focus, lets discuss the solution.
>>>> >> >> >
>>>> >> >> > Best regards,
>>>> >> >> > Pawel Kunio
>>>> >> >> >
>>>> >> >> > czw., 15.04.2021, 23:37 użytkownik Min-Yih Hsu <minyihh at uci.edu> napisał:
>>>> >> >> >>
>>>> >> >> >> You can use `LLVM_USE_LINKER=lld` CMake variable to adopt LLD (to build LLVM). And yes, LLD takes less memory and runs faster. Here are some other tips to save memory:
>>>> >> >> >> 1. You can use `LLVM_PARALLEL_LINK_JOBS=N` (also a cmake variable) to limit the amount of parallel linker jobs to save some memory.
>>>> >> >> >>
>>>> >> >> >> 2. Build libraries as shared library via `BUILD_SHARED_LIBS=ON` CMake variable can dramatically speed up the linking time and save you some memory.
>>>> >> >> >>
>>>> >> >> >> 3. Since you’re building a Debug build (and you’re building on Linux), `LLVM_USE_SPLIT_DWARF` can dramatically reduce the size of debug info, which, to some extend, also save you some memory during link time.
>>>> >> >> >>
>>>> >> >> >> Best,
>>>> >> >> >> -Min
>>>> >> >> >> > On Apr 15, 2021, at 1:05 PM, pawel k. via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>> >> >> >> >
>>>> >> >> >> > Hello,
>>>> >> >> >> > Im trying to build trunk clang in debug version on oldish ubuntu with low mem. Linking lli takes ages and fails on low mem. Is there a chance building would succeed if i used lld instead of ld? If so is there an option either to force lld or whole clang toolchain use in cmake instead of default gcc (both gcc and clang are avail on system)? Otherwise I think ill stick with release.
>>>> >> >> >> >
>>>> >> >> >> > Best regards,
>>>> >> >> >> > Pawel Kunio
>>>> >> >> >> > _______________________________________________
>>>> >> >> >> > LLVM Developers mailing list
>>>> >> >> >> > llvm-dev at lists.llvm.org
>>>> >> >> >> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>> >> >> >>
>>>> >> >> > _______________________________________________
>>>> >> >> > LLVM Developers mailing list
>>>> >> >> > llvm-dev at lists.llvm.org
>>>> >> >> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev