[LLVMdev] Writing unit tests for DWARF?

[Renato Golin]

On 17 March 2011 06:40, Talin <viridia at gmail.com> wrote:
> The code that generates debugging information is quite fragile - you have to
> generate metadata for each of your files, classes, and functions, and do so
> without error, because if you do make a mistake, the only way you'll find
> out is because gdb refuses to debug your program. And as I work on the code,
> occasionally bugs creep in, either from my side or occasionally from the
> LLVM side. The problem is, that I don't always check if the debug
> information is valid, so several weeks can go by and I don't notice
> something broke.

Strongly agree.


> What is needed is some way to write a unit test for DWARF information, so
> that if I broke something I would notice it immediately and could either fix
> it or roll back. Unfortunately, the various DIDescriptor.Verify() methods
> are nowhere near strict enough - you can create completely nonsensical DIEs
> that still pass through Verify(). And even if the Verify() methods were 100%
> reliable, they only test whether the LLVM metadata is valid - they don't
> test whether the actual DWARF embedded in the final executable is correct.

Strongly agree. But I go further...

I could help with the verification process (since it's much better to
fail verification than to fail gdb testuite), but I don't know the
design decisions being taken for debug information/metadata, and they
change too frequently to dig the code to learn. There is no API
documentation and the interface (IR metadata) docs are old and
inaccurate.

I'd say, in order of importance, the three things that need to be done ASAP are:

 1. Stick to one representation and document it (like LangRef), so
other people could help
 2. Enhance Validate() methods to be extremely strict (like Module's),
so it fails straight away
 3. Create tests (unit and regression) and run them during check-all,
so we don't regress

The tests are last because it's much easier to catch an assertion than
a silent codegen error.

After the initial period, we iterate those three steps (and not less!)
again and again, until debug information is good. I see the importance
of changing the IR (as I've requested quite a few times) but I
understand that it's better for every one to have a stable IR. Every
new version can have a few changes, not necessarily backward
compatible, but those also need to be documented beforehand (mailing
list, blog, release notes).

If we follow the three steps above in an iterative way, during every
release, we can achieve stability AND feature completeness. But
(IMHO), stability comes first.

cheers,
--renato