[LLVMdev] RFC: Improving our DWARF (and ELF) emission testing capabilities

[Robinson, Paul]

David Blaikie [dblaikie at gmail.com] wrote:
> On Tue, Jan 22, 2013 at 6:22 PM, Robinson, Paul
> <Paul.Robinson at am.sony.com> wrote:
>> David Blaikie [dblaikie at gmail.com] wrote:
>>> On Tue, Jan 22, 2013 at 3:23 PM, Robinson, Paul
>>> <Paul.Robinson at am.sony.com> wrote:
>>>>>>> 2. Relying of assembly directive emissions (i.e. .cfi_*), which is
>>>>>>> cumbersome and misses a lot of things like actual DWARF encoding.
>>>>>>
>>>>>> I'm not sure what you mean by "actual DWARF encoding" here.
>>>>>> (disclaimer: I've only recently started dabbling with debug info, so I
>>>>>> may be missing obvious things)
>>>>>
>>>>> I mean that it doesn't test the whole way, and there's quite a bit of
>>>>> DWARF-related functionality in MC. So when a test relies on matching
>>>>> directives in ASM output, there's quite a bit of code in MC it doesn't
>>>>> exercise.
>>>>
>>>> Hmmm. "Proper" testing would exercise each component involved, as well
>>>> as possibly longer paths that maybe are not exactly the sum of the parts.
>>>> Debug info changes are quite likely to involve most or all of:
>>>> - Clang's C/C++ to IR (metadata)
>>>
>>> Simple enough: changes to Clang require tests in Clang (OK, so there's
>>> no mechanism in place to avoid tests in Clang testing optimization,
>>> for example, but we try to be good about it)
>>>
>>>> - LLVM's IR to assembler source
>>>> - assembler source to object file
>>>> - LLVM's IR to object file (which partly bypasses or can be different
>>>>   from the previous two paths)
>>>
>>> & these generally go in LLVM - yeah, they could be separate, but I'd
>>> expect the assembler and object file emission to be tested separately
>>> already - the only benefit to testing particular IR->object &
>>> separately testing particular IR->assembly is probably not worthwhile.
>>
>> I cite PR13303/PR14524, where asm and direct-object output differ.
>> This came up early in my LLVM career and has doubtless poisoned my
>> outlook for life....
>>
>> In many cases I think the same test _source_ can be used to check both
>> asm and object, with appropriate RUN lines, and whether you want to
>> count that as the same or separate depends on how you like to game the
>> counts.  What matters to me is both paths get tested.
>
> Sure - I'd just rather see these separated into object emission/asm
> tests if possible, rather than littering the other test cases with two
> modes each. I assume the code is sufficiently factored such that
> testing in this way would be generally fairly reliable (ie: I hope we
> can hit the same code paths that produce .byte from debug info as
> would produce it from anywhere else in the backend and just test that
> once directly)

That works for sections where we do actually produce .byte directives.
.debug_info, .debug_str, .debug_macinfo, and more are like this.

But for .debug_line (the topic of the PRs I cited) and for CFI (which
was what the OP specifically mentioned, if we can remember that far back)
that's not how it works.  The paths get pretty different internally, and
demonstrably can emit different object files.

So, at least in those cases, tests for (a) LLVM producing the correct 
assembler directives, (b) LLVM producing the correct binary encoding, 
and (c) the assembler producing the correct binary encoding, are all 
appropriate and valuable.  Exactly how they get packaged is less
important than that the tests all exist.

--paulr

P.S. as long as I'm replying anyway....
>> I also think I'd spectacularly fail the CSSP test.  :-)
one reason being I can't even spell CSDP correctly!