[llvm] r174874 - AArch64: Add basic relocation processing for llvm-dwarfdump.

[David Blaikie]

On Mon, Feb 11, 2013 at 11:28 AM, Tim Northover <t.p.northover at gmail.com> wrote:
>> I'm of the opinion that dwarfdump test cases should, as much as
>> possible, only test dwarfdump & thus your original version (with a
>> binary file committed as the input).
>
> That was the thinking behind my commit too. It was actually written
> when AArch64 wasn't even being built by default so I definitely wanted
> this tested wherever possible.
>
> I'm rather more relaxed about that now, and would probably want to
> know if the debugging information changed on a file like that,
> regardless of whether it was llc or dwarfdump's fault.

Incidental test coverage is somewhat unfortunate - if there's llc
behavior that needs testing I'd like to see that tested deliberately
rather than by a side effect of a dwarfdump test.

(oddly enough with test tools I'm usually a bit more lenient about
test coverage of them - so if this had been a test that llc produced
the right debug info & used dwarfdump (& even involved changes to
dwarfdump to make it support things as this did) I might've been more
willing to just ignore the lack of testing of dwarfdump. Not that this
is necessarily correct, just a comment about my own testing
mentality/tradeoffs)

>> This way, we get a clear sense of when dwarfdump is broken (hey, the
>> dwarfdump tests failed) as compared to when the product is broken
>> (hey, the tests that use dwarfdump failed - the product must be
>> broken)
>
> A good idea in general, but I'm not convinced it applies to the
> in-tree tests: they should always be passing no matter what.

They should, yes, but:

> More
> isolated tests help with debugging when failures *do* happen, of
> course, but usually not that much. You know what's been poked recently
> and running the steps manually is usually enough for when even that
> fails.

Easier for more familiar developers - much more helpful for less
familiar developers to have well targeted/constrained test cases. (&
even reducing the analysis load for seasoned developers is valuable -
perhaps in just a small way)

The only other argument I could apply here would be test execution
performance: keeping the regression test iteration time low is
important & running one process instead of two would help with that.

(though, arguably, when Eric & I were discussing this earlier we sort
of settled on the fact that this approach would necessitate up to
twice as many tests: one product test (running llc + dwarfdump +
FileCheck) and one dwarfdump test (obj file > dwarfdump + FileCheck)
so it doesn't necessarily result in a smaller/faster test suite)

- David