[LLVMdev] unit tests

[David Blaikie]

[readding llvm-dev]

On Mon, Aug 20, 2012 at 1:31 PM, reed kotler <rkotler at mips.com> wrote:
> Cool. So a few more questions then.
>
> If I add tests, I can easily check them for x86 and for mips.
> Do I need to test them for Arm and others or should I just conditionally
> enter them, I'm assuming there is a way, for Mips and X86 only?

This (& the subsequent) question confuses me a little. Unit tests are
API level, they should be executable (with the same result) on any
hardware. This applies even if the test is intended only to test one
(or some set of) backend - the unit test would just be hardcoded to
invoke whatever parts of the specific backend it wants to use, and
verify the result of that. This test should then be able to be
executed on every test execution no matter the host hardware.

(technically the same is true of even non unit tests - you'll see many
test cases in both Clang and LLVM have hardcoded target triples to
exercise a particular code path - these tests run on any hardware
which means it's easier to avoid regressing platforms you're not
currently developing on. We don't have a rigorous approach to this
(it's not like we hardcode target triples in every test case to ensure
consistent results - so we get some incidental coverage by running the
suite on different hardware & verifying that the output is the same
(without loading down every execution of the tests to run several
architecture variations)) but it is done where necessary/appropriate)

>
> We run our tests usually "cross" to Qemu or to a raw board. Is there a way
> to do this?
>
> We also run them on Mips linux boxes and can even run them on Mips linux
> booted under Qemu.
>
> Is it okay to make a Target subdirectory ?

I don't dabble in the backend much so I couldn't say for sure -
doesn't seem unreasonable to me to do that if you want to write unit
tests against targets.