[llvm] r289648 - [AVR] Add the very first on-target test

[Matthias Braun via llvm-commits]

> On Dec 14, 2016, at 12:48 PM, Dylan McKay via llvm-commits <llvm-commits at lists.llvm.org> wrote:
> 
> > This test will fail as soon as anyone builds the AVR back-end without
> access to AVR hardware.
> 
> I added a config file that ignored the tests if AVR was not built? On top of that, the tests were only supposed to run if 'AVRLIT_BOARD' and 'AVRLIT_PORT' is set. Clearly it didn't work properly (due to my inexperience with Python)
> 
> If I were to fix that, it wouldn't stop the tests from breaking if AVR was made non-experimental?
> 
> > do not add *any* execution tests to the LLVM tree
> 
> Do we have anything formal thing about this? I'd like to have a conversation about it. I agree it isn't something we really do currently, but I believe there is real value in adding these tests in tree.

You could frame this as the typical discussion of an end-to-end test versus a unit test. Executing on the target pulls in all sort of dependencies (you probably need a target libc, you need a special executor, what if someone wants to use a simulator rather than a target etc.) While I think nobody denies that end-to-end tests are good to have, they are not suitable for the llvm repository which should have portable and focused tests that are closer to unit tests.

The end-to-end and execution testing is usually done with the llvm test-suite which features a few hello-world like applications as well scattered accross SingleSource/UnitTest and SingleSource/Regression.

- Matthias

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20161214/3a1732cf/attachment.html>