<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Dec 14, 2016, at 12:48 PM, Dylan McKay via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" class="">llvm-commits@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">> <span style="font-size:12.8px" class="">This test will fail as soon as anyone builds the AVR back-end without</span><br style="font-size:12.8px" class=""><span style="font-size:12.8px" class="">access to AVR hardware.</span><div class=""><span style="font-size:12.8px" class=""><br class=""></span></div><div class=""><span style="font-size:12.8px" class="">I added a config file that ignored the tests if AVR was not built? On top of that, the tests were only supposed to run if 'AVRLIT_BOARD' and 'AVRLIT_PORT' is set. Clearly it didn't work properly (due to my inexperience with Python)</span></div><div class=""><span style="font-size:12.8px" class=""><br class=""></span></div><div class=""><span style="font-size:12.8px" class="">If I were to fix that, it wouldn't stop the tests from breaking if AVR was made non-experimental?</span></div><div class=""><span style="font-size:12.8px" class=""><br class=""></span></div><div class=""><span style="font-size:12.8px" class="">> </span><span style="font-size:12.8px" class="">do not add *any* execution tests to the LLVM tree</span></div><div class=""><span style="font-size:12.8px" class=""><br class=""></span></div><div class=""><span style="font-size:12.8px" class="">Do we have anything formal thing about this? I'd like to have a conversation about it. I agree it isn't something we really do currently, but I believe there is real value in adding these tests in tree.</span></div></div></div></blockquote><div><br class=""></div><div>You could frame this as the typical discussion of an end-to-end test versus a unit test. Executing on the target pulls in all sort of dependencies (you probably need a target libc, you need a special executor, what if someone wants to use a simulator rather than a target etc.) While I think nobody denies that end-to-end tests are good to have, they are not suitable for the llvm repository which should have portable and focused tests that are closer to unit tests.</div><div><br class=""></div><div>The end-to-end and execution testing is usually done with the llvm test-suite which features a few hello-world like applications as well scattered accross SingleSource/UnitTest and SingleSource/Regression.</div><div><br class=""></div><div>- Matthias</div><div><br class=""></div></div></body></html>