<div dir="ltr">Hi Dylan,<div><br></div><div>> <span style="color:rgb(33,33,33);font-size:12.8px">Do we have anything formal thing about this?</span></div><div><span style="color:rgb(33,33,33);font-size:12.8px"><br></span></div><div><span style="color:rgb(33,33,33);font-size:12.8px">Yes. It's well understood that the only thing LIT tests are allowed to rely on are tools built during the compilation process. That means no emulators, so no execution tests. The ethos of the LIT tests are checking compilation, textually, fast.</span></div><div><span style="color:rgb(33,33,33);font-size:12.8px"><br></span></div><div><span style="color:rgb(33,33,33);font-size:12.8px">For on-target tests, we have the test-suite (see the UnitTests subdirectory).</span></div><div><span style="color:rgb(33,33,33);font-size:12.8px"><br></span></div><div><span style="color:rgb(33,33,33);font-size:12.8px">Cheers,</span></div><div><span style="color:rgb(33,33,33);font-size:12.8px"><br></span></div><div><span style="color:rgb(33,33,33);font-size:12.8px">James</span></div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, 14 Dec 2016 at 20:48 Dylan McKay via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="gmail_msg">> <span style="font-size:12.8px" class="gmail_msg">This test will fail as soon as anyone builds the AVR back-end without</span><br style="font-size:12.8px" class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg">access to AVR hardware.</span><div class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg"><br class="gmail_msg"></span></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg">I added a config file that ignored the tests if AVR was not built? On top of that, the tests were only supposed to run if 'AVRLIT_BOARD' and 'AVRLIT_PORT' is set. Clearly it didn't work properly (due to my inexperience with Python)</span></div><div class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg"><br class="gmail_msg"></span></div><div class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg">If I were to fix that, it wouldn't stop the tests from breaking if AVR was made non-experimental?</span></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg"><br class="gmail_msg"></span></div><div class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg">> </span><span style="font-size:12.8px" class="gmail_msg">do not add *any* execution tests to the LLVM tree</span></div><div class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg"><br class="gmail_msg"></span></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg">Do we have anything formal thing about this? I'd like to have a conversation about it. I agree it isn't something we really do currently, but I believe there is real value in adding these tests in tree.</span></div><div class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg"><br class="gmail_msg"></span></div><div class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg">> </span><span style="font-size:12.8px" class="gmail_msg">Reverted in r289651</span></div><div class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg"><br class="gmail_msg"></span></div><div class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg">Thanks for that!</span></div><div class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg"><br class="gmail_msg"></span></div><div class="gmail_msg"><span style="font-size:12.8px" class="gmail_msg"><br class="gmail_msg"></span></div></div><div class="gmail_extra gmail_msg"><br class="gmail_msg"><div class="gmail_quote gmail_msg">On Thu, Dec 15, 2016 at 2:35 AM, Renato Golin <span dir="ltr" class="gmail_msg"><<a href="mailto:renato.golin@linaro.org" class="gmail_msg" target="_blank">renato.golin@linaro.org</a>></span> wrote:<br class="gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="gmail_msg">On 14 December 2016 at 13:32, Renato Golin <<a href="mailto:renato.golin@linaro.org" class="gmail_msg" target="_blank">renato.golin@linaro.org</a>> wrote:<br class="gmail_msg">
> Hi Dylan,<br class="gmail_msg">
><br class="gmail_msg">
> This test will fail as soon as anyone builds the AVR back-end without<br class="gmail_msg">
> access to AVR hardware.<br class="gmail_msg">
><br class="gmail_msg">
> It's also guaranteed to fail whenever the AVR back-end moves out of<br class="gmail_msg">
> experimental (when everyone will build it by default).<br class="gmail_msg">
><br class="gmail_msg">
> Please, revert this patch and do not add *any* execution tests to the LLVM tree.<br class="gmail_msg">
<br class="gmail_msg">
</span>Oh well, it seems I was being optimistic. It breaks now because no one<br class="gmail_msg">
else has the AVRLIT_PORT.<br class="gmail_msg">
<br class="gmail_msg">
<a href="http://lab.llvm.org:8011/builders/clang-cmake-aarch64-42vma/builds/2086/steps/ninja%20check%201/logs/stdio" rel="noreferrer" class="gmail_msg" target="_blank">http://lab.llvm.org:8011/builders/clang-cmake-aarch64-42vma/builds/2086/steps/ninja%20check%201/logs/stdio</a><br class="gmail_msg">
<br class="gmail_msg">
Reverted in r289651.<br class="gmail_msg">
<br class="gmail_msg">
cheers,<br class="gmail_msg">
--renato<br class="gmail_msg">
</blockquote></div><br class="gmail_msg"></div>
_______________________________________________<br class="gmail_msg">
llvm-commits mailing list<br class="gmail_msg">
<a href="mailto:llvm-commits@lists.llvm.org" class="gmail_msg" target="_blank">llvm-commits@lists.llvm.org</a><br class="gmail_msg">
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" class="gmail_msg" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br class="gmail_msg">
</blockquote></div>