<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Jul 13, 2017, at 9:30 AM, Mehdi AMINI via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Copy-pasting a run line to debug a failure is trivial... when you know which line to copy/paste!<div class="">To be the frustration has rather been that lit does not say which of the command fails when there are multiple run lines in a test.</div></div></div></blockquote><div><br class=""></div><div>Somewhat orthogonally to the current discussion I’ve submitted <a href="https://reviews.llvm.org/D35330" class="">https://reviews.llvm.org/D35330</a> which aims to solve this issue.</div><div>[though the patch can of course can be improved]</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><div class=""><br class=""></div><div class="">-- </div><div class="">Mehdi</div><div class=""><br class=""></div><div class=""><br class=""></div></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">2017-07-10 9:04 GMT-07:00 Daniel Dunbar via llvm-dev <span dir="ltr" class=""><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>></span>:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On Jul 8, 2017, at 11:18 AM, David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank" class="">dblaikie@gmail.com</a>> wrote:</div><br class="m_-8552280462058686131Apple-interchange-newline"><div class=""><div dir="ltr" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><br class=""><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Sat, Jul 8, 2017 at 10:07 AM Daniel Dunbar <<a href="mailto:daniel_dunbar@apple.com" target="_blank" class="">daniel_dunbar@apple.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="auto" class=""><div class=""><br class=""></div><div class=""><br class="">On Jul 8, 2017, at 7:32 AM, David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank" class="">dblaikie@gmail.com</a>> wrote:<br class=""><br class=""></div><blockquote type="cite" class=""><div class=""><div dir="ltr" class="">Ideally/the better integration with Buildbot would be to have these outputs referenced as "associated files" (it's been a while since I played with buildbot - I remember finding this and considering how it could be done, but not getting all the way through) so they'd come back as actual files on the build master, linked from the results page that you could click on to view/download.<br class=""><br class="">(similarly in the local output, having these files written to disk and the name of the file mentioned in the output would seem nice to me - some tests use the %t, etc, to create temporary files and you can see their names in the output, but some stream directly - it'd be great if the direct streaming still allowed the user to inspect the files along pipe chain)<br class=""><br class="">(aside: I'd love it if lit would tell me /which/ of the RUN commands it was running when it failed, or which one the output came from (in the case of a single RUN line having multiple commands... ) somehow - would simplify things a bit too)</div></div></blockquote><div class=""><br class=""></div></div><div dir="auto" class="">It actually will already do this if LLVM switched to the "internal test runner" (as opposed to the mode which runs the entire thing as one bash script).</div></blockquote><div class=""><br class="">Is that the default on windows/platforms that might not have bash?<br class=""></div></div></div></div></blockquote><div class=""><br class=""></div></span>Yup.</div><div class=""><br class=""></div><div class=""><span class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><div class="gmail_quote"><div class="">Is there a CMake flag/config for it?<br class=""></div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">It looks like there is an env var for it (from LLVM's lit.cfg):</div><div class="">```</div><div class=""><div class=""># Choose between lit's internal shell pipeline runner and a real shell. If</div><div class=""># LIT_USE_INTERNAL_SHELL is in the environment, we use that as an override.</div><div class="">use_lit_shell = os.environ.get("LIT_USE_<wbr class="">INTERNAL_SHELL")</div><div class="">if use_lit_shell:</div><div class=""> # 0 is external, "" is default, and everything else is internal.</div><div class=""> execute_external = (use_lit_shell == "0")</div><div class="">else:</div><div class=""> # Otherwise we default to internal on Windows and external elsewhere, as</div><div class=""> # bash on Windows is usually very slow.</div><div class=""> execute_external = (not sys.platform in ['win32'])</div><div class="">```</div><div class=""><br class=""></div></div></div><div class=""> - Daniel</div><span class=""><div class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><div class="gmail_quote"><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="auto" class=""><div class=""><br class=""></div><div class=""> - Daniel</div></div><div dir="auto" class=""><div class=""><br class=""><blockquote type="cite" class=""><div class=""><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Fri, Jul 7, 2017 at 4:24 PM George Karpenkov via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class="">What about having an environment variable FILECHECKER_VERBOSE=1?<div class="">This would not require substitutions, and could be even set automatically by “lit” when launched with “-v”.</div><div class="">At least to me that would make debugging tests much easier.</div></div><div style="word-wrap:break-word" class=""><div class=""><div class=""><blockquote type="cite" class=""><div class="">On Jul 7, 2017, at 3:05 PM, Daniel Dunbar <<a href="mailto:daniel_dunbar@apple.com" target="_blank" class="">daniel_dunbar@apple.com</a>> wrote:</div><br class="m_-8552280462058686131m_-4729501545703823629m_6483847269717187904Apple-interchange-newline"><div class=""><div style="word-wrap:break-word" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Jul 7, 2017, at 2:19 PM, Reid Kleckner <<a href="mailto:rnk@google.com" target="_blank" class="">rnk@google.com</a>> wrote:</div><br class="m_-8552280462058686131m_-4729501545703823629m_6483847269717187904Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote">On Fri, Jul 7, 2017 at 1:20 PM, George Karpenkov via llvm-dev<span class="m_-8552280462058686131Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.<wbr class="">org</a>></span><span class="m_-8552280462058686131Apple-converted-space"> </span>wrote:<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Thus, I propose modifying FileCheck default behavior to dump all swallowed output on stderr when the test has failed.<br class="">Would there be any objections to such a change?<br class=""></blockquote><div class=""><br class=""></div><div class="">Yes.</div><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">I understand the concern that log files might become unnecessarily large, but since it would only be done for failed<br class="">test I think the added readability would be worth it.<br class=""></blockquote><div class=""><br class=""></div><div class="">I disagree, it would be too much output. During development, it's pretty common to cause tens of tests to fail. I don't really want 10 entire assembly files dumped into my console during incremental development. Our test output is already long, and I wish it were shorter.</div></div></div></div></div></blockquote><div class=""><br class=""></div>Could this be solved by having lit be intelligent about showing less output when there are large numbers of test failures (w/o other output), and truncating very large outputs?</div><div class=""><br class=""></div><div class="">I do think there are situations where having the output just show up by default locally could prevent needing to rerun a command, which is handy.</div><div class=""><br class=""></div><div class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">I agree that this is a real problem when remote buildbots in different configurations get involved. Locally debugging FileCheck failures is easy, you just copy-paste the command like you said and pipe it to less. It's only a pain when you aren't sure if a failure on a bot will reproduce locally. So, I would be in favor of an option to lit that we enable on buildslaves that dumps the output. We already have a '\bFileCheck\b' substitution in lit. We'd just expand it to 'FileCheck --dump-on-failure' or something on bots.</div></div></div></div></div></blockquote></div><br class=""><div class="">This sounds reasonable to me, no matter what on the above question.</div><div class=""><br class=""></div><div class=""> - Daniel</div><div class=""><br class=""></div></div></div></blockquote></div><br class=""></div></div>______________________________<wbr class="">_________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/<wbr class="">mailman/listinfo/llvm-dev</a></blockquote></div></div></blockquote></div></div></blockquote></div></div></div></blockquote></div><br class=""></span></div><br class="">______________________________<wbr class="">_________________<br class="">
LLVM Developers mailing list<br class="">
<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class="">
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/<wbr class="">mailman/listinfo/llvm-dev</a><br class="">
<br class=""></blockquote></div><br class=""></div>
_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<br class=""></div></blockquote></div><br class=""></body></html>