<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">Am 19.06.20 um 10:32 schrieb Mehdi
AMINI via llvm-dev:<br>
</div>
<blockquote type="cite"
cite="mid:CANF-O=be-kbmaT7gDQu6X+2ps+_K+zOptb4CueEc4Nv7Uisipw@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div dir="ltr"><br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Jun 19, 2020 at
12:56 AM Sjoerd Meijer via llvm-dev <<a
href="mailto:llvm-dev@lists.llvm.org"
moz-do-not-send="true">llvm-dev@lists.llvm.org</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div
style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">Sorry
if I wasn't clear about my use case. In my daily dev
work, I do many local "ninja check"s, or "llvm-lit" on a
subdirectory as a quick(er) smoke test if I am making
changes in that area (e.g. "llvm-lit
../llvm/test/CodeGen"). Nothing wrong here, as indeed
nothing changed here. But in case of a test failure, I
want to run just that test:
<br>
</div>
<div
style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><br>
</div>
<div
style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
bin/llvm-lit ../llvm/test/CodeGen/my_test.ll</div>
<div
style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><br>
</div>
<div
style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">This
only reports success/failure, and doesn't show any cause
for failure , so I run it in verbose mode with:<br>
</div>
<div
style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<div
style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div
style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
bin/llvm-lit -a ../llvm/test/CodeGen/my_test.ll</div>
</div>
<div
style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><br>
</div>
<div
style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">In
a terminal, the new default behaviour of FileCheck has
become pretty unusable IMHO.<br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I don't know how you proceed to debug FileCheck failures,
but for me most of the time I'll have to figure out which
"RUN" line fail and try to execute it manually and then
remove the FileCheck pipe to get the raw input and then
painfully tried to match the FileCheck error to the actual
input. At least that's what I used to do before
check-input=fail: this changed my relationship with
debugging Filecheck failures!</div>
<div><br>
</div>
<div>Matt mentioned that some codegen tests are very long:
reducing the output by being contextual would likely help,
but I'm also not convinced that this is a good practice to
have so huge test in single files in the first place, it
seems to me almost like anti-pattern and if it gets hard to
read it is a good sign that it could benefit from some
sharding.</div>
</div>
</div>
</blockquote>
<p>Sometimes it's not really feasible to break up a test into
multiple smaller tests. For example, for GlobalISel tests are
usually grouped per stage and per instruction (i.e.
legalize-merge-values.mir). Especially those legalizer tests are
often very extensive due to the large number of type combinations.
At least I wouldn't want to break up those tests and clutter my
test folder with tens of files for a single instruction and a
single stage.</p>
<p> I'm also not too happy with the current default behavior. Having
the error message printed last would already go a long way, but
obviously reducing the output in some contextual way would be
best.<br>
</p>
<p>Dominik<br>
</p>
<blockquote type="cite"
cite="mid:CANF-O=be-kbmaT7gDQu6X+2ps+_K+zOptb4CueEc4Nv7Uisipw@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
</div>
<div><br>
</div>
<div>-- </div>
<div>Mehdi</div>
<div><br>
</div>
<div><br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div
style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"></div>
<div
style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><br>
</div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_693392159962186127divRplyFwdMsg"
dir="ltr"><font style="font-size:11pt" face="Calibri,
sans-serif" color="#000000"><b>From:</b> Chris
Tetreault <<a href="mailto:ctetreau@quicinc.com"
target="_blank" moz-do-not-send="true">ctetreau@quicinc.com</a>><br>
<b>Sent:</b> 18 June 2020 20:49<br>
<b>To:</b> Joel E. Denny <<a
href="mailto:jdenny.ornl@gmail.com" target="_blank"
moz-do-not-send="true">jdenny.ornl@gmail.com</a>><br>
<b>Cc:</b> Sjoerd Meijer <<a
href="mailto:Sjoerd.Meijer@arm.com" target="_blank"
moz-do-not-send="true">Sjoerd.Meijer@arm.com</a>>;
<a href="mailto:llvm-dev@lists.llvm.org"
target="_blank" moz-do-not-send="true">llvm-dev@lists.llvm.org</a>
<<a href="mailto:llvm-dev@lists.llvm.org"
target="_blank" moz-do-not-send="true">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> RE: [llvm-dev] FileCheck</font>
<div> </div>
</div>
<div lang="EN-US">
<div>
<p>Sjoerd specifically said “in verbose mode”, which I
interpret to mean “when passing -v or -vv”. If we’re
discussing the default behavior, then that’s a
separate issue. Regardless, my other points stand
independent of that.</p>
<p> </p>
<p><b>From:</b> Joel E. Denny <<a
href="mailto:jdenny.ornl@gmail.com"
target="_blank" moz-do-not-send="true">jdenny.ornl@gmail.com</a>>
<br>
<b>Sent:</b> Thursday, June 18, 2020 12:43 PM<br>
<b>To:</b> Chris Tetreault <<a
href="mailto:ctetreau@quicinc.com" target="_blank"
moz-do-not-send="true">ctetreau@quicinc.com</a>><br>
<b>Cc:</b> Sjoerd Meijer <<a
href="mailto:Sjoerd.Meijer@arm.com"
target="_blank" moz-do-not-send="true">Sjoerd.Meijer@arm.com</a>>;
<a href="mailto:llvm-dev@lists.llvm.org"
target="_blank" moz-do-not-send="true">llvm-dev@lists.llvm.org</a><br>
<b>Subject:</b> [EXT] Re: [llvm-dev] FileCheck</p>
<p> </p>
<div>
<div>
<div>
<p>On Thu, Jun 18, 2020 at 3:37 PM Chris
Tetreault <<a
href="mailto:ctetreau@quicinc.com"
target="_blank" moz-do-not-send="true">ctetreau@quicinc.com</a>>
wrote:</p>
</div>
<blockquote style="border-style:none none none
solid;border-left-width:1pt;border-left-color:rgb(204,204,204);padding:0in
0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p>We’re talking about verbose output right?
Verbose isn’t the default.</p>
</div>
</div>
</blockquote>
<div>
<p> </p>
</div>
<div>
<p>I'm fairly certain the issue in this thread
is just the verbosity of -dump-input=fail.
Yes, -vv makes it even more verbose by
annotating input lines with good matches,
etc., but that's not part of the "new
behaviour" Sjoerd meant, I believe.</p>
</div>
<div>
<p> </p>
</div>
<div>
<p>Joel</p>
</div>
<div>
<p> </p>
</div>
<blockquote style="border-style:none none none
solid;border-left-width:1pt;border-left-color:rgb(204,204,204);padding:0in
0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p> </p>
<p><b>From:</b> Joel E. Denny <<a
href="mailto:jdenny.ornl@gmail.com"
target="_blank" moz-do-not-send="true">jdenny.ornl@gmail.com</a>>
<br>
<b>Sent:</b> Thursday, June 18, 2020 10:54
AM<br>
<b>To:</b> Chris Tetreault <<a
href="mailto:ctetreau@quicinc.com"
target="_blank" moz-do-not-send="true">ctetreau@quicinc.com</a>><br>
<b>Cc:</b> Sjoerd Meijer <<a
href="mailto:Sjoerd.Meijer@arm.com"
target="_blank" moz-do-not-send="true">Sjoerd.Meijer@arm.com</a>>;
<a href="mailto:llvm-dev@lists.llvm.org"
target="_blank" moz-do-not-send="true">llvm-dev@lists.llvm.org</a><br>
<b>Subject:</b> [EXT] Re: [llvm-dev]
FileCheck</p>
<p> </p>
<div>
<div>
<p>Hi Chris,</p>
</div>
<p> </p>
<div>
<div>
<p>On Thu, Jun 18, 2020 at 1:37 PM
Chris Tetreault via llvm-dev <<a
href="mailto:llvm-dev@lists.llvm.org" target="_blank"
moz-do-not-send="true">llvm-dev@lists.llvm.org</a>>
wrote:</p>
</div>
<blockquote style="border-style:none
none none
solid;border-left-width:1pt;padding:0in
0in 0in 6pt;margin:5pt 0in 5pt
4.8pt;border-color:currentcolor
currentcolor currentcolor
rgb(204,204,204)">
<div>
<div>
<p>The thing I use normally only
shows the first N lines by
default (I don’t know off hand
what N is). Honestly, I don’t
feel very strongly about the
specific order, but it’s not
useful when somebody proposes
something on the list, and
nobody voices any dissent
(choosing instead to silently
oppose the change). My requests
would be:</p>
<p> </p>
<ol type="1" start="1">
<li>
The order should be
customizable via command line.</li>
<li>
By default, it should not dump
things to multiple locations.
If I ask for verbose output, I
want to get blasted with all
the stuff.</li>
<li>
The most important thing for
me personally is to see the
input to filecheck (I realize
that this is in conflict with
my earlier point. It’s early
and I hadn’t had my coffee
<span
style="font-family:"Segoe
UI Emoji",sans-serif">😊</span>
). When I get a failure I want
to be able to reproduce it in
an IDE to use a debugger. Any
change should not make this
use case harder.</li>
</ol>
<p> </p>
<p>Personally, I do not find the
argument that the defaults
should be setup to be best for
newcomers to be very compelling;
we are talking about changing
the behavior of a non-default
option after all.</p>
</div>
</div>
</blockquote>
<div>
<p> </p>
</div>
<div>
<p>What do you mean by a "non-default
option"? The default of
-dump-input=never was recently
changed to -dump-input=fail.</p>
</div>
<div>
<p> </p>
</div>
<div>
<p>Joel</p>
</div>
<div>
<p> </p>
</div>
<blockquote style="border-style:none
none none
solid;border-left-width:1pt;padding:0in
0in 0in 6pt;margin:5pt 0in 5pt
4.8pt;border-color:currentcolor
currentcolor currentcolor
rgb(204,204,204)">
<div>
<div>
<p>If just a bare filecheck
invocation doesn’t tell a
newcomer what they need to know,
then they have to do filecheck
-help or google the
documentation anyways. At that
point, they are going to
customize it however they want.
I assume anybody using filecheck
to debug an issue is tech savvy
enough to be able to configure
the options, given reasonable
documentation.</p>
<p> </p>
<p>Thanks,</p>
<p> Christopher Tetreault</p>
<p> </p>
<div>
<div style="border-style:solid
none
none;border-top-width:1pt;padding:3pt
0in
0in;border-color:currentcolor">
<p><b>From:</b> Sjoerd Meijer
<<a
href="mailto:Sjoerd.Meijer@arm.com"
target="_blank"
moz-do-not-send="true">Sjoerd.Meijer@arm.com</a>>
<br>
<b>Sent:</b> Thursday, June
18, 2020 9:45 AM<br>
<b>To:</b> Chris Tetreault
<<a
href="mailto:ctetreau@quicinc.com"
target="_blank"
moz-do-not-send="true">ctetreau@quicinc.com</a>><br>
<b>Cc:</b> <a
href="mailto:llvm-dev@lists.llvm.org"
target="_blank"
moz-do-not-send="true">llvm-dev@lists.llvm.org</a><br>
<b>Subject:</b> [EXT] Re:
[llvm-dev] FileCheck</p>
</div>
</div>
<p> </p>
<div>
<p><span
style="font-size:12pt;color:black"> </span></p>
</div>
<div>
<p><span
style="font-size:12pt;color:black">I
would guess that in a CI
system the order doesn't
matter much because you look
at a webpage? I looked at
some build bots
today/yesterday that now
also show this, and yeah,
it's fine either way, I was
guessing.</span></p>
</div>
<div>
<p><span
style="font-size:12pt;color:black"> </span></p>
</div>
<div>
<p><span
style="font-size:12pt;color:black">My
primary use-case is usage in
a terminal, and displaying
the errors first followed by
all input makes this pretty
unusable.</span></p>
</div>
<div style="text-align:center"
align="center">
<hr width="98%" size="2"
align="center">
</div>
<div
id="gmail-m_693392159962186127x_gmail-m_4777886285961706663gmail-m_3391276456716634261divRplyFwdMsg">
<p><b><span style="color:black">From:</span></b><span
style="color:black"> Chris
Tetreault <</span><a
href="mailto:ctetreau@quicinc.com"
target="_blank"
moz-do-not-send="true">ctetreau@quicinc.com</a><span
style="color:black">><br>
<b>Sent:</b> 18 June 2020
17:34<br>
<b>To:</b> Sjoerd Meijer
<</span><a
href="mailto:Sjoerd.Meijer@arm.com"
target="_blank"
moz-do-not-send="true">Sjoerd.Meijer@arm.com</a><span
style="color:black">><br>
<b>Cc:</b> </span><a
href="mailto:llvm-dev@lists.llvm.org"
target="_blank"
moz-do-not-send="true">llvm-dev@lists.llvm.org</a><span
style="color:black"> <</span><a
href="mailto:llvm-dev@lists.llvm.org" target="_blank"
moz-do-not-send="true">llvm-dev@lists.llvm.org</a><span
style="color:black">><br>
<b>Subject:</b> RE:
[llvm-dev] FileCheck</span>
</p>
<div>
<p> </p>
</div>
</div>
<div>
<div>
<p>For anybody viewing these
failures through some sort
of CI system, showing the
error first then the input
file is more useful for the
same reasons you mentioned.
Personally, I rarely run
filecheck by hand from the
command prompt, so your
change would make my life
worse. Granted, I’m just one
person.</p>
<p> </p>
<p>The point I’m trying to
make is that I don’t think
it’s clear-cut which order
is better, so maybe we
shouldn’t change it. I think
it might be fine to add an
option to swap the order,
but I’d be very sad if it
started dumping to some
random file by default.</p>
<p> </p>
<p>Thanks,</p>
<p> Christopher Tetreault</p>
<p> </p>
<div>
<div
style="border-style:solid
none
none;border-top-width:1pt;padding:3pt
0in
0in;border-color:currentcolor">
<p><b>From:</b> llvm-dev
<<a
href="mailto:llvm-dev-bounces@lists.llvm.org"
target="_blank"
moz-do-not-send="true">llvm-dev-bounces@lists.llvm.org</a>>
<b>On Behalf Of </b>Sjoerd
Meijer via llvm-dev<br>
<b>Sent:</b> Thursday,
June 18, 2020 9:16 AM<br>
<b>To:</b> <a
href="mailto:llvm-dev@lists.llvm.org"
target="_blank"
moz-do-not-send="true">llvm-dev@lists.llvm.org</a><br>
<b>Subject:</b> [EXT]
[llvm-dev] FileCheck</p>
</div>
</div>
<p> </p>
<div>
<p><span
style="font-size:12pt;color:black">Hello,</span></p>
</div>
<div>
<p><span
style="font-size:12pt;color:black"> </span></p>
</div>
<div>
<p><span
style="font-size:12pt;color:black">I
am not sold on
FileCheck's new
behaviour. For failing
tests in verbose mode,
it first dump the actual
error messages, followed
by the annotated input
file to FileCheck. The
result is I can't
immediately see error
messages if the input is
more than just a few
lines long, so I have to
scroll all the way up to
see the errors, then
down again, etc.
</span></p>
</div>
<div>
<p><span
style="font-size:12pt;color:black"> </span></p>
</div>
<div>
<p><span
style="font-size:12pt;color:black">I
do see some advantages
of dumping the input to
FileCheck, but an
improvement for me would
be:</span></p>
</div>
<div>
<ul type="disc">
<li style="color:black">
<span
style="font-size:12pt">to
dump the input first,
then followed by the
error message, so that
I can the errors
first, and then decide
to scroll up if I am
interested to do so.</span></li>
<li style="color:black">
<span
style="font-size:12pt">dump
it to a separate file
(controlled with an
option).</span></li>
</ul>
</div>
<div>
<p><span
style="font-size:12pt;color:black">I
am interested in
changing the behaviour,
because I think I find
setting environment
varibale
"FILECHECK_OPTS="--dump-input
never"" inconvenient. </span></p>
</div>
<div>
<p><span
style="font-size:12pt;color:black"> </span></p>
</div>
<div>
<p><span
style="font-size:12pt;color:black">My
2 pennies.</span></p>
</div>
<div>
<p><span
style="font-size:12pt;color:black">Sjoerd.</span></p>
</div>
</div>
</div>
</div>
</div>
<p>_______________________________________________<br>
LLVM Developers mailing list<br>
<a
href="mailto:llvm-dev@lists.llvm.org"
target="_blank"
moz-do-not-send="true">llvm-dev@lists.llvm.org</a><br>
<a
href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev"
target="_blank"
moz-do-not-send="true">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></p>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank"
moz-do-not-send="true">llvm-dev@lists.llvm.org</a><br>
<a
href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
LLVM Developers mailing list
<a class="moz-txt-link-abbreviated" href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>
<a class="moz-txt-link-freetext" href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
----------------------------------------------------------------------
Dominik Montada Email: <a class="moz-txt-link-abbreviated" href="mailto:dominik.montada@hightec-rt.com">dominik.montada@hightec-rt.com</a>
HighTec EDV-Systeme GmbH Phone: +49 681 92613 19
Europaallee 19 Fax: +49-681-92613-26
D-66113 Saarbrücken WWW: <a class="moz-txt-link-freetext" href="http://www.hightec-rt.com">http://www.hightec-rt.com</a>
Managing Director: Vera Strothmann
Register Court: Saarbrücken, HRB 10445, VAT ID: DE 138344222
This e-mail may contain confidential and/or privileged information. If
you are not the intended recipient please notify the sender immediately
and destroy this e-mail. Any unauthorised copying, disclosure or
distribution of the material in this e-mail is strictly forbidden.
--- </pre>
</body>
</html>