[llvm-dev] FileCheck

Thu Jun 18 12:43:21 PDT 2020

On Thu, Jun 18, 2020 at 3:37 PM Chris Tetreault <ctetreau at quicinc.com>
wrote:

> We’re talking about verbose output right? Verbose isn’t the default.
>

I'm fairly certain the issue in this thread is just the verbosity of
-dump-input=fail.  Yes, -vv makes it even more verbose by annotating input
lines with good matches, etc., but that's not part of the "new behaviour"
Sjoerd meant, I believe.

Joel

>
>
> *From:* Joel E. Denny <jdenny.ornl at gmail.com>
> *Sent:* Thursday, June 18, 2020 10:54 AM
> *To:* Chris Tetreault <ctetreau at quicinc.com>
> *Cc:* Sjoerd Meijer <Sjoerd.Meijer at arm.com>; llvm-dev at lists.llvm.org
> *Subject:* [EXT] Re: [llvm-dev] FileCheck
>
>
>
> Hi Chris,
>
>
>
> On Thu, Jun 18, 2020 at 1:37 PM Chris Tetreault via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> The thing I use normally only shows the first N lines by default (I don’t
> know off hand what N is). Honestly, I don’t feel very strongly about the
> specific order, but it’s not useful when somebody proposes something on the
> list, and nobody voices any dissent (choosing instead to silently oppose
> the change). My requests would be:
>
>
>
>    1. The order should be customizable via command line.
>    2. By default, it should not dump things to multiple locations. If I
>    ask for verbose output, I want to get blasted with all the stuff.
>    3. The most important thing for me personally is to see the input to
>    filecheck (I realize that this is in conflict with my earlier point. It’s
>    early and I hadn’t had my coffee 😊 ). When I get a failure I want to
>    be able to reproduce it in an IDE to use a debugger. Any change should not
>    make this use case harder.
>
>
>
> Personally, I do not find the argument that the defaults should be setup
> to be best for newcomers to be very compelling; we are talking about
> changing the behavior of a non-default option after all.
>
>
>
> What do you mean by a "non-default option"?  The default of
> -dump-input=never was recently changed to -dump-input=fail.
>
>
>
> Joel
>
>
>
> If just a bare filecheck invocation doesn’t tell a newcomer what they need
> to know, then they have to do filecheck -help or google the documentation
> anyways. At that point, they are going to customize it however they want. I
> assume anybody using filecheck to debug an issue is tech savvy enough to be
> able to configure the options, given reasonable documentation.
>
>
>
> Thanks,
>
>    Christopher Tetreault
>
>
>
> *From:* Sjoerd Meijer <Sjoerd.Meijer at arm.com>
> *Sent:* Thursday, June 18, 2020 9:45 AM
> *To:* Chris Tetreault <ctetreau at quicinc.com>
> *Cc:* llvm-dev at lists.llvm.org
> *Subject:* [EXT] Re: [llvm-dev] FileCheck
>
>
>
>
>
> I would guess that in a CI system the order doesn't matter much because
> you look at a webpage? I looked at some build bots today/yesterday that now
> also show this, and yeah, it's fine either way, I was guessing.
>
>
>
> My primary use-case is usage in a terminal, and displaying the errors
> first followed by all input makes this pretty unusable.
> ------------------------------
>
> *From:* Chris Tetreault <ctetreau at quicinc.com>
> *Sent:* 18 June 2020 17:34
> *To:* Sjoerd Meijer <Sjoerd.Meijer at arm.com>
> *Cc:* llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>
> *Subject:* RE: [llvm-dev] FileCheck
>
>
>
> For anybody viewing these failures through some sort of CI system, showing
> the error first then the input file is more useful for the same reasons you
> mentioned. Personally, I rarely run filecheck by hand from the command
> prompt, so your change would make my life worse. Granted, I’m just one
> person.
>
>
>
> The point I’m trying to make is that I don’t think it’s clear-cut which
> order is better, so maybe we shouldn’t change it. I think it might be fine
> to add an option to swap the order, but I’d be very sad if it started
> dumping to some random file by default.
>
>
>
> Thanks,
>
>    Christopher Tetreault
>
>
>
> *From:* llvm-dev <llvm-dev-bounces at lists.llvm.org> *On Behalf Of *Sjoerd
> Meijer via llvm-dev
> *Sent:* Thursday, June 18, 2020 9:16 AM
> *To:* llvm-dev at lists.llvm.org
> *Subject:* [EXT] [llvm-dev] FileCheck
>
>
>
> Hello,
>
>
>
> I am not sold on FileCheck's new behaviour. For failing tests in verbose
> mode, it first dump the actual error messages, followed by the annotated
> input file to FileCheck. The result is I can't immediately see error
> messages if the input is more than just a few lines long, so I have to
> scroll all the way up to see the errors, then down again, etc.
>
>
>
> I do see some advantages of dumping the input to FileCheck, but an
> improvement for me would be:
>
>    - to dump the input first, then followed by the error message, so that
>    I can the errors first, and then decide to scroll up if I am interested to
>    do so.
>    - dump it to a separate file (controlled with an option).
>
> I am interested in changing the behaviour, because I think I find setting
> environment varibale "FILECHECK_OPTS="--dump-input never"" inconvenient.
>
>
>
> My 2 pennies.
>
> Sjoerd.
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200618/f710a110/attachment.html>