[llvm-dev] FileCheck

[Joel E. Denny via llvm-dev]

Thanks for bringing this discussion to the list so we can see a broader set
of opinions on what the default should be.

I like Johannes's suggestion that defaults should consider the new LLVM
user.  For that case, I think verbosity should be low, and diagnostics
should mention how to increase verbosity.

For CI, I agree verbosity should be high because it's not easy for many
LLVM contributors to try again with higher verbosity.

For my personal usage, I don't care what the default behavior is as long as
behavior is easy to customize.  FILECHECK_OPTS lets me do that per test
suite run, or in my ~/.profile, or in my CI configs, etc.

Joel

On Thu, Jun 18, 2020 at 12:45 PM Sjoerd Meijer via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

>
> I would guess that in a CI system the order doesn't matter much because
> you look at a webpage? I looked at some build bots today/yesterday that now
> also show this, and yeah, it's fine either way, I was guessing.
>
> My primary use-case is usage in a terminal, and displaying the errors
> first followed by all input makes this pretty unusable.
> ------------------------------
> *From:* Chris Tetreault <ctetreau at quicinc.com>
> *Sent:* 18 June 2020 17:34
> *To:* Sjoerd Meijer <Sjoerd.Meijer at arm.com>
> *Cc:* llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>
> *Subject:* RE: [llvm-dev] FileCheck
>
>
> For anybody viewing these failures through some sort of CI system, showing
> the error first then the input file is more useful for the same reasons you
> mentioned. Personally, I rarely run filecheck by hand from the command
> prompt, so your change would make my life worse. Granted, I’m just one
> person.
>
>
>
> The point I’m trying to make is that I don’t think it’s clear-cut which
> order is better, so maybe we shouldn’t change it. I think it might be fine
> to add an option to swap the order, but I’d be very sad if it started
> dumping to some random file by default.
>
>
>
> Thanks,
>
>    Christopher Tetreault
>
>
>
> *From:* llvm-dev <llvm-dev-bounces at lists.llvm.org> * On Behalf Of *Sjoerd
> Meijer via llvm-dev
> *Sent:* Thursday, June 18, 2020 9:16 AM
> *To:* llvm-dev at lists.llvm.org
> *Subject:* [EXT] [llvm-dev] FileCheck
>
>
>
> Hello,
>
>
>
> I am not sold on FileCheck's new behaviour. For failing tests in verbose
> mode, it first dump the actual error messages, followed by the annotated
> input file to FileCheck. The result is I can't immediately see error
> messages if the input is more than just a few lines long, so I have to
> scroll all the way up to see the errors, then down again, etc.
>
>
>
> I do see some advantages of dumping the input to FileCheck, but an
> improvement for me would be:
>
>    - to dump the input first, then followed by the error message, so that
>    I can the errors first, and then decide to scroll up if I am interested to
>    do so.
>    - dump it to a separate file (controlled with an option).
>
> I am interested in changing the behaviour, because I think I find setting
> environment varibale "FILECHECK_OPTS="--dump-input never"" inconvenient.
>
>
>
> My 2 pennies.
>
> Sjoerd.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200618/c401ffec/attachment.html>