[llvm-dev] FileCheck

[Joel E. Denny via llvm-dev]

As for the ordering issue... from the command line, I tend to set verbosity
high, and so I pipe to a pager and search for "error:".  I prefer errors
before the input dump then.  My point is that it should be configurable if
we change the default ordering.

Joel

On Thu, Jun 18, 2020 at 1:13 PM Joel E. Denny <jdenny.ornl at gmail.com> wrote:

> Thanks for bringing this discussion to the list so we can see a broader
> set of opinions on what the default should be.
>
> I like Johannes's suggestion that defaults should consider the new LLVM
> user.  For that case, I think verbosity should be low, and diagnostics
> should mention how to increase verbosity.
>
> For CI, I agree verbosity should be high because it's not easy for many
> LLVM contributors to try again with higher verbosity.
>
> For my personal usage, I don't care what the default behavior is as long
> as behavior is easy to customize.  FILECHECK_OPTS lets me do that per test
> suite run, or in my ~/.profile, or in my CI configs, etc.
>
> Joel
>
> On Thu, Jun 18, 2020 at 12:45 PM Sjoerd Meijer via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>>
>> I would guess that in a CI system the order doesn't matter much because
>> you look at a webpage? I looked at some build bots today/yesterday that now
>> also show this, and yeah, it's fine either way, I was guessing.
>>
>> My primary use-case is usage in a terminal, and displaying the errors
>> first followed by all input makes this pretty unusable.
>> ------------------------------
>> *From:* Chris Tetreault <ctetreau at quicinc.com>
>> *Sent:* 18 June 2020 17:34
>> *To:* Sjoerd Meijer <Sjoerd.Meijer at arm.com>
>> *Cc:* llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>
>> *Subject:* RE: [llvm-dev] FileCheck
>>
>>
>> For anybody viewing these failures through some sort of CI system,
>> showing the error first then the input file is more useful for the same
>> reasons you mentioned. Personally, I rarely run filecheck by hand from the
>> command prompt, so your change would make my life worse. Granted, I’m just
>> one person.
>>
>>
>>
>> The point I’m trying to make is that I don’t think it’s clear-cut which
>> order is better, so maybe we shouldn’t change it. I think it might be fine
>> to add an option to swap the order, but I’d be very sad if it started
>> dumping to some random file by default.
>>
>>
>>
>> Thanks,
>>
>>    Christopher Tetreault
>>
>>
>>
>> *From:* llvm-dev <llvm-dev-bounces at lists.llvm.org> * On Behalf Of *Sjoerd
>> Meijer via llvm-dev
>> *Sent:* Thursday, June 18, 2020 9:16 AM
>> *To:* llvm-dev at lists.llvm.org
>> *Subject:* [EXT] [llvm-dev] FileCheck
>>
>>
>>
>> Hello,
>>
>>
>>
>> I am not sold on FileCheck's new behaviour. For failing tests in verbose
>> mode, it first dump the actual error messages, followed by the annotated
>> input file to FileCheck. The result is I can't immediately see error
>> messages if the input is more than just a few lines long, so I have to
>> scroll all the way up to see the errors, then down again, etc.
>>
>>
>>
>> I do see some advantages of dumping the input to FileCheck, but an
>> improvement for me would be:
>>
>>    - to dump the input first, then followed by the error message, so
>>    that I can the errors first, and then decide to scroll up if I am
>>    interested to do so.
>>    - dump it to a separate file (controlled with an option).
>>
>> I am interested in changing the behaviour, because I think I find setting
>> environment varibale "FILECHECK_OPTS="--dump-input never"" inconvenient.
>>
>>
>>
>> My 2 pennies.
>>
>> Sjoerd.
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200618/bf7f3742/attachment.html>