[LLVMdev] lit: deprecating trailing \ in RUN lines

[Chandler Carruth]

On Sun, Dec 8, 2013 at 2:11 AM, Alp Toker <alp at nuanti.com> wrote:

>  I'd like to propose deprecating and shortly thereafter removing the lit
> test runner feature that concatenates RUN lines ending in a trailing \
>

I'm really opposed to this. Especially for the Clang test suite where run
lines are often *very* long and hard to organize, read, and edit without
this feature.


>
> *Rationale:*
>
>
>    - Trailing \ has a special meaning in various language standards that
>    we support nowadays. In the C preprocessor, for example, it's handled
>    _before_ comments. Various compilers handle this differently and it
>    introduces needless incompatibilities.
>
> What incompatibilities? I've never had this be an issue.


>
>    - Forgetting the trailing \ between two RUN lines causes the lines to
>    be run individually. People have checked in tests which they believed were
>    getting run whereas the features being tested were actually silently
>    broken. I've been committing fixes for some of these but it's exceptionally
>    time-consuming to hunt them down after the fact.
>
> I'd like to understand the rate at which this happens (per RUN-line? per
test-file?). It's never been a problem for me, but that is in part because
I check that my tests fail without my change in addition to passing with my
change.


>
>    - Removing trailing \ will introduce the neat property that one RUN
>    line corresponds precisely to one command that's executed. This is good for
>    humans and will enable simplifications in the test runner.
>
> FWIW, I've never really had a problem that needed this. The RUN: forms a
prefix of a shell script in my head, and I know how to read shell scripts
including multiple lines.


>
>    - Eliminating the trailing \ syntax will unblock work on improved
>    failure source locations in lit. Right now, when the builders encounter a
>    test RUN failure it's a matter of guesswork as to which RUN line is
>    failing, and the cycle of commit-fix-and-watch-buildbots is laborious.
>    We've all wasted time with this at some point and can totally do better.
>
> While I would very much enjoy better failure reporting, I don't really
understand why it needs this. We have a in-python parser for the RUN: lines
which understands what lines a "command" spans?

Anyways, even though I would *really* like better failure reporting from
lit, not at the cost of less readable tests. That's the tail wagging the
dog IMO.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131208/190321e6/attachment.html>