[LLVMdev] lit: deprecating trailing \ in RUN lines

[Alp Toker]

On 08/12/2013 13:12, Chandler Carruth wrote:
>
>>           * Removing trailing \ will introduce the neat property that
>>             one RUN line corresponds precisely to one command that's
>>             executed. This is good for humans and will enable
>>             simplifications in the test runner.
>>
>>     FWIW, I've never really had a problem that needed this. The RUN:
>>     forms a prefix of a shell script in my head, and I know how to
>>     read shell scripts including multiple lines.
>
>     The transformations lit does are really too complex and there's at
>     least one known bug to do with closed pipes that's contributing to
>     no-op tests (think the discussion thread was on cfe-dev).
>
>     In a nutshell, the script output lit forms right now is not likely
>     not the pipeline you had in your head ;-)
>
>
> I understand that you think this is too complex, but I'm suggesting 
> that this particular aspect of lit does not seem too complex to at 
> least one other developer, and thus you shouldn't assume it to be true.


It's great if we've made it all look simple to you. Unfortunately for 
the developers there's an ongoing problem with algorithmic complexity in 
lit hiding problems that lead to broken tests.

In particular, there are constructs that would error out in a shell but 
get silently accepted by the lit runner.

See Sean Silva's observation of one of these cases in the thread on 
no-op tests on cfe-dev:

On 08/11/2013 06:48, Sean Silva wrote:
> Although it doesn't eliminate the hassle of having to manually fix 
> this, it seems like at least one issue worth fixing in its own right 
> is the fact that RUN lines ending with a | are silently accepted by 
> our test infrastructure.

This proposal is about resolving a class of these problems, and I'd like 
to see if we can get it done early in the 3.5 cycle given there's a 
degree of churn.

It's ultimately about weighing up the cost/benefit of wrapped RUN lines. 
The benefit you pointed out so far has been visual wrapping in the 
editor, but the cost of that is very high. Perhaps your editor has a 
virtual wrapping mode?

With one-to-one mapping, it becomes possible to use simple tools like 
grep to validate common mistakes like %clang / %clang_cc1 mixups, a 
missing -o flag and so on.

Right now there's no obvious way to do those checks and we've ended up 
without an easy way to lint for broken tests as a result. Each broken 
test has a high cost so we need to continually look at ways to improve 
the situation.

Alp.

-- 
http://www.nuanti.com
the browser experts