[LLVMdev] [RFC] Switching make check to use 'set -o pipefail'

[Tom Stellard]

Hi,

I am in favor of using pipefail by default since it will reduce the
potential for false positives in the test suite.


On Mon, Jul 08, 2013 at 11:07:14AM -0400, Rafael Espíndola wrote:
> > Hmm, I don't know LLVM's Makefile system well enough to know the
> > easiest way to implement an option; if it's non-trivial then maybe
> > it's not worth it.
> 
> That is my impression at least. These errors are somewhat easy to
> introduce, but also easy to fix.
> 

FWIW, I don't think there is much value in adding a command line option
for disabling pipefail.  It shouldn't be too difficult for out of tree
targets to fix their tests.

-Tom

> > I also don't know the workflow of most people doing out-of-tree work,
> > so I'm not sure how much impact this might have. It can obviously be
> > temporarily reverted locally pretty easily, but it assumes people are
> > paying attention to LLVMDev/llvm-commits and know what's going on.
> > Also, the commit causing all the new failures might not be as obvious
> > down the line to someone updating their tree irregularly (which is
> > probably true for a lot of academic LLVM users, I'm guessing.)
> >
> > Just curious, does using pipefail give any information about where in
> > the pipe the failure actually comes from? Some kind of message would
> > be useful for debugging purposes, in addition to explaining what's
> > going on to someone who wasn't watching dev lists and commit messages
> > carefully.
> 
> With the external (shell based) testing we don't get anything. With
> the internal (python based) one we get the last failing command line.
> It doesn't say which component of it failed, but the vast majority of
> the pipes we have are of the form "command | FileCheck", so it is
> clear that "command" must be the one failing since the pipe was
> passing before enabling pipefail.
> 
> Cheers,
> Rafael
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev