[PATCH] D65156: [lit] Protect full test suite from FILECHECK_OPTS

[Joel E. Denny via llvm-commits]

lit's internal shell has its own env implementation.  I think that's what
runs for me (in Ubuntu).  It seems your bot is somehow using the external
shell and thus uses the system env.  Michal's bot must be too because lit's
env supports -u.  Why would your bots be using the external shell?

Joel

On Fri, Jul 26, 2019 at 1:08 PM <paul.robinson at sony.com> wrote:

> googletest-format.py generates a %t.out, which among other things emits
> these two lines:
>
> I am subTest B, I FAIL
>
> And I have two lines of output
>
>
>
> A hexdump of the .out file was very informative.
>
> If there is no 'env' in front of the %{python} in the %{lit} substitution,
> those lines are separate by CR CR LF.
>
> If there IS 'env' in front of %{python}, those lines are separated by CR
> CR CR LF.  And that causes the CHECK-NEXT of the second line of output to
> fail.
>
>
>
> This is likely some peculiar interaction of python and GnuWin32 'env' on
> Windows.  It still bugs me that Windows bots are not running into this.
> Yes we are pursuing that internally.  We might decide to XFAIL a bunch of
> the lit test suite tests just to get our auto-integration moving again,
> although I am not too happy about that tactic.
>
>
>
> Thanks,
>
> --paulr
>
>
>
> *From:* Joel E. Denny [mailto:jdenny.ornl at gmail.com]
> *Sent:* Friday, July 26, 2019 12:35 PM
> *To:* Robinson, Paul
> *Cc:* mgorny at gentoo.org;
> reviews+D65156+public+5784c7837e421597 at reviews.llvm.org; Dan Liew; Thomas
> Preud'homme; LLVM Commits
> *Subject:* Re: [PATCH] D65156: [lit] Protect full test suite from
> FILECHECK_OPTS
>
>
>
> Hi Paul,
>
>
>
> On Fri, Jul 26, 2019 at 11:56 AM <paul.robinson at sony.com> wrote:
>
> > We have had an internal bot get 9 failures in the lit test suite
> > since this patch went in; I will see if the suggested patches help.
> > --paulr
>
> They don't help.  It appears the gnuwin32 'env' does understand -u so
> that wasn't the problem.
>
>
>
> Bummer.
>
>
>
>
> Picking apart 'googletest-format.py' and doing the steps manually,
> so that I can get FileCheck to tell me what's going on, it appears
> that the .out file generated by the test has an extra line terminator,
> so a CHECK-NEXT on the .out file fails.
>
>
>
> Where does that line terminator appear?
>
>
>
>
> If I hack the lit.cfg so it's just 'env %{python} etc' (not trying
> to set or unset) the same failure occurs; so, something about running
> python or lit.py under GnuWin32 'env' is making this occur.
>
> env version 5.3.0
> python version 2.7.12 on Windows version 7
> python version 2.7.14 on Windows version 10
>
>
>
> If you're pursuing this, let me know.  Otherwise, I'll move to windows and
> try to figure this out.
>
>
>
> Thanks for the details.
>
>
>
> Joel
>
>
>
>
> --paulr
>
> >
> > > -----Original Message-----
> > > From: Michał Górny [mailto:mgorny at gentoo.org]
> > > Sent: Friday, July 26, 2019 10:28 AM
> > > To: Joel E. Denny;
> > reviews+D65156+public+5784c7837e421597 at reviews.llvm.org
> > > Cc: Robinson, Paul; Dan Liew; Thomas Preud'homme; LLVM Commits
> > > Subject: Re: [PATCH] D65156: [lit] Protect full test suite from
> > > FILECHECK_OPTS
> > >
> > > On Fri, 2019-07-26 at 10:17 -0400, Joel E. Denny wrote:
> > > > Hi Michał,
> > > >
> > > > On Fri, Jul 26, 2019 at 10:12 AM Michał Górny via Phabricator <
> > > > reviews at reviews.llvm.org> wrote:
> > > >
> > > > > mgorny added a comment.
> > > > >
> > > > > This change broke NetBSD buildbots, e.g.:
> > > > > http://lab.llvm.org:8011/builders/netbsd-
> > >
> >
> amd64/builds/21152/steps/run%20unit%20tests/logs/FAIL%3A%20lit%3A%3A%20goo
> > > gletest-upstream-format.py
> > > >
> > > > I didn't receive an automated email for this.  Should I have?
> > > >
> > >
> > > To be honest, I don't really know.  The bot hasn't managed to get green
> > > for quite some time already, and I don't know if it pings you if there
> > > is a constant breakage.
> > >
> > > --
> > > Best regards,
> > > Michał Górny
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20190726/59d3c42b/attachment.html>