[llvm-dev] [RFC] Require PRs for XFAILing tests

[David Blaikie via llvm-dev]

On Wed, Sep 28, 2016 at 11:58 AM Robinson, Paul via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> On 28 September 2016 at 10:08, Chris Bieneman via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> > I cannot think of any situation where a universally failing test
> > should be in-tree unless it is a bug that someone is expecting to fix.
>
> It seems moderately common to mark something XFAIL temporarily to get
> the bots green while then going ahead to fix the issue.  Your proposal
> would add extra overhead to that flow by requiring a PR as well.  This
> has value when it turns out that fix can't happen in the short term for
> any reason.  I don't have a feel for how common that is, although I'm
> sure it does happen.
> I think the overhead is worth the added value, but then I'm a process
> kind of guy.
>
>
> On 28 September 2016 at 10:28, Renato Golin via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> > We already have an unwritten rule to create PRs for XFAILs, and we
> > normally don't XFAIL lightly (I don't, at least). But creating one PR
> > for every existing XFAIL may end up as a long list of never looked
> > PRs. :)
>
> As opposed to the other ~9000 open PRs?  At least they would be tracked.
>

I'd be inclined to agree (or at least voice the same concern) as Renato
here - as a project we don't really have very good bug hygiene, so adding
more bug filing doesn't seem to me like it'd give us much value.

Auditing existing XFAILs can be done today without filing bugs for them.

And we still always have the option to (& in many cases do) file bugs for
XFAILs to discuss them, etc.

But I don't feel strongly about it either way, so happy to leave the folks
who do to make the call/do the work.

- Dave


> --paulr
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160929/a74298f3/attachment.html>