[llvm-dev] [RFC] Require PRs for XFAILing tests
Matthias Braun via llvm-dev
llvm-dev at lists.llvm.org
Wed Sep 28 13:16:43 PDT 2016
> On Sep 28, 2016, at 1:01 PM, Chris Bieneman via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> I think there are a few interesting things that could follow from solidifying a policy requiring PRs for XFAILs.
> First and foremost bugs can have way more context than than you would often find in a test case comment. That would make it a lot easier to audit XFAILs in the future and help keep the number of XFAILs to a minimum. I think this is important because many of our XFAILs are really old, and I’m not convinced that we shouldn’t just be deleting some of these tests.
> For example, 2008-12-14-StrideAndSigned.ll and 7 other tests were marked with “XFAIL: *” in 2009, and the commit message doesn’t really explain what was going on:
>> commit 789558db70d9513a017c11c5be30945839fdff1c
>> Author: Nick Lewycky <nicholas at mxc.ca>
>> Date: Tue Jan 13 09:18:58 2009 +0000
>> Wind SCEV back in time, to Nov 18th. This 'fixes' PR3275, PR3294, PR3295,
>> PR3296 and PR3302.
>> git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@62160 91177308-0d34-0410-b5e6-96231b3b80d8
> Requiring a PR doesn’t necessarily fix this problem because you could list the wrong PR, or even just a generic stub PR that didn’t add meaningful value, but I think our community is pretty good at keeping people honest via code reviews.
Other important aspects:
- We can immediately start a specific and meaningful conversation about the problem
- Progress/blockers in solving the problem can be documented if necessary
- It is actually easy to find the corresponding discussion to a problem (the alternative of looking up the commit adding the XFAIL and then searching the commits mailing list for threads on that commit is more trouble)
More information about the llvm-dev