[LLVMdev] make check issue with llvm-cov

[Reed Kotler]

On 02/21/2014 02:58 AM, Daniel Sanders wrote:
>>> And in the test file there is a line:
>>> XFAIL: powerpc64, s390x, mips, sparc
>> This is a crude attempt at "XFAIL: big-endian". The mips entry here is just
>> wrong if the system is little-endian - the test passes on little-endian machines
>> and fails on big-endian. This is obviously a problem.
> 'XFAIL: mips' counts as an XFAIL for all mips targets because lit is checking for substrings (using the 'in' operator) of the target triple and 'mips' is a substring of 'mipsel-...'. I haven't tested this but I think that we need 'XFAIL: mips-, mips64-' to distinguish little/big-endian correctly for MIPS.
> The relevant code is the isExpectedToFail() function in utils/lit/lit/Test.py.
In this case what seems to be happening is the llvm-cov is NOT failing 
and it should.
That causes an unexpected pass which is what is making "make check" fail.

>> -----Original Message-----
>> From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu]
>> On Behalf Of Justin Bogner
>> Sent: 21 February 2014 08:21
>> To: Reed Kotler
>> Cc: LLVMdev at cs.uiuc.edu
>> Subject: Re: [LLVMdev] make check issue with llvm-cov
>>
>> reed kotler <rkotler at mips.com> writes:
>>> On a native mips linux machine, we are failing a make check test which
>>> seems like it is supposed to already xfail.
>> I think you're mistaken. The test below looks like it's passing.
>>
>>> For one, the file test.c does not even exist (it is called test.cpp)
>> This should probably be test.o, rather than test.c or test.cpp, but the
>> interface to llvm-cov is compatible with gcov, which completely ignores the
>> file given as an argument other than to base the .gcno and .gcno file names
>> on it.
>>
>>> And in the test file there is a line:
>>> XFAIL: powerpc64, s390x, mips, sparc
>> This is a crude attempt at "XFAIL: big-endian". The mips entry here is just
>> wrong if the system is little-endian - the test passes on little-endian machines
>> and fails on big-endian. This is obviously a problem.
>>
>>> The script in make check thinks that it passed but clearly it does not.
>>> llvm-cov is returning 0 in this case.
>> I'm not sure which case you're referring to. This test file really has too many
>> tests in it, which makes it confusing to debug failures.
>> Someone (me?) should probably split this into several tests that invoke llvm-
>> cov once each. The use of diff should be replaced with FileCheck as well, so
>> that error messages are clearer.
>>
>> In any case, all of the llvm-cov calls except the read/checksum fail invocations
>> should (and do) return zero on little-endian. I think the only problem here is
>> that the XFAIL line is wrong.
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev