[LLVMdev] make check issue with llvm-cov

[Reed Kotler]

The immediate issue her for me is that llvm-cov is clearly failing but it returns 0.

This causes make check to fail because then it becomes an unexpected pass because
this test is xfailed for mips.

________________________________________
From: reed kotler [rkotler at mips.com]
Sent: Thursday, February 20, 2014 4:58 PM
To: LLVMdev at cs.uiuc.edu
Subject: make check issue with llvm-cov

rkotler at mipsswbrd006-le:~/caviumllvm/build/test$ make
Making LLVM 'lit.site.cfg' file...
Making LLVM unittest 'lit.site.cfg' file...
( ulimit -t 600 ; ulimit -d 512000 ; ulimit -m 512000 ; ulimit -s 8192 ; \
       /usr/bin/python /home/rkotler/workspace/llvm/utils/lit/lit.py -s
-v . )
XPASS: LLVM :: tools/llvm-cov/llvm-cov.test (8916 of 9784)
******************** TEST 'LLVM :: tools/llvm-cov/llvm-cov.test' FAILED
********************
Script:
--
rm -rf
/home/rkotler/caviumllvm/build/test/tools/llvm-cov/Output/llvm-cov.test.tmp
mkdir
/home/rkotler/caviumllvm/build/test/tools/llvm-cov/Output/llvm-cov.test.tmp
cd
/home/rkotler/caviumllvm/build/test/tools/llvm-cov/Output/llvm-cov.test.tmp
cp /home/rkotler/workspace/llvm/test/tools/llvm-cov/Inputs/test* .
/home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov test.c | diff
-u test_no_options.output -
diff -aub test_no_options.cpp.gcov test.cpp.gcov
diff -aub test_no_options.h.gcov test.h.gcov
mkdir -p
/home/rkotler/caviumllvm/build/test/tools/llvm-cov/Output/llvm-cov.test.tmp/objdir
cp test.gcno test.gcda
/home/rkotler/caviumllvm/build/test/tools/llvm-cov/Output/llvm-cov.test.tmp/objdir
/home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov -o objdir
test.c | diff -u test_no_options.output -
diff -aub test_objdir.cpp.gcov test.cpp.gcov
diff -aub test_objdir.h.gcov test.h.gcov
/home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov -o
objdir/test.o test.c | diff -u test_no_options.output -
diff -aub test_objdir.cpp.gcov test.cpp.gcov
diff -aub test_objdir.h.gcov test.h.gcov
/home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov -o objdir/test
test.c | diff -u test_no_options.output -
diff -aub test_objdir.cpp.gcov test.cpp.gcov
diff -aub test_objdir.h.gcov test.h.gcov
mkdir -p
/home/rkotler/caviumllvm/build/test/tools/llvm-cov/Output/llvm-cov.test.tmp/srcdir/nested_dir
cp test.cpp test.h
/home/rkotler/caviumllvm/build/test/tools/llvm-cov/Output/llvm-cov.test.tmp/srcdir
/home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov -p
test_paths.cpp | diff -u test_preserve_paths.output -
diff -aub test_paths.cpp.gcov srcdir#nested_dir#^#test.cpp.gcov
diff -aub test_paths.h.gcov srcdir#nested_dir#^#test.h.gcov
/home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov test_paths.cpp
| diff -u test_no_preserve_paths.output -
diff -aub test_paths.cpp.gcov test.cpp.gcov
diff -aub test_paths.h.gcov test.h.gcov
/home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov test.c -f |
diff -u test_-f.output -
diff -aub test_no_options.cpp.gcov test.cpp.gcov
diff -aub test_no_options.h.gcov test.h.gcov
/home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov test.c -a |
diff -u test_no_options.output -
diff -aub test_-a.cpp.gcov test.cpp.gcov
diff -aub test_-a.h.gcov test.h.gcov
/home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov test.c -a -b |
diff -u test_-b.output -
diff -aub test_-a_-b.cpp.gcov test.cpp.gcov
diff -aub test_-a_-b.h.gcov test.h.gcov
On a native mips linux machine, we are failing a make check test which
seems like it is supposed to already xfail.

For one, the file test.c does not even exist (it is called test.cpp)

And in the test file there is a line:
XFAIL: powerpc64, s390x, mips, sparc

The script in make check thinks that it passed but clearly it does not.
llvm-cov is returning 0 in this case.

Anybody know about this issue?


/home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov test.c -a -b
-f | /home/rkotler/caviumllvm/build/Debug+Asserts/bin/not diff -u
test_-b_-f.output - >/dev/null
diff -aub test_-a_-b.cpp.gcov test.cpp.gcov
diff -aub test_-a_-b.h.gcov test.h.gcov
/home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov test.c -a -b
-u | diff -u test_-b.output -
diff -aub test_-a_-b_-u.cpp.gcov test.cpp.gcov
diff -aub test_-a_-b_-u.h.gcov test.h.gcov
/home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov test.c -a -b
-c -u | diff -u test_-b.output -
diff -aub test_-a_-b_-c_-u.cpp.gcov test.cpp.gcov
diff -aub test_-a_-b_-c_-u.h.gcov test.h.gcov
/home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov test.c
-gcda=no_such_gcda_file | diff -u test_no_gcda.output -
diff -aub test_no_gcda.cpp.gcov test.cpp.gcov
diff -aub test_no_gcda.h.gcov test.h.gcov
not /home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov test.c
-gcno=test_read_fail.gcno
not /home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov test.c
-gcda=test_file_checksum_fail.gcda
not /home/rkotler/caviumllvm/build/Debug+Asserts/bin/llvm-cov test.c
-gcda=test_func_checksum_fail.gcda
--
Exit Code: 0

Command Output (stderr):
--
Unexpected end of memory buffer: 72.
Invalid .gcno File!
File checksums do not match: 557866544 != 1280071245.
Invalid .gcda File!
File checksums do not match: 557866544 != 3906235290.
Invalid .gcda File!

--

********************
Testing Time: 418.75s
********************
Unexpected Passing Tests (1):
     LLVM :: tools/llvm-cov/llvm-cov.test

   Expected Passes    : 9678
   Expected Failures  : 70
   Unsupported Tests  : 35
   Unexpected Passes  : 1
make: *** [check-local] Error 1