[LLVMdev] Running gcc tests using Clang

[Daniel Dunbar]

Hi John,

On Wed, Sep 22, 2010 at 11:32 AM, John Thompson
<john.thompson.jtsoftware at gmail.com> wrote:
> I've run the tests from clang-tests/trunk/gcc-4_2-testsuite on a Ubuntu
> x86-64 Linux box with the following results:
>
>         === gcc Summary ===
>
> # of expected passes        29946
> # of unexpected failures    9938
> # of unexpected successes    29
> # of expected failures        28
> # of unresolved testcases    1451
> # of untested testcases        273
> # of unsupported tests        811
> pid is 4456 -4456
> output is clang version 2.9 (trunk 113841)
> Target: x86_64-unknown-linux-gnu
> Thread model: posix
>  status 0
> /home/john/llvm/Release+Asserts/bin/clang  version 2.9 (trunk 113841)
>
> This seems to be an improvement over the results I had previously when using
> the i386 configuration.  However, it sounded like you had more success on a
> Darwin box, with only 1 unexpected failure (mine, which should be fixed
> now).  Should I expect that with this configuration?

No, that is unfortunately not the case.

What I do for testing is keep arounds results (see the 'make
save-results' utility target) and compare the diffs from run to run.
This is a real pain, but the best tool we have at the moment.

 - Daniel

> I found that I had to exit most other running applications to be able to run
> the tests without running out of memory.
>
> I've enclosed my hacked makefile, hacked to change the configuration and add
> the -v (verbose) flag.
>
> I have a 6 mb log file from this run if needed.
>
> ~/gcc-4_2-testsuite$ find . -print | wc
>   36392   36392 1687901
>
> Some output from the first few failed tests:
>
> Testing gcc.apple/5490617.c
> doing compile
> pid is 17502 -17502
> output is  status 0
> FAIL: gcc.apple/5490617.c scan-assembler .lcomm __my_endbss
>
> Testing gcc.apple/4104248.c
> doing compile
> pid is 17445 -17445
> close result is 17445 exp8 0 1
> output is ilp3217400.c:2: error: array size is negative
>  status 1
>
> Testing gcc.apple/4641942.c
> doing compile
> pid is 17481 -17481
> output is  status 0
> FAIL: gcc.apple/4641942.c  (test for warnings, line 20)
>
> Testing gcc.apple/5597292.c
> doing compile
> pid is 17507 -17507
> close result is 17507 exp8 0 1
> output is /home/john/gcc-4_2-testsuite/src/gcc.apple/5597292.c:7: error:
> fields must have a constant size: 'variable length array in structure'
> extension will never be supported
>  status 1
> compiler exited with status 1
> FAIL: gcc.apple/5597292.c (test for excess errors)
>
> Testing gcc.apple/6286881.c
> doing compile
> pid is 17572 -17572
> close result is 17572 exp8 0 1
> output is /home/john/gcc-4_2-testsuite/src/gcc.apple/6286881.c:8: warning: ^
> has lower precedence than !=; != will be evaluated first [-Wparentheses]
> /home/john/gcc-4_2-testsuite/src/gcc.apple/6286881.c:8: note: place
> parentheses around the ^ expression to evaluate it first
> /home/john/gcc-4_2-testsuite/src/gcc.apple/6286881.c:8: note: place
> parentheses around the != expression to silence this warning
> /home/john/gcc-4_2-testsuite/src/gcc.apple/6286881.c:8: error: address
> expression must be an lvalue or a function designator
>  status 1
> compiler exited with status 1
> FAIL: gcc.apple/6286881.c  (test for errors, line 8)
>
> etc.
>
> -John
>
> --
> John Thompson
> John.Thompson.JTSoftware at gmail.com
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>