[LLVMdev] LLVM / CLANG Test Infrastructure Question

[Sundeep]

Hi All,

I am working on a bug in clang. I already have a fix for it and I am going
through the "LLVM Testing Infrastructure Guide" to make sure I haven't
broken anything else. I have few questions regarding the test suite
infrastructure that I hope someone can answer.

1. I checked out and built llvm, clang, and test-suite from svn tip. When I
run llvm/test *without* my changes on x86, I see 3 failures. Is it expected?
I was expecting svn tip on x86 to be clean. Is there a nightly regression
result available somewhere that I can check for broken tests. Here is the
report from my log:

Failing Tests (3):
    Clang :: Analysis/iterators.cpp
    Clang :: Driver/hello.c
    Clang :: Preprocessor/macro_paste_c_block_comment.c

  Expected Passes    : 8757
  Expected Failures  : 69
  Unsupported Tests  : 551
  Unexpected Failures: 3

2. I see clang has its own test suite under clang/test. I assume when I run
llvm/test, the test infrastructure also runs clang/test.

3. For running projects/test-suite, the instruction guide directs to build
llvm-gcc. Why is llvm-gcc required? Why is clang not being used?

4. I see many compile errors when running projects/test-suite. Is it
expected?

5. I used "make -i" to ignore compile erros and execute all tests under
projects/test-suite. I finally get a report something like this:

Program                                                                  |
GCCAS   Bytecode LLC compile LLC-BETA compile JIT codegen | GCC      CBE
LLC     LLC-BETA JIT      | GCC/CBE GCC/LLC
GMultiSource/Applications/Burg/burg                                       |
5.5484  115468   2.9722      *                3.0922      |  0.0100  *
0.0000 *         3.2400  | n/a     -       n
MultiSource/Applications/ClamAV/clamscan                                 |
41.8306 1335184  34.4382     *                18.6692     |  0.1200  *
0.1300 *        19.5200  | n/a     0.92    n

I was also expecting to see a list of passing/failing test cases similar to
llvm/test runs. Is there a make target to do so?

I apologies for asking so many questions. I followed directions from the
"LLVM Testing Infrastructure Guide" but it doesn't address all these issues.
I will appreciate your help to resolve these.

Thanks,
Sundeep