[cfe-dev] Running Test Cases

[Douglas Gregor]

On Sep 29, 2009, at 7:02 AM, Daniel Dilts wrote:

> I feel like a complete idiot.  I can't figure out how to run the test
> cases for clang.
>
> In the test subdirectory I have run make all.

I assume you mean tools/clang/test

> That shows a pretty
> progress bar, but as far as I can tell nothing is actually output.

You'll typically get something like this:

Making Clang 'lit.site.cfg' file...
--- Running clang tests for x86_64-apple-darwin10.0.0 ---
lit.py: lit.cfg:146: note: using clang: '/Users/dgregor/Projects/llvm- 
autotools-make/Release-Asserts/bin/clang'
lit.py: lit.cfg:151: note: using clang-cc: '/Users/dgregor/Projects/ 
llvm-autotools-make/Release-Asserts/bin/clang-cc'
                       -- Testing: 1625 tests, 2 threads --
  82% [================================================-----------]  
ETA: 00:00:10
Clang::SemaObjC/check-dup-objc-decls-1.m

once the tests finish, the progress bar will go away and you'll get a  
summary:

Testing Time: 58.61s
   Expected Passes    : 1610
   Expected Failures  : 15

there's no other output unless there is a failure, which shouldn't  
happen. If you really want to see every test go by, pass the make  
argument VERBOSE=1

> I can't figure out what to do with TestRunner.sh.

Ignore it. At some point, it will disappear. llvm/utils/lit/lit.py is  
what drives Clang testing now ("make all" in tools/clang/test just  
runs lit.py).

> I can see that in the test/CXX directory there are numerous
> subdirectories that appear to have test cases (.cpp) and scripts
> (.script).  The content of the .script files looks to be a simple
> command line, but I can't figure out how to run them in mass and/or
> get output for what passes/fails.


The verbose output shows all passes/fails. Generally, it's only the  
fails that matter, and there should never be any not marked XFAIL.

	- Doug