[PATCH] [LIT] Add SuffixDispatchTest for use in libc++ and libc++abi tests.

[Eric Fiselier]

Hi ddunbar, jroelofs, danalbert,

This is an attempt to lift common functionality needed by libc++ and libc++abi into LIT proper. 

`SuffixDispatchTest` handles:
* Reading `XFAIL`, `REQUIRES`, `UNSUPPORTED`, `REQUIRES-XFAIL` and `UNSUPPORTED-XFAIL` tags from the source file.
* Calculating if a test is unsupported.
* Invoking the test_runner based on the tests suffix.
* Translating the result based on `UNSUPPORTED-XFAIL` and `REQUIRES-XFAIL`.

The tags have the following semantics:
* `XFAIL`: Tests that are known failures and should be fixed.
* `UNSUPPORTED`: Tests that are not relevant given a set of features.
* `REQUIRES`: Mirrors `UNSUPPORTED`
* `UNSUPPORTED-XFAIL`: Tests that are not relevant given a set of features but are still required to fail. The rest is reported as `UNSUPPORTED` if it fails and `XPASS` if it passes.
* `REQUIRES-XFAIL`: Mirrors `UNSUPPORTED-XFAIL`.

The `UNSUPPORTED-XFAIL`  tag helps to ensure a test fails while still reporting it as UNSUPPORTED (instead of XFAIL). This is useful when running std::tuple tests in C++03 mode for example. We want to say that std::tuple is *unsupported* in c++03 while ensuring the test continues to fail.

SuffixDispatchTest allows tests with different suffixes to be handled in different ways. In libc++ `.pass.cpp` tests must compile and execute successfully and `.fail.cpp` tests must fail to compile. We can provide two different handlers to SuffixDispatchTest as follows:
```
test_format = SuffixDispatchTest()
test_format.add_suffix_test('.pass.cpp', PassTestHandler(...))
test_format.add_suffix_test('.fail.cpp', FailTestHandler(...))
```

Test handlers are expected to provide the following signature:
```
result,report,msg = handler(test, lit_config)
```
Where:
* result is a instance of ResultCode
* report is a string representing the test that was run and the output.
* msg is a message about the result.
* test is an instance of lit.Test
* lit_config is the global lit configuration object.


`CompileAndExecuteTestRunner` is a test runner for use with `SuffixDispatchTest`. It attempts to compile and run an executable.


I have patches to move libc++ and libc++abi over to the new format once this is accepted.

http://reviews.llvm.org/D6206

Files:
  utils/lit/lit/formats/__init__.py
  utils/lit/lit/formats/base.py
  utils/lit/lit/formats/suffixdispatchtest.py
  utils/lit/lit/util.py
-------------- next part --------------
A non-text attachment was scrubbed...
Name: D6206.16020.patch
Type: text/x-patch
Size: 14137 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20141111/7f0920d5/attachment.bin>