[LLVMdev] Poll: LLVM test-suite project usage
criswell at illinois.edu
Thu Oct 25 21:39:41 PDT 2012
On 10/25/12 6:04 PM, Daniel Dunbar wrote:
> Hi all,
> I would like to get some feedback on who in the community is using the
> LLVM test-suite repository, and in what ways.
> I am talking specifically about the "test-suite" SVN project, e.g.:
> not about the "test (no-dash) suite" that is part of LLVM.
> If you are using the LLVM test-suite project directly (as opposed to
> via LNT), I'd appreciate it if you could reply to me personally with
> the following information:
> 1. What facilities of the project do you use:
> [ X] User defined test functionality
> [ X] Ability to execute two compilers (baseline, modified)
> [ ] Ability to use Makefile's to modify and rerun small parts of tests
> [ X] Test suite ability to work with LLVM bit code files and tools (as
> opposed to native object files)
> [ X] Ability to test SPEC
> [ ] Ability to execute tests on a remote target
> [ X] Ability to test Fortran codes
The ability to run Fortran code is something we've used in the past, and
we may want to use it in the future (at least if there's Fortran code in
SPEC 2006). However, I think it's a very low priority for us.
> 2. What compilers do you use the project to test:
> [ X] clang
> [ ] llvm-gcc
> [ ] dragonegg
> [ X] Other LLVM compilers (?)
By other compilers, I mean research prototypes. Some are modified
versions of Clang; others are passes that are loaded into opt. Others
might use llvm-gcc (although I don't know of any off-hand that do that
we care about).
> [ ] Other non-LLVM compilers
> 3. What TEST variants do you use:
> [ ] I do not know what this means
> [ ] nightly
> [ ] simple
> [ X] Other, please specify: _________
We use hand-written variants in SAFECode, Poolalloc, and in internal
research projects. These Makefiles are out-of-tree but plug into the
test suite using an extension mechanism that Chris Lattner originally
wrote for the DSA/Poolalloc project.
We have used test-suite heavily in our research experiments and
open-source project testing. The two features that are most beneficial
to us are:
1) We can plug into the LLVM test suite to do customized experiments
without modifying the core test-suite Makefiles.
2) The API doesn't change, so we continue using the same infrastructure
with each new LLVM release. The SAFECode and Poolalloc tests were
written years ago but have continued working with only minor
modification. Many of our internal projects have just copied and
modified what they've done. Experimental infrastructure is a pain to
set up, so having most of the work done is a real benefit.
-- John T.
> - Daniel
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
More information about the llvm-dev