[llvm-commits] CVS: llvm/docs/TestingGuide.html
Misha Brukman
brukman at cs.uiuc.edu
Mon Mar 1 12:22:01 PST 2004
Changes in directory llvm/docs:
TestingGuide.html updated: 1.6 -> 1.7
---
Log message:
* HTML 4.01 Strict compliance
* Removed unnecessary tabs in the entire file
---
Diffs of the changes: (+355 -401)
Index: llvm/docs/TestingGuide.html
diff -u llvm/docs/TestingGuide.html:1.6 llvm/docs/TestingGuide.html:1.7
--- llvm/docs/TestingGuide.html:1.6 Mon Nov 3 08:59:59 2003
+++ llvm/docs/TestingGuide.html Mon Mar 1 12:21:04 2004
@@ -1,11 +1,10 @@
-<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
+ "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
- <link rel="stylesheet" href="llvm.css" type="text/css" media="screen" />
- <title>LLVM Test Suite Guide</title>
+ <title>LLVM Test Suite Guide</title>
+ <link rel="stylesheet" href="llvm.css" type="text/css">
</head>
-
<body>
<div class="doc_title">
@@ -16,414 +15,369 @@
<li><a href="#overview">Overview</a></li>
<li><a href="#Requirements">Requirements</a></li>
<li><a href="#quick">Quick Start</a></li>
-<li><a href="#org">LLVM Test Suite Organization</a></li>
+<li><a href="#org">LLVM Test Suite Organization</a>
<ul>
<li><a href="#codefragments">Code Fragments</a></li>
<li><a href="#wholeprograms">Whole Programs</a></li>
-</ul>
+</ul></li>
<li><a href="#tree">LLVM Test Suite Tree</a></li>
<li><a href="#qmstructure">QMTest Structure</a></li>
<li><a href="#progstructure">Programs Structure</a></li>
<li><a href="#run">Running the LLVM Tests</a></li>
-<p><b>Written by John T. Criswell</b></p>
</ol>
- <!--===============================================================-->
- <div class="doc_section"><a name="overview">Overview</a></div>
- <!--===============================================================-->
-
- <div class="doc_text">
- <p>
- This document is the reference manual for the LLVM test suite. It
- documents the structure of the LLVM test suite, the tools needed to
- use it, and how to add and run tests.
- </p>
- </div>
-
- <!--===============================================================-->
- <div class="doc_section"><a name="Requirements">Requirements</a></div>
- <!--===============================================================-->
-
- <div class="doc_text">
- <p>
- In order to use the LLVM test suite, you will need all of the software
- required to build LLVM, plus the following:
- </p>
- <dl compact>
- <dt><A HREF="http://www.qmtest.com">QMTest</A></dt>
- <dd>The LLVM test suite uses QMTest to organize and
- run tests.</dd>
-
- <dt><A HREF="http://www.python.org">Python</A></dt>
- <dd>You will need a Python interpreter that works with
- QMTest. Python will need zlib and SAX support
- enabled.</dd>
- </dl>
- </div>
-
- <!--===============================================================-->
- <div class="doc_section"><a name="quick">Quick Start</a></div>
- <!--===============================================================-->
-
- <div class="doc_text">
- <p>
- The tests are located in the LLVM source tree under the directory
- <tt>llvm/test</tt>. To run all of the tests in LLVM, use the Master
- Makefile in that directory:
- </p>
- <pre>
- % gmake -C llvm/test
- </pre>
-
- <p>
- To run only the code fragment tests (i.e. those that do basic testing of
- LLVM), run the tests organized by QMTest:
- </p>
-
- <pre>
- % gmake -C llvm/test qmtest
- </pre>
-
- <p>
- To run only the tests that compile and execute whole programs, run the
- Programs tests:
- </p>
-
- <pre>
- % gmake -C llvm/test/Programs
- </pre>
- </div>
-
- <!--===============================================================-->
- <div class="doc_section"><h2><a name="org">LLVM Test Suite
- Organization </a></h2></div>
- <!--===============================================================-->
-
- <div class="doc_text">
- <p>The LLVM test suite contains two major categories of tests: code
- fragments and whole programs.</p>
- </div>
-
- <div class="doc_subsection"><a name="codefragments">Code Fragments</a>
- </div>
-
- <div class="doc_text">
- <p>
- Code fragments are small pieces of code that test a specific
- feature of LLVM or trigger a specific bug in LLVM. They are
- usually written in LLVM assembly language, but can be
- written in other languages if the test targets a
- particular language front end.
- </p><p>
- Code fragments are not complete programs, and they are
- never executed to determine correct behavior.
- </p><p>
- The tests in the Features and
- Regression directories contain code fragments.
- </p>
- </div>
-
- <div class="doc_subsection"><a name="wholeprograms">Whole Programs</a>
- </div>
-
- <div class="doc_text">
- <p>
- Whole Programs are pieces of code which can be compiled and
- linked into a stand-alone program that can be executed. These
- programs are generally written in high level languages such as C
- or C++, but sometimes they are written straight in LLVM
- assembly.
- </p><p>
- These programs are compiled and then executed using several
- different methods (native compiler, LLVM C backend, LLVM JIT,
- LLVM native code generation, etc). The output of these programs
- is compared to ensure that LLVM is compiling the program
- correctly.
- </p><p>
- In addition to compiling and executing programs, whole program
- tests serve as a way of benchmarking LLVM performance, both in
- terms of the efficiency of the programs generated as well as the
- speed with which LLVM compiles, optimizes, and generates code.
- </p><p>
- The Programs directory contains all tests which compile and
- benchmark whole programs.
- </p>
- </div>
-
- <!--===============================================================-->
- <div class="doc_section"><h2><a name="tree">LLVM Test Suite Tree</a>
- </div>
- <!--===============================================================-->
-
- <div class="doc_text">
- <p>Each type of test in the LLVM test suite has its own directory. The
- major subtrees of the test suite directory tree are as follows:</p>
+<p><b>Written by John T. Criswell</b></p>
+
+<!--===============================================================-->
+<div class="doc_section"><a name="overview">Overview</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p>This document is the reference manual for the LLVM test suite. It documents
+the structure of the LLVM test suite, the tools needed to use it, and how to add
+and run tests.</p>
+
+</div>
+
+<!--===============================================================-->
+<div class="doc_section"><a name="Requirements">Requirements</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p>In order to use the LLVM test suite, you will need all of the software
+required to build LLVM, plus the following:</p>
+
+<dl>
+ <dt><a href="http://www.qmtest.com">QMTest</A></dt>
+ <dd>The LLVM test suite uses QMTest to organize and run tests.</dd>
+
+ <dt><a href="http://www.python.org">Python</A></dt>
+ <dd>You will need a Python interpreter that works with QMTest. Python will
+ need zlib and SAX support enabled.</dd>
+</dl>
+
+</div>
+
+<!--===============================================================-->
+<div class="doc_section"><a name="quick">Quick Start</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p> The tests are located in the LLVM source tree under the directory
+<tt>llvm/test</tt>. To run all of the tests in LLVM, use the Master Makefile in
+that directory:</p>
+
+<pre>
+ % gmake -C llvm/test
+</pre>
+
+<p>To run only the code fragment tests (i.e. those that do basic testing of
+LLVM), run the tests organized by QMTest:</p>
+
+<pre>
+ % gmake -C llvm/test qmtest
+</pre>
+
+<p>To run only the tests that compile and execute whole programs, run the
+Programs tests:</p>
+
+<pre>
+ % gmake -C llvm/test/Programs
+</pre>
+
+</div>
+
+<!--===============================================================-->
+<div class="doc_section"><a name="org">LLVM Test Suite Organization</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p>The LLVM test suite contains two major categories of tests: code
+fragments and whole programs.</p>
+
+</div>
+
+<div class="doc_subsection"><a name="codefragments">Code Fragments</a>
+</div>
+
+<div class="doc_text">
+
+<p>Code fragments are small pieces of code that test a specific feature of LLVM
+or trigger a specific bug in LLVM. They are usually written in LLVM assembly
+language, but can be written in other languages if the test targets a particular
+language front end.</p>
+
+<p>Code fragments are not complete programs, and they are never executed to
+determine correct behavior.</p>
+
+<p>The tests in the Features and Regression directories contain code
+fragments.</p>
+
+</div>
+
+<div class="doc_subsection"><a name="wholeprograms">Whole Programs</a></div>
+
+<div class="doc_text">
+
+<p>Whole Programs are pieces of code which can be compiled and linked into a
+stand-alone program that can be executed. These programs are generally written
+in high level languages such as C or C++, but sometimes they are written
+straight in LLVM assembly.</p>
+
+<p>These programs are compiled and then executed using several different
+methods (native compiler, LLVM C backend, LLVM JIT, LLVM native code generation,
+etc). The output of these programs is compared to ensure that LLVM is compiling
+the program correctly.</p>
+
+<p>In addition to compiling and executing programs, whole program tests serve as
+a way of benchmarking LLVM performance, both in terms of the efficiency of the
+programs generated as well as the speed with which LLVM compiles, optimizes, and
+generates code.</p>
+
+<p>The Programs directory contains all tests which compile and benchmark whole
+programs.</p>
+
+</div>
+
+<!--===============================================================-->
+<div class="doc_section"><a name="tree">LLVM Test Suite Tree</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p>Each type of test in the LLVM test suite has its own directory. The major
+subtrees of the test suite directory tree are as follows:</p>
+
+<ul>
+ <li>Features
+ <p>This directory contains sample codes that test various features of the
+ LLVM language. These pieces of sample code are run through various
+ assembler, disassembler, and optimizer passes.</p>
+
+ <li>Regression
+ <p>This directory contains regression tests for LLVM. When a bug is found
+ in LLVM, a regression test containing just enough code to reproduce the
+ problem should be written and placed somewhere underneath this directory.
+ In most cases, this will be a small piece of LLVM assembly language code,
+ often distilled from an actual application or benchmark.</p>
+
+ <li>Programs
+ <p>The Programs directory contains programs that can be compiled with LLVM
+ and executed. These programs are compiled using the native compiler and
+ various LLVM backends. The output from the program compiled with the native
+ compiler is assumed correct; the results from the other programs are
+ compared to the native program output and pass if they match. </p>
+
+ <p> In addition for testing correctness, the Programs directory also
+ performs timing tests of various LLVM optimizations. It also records
+ compilation times for the compilers and the JIT. This information can be
+ used to compare the effectiveness of LLVM's optimizations and code
+ generation.</p>
+
+ <p>The Programs directory is subdivided into several smaller subdirectories:
+ </p>
+
+ <ul>
+ <li>Programs/SingleSource
+ <p>The SingleSource directory contains test programs that are only a
+ single source file in size. These are usually small benchmark programs
+ or small programs that calculate a particular value. Several such
+ programs are grouped together in each directory.</p></li>
+
+ <li>Programs/MultiSource
+ <p>The MultiSource directory contains subdirectories which contain
+ entire programs with multiple source files. Large benchmarks and whole
+ applications go here.</p></li>
+
+ <li>Programs/External
+ <p>The External directory contains Makefiles for building code that is
+ external to (i.e. not distributed with) LLVM. The most prominent member
+ of this directory is the SPEC 2000 benchmark suite. The presence and
+ location of these external programs is configured by the LLVM
+ <tt>configure</tt> script.</p></li>
- <ul>
- <li>Features
- <p>
- This directory contains sample codes that test various features
- of the LLVM language. These pieces of sample code are run
- through various assembler, disassembler, and optimizer passes.
- </p>
-
- <li>Regression
- <p>
- This directory contains regression tests for LLVM. When a bug
- is found in LLVM, a regression test containing just enough
- code to reproduce the problem should be written and placed
- somewhere underneath this directory. In most cases, this
- will be a small piece of LLVM assembly language code, often
- distilled from an actual application or benchmark.
- </p>
-
- <li>Programs
- <p>
- The Programs directory contains programs that can be compiled
- with LLVM and executed. These programs are compiled using the
- native compiler and various LLVM backends. The output from the
- program compiled with the native compiler is assumed correct;
- the results from the other programs are compared to the native
- program output and pass if they match.
- </p><p>
- In addition for testing correctness, the Programs directory
- also performs timing tests of various LLVM optimizations.
- It also records compilation times for the compilers and the
- JIT. This information can be used to compare the
- effectiveness of LLVM's optimizations and code generation.
- </p><p>
- The Programs directory is subdivided into several smaller
- subdirectories:
- </p>
-
- <ul>
- <li>Programs/SingleSource
- <p>
- The SingleSource directory contains test programs that
- are only a single source file in size. These are
- usually small benchmark programs or small programs that
- calculate a particular value. Several such programs are
- grouped together in each directory.
- </p>
-
- <li>Programs/MultiSource
- <p>
- The MultiSource directory contains subdirectories which
- contain entire programs with multiple source files.
- Large benchmarks and whole applications go here.
- </p>
-
- <li>Programs/External
- <p>
- The External directory contains Makefiles for building
- code that is external to (i.e. not distributed with)
- LLVM. The most prominent member of this directory is
- the SPEC 2000 benchmark suite. The presence and
- location of these external programs is configured by the
- LLVM <tt>configure</tt> script.
- </p>
- </ul>
-
- <p>
-
- <li>QMTest
- <p>
- This directory contains the QMTest information files. Inside
- this directory are QMTest administration files and the Python
- code that implements the LLVM test and database classes.
- </p>
- </ul>
- </div>
-
- <!--===============================================================-->
- <div class="doc_section"><h2><a name="qmstructure">QMTest Structure</a>
- </div>
- <!--===============================================================-->
-
- <div class="doc_text">
- <p>
- The LLVM test suite is partially driven by QMTest and partially
- driven by GNU Make. Specifically, the Features and Regression tests
- are all driven by QMTest. The Programs directory is currently
- driven by a set of Makefiles.
- </p><p>
- The QMTest system needs to have several pieces of information
- available; these pieces of configuration information are known
- collectively as the "context" in QMTest parlance. Since the context
- for LLVM is relatively large, the master Makefile in llvm/test
- sets it for you.
- </p><p>
- The LLVM database class makes the subdirectories of llvm/test a
- QMTest test database. For each directory that contains tests driven by
- QMTest, it knows what type of test the source file is and how to run it.
- </p><p>
- Hence, the QMTest namespace is essentially what you see in the
- Feature and Regression directories, but there is some magic that
- the database class performs (as described below).
- </p><p>
- The QMTest namespace is currently composed of the following tests and
- test suites:
- </p>
-
- <ul>
- <li>Feature
- <p>
- These are the feature tests found in the Feature directory.
- They are broken up into the following categories:
- </p>
- <ul>
- <li>ad
- <p>
- Assembler/Disassembler tests. These tests verify that a
- piece of LLVM assembly language can be assembled into
- bytecode and then disassembled into the original
- assembly language code. It does this several times to
- ensure that assembled output can be disassembled and
- disassembler output can be assembled. It also verifies
- that the give assembly language file can be assembled
- correctly.
- </p>
-
- <li>opt
- <p>
- Optimizer tests. These tests verify that two of the
- optimizer passes completely optimize a program (i.e.
- after a single pass, they cannot optimize a program
- any further).
- </p>
-
- <li>mc
- <p>
- Machine code tests. These tests verify that the LLVM
- assembly language file can be translated into native
- assembly code.
- </p>
-
- <li>cc
- <p>
- C code tests. These tests verify that the specified
- LLVM assembly code can be converted into C source code
- using the C backend.
- </p>
- </ul>
-
- <p>
- The LLVM database class looks at every file in the Feature
- directory and creates a fake test hierarchy containing
- <tt>Feature.<testtype>.<testname></tt>. So, if you
- add an LLVM assembly language file to the Feature directory, it
- actually creates 5 new tests: assembler/disassembler, assembler,
- optimizer, machine code, and C code.
- </p>
-
- <li>Regression
- <p>
- These are the regression tests. There is one suite for each
- subdirectory of the Regression directory. If you add a new
- subdirectory there, you will need to modify, at least, the
- <tt>RegressionMap</tt> variable in <tt>QMTest/llvmdb.py</tt> so
- that QMTest knows how to run the tests in the new subdirectory.
- </p>
- </ul>
- </div>
-
- <!--===============================================================-->
- <div class="doc_section"><h2><a name="progstructure">Programs
- Structure</a></div>
- <!--===============================================================-->
-
- <div class="doc_text">
- <p>
- As mentioned previously, the Programs tree in llvm/test provides three
- types of tests: MultiSource, SingleSource, and External. Each tree is
- then subdivided into several categories, including applications,
- benchmarks, regression tests, code that is strange grammatically, etc.
- These organizations should be relatively self explanatory.
- </p><p>
- In addition to the regular Programs tests, the Programs tree also
- provides a mechanism for compiling the programs in different ways. If
- the variable TEST is defined on the gmake command line, the test system
- will include a Makefile named <tt>TEST.<value of TEST
- variable>.Makefile</tt>. This Makefile can modify build rules to
- yield different results.
- </p><p>
- For example, the LLVM nightly tester uses <tt>TEST.nightly.Makefile</tt>
- to create the nightly test reports. To run the nightly tests, run
- <tt>gmake TEST=nightly</tt>.
- </p><p>
- There are several TEST Makefiles available in the tree. Some of them
- are designed for internal LLVM research and will not work outside of the
- LLVM research group. They may still be valuable, however, as a guide to
- writing your own TEST Makefile for any optimization or analysis passes
- that you develop with LLVM.
- </p>
- </div>
-
- <!--===============================================================-->
- <div class="doc_section"><h2><a name="run">Running the LLVM Tests</a>
- </div>
- <!--===============================================================-->
-
- <div class="doc_text">
- <p>
- First, all tests are executed within the LLVM object directory tree.
- They <i>are not</i> executed inside of the LLVM source tree. This is
- because the test suite creates temporary files during execution.
- </p><p>
- The master Makefile in llvm/test is capable of running both the
- QMTest driven tests and the Programs tests. By default, it will run
- all of the tests.
- </p><p>
- To run only the QMTest driven tests, run <tt>gmake qmtest</tt> at the
- command line in llvm/tests. To run a specific qmtest, suffix the test
- name with ".t" when running gmake.
- </p><p>
- For example, to run the Regression.LLC tests, type
- <tt>gmake Regression.LLC.t</tt> in llvm/tests.
- </p><p>
- Note that the Makefiles in llvm/test/Features and llvm/test/Regression
- are gone. You must now use QMTest from the llvm/test directory to run
- them.
- </p><p>
- To run the Programs test, cd into the llvm/test/Programs directory and
- type <tt>gmake</tt>. Alternatively, you can type <tt>gmake
- TEST=<type> test</tt> to run one of the specialized tests in
- llvm/test/Programs/TEST.<type>.Makefile. For example, you could
- run the nightly tester tests using the following commands:
- </p>
-
- <pre>
- % cd llvm/test/Programs
- % gmake TEST=nightly test
- </pre>
-
- <p>
- Regardless of which test you're running, the results are printed on
- standard output and standard error. You can redirect these results to a
- file if you choose.
- </p><p>
- Some tests are known to fail. Some are bugs that we have not fixed yet;
- others are features that we haven't added yet (or may never add). In
- QMTest, the result for such tests will be XFAIL (eXpected FAILure). In
- this way, you can tell the difference between an expected and unexpected
- failure.
- </p><p>
- The Programs tests have no such feature as of this time. If the test
- passes, only warnings and other miscellaneous output will be generated.
- If a test fails, a large <program> FAILED message will be
- displayed. This will help you separate benign warnings from actual test
- failures.
- </p>
- </div>
+ </ul></li>
-<!-- *********************************************************************** -->
+ <li>QMTest
+ <p>This directory contains the QMTest information files. Inside this
+ directory are QMTest administration files and the Python code that
+ implements the LLVM test and database classes.</p>
+
+</ul>
+
+</div>
+
+<!--===============================================================-->
+<div class="doc_section"><a name="qmstructure">QMTest Structure</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p>The LLVM test suite is partially driven by QMTest and partially
+driven by GNU Make. Specifically, the Features and Regression tests
+are all driven by QMTest. The Programs directory is currently
+driven by a set of Makefiles.</p>
+
+<p>The QMTest system needs to have several pieces of information
+available; these pieces of configuration information are known
+collectively as the "context" in QMTest parlance. Since the context
+for LLVM is relatively large, the master Makefile in llvm/test
+sets it for you.</p>
+
+<p>The LLVM database class makes the subdirectories of llvm/test a
+QMTest test database. For each directory that contains tests driven by
+QMTest, it knows what type of test the source file is and how to run it.</p>
+
+<p>Hence, the QMTest namespace is essentially what you see in the
+Feature and Regression directories, but there is some magic that
+the database class performs (as described below).</p>
+
+<p>The QMTest namespace is currently composed of the following tests and test
+suites:</p>
+
+<ul>
+ <li>Feature
+ <p>
+ These are the feature tests found in the Feature directory.
+ They are broken up into the following categories:
+ </p>
+ <ul>
+ <li>ad
+ <p>Assembler/Disassembler tests. These tests verify that a piece of LLVM
+ assembly language can be assembled into bytecode and then disassembled
+ into the original assembly language code. It does this several times to
+ ensure that assembled output can be disassembled and disassembler output
+ can be assembled. It also verifies that the give assembly language file
+ can be assembled correctly.</p></li>
+
+ <li>opt
+ <p>Optimizer tests. These tests verify that two of the optimizer passes
+ completely optimize a program (i.e. after a single pass, they cannot
+ optimize a program any further).</p></li>
+
+ <li>mc
+ <p> Machine code tests. These tests verify that the LLVM assembly
+ language file can be translated into native assembly code.</p></li>
+
+ <li>cc
+ <p>C code tests. These tests verify that the specified LLVM assembly
+ code can be converted into C source code using the C backend.</p></li>
+ </ul>
+
+ <p>The LLVM database class looks at every file in the Feature directory and
+ creates a fake test hierarchy containing
+ <tt>Feature.<testtype>.<testname></tt>. So, if you add an LLVM
+ assembly language file to the Feature directory, it actually creates 5 new
+ tests: assembler/disassembler, assembler, optimizer, machine code, and C code.
+ </p>
+
+ <li>Regression
+ <p>These are the regression tests. There is one suite for each
+ subdirectory of the Regression directory. If you add a new subdirectory
+ there, you will need to modify, at least, the <tt>RegressionMap</tt>
+ variable in <tt>QMTest/llvmdb.py</tt> so that QMTest knows how to run the
+ tests in the new subdirectory.</p>
+
+</ul>
+
+</div>
+
+<!--===============================================================-->
+<div class="doc_section"><a name="progstructure">Programs Structure</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p>As mentioned previously, the Programs tree in llvm/test provides three types
+of tests: MultiSource, SingleSource, and External. Each tree is then subdivided
+into several categories, including applications, benchmarks, regression tests,
+code that is strange grammatically, etc. These organizations should be
+relatively self explanatory.</p>
+
+<p>In addition to the regular Programs tests, the Programs tree also provides a
+mechanism for compiling the programs in different ways. If the variable TEST is
+defined on the gmake command line, the test system will include a Makefile named
+<tt>TEST.<value of TEST variable>.Makefile</tt>. This Makefile can modify
+build rules to yield different results.</p>
+
+<p>For example, the LLVM nightly tester uses <tt>TEST.nightly.Makefile</tt> to
+create the nightly test reports. To run the nightly tests, run <tt>gmake
+TEST=nightly</tt>.</p>
+
+<p>There are several TEST Makefiles available in the tree. Some of them are
+designed for internal LLVM research and will not work outside of the LLVM
+research group. They may still be valuable, however, as a guide to writing your
+own TEST Makefile for any optimization or analysis passes that you develop with
+LLVM.</p>
+
+</div>
-<hr><font size="-1">
-<address>John T. Criswell</address>
-<a href="http://llvm.cs.uiuc.edu">The LLVM Compiler Infrastructure</a>
-<br>
-Last modified: $Date: 2003/11/03 14:59:59 $
-</font>
+<!--===============================================================-->
+<div class="doc_section"><a name="run">Running the LLVM Tests</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p>First, all tests are executed within the LLVM object directory tree. They
+<i>are not</i> executed inside of the LLVM source tree. This is because the
+test suite creates temporary files during execution. </p>
+
+<p>The master Makefile in llvm/test is capable of running both the QMTest driven
+tests and the Programs tests. By default, it will run all of the tests.</p>
+
+<p>To run only the QMTest driven tests, run <tt>gmake qmtest</tt> at the
+command line in llvm/tests. To run a specific qmtest, suffix the test name with
+".t" when running gmake.</p>
+
+<p>For example, to run the Regression.LLC tests, type <tt>gmake
+Regression.LLC.t</tt> in llvm/tests.</p>
+
+<p>Note that the Makefiles in llvm/test/Features and llvm/test/Regression are
+gone. You must now use QMTest from the llvm/test directory to run them.</p>
+
+<p>To run the Programs test, cd into the llvm/test/Programs directory and type
+<tt>gmake</tt>. Alternatively, you can type <tt>gmake TEST=<type>
+test</tt> to run one of the specialized tests in
+llvm/test/Programs/TEST.<type>.Makefile. For example, you could run the
+nightly tester tests using the following commands:</p>
+
+<pre>
+ % cd llvm/test/Programs
+ % gmake TEST=nightly test
+</pre>
+
+<p>Regardless of which test you're running, the results are printed on standard
+output and standard error. You can redirect these results to a file if you
+choose.</p>
+
+<p>Some tests are known to fail. Some are bugs that we have not fixed yet;
+others are features that we haven't added yet (or may never add). In QMTest,
+the result for such tests will be XFAIL (eXpected FAILure). In this way, you
+can tell the difference between an expected and unexpected failure.</p>
+
+<p>The Programs tests have no such feature as of this time. If the test passes,
+only warnings and other miscellaneous output will be generated. If a test
+fails, a large <program> FAILED message will be displayed. This will help
+you separate benign warnings from actual test failures.</p>
+
+</div>
+
+<!-- *********************************************************************** -->
+<hr>
+<address>
+ <a href="http://jigsaw.w3.org/css-validator/check/referer"><img
+ src="http://jigsaw.w3.org/css-validator/images/vcss" alt="Valid CSS!"></a>
+ <a href="http://validator.w3.org/check/referer"><img
+ src="http://www.w3.org/Icons/valid-html401" alt="Valid HTML 4.01!" /></a>
+
+ John T. Criswell<br>
+ <a href="http://llvm.cs.uiuc.edu">The LLVM Compiler Infrastructure</a><br>
+ Last modified: $Date: 2004/03/01 18:21:04 $
+</address>
</body>
</html>
More information about the llvm-commits
mailing list