[llvm-commits] CVS: llvm/docs/TestingGuide.html

Misha Brukman brukman at cs.uiuc.edu
Mon Mar 1 12:22:01 PST 2004


Changes in directory llvm/docs:

TestingGuide.html updated: 1.6 -> 1.7

---
Log message:

* HTML 4.01 Strict compliance
* Removed unnecessary tabs in the entire file


---
Diffs of the changes:  (+355 -401)

Index: llvm/docs/TestingGuide.html
diff -u llvm/docs/TestingGuide.html:1.6 llvm/docs/TestingGuide.html:1.7
--- llvm/docs/TestingGuide.html:1.6	Mon Nov  3 08:59:59 2003
+++ llvm/docs/TestingGuide.html	Mon Mar  1 12:21:04 2004
@@ -1,11 +1,10 @@
-<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
+                      "http://www.w3.org/TR/html4/strict.dtd">
 <html>
 <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
-    <link rel="stylesheet" href="llvm.css" type="text/css" media="screen" />
-    <title>LLVM Test Suite Guide</title>
+  <title>LLVM Test Suite Guide</title>
+  <link rel="stylesheet" href="llvm.css" type="text/css">
 </head>
-
 <body>
       
 <div class="doc_title">
@@ -16,414 +15,369 @@
 <li><a href="#overview">Overview</a></li>
 <li><a href="#Requirements">Requirements</a></li>
 <li><a href="#quick">Quick Start</a></li>
-<li><a href="#org">LLVM Test Suite Organization</a></li>
+<li><a href="#org">LLVM Test Suite Organization</a>
 <ul>
   <li><a href="#codefragments">Code Fragments</a></li>
   <li><a href="#wholeprograms">Whole Programs</a></li>
-</ul>
+</ul></li>
 <li><a href="#tree">LLVM Test Suite Tree</a></li>
 <li><a href="#qmstructure">QMTest Structure</a></li>
 <li><a href="#progstructure">Programs Structure</a></li>
 <li><a href="#run">Running the LLVM Tests</a></li>
-<p><b>Written by John T. Criswell</b></p>
 </ol>
 
-	<!--===============================================================-->
-	<div class="doc_section"><a name="overview">Overview</a></div>
-	<!--===============================================================-->
-
-        <div class="doc_text">
-	<p>
-        This document is the reference manual for the LLVM test suite.  It
-	documents the structure of the LLVM test suite, the tools needed to
-	use it, and how to add and run tests.
-        </p>
-        </div>
-
-	<!--===============================================================-->
-	<div class="doc_section"><a name="Requirements">Requirements</a></div>
-	<!--===============================================================-->
-
-        <div class="doc_text">
-        <p>
-	In order to use the LLVM test suite, you will need all of the software
-	required to build LLVM, plus the following:
-        </p>
-	<dl compact>
-		<dt><A HREF="http://www.qmtest.com">QMTest</A></dt>
-		<dd>The LLVM test suite uses QMTest to organize and
-                run tests.</dd>
-
-		<dt><A HREF="http://www.python.org">Python</A></dt>
-		<dd>You will need a Python interpreter that works with
-                QMTest. Python will need zlib and SAX support
-                enabled.</dd>
-	</dl>
-        </div>
-
-	<!--===============================================================-->
-	<div class="doc_section"><a name="quick">Quick Start</a></div>
-	<!--===============================================================-->
-
-        <div class="doc_text">
-        <p>
-        The tests are located in the LLVM source tree under the directory
-        <tt>llvm/test</tt>. To run all of the tests in LLVM, use the Master
-        Makefile in that directory:
-	</p>
-	<pre>
-	 % gmake -C llvm/test
-	</pre>
-
-	<p>
-	To run only the code fragment tests (i.e. those that do basic testing of
-	LLVM), run the tests organized by QMTest:
-	</p>
-
-	<pre>
-	 % gmake -C llvm/test qmtest
-	</pre>
-
-	<p>
-	To run only the tests that compile and execute whole programs, run the
-	Programs tests:
-	</p>
-
-	<pre>
-	 % gmake -C llvm/test/Programs
-	</pre>
-        </div>
-
-	<!--===============================================================-->
-	<div class="doc_section"><h2><a name="org">LLVM Test Suite
-        Organization </a></h2></div>
-	<!--===============================================================-->
-
-        <div class="doc_text">
-	<p>The LLVM test suite contains two major categories of tests: code
-        fragments and whole programs.</p>
-        </div>
-
-        <div class="doc_subsection"><a name="codefragments">Code Fragments</a> 
-        </div>
-
-        <div class="doc_text">
-                <p>
-		Code fragments are small pieces of code that test a specific
-		feature of LLVM or trigger a specific bug in LLVM.  They are
-		usually written in LLVM assembly language, but can be
-		written in other languages if the test targets a
-                particular language front end.
-		</p><p>
-		Code fragments are not complete programs, and they are
-                never executed to determine correct behavior.
-		</p><p>
-		The tests in the Features and
-                Regression directories contain code fragments.
-                </p>
-        </div>
-
-        <div class="doc_subsection"><a name="wholeprograms">Whole Programs</a> 
-        </div>
-
-        <div class="doc_text">
-		<p>
-                Whole Programs are pieces of code which can be compiled and
-                linked into a stand-alone program that can be executed.  These
-                programs are generally written in high level languages such as C
-                or C++, but sometimes they are written straight in LLVM
-                assembly.
-		</p><p>
-		These programs are compiled and then executed using several
-		different methods (native compiler, LLVM C backend, LLVM JIT,
-		LLVM native code generation, etc).  The output of these programs
-		is compared to ensure that LLVM is compiling the program
-		correctly.
-		</p><p>
-		In addition to compiling and executing programs, whole program
-		tests serve as a way of benchmarking LLVM performance, both in
-		terms of the efficiency of the programs generated as well as the
-		speed with which LLVM compiles, optimizes, and generates code.
-		</p><p>
-		The Programs directory contains all tests which compile and
-		benchmark whole programs.
-                </p>
-        </div>
-
-	<!--===============================================================-->
-	<div class="doc_section"><h2><a name="tree">LLVM Test Suite Tree</a>
-        </div>
-	<!--===============================================================-->
-
-        <div class="doc_text">
-	<p>Each type of test in the LLVM test suite has its own directory. The
-        major subtrees of the test suite directory tree are as follows:</p>
+<p><b>Written by John T. Criswell</b></p>
+
+<!--===============================================================-->
+<div class="doc_section"><a name="overview">Overview</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p>This document is the reference manual for the LLVM test suite.  It documents
+the structure of the LLVM test suite, the tools needed to use it, and how to add
+and run tests.</p>
+
+</div>
+
+<!--===============================================================-->
+<div class="doc_section"><a name="Requirements">Requirements</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p>In order to use the LLVM test suite, you will need all of the software
+required to build LLVM, plus the following:</p>
+
+<dl>
+  <dt><a href="http://www.qmtest.com">QMTest</A></dt>
+  <dd>The LLVM test suite uses QMTest to organize and run tests.</dd>
+
+  <dt><a href="http://www.python.org">Python</A></dt>
+  <dd>You will need a Python interpreter that works with QMTest. Python will
+  need zlib and SAX support enabled.</dd>
+</dl>
+
+</div>
+
+<!--===============================================================-->
+<div class="doc_section"><a name="quick">Quick Start</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p> The tests are located in the LLVM source tree under the directory
+<tt>llvm/test</tt>. To run all of the tests in LLVM, use the Master Makefile in
+that directory:</p>
+
+<pre>
+ % gmake -C llvm/test
+</pre>
+
+<p>To run only the code fragment tests (i.e. those that do basic testing of
+LLVM), run the tests organized by QMTest:</p>
+
+<pre>
+ % gmake -C llvm/test qmtest
+</pre>
+
+<p>To run only the tests that compile and execute whole programs, run the
+Programs tests:</p>
+
+<pre>
+ % gmake -C llvm/test/Programs
+</pre>
+
+</div>
+
+<!--===============================================================-->
+<div class="doc_section"><a name="org">LLVM Test Suite Organization</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p>The LLVM test suite contains two major categories of tests: code
+fragments and whole programs.</p>
+
+</div>
+
+<div class="doc_subsection"><a name="codefragments">Code Fragments</a> 
+</div>
+
+<div class="doc_text">
+
+<p>Code fragments are small pieces of code that test a specific feature of LLVM
+or trigger a specific bug in LLVM.  They are usually written in LLVM assembly
+language, but can be written in other languages if the test targets a particular
+language front end.</p>
+
+<p>Code fragments are not complete programs, and they are never executed to
+determine correct behavior.</p> 
+
+<p>The tests in the Features and Regression directories contain code
+fragments.</p>
+
+</div>
+
+<div class="doc_subsection"><a name="wholeprograms">Whole Programs</a></div>
+
+<div class="doc_text">
+
+<p>Whole Programs are pieces of code which can be compiled and linked into a
+stand-alone program that can be executed.  These programs are generally written
+in high level languages such as C or C++, but sometimes they are written
+straight in LLVM assembly.</p>
+  
+<p>These programs are compiled and then executed using several different
+methods (native compiler, LLVM C backend, LLVM JIT, LLVM native code generation,
+etc).  The output of these programs is compared to ensure that LLVM is compiling
+the program correctly.</p>
+
+<p>In addition to compiling and executing programs, whole program tests serve as
+a way of benchmarking LLVM performance, both in terms of the efficiency of the
+programs generated as well as the speed with which LLVM compiles, optimizes, and
+generates code.</p>
+
+<p>The Programs directory contains all tests which compile and benchmark whole
+programs.</p> 
+
+</div>
+
+<!--===============================================================-->
+<div class="doc_section"><a name="tree">LLVM Test Suite Tree</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p>Each type of test in the LLVM test suite has its own directory. The major
+subtrees of the test suite directory tree are as follows:</p>
+      
+<ul>
+  <li>Features 
+    <p>This directory contains sample codes that test various features of the
+    LLVM language.  These pieces of sample code are run through various
+    assembler, disassembler, and optimizer passes.</p>
+
+  <li>Regression
+    <p>This directory contains regression tests for LLVM.  When a bug is found
+    in LLVM, a regression test containing just enough code to reproduce the
+    problem should be written and placed somewhere underneath this directory.
+    In most cases, this will be a small piece of LLVM assembly language code,
+    often distilled from an actual application or benchmark.</p>
+
+  <li>Programs
+    <p>The Programs directory contains programs that can be compiled with LLVM
+    and executed.  These programs are compiled using the native compiler and
+    various LLVM backends.  The output from the program compiled with the native
+    compiler is assumed correct; the results from the other programs are
+    compared to the native program output and pass if they match.  </p>
+  
+    <p> In addition for testing correctness, the Programs directory also
+    performs timing tests of various LLVM optimizations.  It also records
+    compilation times for the compilers and the JIT.  This information can be
+    used to compare the effectiveness of LLVM's optimizations and code
+    generation.</p>
+    
+    <p>The Programs directory is subdivided into several smaller subdirectories:
+    </p>
+
+    <ul>
+      <li>Programs/SingleSource
+        <p>The SingleSource directory contains test programs that are only a
+        single source file in size.  These are usually small benchmark programs
+        or small programs that calculate a particular value.  Several such
+        programs are grouped together in each directory.</p></li>
+
+      <li>Programs/MultiSource
+        <p>The MultiSource directory contains subdirectories which contain
+        entire programs with multiple source files.  Large benchmarks and whole
+        applications go here.</p></li>
+
+      <li>Programs/External
+        <p>The External directory contains Makefiles for building code that is
+        external to (i.e. not distributed with) LLVM.  The most prominent member
+        of this directory is the SPEC 2000 benchmark suite.  The presence and
+        location of these external programs is configured by the LLVM
+        <tt>configure</tt> script.</p></li>
         
-	<ul>
-		<li>Features
-		<p>
-                This directory contains sample codes that test various features
-		of the LLVM language.  These pieces of sample code are run
-		through various assembler, disassembler, and optimizer passes.
-		</p>
-
-		<li>Regression
-                <p>
-		This directory contains regression tests for LLVM.  When a bug
-		is found in LLVM, a regression test containing just enough
-		code to reproduce the problem should be written and placed
-		somewhere underneath this directory.  In most cases, this
-		will be a small piece of LLVM assembly language code, often
-		distilled from an actual application or benchmark.
-		</p>
-
-		<li>Programs
-		<p>
-                The Programs directory contains programs that can be compiled
-		with LLVM and executed.  These programs are compiled using the
-		native compiler and various LLVM backends.  The output from the
-		program compiled with the native compiler is assumed correct;
-		the results from the other programs are compared to the native
-		program output and pass if they match.
-		</p><p>
-		In addition for testing correctness, the Programs directory
-		also performs timing tests of various LLVM optimizations.
-		It also records compilation times for the compilers and the
-		JIT.  This information can be used to compare the
-		effectiveness of LLVM's optimizations and code generation.
-		</p><p>
-		The Programs directory is subdivided into several smaller
-		subdirectories:
-                </p>
-
-		<ul>
-			<li>Programs/SingleSource
-                        <p>
-			The SingleSource directory contains test programs that
-			are only a single source file in size.  These are
-			usually small benchmark programs or small programs that
-			calculate a particular value.  Several such programs are
-			grouped together in each directory.
-			</p>
-
-			<li>Programs/MultiSource
-                        <p>
-                        The MultiSource directory contains subdirectories which
-                        contain entire programs with multiple source files.
-                        Large benchmarks and whole applications go here.
-			</p>
-
-			<li>Programs/External
-                        <p>
-			The External directory contains Makefiles for building
-			code that is external to (i.e. not distributed with)
-			LLVM.  The most prominent member of this directory is
-			the SPEC 2000 benchmark suite.  The presence and
-			location of these external programs is configured by the
-			LLVM <tt>configure</tt> script.
-                        </p>
-		</ul>
-
-		<p>
-
-		<li>QMTest
-                <p>
-		This directory contains the QMTest information files.  Inside
-		this directory are QMTest administration files and the Python
-		code that implements the LLVM test and database classes.
-                </p>
-	</ul>
-        </div>
-
-	<!--===============================================================-->
-	<div class="doc_section"><h2><a name="qmstructure">QMTest Structure</a>
-        </div>
-	<!--===============================================================-->
-
-        <div class="doc_text">
-	<p>
-        The LLVM test suite is partially driven by QMTest and partially
-	driven by GNU Make.  Specifically, the Features and Regression tests
-	are all driven by QMTest.  The Programs directory is currently
-	driven by a set of Makefiles.
-	</p><p>
-	The QMTest system needs to have several pieces of information
-	available; these pieces of configuration information are known
-	collectively as the "context" in QMTest parlance.  Since the context
-	for LLVM is relatively large, the master Makefile in llvm/test
-	sets it for you.
-	</p><p>
-	The LLVM database class makes the subdirectories of llvm/test a
-	QMTest test database.  For each directory that contains tests driven by
-	QMTest, it knows what type of test the source file is and how to run it.
-	</p><p>
-	Hence, the QMTest namespace is essentially what you see in the
-	Feature and Regression directories, but there is some magic that
-	the database class performs (as described below).
-	</p><p>
-	The QMTest namespace is currently composed of the following tests and
-	test suites:
-        </p>
-
-	<ul>
-		<li>Feature
-                <p>
-		These are the feature tests found in the Feature directory.
-		They are broken up into the following categories:
-                </p>
-		<ul>
-			<li>ad
-			<p>
-			Assembler/Disassembler tests.  These tests verify that a
-			piece of LLVM assembly language can be assembled into
-			bytecode and then disassembled into the original
-			assembly language code.  It does this several times to
-			ensure that assembled output can be disassembled and
-			disassembler output can be assembled.  It also verifies
-			that the give assembly language file can be assembled
-			correctly.
-                        </p>
-
-			<li>opt
-			<p>
-			Optimizer tests.  These tests verify that two of the
-			optimizer passes completely optimize a program (i.e.
-			after a single pass, they cannot optimize a program
-			any further).
-			</p>
-
-			<li>mc
-			<p>
-			Machine code tests.  These tests verify that the LLVM
-			assembly language file can be translated into native
-			assembly code.
-			</p>
-
-			<li>cc
-			<p>
-			C code tests.  These tests verify that the specified
-			LLVM assembly code can be converted into C source code
-			using the C backend.
-                        </p>
-		</ul>
-
-		<p>
-		The LLVM database class looks at every file in the Feature
-		directory and creates a fake test hierarchy containing
-		<tt>Feature.<testtype>.<testname></tt>.  So, if you
-		add an LLVM assembly language file to the Feature directory, it
-		actually creates 5 new tests: assembler/disassembler, assembler,
-		optimizer, machine code, and C code.
-                </p>
-
-		<li>Regression
-                <p>
-		These are the regression tests.  There is one suite for each
-		subdirectory of the Regression directory.  If you add a new
-		subdirectory there, you will need to modify, at least, the
-		<tt>RegressionMap</tt> variable in <tt>QMTest/llvmdb.py</tt> so
-		that QMTest knows how to run the tests in the new subdirectory.
-                </p>
-	</ul>
-        </div>
-
-	<!--===============================================================-->
-	<div class="doc_section"><h2><a name="progstructure">Programs
-        Structure</a></div>
-	<!--===============================================================-->
-
-        <div class="doc_text">
-        <p>
-	As mentioned previously, the Programs tree in llvm/test provides three
-	types of tests: MultiSource, SingleSource, and External.  Each tree is
-	then subdivided into several categories, including applications,
-	benchmarks, regression tests, code that is strange grammatically, etc.
-	These organizations should be relatively self explanatory.
-	</p><p>
-	In addition to the regular Programs tests, the Programs tree also
-	provides a mechanism for compiling the programs in different ways.  If
-	the variable TEST is defined on the gmake command line, the test system
-	will include a Makefile named <tt>TEST.<value of TEST
-	variable>.Makefile</tt>.  This Makefile can modify build rules to
-        yield different results.
-	</p><p>
-	For example, the LLVM nightly tester uses <tt>TEST.nightly.Makefile</tt>
-        to create the nightly test reports.  To run the nightly tests, run
-	<tt>gmake TEST=nightly</tt>.
-	</p><p>
-	There are several TEST Makefiles available in the tree.  Some of them
-	are designed for internal LLVM research and will not work outside of the
-	LLVM research group.  They may still be valuable, however, as a guide to
-	writing your own TEST Makefile for any optimization or analysis passes
-	that you develop with LLVM.
-        </p>
-        </div>
-
-	<!--===============================================================-->
-	<div class="doc_section"><h2><a name="run">Running the LLVM Tests</a>
-        </div>
-	<!--===============================================================-->
-
-        <div class="doc_text">
-        <p>
-	First, all tests are executed within the LLVM object directory tree.
-	They <i>are not</i> executed inside of the LLVM source tree.  This is
-	because the test suite creates temporary files during execution.
-	</p><p>
-	The master Makefile in llvm/test is capable of running both the
-	QMTest driven tests and the Programs tests.  By default, it will run
-	all of the tests.
-	</p><p>
-	To run only the QMTest driven tests, run <tt>gmake qmtest</tt> at the
-	command line in llvm/tests.  To run a specific qmtest, suffix the test
-	name with ".t" when running gmake.
-	</p><p>
-	For example, to run the Regression.LLC tests, type
-	<tt>gmake Regression.LLC.t</tt> in llvm/tests.
-	</p><p>
-	Note that the Makefiles in llvm/test/Features and llvm/test/Regression
-	are gone.  You must now use QMTest from the llvm/test directory to run
-	them.
-	</p><p>
-	To run the Programs test, cd into the llvm/test/Programs directory and
-	type <tt>gmake</tt>.  Alternatively, you can type <tt>gmake
-	TEST=<type> test</tt> to run one of the specialized tests in
-	llvm/test/Programs/TEST.<type>.Makefile.  For example, you could
-	run the nightly tester tests using the following commands:
-	</p>
-
-	<pre>
-	 % cd llvm/test/Programs
-	 % gmake TEST=nightly test
-	</pre>
-
-	<p>
-	Regardless of which test you're running, the results are printed on
-	standard output and standard error.  You can redirect these results to a
-	file if you choose.
-	</p><p>
-	Some tests are known to fail.  Some are bugs that we have not fixed yet;
-	others are features that we haven't added yet (or may never add).  In
-	QMTest, the result for such tests will be XFAIL (eXpected FAILure).  In
-	this way, you can tell the difference between an expected and unexpected
-	failure.
-	</p><p>
-	The Programs tests have no such feature as of this time.  If the test
-	passes, only warnings and other miscellaneous output will be generated.
-	If a test fails, a large <program> FAILED message will be
-	displayed.  This will help you separate benign warnings from actual test
-	failures.
-        </p>
-        </div>
+    </ul></li>
 
-<!-- *********************************************************************** -->
+  <li>QMTest
+    <p>This directory contains the QMTest information files.  Inside this
+    directory are QMTest administration files and the Python code that
+    implements the LLVM test and database classes.</p>
+
+</ul>
+
+</div>
+
+<!--===============================================================-->
+<div class="doc_section"><a name="qmstructure">QMTest Structure</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p>The LLVM test suite is partially driven by QMTest and partially
+driven by GNU Make.  Specifically, the Features and Regression tests
+are all driven by QMTest.  The Programs directory is currently
+driven by a set of Makefiles.</p>
+
+<p>The QMTest system needs to have several pieces of information
+available; these pieces of configuration information are known
+collectively as the "context" in QMTest parlance.  Since the context
+for LLVM is relatively large, the master Makefile in llvm/test
+sets it for you.</p>
+
+<p>The LLVM database class makes the subdirectories of llvm/test a
+QMTest test database.  For each directory that contains tests driven by
+QMTest, it knows what type of test the source file is and how to run it.</p>
+
+<p>Hence, the QMTest namespace is essentially what you see in the
+Feature and Regression directories, but there is some magic that
+the database class performs (as described below).</p>
+
+<p>The QMTest namespace is currently composed of the following tests and test
+suites:</p>
+
+<ul>
+  <li>Feature
+              <p>
+  These are the feature tests found in the Feature directory.
+  They are broken up into the following categories:
+              </p>
+  <ul>
+    <li>ad
+      <p>Assembler/Disassembler tests.  These tests verify that a piece of LLVM
+      assembly language can be assembled into bytecode and then disassembled
+      into the original assembly language code.  It does this several times to
+      ensure that assembled output can be disassembled and disassembler output
+      can be assembled.  It also verifies that the give assembly language file
+      can be assembled correctly.</p></li>
+
+    <li>opt
+      <p>Optimizer tests.  These tests verify that two of the optimizer passes
+      completely optimize a program (i.e.  after a single pass, they cannot
+      optimize a program any further).</p></li>
+
+    <li>mc
+      <p> Machine code tests.  These tests verify that the LLVM assembly
+      language file can be translated into native assembly code.</p></li>
+
+    <li>cc
+      <p>C code tests.  These tests verify that the specified LLVM assembly
+      code can be converted into C source code using the C backend.</p></li>
+  </ul>
+
+  <p>The LLVM database class looks at every file in the Feature directory and
+  creates a fake test hierarchy containing
+  <tt>Feature.<testtype>.<testname></tt>.  So, if you add an LLVM
+  assembly language file to the Feature directory, it actually creates 5 new
+  tests: assembler/disassembler, assembler, optimizer, machine code, and C code.
+  </p>
+
+  <li>Regression
+    <p>These are the regression tests.  There is one suite for each
+    subdirectory of the Regression directory.  If you add a new subdirectory
+    there, you will need to modify, at least, the <tt>RegressionMap</tt>
+    variable in <tt>QMTest/llvmdb.py</tt> so that QMTest knows how to run the
+    tests in the new subdirectory.</p>
+
+</ul>
+      
+</div>
+
+<!--===============================================================-->
+<div class="doc_section"><a name="progstructure">Programs Structure</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p>As mentioned previously, the Programs tree in llvm/test provides three types
+of tests: MultiSource, SingleSource, and External.  Each tree is then subdivided
+into several categories, including applications, benchmarks, regression tests,
+code that is strange grammatically, etc.  These organizations should be
+relatively self explanatory.</p>
+
+<p>In addition to the regular Programs tests, the Programs tree also provides a
+mechanism for compiling the programs in different ways.  If the variable TEST is
+defined on the gmake command line, the test system will include a Makefile named
+<tt>TEST.<value of TEST variable>.Makefile</tt>.  This Makefile can modify
+build rules to yield different results.</p>
+
+<p>For example, the LLVM nightly tester uses <tt>TEST.nightly.Makefile</tt> to
+create the nightly test reports.  To run the nightly tests, run <tt>gmake
+TEST=nightly</tt>.</p>
+
+<p>There are several TEST Makefiles available in the tree.  Some of them are
+designed for internal LLVM research and will not work outside of the LLVM
+research group.  They may still be valuable, however, as a guide to writing your
+own TEST Makefile for any optimization or analysis passes that you develop with
+LLVM.</p>
+      
+</div>
 
-<hr><font size="-1">
-<address>John T. Criswell</address>
-<a href="http://llvm.cs.uiuc.edu">The LLVM Compiler Infrastructure</a>
-<br>
-Last modified: $Date: 2003/11/03 14:59:59 $
-</font>
+<!--===============================================================-->
+<div class="doc_section"><a name="run">Running the LLVM Tests</a></div>
+<!--===============================================================-->
+
+<div class="doc_text">
+
+<p>First, all tests are executed within the LLVM object directory tree.  They
+<i>are not</i> executed inside of the LLVM source tree.  This is because the
+test suite creates temporary files during execution.  </p>
+
+<p>The master Makefile in llvm/test is capable of running both the QMTest driven
+tests and the Programs tests.  By default, it will run all of the tests.</p>
+
+<p>To run only the QMTest driven tests, run <tt>gmake qmtest</tt> at the
+command line in llvm/tests.  To run a specific qmtest, suffix the test name with
+".t" when running gmake.</p>
+
+<p>For example, to run the Regression.LLC tests, type <tt>gmake
+Regression.LLC.t</tt> in llvm/tests.</p>
+
+<p>Note that the Makefiles in llvm/test/Features and llvm/test/Regression are
+gone. You must now use QMTest from the llvm/test directory to run them.</p>
+
+<p>To run the Programs test, cd into the llvm/test/Programs directory and type
+<tt>gmake</tt>.  Alternatively, you can type <tt>gmake TEST=<type>
+test</tt> to run one of the specialized tests in
+llvm/test/Programs/TEST.<type>.Makefile. For example, you could run the
+nightly tester tests using the following commands:</p>
+
+<pre>
+ % cd llvm/test/Programs
+ % gmake TEST=nightly test
+</pre>
+
+<p>Regardless of which test you're running, the results are printed on standard
+output and standard error.  You can redirect these results to a file if you
+choose.</p>
+
+<p>Some tests are known to fail.  Some are bugs that we have not fixed yet;
+others are features that we haven't added yet (or may never add).  In QMTest,
+the result for such tests will be XFAIL (eXpected FAILure).  In this way, you
+can tell the difference between an expected and unexpected failure.</p>
+
+<p>The Programs tests have no such feature as of this time. If the test passes,
+only warnings and other miscellaneous output will be generated.  If a test
+fails, a large <program> FAILED message will be displayed.  This will help
+you separate benign warnings from actual test failures.</p>
+      
+</div>
+
+<!-- *********************************************************************** -->
 
+<hr>
+<address>
+  <a href="http://jigsaw.w3.org/css-validator/check/referer"><img
+  src="http://jigsaw.w3.org/css-validator/images/vcss" alt="Valid CSS!"></a>
+  <a href="http://validator.w3.org/check/referer"><img
+  src="http://www.w3.org/Icons/valid-html401" alt="Valid HTML 4.01!" /></a>
+
+  John T. Criswell<br>
+  <a href="http://llvm.cs.uiuc.edu">The LLVM Compiler Infrastructure</a><br>
+  Last modified: $Date: 2004/03/01 18:21:04 $
+</address>
 </body>
 </html>





More information about the llvm-commits mailing list