[llvm-commits] [zorg] r141406 - in /zorg/trunk/lnt: docs/tests.rst lnt/tests/nt.py
Daniel Dunbar
daniel at zuster.org
Fri Oct 7 12:55:18 PDT 2011
Author: ddunbar
Date: Fri Oct 7 14:55:18 2011
New Revision: 141406
URL: http://llvm.org/viewvc/llvm-project?rev=141406&view=rev
Log:
LNT/nt: Pass a few more parameters to the test module (for supporting tests that want to build code) and add documentation on the parameters passed to test modules.
Modified:
zorg/trunk/lnt/docs/tests.rst
zorg/trunk/lnt/lnt/tests/nt.py
Modified: zorg/trunk/lnt/docs/tests.rst
URL: http://llvm.org/viewvc/llvm-project/zorg/trunk/lnt/docs/tests.rst?rev=141406&r1=141405&r2=141406&view=diff
==============================================================================
--- zorg/trunk/lnt/docs/tests.rst (original)
+++ zorg/trunk/lnt/docs/tests.rst Fri Oct 7 14:55:18 2011
@@ -157,5 +157,59 @@
passed the user configuration parameters for a test run and expected to return
back the test results in the LNT native format.
+Test modules are defined by providing a ``TestModule`` file in a subdirectory of
+the ``LNTBased`` root directory inside the LLVM test-suite repository. The
+``TestModule`` file is expected to be a well-formed Python module that provides
+a ``test_class`` global variable which should be a subclass of the
+``lnt.tests.nt.TestModule`` abstract base class.
+
+The test class should override the ``execute_test`` method which is passed an
+options dictionary containg the NT user parameters which apply to test
+execution, and the test should return the test results as a list of
+``lnt.testing.TestSamples`` objects.
+
+The ``execute_test`` method is passed the following options which apply to how
+tests should be executed:
+
+ * ``THREADS`` - The number of parallel processes to run during testing.
+
+ * ``BUILD_THREADS`` - The number of parallel processes to use while building
+ tests (if applicable).
+
+The method is passed the following options which specify how and whether tests
+should be executed remotely. If any of these parameters are present then all are
+guaranteed to be present.
+
+ * ``REMOTE_HOST`` - The host name of the remote machine to execute tests on.
+
+ * ``REMOTE_USER`` - The user to log in to the remote machine as.
+
+ * ``REMOTE_PORT`` - The port to connect to the remote machine on.
+
+ * ``REMOTE_CLIENT`` - The ``rsh`` compatible client to use to connect to the
+ remote machine with.
+
+The method is passed the following options which specify how to build the tests:
+
+ * ``CC`` - The C compiler command to use.
+
+ * ``CXX`` - The C++ compiler command to use.
+
+ * ``CFLAGS`` - The compiler flags to use for building C code.
+
+ * ``CXXFLAGS`` - The compiler flags to use for building C++ code.
+
+The method is passed the following optional parameters which specify the
+environment to use for various commands:
+
+ * ``COMPILE_ENVIRONMENT_OVERRIDES`` [optional] - If given, a ``env`` style list
+ of environment overrides to use when compiling.
+
+ * ``LINK_ENVIRONMENT_OVERRIDES`` [optional] - If given, a ``env`` style list of
+ environment overrides to use when linking.
+
+ * ``EXECUTION_ENVIRONMENT_OVERRIDES`` [optional] - If given, a ``env`` style list of
+ environment overrides to use when executing tests.
+
For more information, see the example tests in the LLVM test-suite repository
under the ``LNT/Examples`` directory.
Modified: zorg/trunk/lnt/lnt/tests/nt.py
URL: http://llvm.org/viewvc/llvm-project/zorg/trunk/lnt/lnt/tests/nt.py?rev=141406&r1=141405&r2=141406&view=diff
==============================================================================
--- zorg/trunk/lnt/lnt/tests/nt.py (original)
+++ zorg/trunk/lnt/lnt/tests/nt.py Fri Oct 7 14:55:18 2011
@@ -13,7 +13,7 @@
import lnt.testing.util.compilers
from lnt.testing.util.commands import note, warning, error, fatal
-from lnt.testing.util.commands import capture, which
+from lnt.testing.util.commands import capture, mkdir_p, which
from lnt.testing.util.rcs import get_source_version
###
@@ -46,6 +46,7 @@
# First, load the test module file.
locals = globals = {}
test_path = os.path.join(opts.test_suite_root, 'LNTBased', name)
+ test_obj_path = os.path.join(basedir, 'LNTBased', name)
module_path = os.path.join(test_path, 'TestModule')
module_file = open(module_path)
try:
@@ -63,9 +64,19 @@
except:
fatal("unable to instantiate test class for: %r" % module_path)
+ if not isinstance(test_instance, TestModule):
+ fatal("invalid test class (expected lnt.tests.nt.TestModule "
+ "subclass) for: %r" % module_path)
+
+ # Create the per test variables, and ensure the output directory exists.
+ variables = test_module_variables.copy()
+ variables['SRCROOT'] = test_path
+ variables['OBJROOT'] = test_obj_path
+ mkdir_p(test_obj_path)
+
# Execute the tests.
try:
- test_samples = test_instance.execute_test(test_module_variables)
+ test_samples = test_instance.execute_test(variables)
except:
info = traceback.format_exc()
fatal("exception executing tests for: %r\n%s" % (
@@ -95,9 +106,10 @@
test_module_variables = {
'CC' : make_variables['TARGET_CC'],
'CXX' : make_variables['TARGET_CXX'],
- 'CFLAGS' : make_variables['TARGET_FLAGS'],
- 'CXXFLAGS' : make_variables['TARGET_FLAGS'],
- 'OPTFLAGS' : make_variables['OPTFLAGS'] }
+ 'CFLAGS' : (make_variables['TARGET_FLAGS'] + ' ' +
+ make_variables['OPTFLAGS']),
+ 'CXXFLAGS' : (make_variables['TARGET_FLAGS'] + ' ' +
+ make_variables['OPTFLAGS']) }
# Add the remote execution variables.
if opts.remote:
@@ -122,6 +134,11 @@
test_module_variables['EXECUTION_ENVIRONMENT_OVERRIDES'] = \
make_variables['EXECUTION_ENVIRONMENT_OVERRIDES']
+ # We pass the test execution values as variables too, this might be better
+ # passed as actual arguments.
+ test_module_variables['THREADS'] = opts.threads
+ test_module_variables['BUILD_THREADS'] = opts.build_threads or opts.threads
+
return test_module_variables
def execute_nt_tests(test_log, make_variables, basedir, opts):
More information about the llvm-commits
mailing list