<div dir="ltr">Hi Alexey,<div><br></div><div style>This shouldn't require adding a new lit option.</div><div style><br></div><div style>Ideally, I would propose the right way to handle this would be to have some way to detect that the compile is done with ASAN support enabled (if the support is enabled with a configure flag, just set a new build variable). For example, see how "llvm_use_intel_jit" gets added to the lit features for LLVM. This way things will "just work" if people build with ASAN enabled.</div>
<div style><br></div><div style>Is there a reason that approach doesn't work here? If not, then I still would prefer using the single existing lit mechanism --param to pass in the parameter.</div><div style><br></div>
<div style> - Daniel</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Feb 15, 2013 at 6:57 AM, Alexey Samsonov <span dir="ltr"><<a href="mailto:samsonov@google.com" target="_blank">samsonov@google.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi ddunbar,<br>
<br>
This patch adds a new option for lit test runner: --exec-feature=<feature><br>
This option allows to specify test execution mode and can be used in REQUIRES:<br>
and XFAIL: lines. Particular use case: we are running LLVM/Clang tests under<br>
AddressSanitizer to find memory errors, and there are some "known" buganized<br>
failures like <a href="http://llvm.org/bugs/show_bug.cgi?id=15130" target="_blank">http://llvm.org/bugs/show_bug.cgi?id=15130</a>. With this patch applied,<br>
we can mark these tests "XFAIL: asan", specify LLVM_LIT_ARGS="--exec-feature=asan"<br>
at configuration time, and keep the build clean.<br>
<br>
<a href="http://llvm-reviews.chandlerc.com/D413" target="_blank">http://llvm-reviews.chandlerc.com/D413</a><br>
<br>
Files:<br>
docs/CommandGuide/lit.rst<br>
utils/lit/lit/TestingConfig.py<br>
utils/lit/lit/discovery.py<br>
utils/lit/lit/LitConfig.py<br>
utils/lit/lit/main.py<br>
<br>
Index: docs/CommandGuide/lit.rst<br>
===================================================================<br>
--- docs/CommandGuide/lit.rst<br>
+++ docs/CommandGuide/lit.rst<br>
@@ -122,6 +122,12 @@<br>
suite take the most time to execute. Note that this option is most useful<br>
with ``-j 1``.<br>
<br>
+.. option:: --exec-feature=FEATURE<br>
+<br>
+ Specify an additional feature of test execution that can be matched in REQUIRES<br>
+ and XFAIL lines. For example, if you run tests under AddressSanitizer, you can<br>
+ specify ``--exec-feature=asan``.<br>
+<br>
.. _selection-options:<br>
<br>
SELECTION OPTIONS<br>
Index: utils/lit/lit/TestingConfig.py<br>
===================================================================<br>
--- utils/lit/lit/TestingConfig.py<br>
+++ utils/lit/lit/TestingConfig.py<br>
@@ -29,13 +29,6 @@<br>
'TMP' : os.environ.get('TMP',''),<br>
})<br>
<br>
- # Set the default available features based on the LitConfig.<br>
- available_features = []<br>
- if litConfig.useValgrind:<br>
- available_features.append('valgrind')<br>
- if litConfig.valgrindLeakCheck:<br>
- available_features.append('vg_leak')<br>
-<br>
config = TestingConfig(parent,<br>
name = '<unnamed>',<br>
suffixes = set(),<br>
@@ -47,7 +40,7 @@<br>
test_exec_root = None,<br>
test_source_root = None,<br>
excludes = [],<br>
- available_features = available_features)<br>
+ available_features = litConfig.execFeatures)<br>
<br>
if os.path.exists(path):<br>
# FIXME: Improve detection and error reporting of errors in the<br>
Index: utils/lit/lit/discovery.py<br>
===================================================================<br>
--- utils/lit/lit/discovery.py<br>
+++ utils/lit/lit/discovery.py<br>
@@ -222,6 +222,7 @@<br>
valgrindLeakCheck = False,<br>
valgrindArgs = [],<br>
noExecute = False,<br>
+ execFeatures = [],<br>
ignoreStdErr = False,<br>
debug = False,<br>
isWindows = (platform.system()=='Windows'),<br>
Index: utils/lit/lit/LitConfig.py<br>
===================================================================<br>
--- utils/lit/lit/LitConfig.py<br>
+++ utils/lit/lit/LitConfig.py<br>
@@ -19,7 +19,7 @@<br>
<br>
def __init__(self, progname, path, quiet,<br>
useValgrind, valgrindLeakCheck, valgrindArgs,<br>
- noExecute, ignoreStdErr, debug, isWindows,<br>
+ noExecute, execFeatures, ignoreStdErr, debug, isWindows,<br>
params, config_prefix = None):<br>
# The name of the test runner.<br>
self.progname = progname<br>
@@ -30,6 +30,7 @@<br>
self.valgrindLeakCheck = bool(valgrindLeakCheck)<br>
self.valgrindUserArgs = list(valgrindArgs)<br>
self.noExecute = noExecute<br>
+ self.execFeatures = list(execFeatures)<br>
self.ignoreStdErr = ignoreStdErr<br>
self.debug = debug<br>
self.isWindows = bool(isWindows)<br>
@@ -47,10 +48,12 @@<br>
<br>
self.valgrindArgs = []<br>
if self.useValgrind:<br>
+ self.execFeatures.append('valgrind')<br>
self.valgrindArgs = ['valgrind', '-q', '--run-libc-freeres=no',<br>
'--tool=memcheck', '--trace-children=yes',<br>
'--error-exitcode=123']<br>
if self.valgrindLeakCheck:<br>
+ self.execFeatures.append('vg_leak')<br>
self.valgrindArgs.append('--leak-check=full')<br>
else:<br>
# The default is 'summary'.<br>
Index: utils/lit/lit/main.py<br>
===================================================================<br>
--- utils/lit/lit/main.py<br>
+++ utils/lit/lit/main.py<br>
@@ -207,6 +207,10 @@<br>
group.add_option("", "--no-execute", dest="noExecute",<br>
help="Don't execute any tests (assume PASS)",<br>
action="store_true", default=False)<br>
+ group.add_option("", "--exec-feature", dest="execFeatures",<br>
+ metavar="FEATURE",<br>
+ help="Add user-defined feature of test execution",<br>
+ type=str, action="append", default=[])<br>
parser.add_option_group(group)<br>
<br>
group = OptionGroup(parser, "Test Selection")<br>
@@ -271,6 +275,7 @@<br>
valgrindLeakCheck = opts.valgrindLeakCheck,<br>
valgrindArgs = opts.valgrindArgs,<br>
noExecute = opts.noExecute,<br>
+ execFeatures = opts.execFeatures,<br>
ignoreStdErr = False,<br>
debug = opts.debug,<br>
isWindows = (platform.system()=='Windows'),<br>
</blockquote></div><br></div>