On Thu, Oct 18, 2012 at 1:43 PM, Daniel Dunbar <span dir="ltr"><<a href="mailto:daniel@zuster.org" target="_blank" class="cremed">daniel@zuster.org</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: ddunbar<br>
Date: Thu Oct 18 15:43:04 2012<br>
New Revision: 166224<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=166224&view=rev" target="_blank" class="cremed">http://llvm.org/viewvc/llvm-project?rev=166224&view=rev</a><br>
Log:<br>
lit: Allow XFAIL: lines to also refer to "features".<br></blockquote><div><br></div><div>What's the motivation for this patch?</div><div><br></div><div>I would much rather continue to use directory structures and/or REQUIRES to represent feature differences. This seems to add still more complexity to the interface for writing a lit tests, and I'm worried about that already and so interested in scrutinizing the use cases here....</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Added:<br>
llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c<br>
Modified:<br>
llvm/trunk/docs/TestingGuide.html<br>
llvm/trunk/utils/lit/lit/TestRunner.py<br>
<br>
Modified: llvm/trunk/docs/TestingGuide.html<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/TestingGuide.html?rev=166224&r1=166223&r2=166224&view=diff" target="_blank" class="cremed">http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/TestingGuide.html?rev=166224&r1=166223&r2=166224&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/docs/TestingGuide.html (original)<br>
+++ llvm/trunk/docs/TestingGuide.html Thu Oct 18 15:43:04 2012<br>
@@ -798,14 +798,15 @@<br>
<p>Sometimes it is necessary to mark a test case as "expected fail" or XFAIL.<br>
You can easily mark a test as XFAIL just by including <tt>XFAIL: </tt> on a<br>
line near the top of the file. This signals that the test case should succeed<br>
- if the test fails. Such test cases are counted separately by the testing tool. To<br>
- specify an expected fail, use the XFAIL keyword in the comments of the test<br>
- program followed by a colon and one or more regular expressions (separated by<br>
- a comma). The regular expressions allow you to XFAIL the test conditionally by<br>
- host platform. The regular expressions following the : are matched against the<br>
- target triplet for the host machine. If there is a match, the test is expected<br>
- to fail. If not, the test is expected to succeed. To XFAIL everywhere just<br>
- specify <tt>XFAIL: *</tt>. Here is an example of an <tt>XFAIL</tt> line:</p><br>
+ if the test fails. Such test cases are counted separately by the testing<br>
+ tool. To specify an expected fail, use the XFAIL keyword in the comments of<br>
+ the test program followed by a colon and one or more failure patterns. Each<br>
+ failure pattern can be either '*' (to specify fail everywhere), or a part of a<br>
+ target triple (indicating the test should fail on that platfomr), or the name<br>
+ of a configurable feature (for example, "loadable_module").. If there is a<br>
+ match, the test is expected to fail. If not, the test is expected to<br>
+ succeed. To XFAIL everywhere just specify <tt>XFAIL: *</tt>. Here is an<br>
+ example of an <tt>XFAIL</tt> line:</p><br>
<br>
<div class="doc_code"><br>
<pre><br>
<br>
Added: llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c?rev=166224&view=auto" target="_blank" class="cremed">http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c?rev=166224&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c (added)<br>
+++ llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c Thu Oct 18 15:43:04 2012<br>
@@ -0,0 +1,4 @@<br>
+// This test should XPASS.<br>
+<br>
+// RUN: true<br>
+// XFAIL: some-feature-name<br>
<br>
Modified: llvm/trunk/utils/lit/lit/TestRunner.py<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/TestRunner.py?rev=166224&r1=166223&r2=166224&view=diff" target="_blank" class="cremed">http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/TestRunner.py?rev=166224&r1=166223&r2=166224&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/utils/lit/lit/TestRunner.py (original)<br>
+++ llvm/trunk/utils/lit/lit/TestRunner.py Thu Oct 18 15:43:04 2012<br>
@@ -370,10 +370,15 @@<br>
<br>
return executeCommand(command, cwd=cwd, env=test.config.environment)<br>
<br>
-def isExpectedFail(xfails, xtargets, target_triple):<br>
- # Check if any xfail matches this target.<br>
+def isExpectedFail(test, xfails, xtargets):<br>
+ # If the xfail matches an available feature, it always fails.<br>
for item in xfails:<br>
- if item == '*' or item in target_triple:<br>
+ if item in test.config.available_features:<br>
+ return True<br>
+<br>
+ # Otherwise, check if any xfail matches this target.<br>
+ for item in xfails:<br>
+ if item == '*' or item in test.suite.config.target_triple:<br>
break<br>
else:<br>
return False<br>
@@ -382,7 +387,7 @@<br>
#<br>
# FIXME: Rename XTARGET to something that makes sense, like XPASS.<br>
for item in xtargets:<br>
- if item == '*' or item in target_triple:<br>
+ if item == '*' or item in test.suite.config.target_triple:<br>
return False<br>
<br>
return True<br>
@@ -491,7 +496,7 @@<br>
return (Test.UNSUPPORTED,<br>
"Test requires the following features: %s" % msg)<br>
<br>
- isXFail = isExpectedFail(xfails, xtargets, test.suite.config.target_triple)<br>
+ isXFail = isExpectedFail(test, xfails, xtargets)<br>
return script,isXFail,tmpBase,execdir<br>
<br>
def formatTestOutput(status, out, err, exitCode, failDueToStderr, script):<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@cs.uiuc.edu" class="cremed">llvm-commits@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank" class="cremed">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a><br>
</blockquote></div><br></div>