[llvm-commits] [llvm] r166224 - in /llvm/trunk: docs/TestingGuide.html utils/lit/lit/ExampleTests/xfail-feature.c utils/lit/lit/TestRunner.py

Daniel Dunbar daniel at zuster.org
Fri Oct 19 10:24:01 PDT 2012


On Fri, Oct 19, 2012 at 1:11 AM, Chandler Carruth <chandlerc at google.com> wrote:
> On Thu, Oct 18, 2012 at 1:43 PM, Daniel Dunbar <daniel at zuster.org> wrote:
>>
>> Author: ddunbar
>> Date: Thu Oct 18 15:43:04 2012
>> New Revision: 166224
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=166224&view=rev
>> Log:
>> lit: Allow XFAIL: lines to also refer to "features".
>
>
> What's the motivation for this patch?

The specific use case was wanting to XFAIL something that is broken
when the compiler is built with LTO, but the reason I went this route
was it seemed like a reasonably sensical and orthogonal feature.

> I would much rather continue to use directory structures and/or REQUIRES to
> represent feature differences.

That is a different use case, that indicates the test needs a
particular feature, this indicates a test fails with a particular
feature.

>From a design standpoint, it actually seems cleaner to have all of the
REQUIRES, XFAIL, XTARGET lines draw from the same namespace, and maybe
move to treating triples as just another "feature". This is just a
step closer to that.

 - Daniel

> This seems to add still more complexity to
> the interface for writing a lit tests, and I'm worried about that already
> and so interested in scrutinizing the use cases here....
>
>>
>>
>> Added:
>>     llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c
>> Modified:
>>     llvm/trunk/docs/TestingGuide.html
>>     llvm/trunk/utils/lit/lit/TestRunner.py
>>
>> Modified: llvm/trunk/docs/TestingGuide.html
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/TestingGuide.html?rev=166224&r1=166223&r2=166224&view=diff
>>
>> ==============================================================================
>> --- llvm/trunk/docs/TestingGuide.html (original)
>> +++ llvm/trunk/docs/TestingGuide.html Thu Oct 18 15:43:04 2012
>> @@ -798,14 +798,15 @@
>>    <p>Sometimes it is necessary to mark a test case as "expected fail" or
>> XFAIL.
>>    You can easily mark a test as XFAIL just by including <tt>XFAIL: </tt>
>> on a
>>    line near the top of the file. This signals that the test case should
>> succeed
>> -  if the test fails. Such test cases are counted separately by the
>> testing tool. To
>> -  specify an expected fail, use the XFAIL keyword in the comments of the
>> test
>> -  program followed by a colon and one or more regular expressions
>> (separated by
>> -  a comma). The regular expressions allow you to XFAIL the test
>> conditionally by
>> -  host platform. The regular expressions following the : are matched
>> against the
>> -  target triplet for the host machine. If there is a match, the test is
>> expected
>> -  to fail. If not, the test is expected to succeed. To XFAIL everywhere
>> just
>> -  specify <tt>XFAIL: *</tt>. Here is an example of an <tt>XFAIL</tt>
>> line:</p>
>> +  if the test fails. Such test cases are counted separately by the
>> testing
>> +  tool. To specify an expected fail, use the XFAIL keyword in the
>> comments of
>> +  the test program followed by a colon and one or more failure patterns.
>> Each
>> +  failure pattern can be either '*' (to specify fail everywhere), or a
>> part of a
>> +  target triple (indicating the test should fail on that platfomr), or
>> the name
>> +  of a configurable feature (for example, "loadable_module").. If there
>> is a
>> +  match, the test is expected to fail. If not, the test is expected to
>> +  succeed. To XFAIL everywhere just specify <tt>XFAIL: *</tt>. Here is an
>> +  example of an <tt>XFAIL</tt> line:</p>
>>
>>  <div class="doc_code">
>>  <pre>
>>
>> Added: llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c?rev=166224&view=auto
>>
>> ==============================================================================
>> --- llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c (added)
>> +++ llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c Thu Oct 18
>> 15:43:04 2012
>> @@ -0,0 +1,4 @@
>> +// This test should XPASS.
>> +
>> +// RUN: true
>> +// XFAIL: some-feature-name
>>
>> Modified: llvm/trunk/utils/lit/lit/TestRunner.py
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/TestRunner.py?rev=166224&r1=166223&r2=166224&view=diff
>>
>> ==============================================================================
>> --- llvm/trunk/utils/lit/lit/TestRunner.py (original)
>> +++ llvm/trunk/utils/lit/lit/TestRunner.py Thu Oct 18 15:43:04 2012
>> @@ -370,10 +370,15 @@
>>
>>      return executeCommand(command, cwd=cwd, env=test.config.environment)
>>
>> -def isExpectedFail(xfails, xtargets, target_triple):
>> -    # Check if any xfail matches this target.
>> +def isExpectedFail(test, xfails, xtargets):
>> +    # If the xfail matches an available feature, it always fails.
>>      for item in xfails:
>> -        if item == '*' or item in target_triple:
>> +        if item in test.config.available_features:
>> +            return True
>> +
>> +    # Otherwise, check if any xfail matches this target.
>> +    for item in xfails:
>> +        if item == '*' or item in test.suite.config.target_triple:
>>              break
>>      else:
>>          return False
>> @@ -382,7 +387,7 @@
>>      #
>>      # FIXME: Rename XTARGET to something that makes sense, like XPASS.
>>      for item in xtargets:
>> -        if item == '*' or item in target_triple:
>> +        if item == '*' or item in test.suite.config.target_triple:
>>              return False
>>
>>      return True
>> @@ -491,7 +496,7 @@
>>          return (Test.UNSUPPORTED,
>>                  "Test requires the following features: %s" % msg)
>>
>> -    isXFail = isExpectedFail(xfails, xtargets,
>> test.suite.config.target_triple)
>> +    isXFail = isExpectedFail(test, xfails, xtargets)
>>      return script,isXFail,tmpBase,execdir
>>
>>  def formatTestOutput(status, out, err, exitCode, failDueToStderr,
>> script):
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>
>



More information about the llvm-commits mailing list