[llvm-commits] [llvm] r166224 - in /llvm/trunk: docs/TestingGuide.html utils/lit/lit/ExampleTests/xfail-feature.c utils/lit/lit/TestRunner.py

[Chandler Carruth]

On Thu, Oct 18, 2012 at 1:43 PM, Daniel Dunbar <daniel at zuster.org> wrote:

> Author: ddunbar
> Date: Thu Oct 18 15:43:04 2012
> New Revision: 166224
>
> URL: http://llvm.org/viewvc/llvm-project?rev=166224&view=rev
> Log:
> lit: Allow XFAIL: lines to also refer to "features".
>

What's the motivation for this patch?

I would much rather continue to use directory structures and/or REQUIRES to
represent feature differences. This seems to add still more complexity to
the interface for writing a lit tests, and I'm worried about that already
and so interested in scrutinizing the use cases here....


>
> Added:
>     llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c
> Modified:
>     llvm/trunk/docs/TestingGuide.html
>     llvm/trunk/utils/lit/lit/TestRunner.py
>
> Modified: llvm/trunk/docs/TestingGuide.html
> URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/TestingGuide.html?rev=166224&r1=166223&r2=166224&view=diff
>
> ==============================================================================
> --- llvm/trunk/docs/TestingGuide.html (original)
> +++ llvm/trunk/docs/TestingGuide.html Thu Oct 18 15:43:04 2012
> @@ -798,14 +798,15 @@
>    <p>Sometimes it is necessary to mark a test case as "expected fail" or
> XFAIL.
>    You can easily mark a test as XFAIL just by including <tt>XFAIL: </tt>
> on a
>    line near the top of the file. This signals that the test case should
> succeed
> -  if the test fails. Such test cases are counted separately by the
> testing tool. To
> -  specify an expected fail, use the XFAIL keyword in the comments of the
> test
> -  program followed by a colon and one or more regular expressions
> (separated by
> -  a comma). The regular expressions allow you to XFAIL the test
> conditionally by
> -  host platform. The regular expressions following the : are matched
> against the
> -  target triplet for the host machine. If there is a match, the test is
> expected
> -  to fail. If not, the test is expected to succeed. To XFAIL everywhere
> just
> -  specify <tt>XFAIL: *</tt>. Here is an example of an <tt>XFAIL</tt>
> line:</p>
> +  if the test fails. Such test cases are counted separately by the testing
> +  tool. To specify an expected fail, use the XFAIL keyword in the
> comments of
> +  the test program followed by a colon and one or more failure patterns.
> Each
> +  failure pattern can be either '*' (to specify fail everywhere), or a
> part of a
> +  target triple (indicating the test should fail on that platfomr), or
> the name
> +  of a configurable feature (for example, "loadable_module").. If there
> is a
> +  match, the test is expected to fail. If not, the test is expected to
> +  succeed. To XFAIL everywhere just specify <tt>XFAIL: *</tt>. Here is an
> +  example of an <tt>XFAIL</tt> line:</p>
>
>  <div class="doc_code">
>  <pre>
>
> Added: llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c
> URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c?rev=166224&view=auto
>
> ==============================================================================
> --- llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c (added)
> +++ llvm/trunk/utils/lit/lit/ExampleTests/xfail-feature.c Thu Oct 18
> 15:43:04 2012
> @@ -0,0 +1,4 @@
> +// This test should XPASS.
> +
> +// RUN: true
> +// XFAIL: some-feature-name
>
> Modified: llvm/trunk/utils/lit/lit/TestRunner.py
> URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/TestRunner.py?rev=166224&r1=166223&r2=166224&view=diff
>
> ==============================================================================
> --- llvm/trunk/utils/lit/lit/TestRunner.py (original)
> +++ llvm/trunk/utils/lit/lit/TestRunner.py Thu Oct 18 15:43:04 2012
> @@ -370,10 +370,15 @@
>
>      return executeCommand(command, cwd=cwd, env=test.config.environment)
>
> -def isExpectedFail(xfails, xtargets, target_triple):
> -    # Check if any xfail matches this target.
> +def isExpectedFail(test, xfails, xtargets):
> +    # If the xfail matches an available feature, it always fails.
>      for item in xfails:
> -        if item == '*' or item in target_triple:
> +        if item in test.config.available_features:
> +            return True
> +
> +    # Otherwise, check if any xfail matches this target.
> +    for item in xfails:
> +        if item == '*' or item in test.suite.config.target_triple:
>              break
>      else:
>          return False
> @@ -382,7 +387,7 @@
>      #
>      # FIXME: Rename XTARGET to something that makes sense, like XPASS.
>      for item in xtargets:
> -        if item == '*' or item in target_triple:
> +        if item == '*' or item in test.suite.config.target_triple:
>              return False
>
>      return True
> @@ -491,7 +496,7 @@
>          return (Test.UNSUPPORTED,
>                  "Test requires the following features: %s" % msg)
>
> -    isXFail = isExpectedFail(xfails, xtargets,
> test.suite.config.target_triple)
> +    isXFail = isExpectedFail(test, xfails, xtargets)
>      return script,isXFail,tmpBase,execdir
>
>  def formatTestOutput(status, out, err, exitCode, failDueToStderr, script):
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20121019/ca15a7ca/attachment.html>