[PATCH] D140479: [docs] Add tips on writing test constraints

[Paul Robinson via Phabricator via llvm-commits]

probinson created this revision.
probinson added reviewers: ldionne, dblaikie.
Herald added a project: All.
probinson requested review of this revision.
Herald added a project: LLVM.

Guidance on using REQUIRES, UNSUPPORTED, XFAIL, and the `target=...` feature.

Things I thought of while updating lots and lots of lit tests.


https://reviews.llvm.org/D140479

Files:
  llvm/docs/TestingGuide.rst


Index: llvm/docs/TestingGuide.rst
===================================================================
--- llvm/docs/TestingGuide.rst
+++ llvm/docs/TestingGuide.rst
@@ -533,6 +533,46 @@
     ; XFAIL: target=powerpc{{.*}}, system-darwin
 
 
+Tips for writing constraints
+----------------------------
+
+**``REQUIRES`` and ``UNSUPPORTED``**
+
+These are logical inverses. In principle, ``UNSUPPORTED`` isn't absolutely
+necessary (the logical negation could be used with ``REQUIRES`` to get
+exactly the same effect), but it can make these clauses easier to read and
+understand. Generally, people use ``REQUIRES`` to state things that the test
+depends on to operate correctly, and ``UNSUPPORTED`` to exclude cases where
+the test is expected never to work.
+
+**``UNSUPPORTED`` and ``XFAIL``**
+
+Both of these indicate that the test isn't expected to work; however, they
+have different effects. ``UNSUPPORTED`` causes the test to be skipped;
+this saves execution time, but then you'll never know whether the test
+actually would start working. Conversely, ``XFAIL`` actually runs the test
+but expects a failure output, taking extra execution time but alerting you
+if/when the test begins to behave correctly (an XPASS test result). You
+need to decide which is more appropriate in each case.
+
+**Using ``target=...``**
+
+Checking the target triple can be tricky; it's easy to mis-specify. For
+example, ``target=mips{{.*}}`` will match not only mips, but also mipsel,
+mips64, and mips64el. ``target={{.*}}-linux-gnu`` will match
+x86_64-unknown-linux-gnu, but not armv8l-unknown-linux-gnueabihf.
+Prefer to use hyphens to delimit triple components (``target=mips-{{.*}}``)
+and it's generally a good idea to use a trailing wildcard to allow for
+unexpected suffixes.
+
+Also, it's generally better to write regular expressions that use entire
+triple components, than to do something clever to shorten them. For
+example, to match both freebsd and netbsd in an expression, you could write
+``target={{.*(free|net)bsd.*}}`` and that would work. However, it would
+prevent a ``grep freebsd`` from finding this test. Better to use 
+``target={{.*(freebsd|netbsd).*}}`` in this case.
+
+
 Substitutions
 -------------
 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D140479.484572.patch
Type: text/x-patch
Size: 2224 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20221221/5d2ad8a4/attachment.bin>