[libcxx-commits] [libcxx] [libcxx][docs] Make test name pattern documentation more obvious (PR #73136)
Will Hawkins via libcxx-commits
libcxx-commits at lists.llvm.org
Wed Nov 22 18:21:22 PST 2023
https://github.com/hawkinsw updated https://github.com/llvm/llvm-project/pull/73136
>From b46ef9e42a525433eeb3959b0c92468120e63ae4 Mon Sep 17 00:00:00 2001
From: Will Hawkins <hawkinsw at obs.cr>
Date: Wed, 22 Nov 2023 10:12:04 -0500
Subject: [PATCH 1/2] [libcxx][docs] Make test name pattern documentation more
obvious
As a new contributor, I found it hard to find the documentation for the
meaning of the names of different tests and how those names translate
to Lit.
This makes that documentation more explicit.
---
libcxx/docs/TestingLibcxx.rst | 49 ++++++++++++++++++++++++++++++-----
1 file changed, 42 insertions(+), 7 deletions(-)
diff --git a/libcxx/docs/TestingLibcxx.rst b/libcxx/docs/TestingLibcxx.rst
index 44f3904f4e426a0..5783cc6a21922cc 100644
--- a/libcxx/docs/TestingLibcxx.rst
+++ b/libcxx/docs/TestingLibcxx.rst
@@ -330,13 +330,48 @@ additional headers.
taken to make it work in earlier language versions.
-Additional reading
-------------------
-
-The function ``CxxStandardLibraryTest`` in the file
-``libcxx/utils/libcxx/test/format.py`` has documentation about writing test. It
-explains the difference between the test named ``foo.pass.cpp`` and named
-``foo.verify.cpp`` are.
+Test names
+----------
+
+The names of test files have meaning for the libcxx-specific configuration of
+Lit. A complete description can be found in the docstring for the function
+``CxxStandardLibraryTest`` in the file ``libcxx/utils/libcxx/test/format.py``.
+
+For quick reference, here is an overview of some of the most useful patterns:
+
+.. list-table:: Lit Meaning of libcxx Test Filenames
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Name Pattern
+ - Meaning
+ * - ``FOO.pass.cpp``
+ - Compiles, links and runs successfully
+ * - ``FOO.pass.mm``
+ - Same as ``FOO.pass.cpp``, but for Objective-C++
+
+ * - ``FOO.compile.pass.cpp``
+ - Compiles successfully, link and run not attempted
+ * - ``FOO.compile.pass.mm``
+ - Same as ``FOO.compile.pass.cpp``, but for Objective-C++
+ * - ``FOO.compile.fail.cpp``
+ - Does not compile successfully
+
+ * - ``FOO.link.pass.cpp``
+ - Compiles and links successfully, run not attempted
+ * - ``FOO.link.pass.mm``
+ - Same as ``FOO.link.pass.cpp``, but for Objective-C++
+ * - ``FOO.link.fail.cpp``
+ - Compiles successfully, but fails to link
+
+ * - ``FOO.sh.<anything>``
+ - A builtin Lit Shell test
+
+ * - ``FOO.gen.<anything>``
+ - A ``.sh`` test that generates one or more Lit tests on the fly. Executing this test must generate one or more files as expected by LLVM split-file, and each generated file leads to a separate Lit test that runs that file as defined by the test format. This can be used to generate multiple Lit tests from a single source file, which is useful for testing repetitive properties in the library. Be careful not to abuse this since this is not a replacement for usual code reuse techniques.
+
+ * - ``FOO.verify.cpp``
+ - Compiles with clang-verify. This type of test is automatically marked as UNSUPPORTED if the compiler does not support clang-verify.
Benchmarks
==========
>From c148c822407d5ee402364a88dd55bb503fa1b67c Mon Sep 17 00:00:00 2001
From: Will Hawkins <hawkinsw at obs.cr>
Date: Wed, 22 Nov 2023 21:21:02 -0500
Subject: [PATCH 2/2] fixup! [libcxx][docs] Make test name pattern
documentation more obvious
Update to consolidate lit test file name pattern semantic documentation to docs/
---
libcxx/utils/libcxx/test/format.py | 36 ++++--------------------------
1 file changed, 4 insertions(+), 32 deletions(-)
diff --git a/libcxx/utils/libcxx/test/format.py b/libcxx/utils/libcxx/test/format.py
index 52c6f9cd8f2ef22..dc5291a62464aad 100644
--- a/libcxx/utils/libcxx/test/format.py
+++ b/libcxx/utils/libcxx/test/format.py
@@ -155,38 +155,10 @@ class CxxStandardLibraryTest(lit.formats.FileBasedTest):
"""
Lit test format for the C++ Standard Library conformance test suite.
- This test format is based on top of the ShTest format -- it basically
- creates a shell script performing the right operations (compile/link/run)
- based on the extension of the test file it encounters. It supports files
- with the following extensions:
-
- FOO.pass.cpp - Compiles, links and runs successfully
- FOO.pass.mm - Same as .pass.cpp, but for Objective-C++
-
- FOO.compile.pass.cpp - Compiles successfully, link and run not attempted
- FOO.compile.pass.mm - Same as .compile.pass.cpp, but for Objective-C++
- FOO.compile.fail.cpp - Does not compile successfully
-
- FOO.link.pass.cpp - Compiles and links successfully, run not attempted
- FOO.link.pass.mm - Same as .link.pass.cpp, but for Objective-C++
- FOO.link.fail.cpp - Compiles successfully, but fails to link
-
- FOO.sh.<anything> - A builtin Lit Shell test
-
- FOO.gen.<anything> - A .sh test that generates one or more Lit tests on the
- fly. Executing this test must generate one or more files
- as expected by LLVM split-file, and each generated file
- leads to a separate Lit test that runs that file as
- defined by the test format. This can be used to generate
- multiple Lit tests from a single source file, which is
- useful for testing repetitive properties in the library.
- Be careful not to abuse this since this is not a replacement
- for usual code reuse techniques.
-
- FOO.verify.cpp - Compiles with clang-verify. This type of test is
- automatically marked as UNSUPPORTED if the compiler
- does not support Clang-verify.
-
+ Lit tests are contained in files that follow a certain pattern. That
+ pattern determines the semantics of the test. See
+ libcxx/docs/TestingLibcxx.rst or https://libcxx.llvm.org/TestingLibcxx.html
+ for a complete description of those semantics.
Substitution requirements
===============================
More information about the libcxx-commits
mailing list