[libcxx-commits] [libcxx] [libcxx][docs] Make test name pattern documentation more obvious (PR #73136)

[Louis Dionne via libcxx-commits]

================
@@ -330,13 +330,83 @@ additional headers.
           taken to make it work in earlier language versions.
 
 
-Additional reading
-------------------
-
-The function ``CxxStandardLibraryTest`` in the file
-``libcxx/utils/libcxx/test/format.py`` has documentation about writing test. It
-explains the difference between the test named  ``foo.pass.cpp`` and named
-``foo.verify.cpp`` are.
+Test names
+----------
+
+The names of test files have meaning for the libcxx-specific configuration of 
+Lit. Based on the pattern that matches the name of a test file, Lit will test
+the code contained therein in different ways. Refer to the `Lit Meaning of libcxx
+Test Filenames`_ when
+determining the names for new test files.
+
+.. _Lit Meaning of libcxx Test Filenames:
+.. list-table:: Lit Meaning of libcxx Test Filenames
+   :widths: 25 75
+   :header-rows: 1
+
+   * - Name Pattern
+     - Meaning
+   * - ``FOO.pass.cpp``
+     - Checks whether the C++ code in the file compiles, links and runs successfully.
+   * - ``FOO.pass.mm``
+     - Same as ``FOO.pass.cpp``, but for Objective-C++.
+
+   * - ``FOO.compile.pass.cpp``
+     - Checks whether the C++ code in the file compiles successfully.
+   * - ``FOO.compile.pass.mm``
+     - Same as ``FOO.compile.pass.cpp``, but for Objective-C++.
+   * - ``FOO.compile.fail.cpp``
+     - Checks that the code in the file does *not* compile successfully.
+
+   * - ``FOO.verify.cpp``
+     - Compiles with clang-verify. This type of test is automatically marked as UNSUPPORTED if the compiler does not support clang-verify.
+
+   * - ``FOO.link.pass.cpp``
+     - Checks that the C++ code in the file compiles and links successfully -- no run attempted.
+   * - ``FOO.link.pass.mm``
+     - Same as ``FOO.link.pass.cpp``, but for Objective-C++.
+   * - ``FOO.link.fail.cpp``
+     - Checks whether the C++ code in the file fails to link after successful compilation.
+   * - ``FOO.link.fail.mm``
+     - Same as ``FOO.link.fail.cpp``, but for Objective-C++.
+
+   * - ``FOO.sh.<anything>``
+     - A *builtin Lit Shell* test.
+   * - ``FOO.gen.<anything>``
+     - A variant of a *Lit Shell* test that generates one or more Lit tests on the fly. Executing this test must generate one or more files as expected
+       by LLVM split-file. Each generated file will drive an invocation of a separate Lit test. The format of the generated file will determine the type
+       of Lit test to be executed. This can be used to generate multiple Lit tests from a single source file, which is useful for testing repetitive properties
+       in the library. Be careful not to abuse this since this is not a replacement for usual code reuse techniques.
+
+
+libc++-Specific Lit Features
+----------------------------
+
+Custom Directives
+~~~~~~~~~~~~~~~~~
+
+Lit has many directives built in (e.g., ``DEFINE``, ``UNSUPPORTED``). In addition to those directives, libcxx adds two additional libc++-specific directives that makes
+writing tests easier. See `libc++-specific Lit Directives`_ for more information about the ``FILE_DEPENDENCIES`` and ``ADDITIONAL_COMPILE_FLAGS`` libc++-specific directives.
+
+.. _libc++-specific Lit Directives:
+.. list-table:: libc++-specific Lit Directives
+   :widths: 20 35 45
+   :header-rows: 1
+
+   * - Directive
+     - Parameters
+     - Usage
+   * - ``FILE_DEPENDENCIES``
+     - ``// FILE_DEPENDENCIES: file, directory, /path/to/file, ...``
+     - The paths given to the ``FILE_DEPENDENCIES`` directive can specify directories or specific files upon which a given test depend. Copies of the files individually specified and
+       *all* the files in any specified directories will be placed in the *current working directory* of the test when it executes. All dependency directories and files
+       specified using relative paths will be anchored to the directory specified by the ``%S`` substitution (i.e., the source directory of the test being executed.).
+   * - ``ADDITIONAL_COMPILE_FLAGS``
+     - ``// ADDITIONAL_COMPILE_FLAGS: flag1, flag2, ...``
+     - The additional compiler flags specified using the ``ADDITIONAL_COMPILE_FLAGS`` libc++-specific Lit directive will be added to the invocation of ``clang`` used to build
----------------
ldionne wrote:

This new documentation is not accurate. They don't apply to Clang only, and they do affect `.sh.cpp` tests too. I suggest we stick to the previous documentation:

```
This directive will cause the provided flags to be added to the
%{compile_flags} substitution for the test that contains it. This
allows adding special compilation flags without having to use a
.sh.cpp test, which would be more powerful but perhaps overkill.
```

https://github.com/llvm/llvm-project/pull/73136