[llvm] c2bed2a - [UTC] Add do-not-autogenerate capability

[Paul Robinson via llvm-commits]

Author: Paul Robinson
Date: 2023-07-06T10:56:42-07:00
New Revision: c2bed2a1703a8187c231fdb2926714e877221d85

URL: https://github.com/llvm/llvm-project/commit/c2bed2a1703a8187c231fdb2926714e877221d85
DIFF: https://github.com/llvm/llvm-project/commit/c2bed2a1703a8187c231fdb2926714e877221d85.diff

LOG: [UTC] Add do-not-autogenerate capability

Differential Revision: https://reviews.llvm.org/D154383

Added: 
    

Modified: 
    llvm/docs/TestingGuide.rst
    llvm/utils/UpdateTestChecks/common.py

Removed: 
    


################################################################################
diff  --git a/llvm/docs/TestingGuide.rst b/llvm/docs/TestingGuide.rst
index 05b8d4ddd9d812..56e923affafe44 100644
--- a/llvm/docs/TestingGuide.rst
+++ b/llvm/docs/TestingGuide.rst
@@ -283,14 +283,23 @@ In that case to reduce the human work we can use the scripts available in
 llvm/utils/ to generate the assertions.
 
 For example to generate assertions in an :program:`llc`-based test, after
-adding RUN line use:
+adding one or more RUN lines use:
 
  .. code-block:: bash
 
      % llvm/utils/update_llc_test_checks.py --llc-binary build/bin/llc test.ll
 
-And if you want to update assertions in an existing test case, pass `-u` option
-which first check the ``NOTE:`` line exists and matches the script name.
+This will generate FileCheck assertions, and insert a ``NOTE:`` line at the
+top to indicate that assertions were automatically generated.
+
+If you want to update assertions in an existing test case, pass the `-u` option
+which first checks the ``NOTE:`` line exists and matches the script name.
+
+Sometimes a test absolutely depends on hand-written assertions and should not
+have assertions automatically generated. In that case, add the text ``NOTE: Do
+not autogenerate`` to the first line, and the scripts will skip that test. It
+is a good idea to explain why generated assertions will not work for the test
+so future developers will understand what is going on.
 
 These are the most common scripts and their purposes/applications in generating
 assertions:

diff  --git a/llvm/utils/UpdateTestChecks/common.py b/llvm/utils/UpdateTestChecks/common.py
index 81b1335d78cbdc..6f8180540cd9e9 100644
--- a/llvm/utils/UpdateTestChecks/common.py
+++ b/llvm/utils/UpdateTestChecks/common.py
@@ -327,6 +327,9 @@ def itertests(
             with open(test) as f:
                 input_lines = [l.rstrip() for l in f]
             first_line = input_lines[0] if input_lines else ""
+            if UTC_AVOID in first_line:
+              warn("Skipping test that must not be autogenerated: " + test)
+              continue
             is_regenerate = UTC_ADVERT in first_line
 
             # If we're generating a new test, set the default version to the latest.
@@ -468,6 +471,7 @@ def invoke_tool(exe, cmd_args, ir, preprocess_cmd=None, verbose=False):
 UTC_ARGS_KEY = "UTC_ARGS:"
 UTC_ARGS_CMD = re.compile(r".*" + UTC_ARGS_KEY + "\s*(?P<cmd>.*)\s*$")
 UTC_ADVERT = "NOTE: Assertions have been autogenerated by "
+UTC_AVOID = "NOTE: Do not autogenerate"
 UNUSED_NOTE = "NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:"
 
 OPT_FUNCTION_RE = re.compile(