[PATCH] D142441: [Docs] Add best practices for regression tests

[Nikita Popov via Phabricator via llvm-commits]

nikic created this revision.
nikic added reviewers: spatel, reames, jdoerfert, arsenm.
Herald added a subscriber: StephenFan.
Herald added a project: All.
nikic requested review of this revision.
Herald added subscribers: llvm-commits, wdng.
Herald added a project: LLVM.

There are a lot of conventions for writing tests that don't seem to be documented anywhere right now, so this takes a stab at writing down some "best practices".


https://reviews.llvm.org/D142441

Files:
  llvm/docs/TestingGuide.rst


Index: llvm/docs/TestingGuide.rst
===================================================================
--- llvm/docs/TestingGuide.rst
+++ llvm/docs/TestingGuide.rst
@@ -315,6 +315,28 @@
   update_test_checks.py
   opt
 
+Best practices for regression tests
+-----------------------------------
+
+- Unless the test is for a crash or assertion failure, first commit the test
+  with baseline check lines, i.e. either showing a miscompile or missing
+  optimization. Such test additions can be committed without review. Your
+  actual patch will then only contain check line diff, which makes it easier
+  to see the effect of your patch.
+- Use auto-generated check lines (produced by the scripts in the previous
+  section) whenever feasible.
+- Especially for regression tests, include a comment on what is being tested,
+  and include a reference to any relevant issue on the bug tracker.
+- Avoid undefined behavior unless necessary. For example, do not use patterns
+  like ``br i1 undef``, which are likely to break as a result of future
+  optimizations.
+- Minimize regression tests by removing unnecessary instructions, metadata,
+  attributes, etc. Tools like ``llvm-reduce`` can help automate this.
+- Avoid unnamed instructions/blocks (such as ``%0`` or ``1:``). These can be
+  removed by running the test through ``opt -S -passes=instnamer``.
+- Try to give SSA variables meaningful names, and avoid retaining complex names
+  generated by the optimization pipeline (such as ``%foo.0.0.0.0.0.0``).
+
 Extra files
 -----------
 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D142441.491665.patch
Type: text/x-patch
Size: 1540 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20230124/0d9c5892/attachment.bin>