[all-commits] [llvm/llvm-project] a5622c: [clang][test] add testing for the AST matcher refe...

Fri Jul 12 15:02:04 PDT 2024

  Branch: refs/heads/users/5chmidti/add_testing_for_the_AST_matcher_reference
  Home:   https://github.com/llvm/llvm-project
  Commit: a5622c4dbc623deacec0fede97733c09a7cd9fe5
      https://github.com/llvm/llvm-project/commit/a5622c4dbc623deacec0fede97733c09a7cd9fe5
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-07-13 (Sat, 13 Jul 2024)

  Changed paths:
    M clang/docs/LibASTMatchersReference.html
    M clang/docs/ReleaseNotes.rst
    M clang/docs/doxygen.cfg.in
    M clang/docs/tools/dump_ast_matchers.py
    M clang/include/clang/ASTMatchers/ASTMatchers.h
    M clang/unittests/ASTMatchers/ASTMatchersTest.h
    M clang/unittests/ASTMatchers/CMakeLists.txt
    A clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  [clang][test] add testing for the AST matcher reference

Previously, the examples in the AST matcher reference, which gets
generated by the doxygen comments in `ASTMatchers.h`, were untested
and best effort.
Some of the matchers had no or wrong examples of how to use the matcher.

This patch introduces a simple DSL around doxygen commands to enable
testing the AST matcher documentation in a way that should be relatively
easy.
In `ASTMatchers.h`, most matchers are documented with a doxygen comment.
Most of these also have a code example that aims to show what the
matcher will match, given a matcher somewhere in the documentation text.
The way that testing the documentation is done, is by using doxygens
alias feature to declare custom aliases. These aliases forward to
`<tt>text</tt>` (which is what doxygens \c does, but for multiple words).
Using the doxygen aliases was the obvious choice, because there are
(now) four consumers:
 - people reading the header/using signature help
 - the doxygen generated documentation
 - the generated html AST matcher reference
 - (new) the generated matcher tests

This patch rewrites/extends the documentation such that all matchers
have a documented example.
The new `generate_ast_matcher_doc_tests.py` script will warn on any
undocumented matchers (but not on matchers without a doxygen comment)
and provides diagnostics and statistics about the matchers.
Below is a file-level comment from the test generation script that
describes how documenting matchers to be tested works on a slightly more
technical level. In general, the new comments can be used as a reference
for how to implement a tested documentation.

The current statistics emitted by the parser are:

```text
Statistics:
        doxygen_blocks                :   519
        missing_tests                 :    10
        skipped_objc                  :    42
        code_snippets                 :   503
        matches                       :   820
        matchers                      :   580
        tested_matchers               :   574
        none_type_matchers            :     6
```

The tests are generated during building and the script will only print
something if it found an issue (compile failure, parsing issues,
the expected and actual number of failures differs).

DSL for generating the tests from documentation.

TLDR:
The order for a single code snippet example is:

  \header{a.h}
  \endheader     <- zero or more header

  \code
    int a = 42;
  \endcode
  \compile_args{-std=c++,c23-or-later} <- optional, supports std ranges and
                                          whole languages

  \matcher{expr()} <- one or more matchers in succession
  \match{42}   <- one ore more matches in succession

  \matcher{varDecl()} <- new matcher resets the context, the above
                         \match will not count for this new
                         matcher(-group)
  \match{int a  = 42} <- only applies to the previous matcher (no the
                         previous case)

The above block can be repeated inside of a doxygen command for multiple
code examples.

Language Grammar:
  [] denotes an optional, and <> denotes user-input

  compile_args j:= \compile_args{[<compile_arg>;]<compile_arg>}
  matcher_tag_key ::= type
  match_tag_key ::= type || std || count
  matcher_tags ::= [matcher_tag_key=<value>;]matcher_tag_key=<value>
  match_tags ::= [match_tag_key=<value>;]match_tag_key=<value>
  matcher ::= \matcher{[matcher_tags$]<matcher>}
  matchers ::= [matcher] matcher
  match ::= \match{[match_tags$]<match>}
  matches ::= [match] match
  case ::= matchers matches
  cases ::= [case] case
  header-block ::= \header{<name>} <code> \endheader
  code-block ::= \code <code> \endcode
  testcase ::= code-block [compile_args] cases

The 'std' tag and '\compile_args' support specifying a specific
language version, a whole language and all of it's versions, and thresholds
(implies ranges). Multiple arguments are passed with a ',' seperator.
For a language and version to execute a tested matcher, it has to match
the specified '\compile_args' for the code, and the 'std' tag for the matcher.
Predicates for the 'std' compiler flag are used with disjunction between
languages (e.g. 'c || c++') and conjunction for all predicates specific
to each language (e.g. 'c++11-or-later && c++23-or-earlier').

Examples:
 - c                                    all available versions of C
 - c++11                                only C++11
 - c++11-or-later                       C++11 or later
 - c++11-or-earlier                     C++11 or earlier
 - c++11-or-later,c++23-or-earlier,c    all of C and C++ between 11 and
                                          23 (inclusive)
 - c++11-23,c                             same as above

Tags:

  Type:
  Match types are used to select where the string that is used to check if
  a node matches comes from.
  Available: code, name, typestr, typeofstr.
  The default is 'code'.

  Matcher types are used to mark matchers as submatchers with 'sub' or as
  deactivated using 'none'. Testing submatchers is not implemented.

  Count:
  Specifying a 'count=n' on a match will result in a test that requires that
  the specified match will be matched n times. Default is 1.

  Std:
  A match allows specifying if it matches only in specific language versions.
  This may be needed when the AST differs between language versions.

Fixes #57607
Fixes #63748

  Commit: 3bfe588f6371e50239d531274ffadbc84641a8ca
      https://github.com/llvm/llvm-project/commit/3bfe588f6371e50239d531274ffadbc84641a8ca
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-07-13 (Sat, 13 Jul 2024)

  Changed paths:
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  fix cuda tests

  Commit: 50f3a8c385c4a2488d6396620febf11586bae94b
      https://github.com/llvm/llvm-project/commit/50f3a8c385c4a2488d6396620febf11586bae94b
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-07-13 (Sat, 13 Jul 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  fix cuda tests

  Commit: 7101371c4433e4afcd0659d42fc0ba8087560a69
      https://github.com/llvm/llvm-project/commit/7101371c4433e4afcd0659d42fc0ba8087560a69
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-07-13 (Sat, 13 Jul 2024)

  Changed paths:
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  fix f-string containing backslash

  Commit: b32adfebaabfae87f50326a33159c92b11f45e25
      https://github.com/llvm/llvm-project/commit/b32adfebaabfae87f50326a33159c92b11f45e25
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-07-13 (Sat, 13 Jul 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h
    M clang/unittests/ASTMatchers/ASTMatchersTest.h
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  rm not needed typeofstr match kind

  Commit: a206127a030474a615c5e0a70d05b57c288bd5f4
      https://github.com/llvm/llvm-project/commit/a206127a030474a615c5e0a70d05b57c288bd5f4
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-07-13 (Sat, 13 Jul 2024)

  Changed paths:
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  move some FIXME's to the main function

  Commit: e84f21a001599f81f4e3bc4943ac7bfaf081ad08
      https://github.com/llvm/llvm-project/commit/e84f21a001599f81f4e3bc4943ac7bfaf081ad08
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-07-13 (Sat, 13 Jul 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h

  Log Message:
  -----------
  replace some type=name matches with explicit code matches to be more expressive

  Commit: c92804507e9581855d09fc6d8963899f8d6e605a
      https://github.com/llvm/llvm-project/commit/c92804507e9581855d09fc6d8963899f8d6e605a
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-07-13 (Sat, 13 Jul 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h

  Log Message:
  -----------
  add missing comments on some matchers on the number of matches generated

  Commit: 82564f2de422f682b7ab8a75d6211191e15ffc94
      https://github.com/llvm/llvm-project/commit/82564f2de422f682b7ab8a75d6211191e15ffc94
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-07-13 (Sat, 13 Jul 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h

  Log Message:
  -----------
  add documentation (pr desc) to ASTMatchers.h's file comment

  Commit: fe4328cd27e945a2f71e0c2102616a5216137602
      https://github.com/llvm/llvm-project/commit/fe4328cd27e945a2f71e0c2102616a5216137602
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-07-13 (Sat, 13 Jul 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h

  Log Message:
  -----------
  fix empty enum decl

Compare: https://github.com/llvm/llvm-project/compare/c4a014ec485c...fe4328cd27e9

To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications