[all-commits] [llvm/llvm-project] 9f4aff: [clang][test] add testing for the AST matcher refe...

Sat Aug 24 04:14:37 PDT 2024

  Branch: refs/heads/users/5chmidti/add_testing_for_the_AST_matcher_reference
  Home:   https://github.com/llvm/llvm-project
  Commit: 9f4aff21b7ef4b2b9b4d5fc6eb606805c6ecd609
      https://github.com/llvm/llvm-project/commit/9f4aff21b7ef4b2b9b4d5fc6eb606805c6ecd609
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-08-24 (Sat, 24 Aug 2024)

  Changed paths:
    M clang/docs/LibASTMatchersReference.html
    M clang/docs/ReleaseNotes.rst
    M clang/docs/doxygen.cfg.in
    M clang/docs/tools/dump_ast_matchers.py
    M clang/include/clang/ASTMatchers/ASTMatchers.h
    M clang/unittests/ASTMatchers/ASTMatchersTest.h
    M clang/unittests/ASTMatchers/CMakeLists.txt
    A clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  [clang][test] add testing for the AST matcher reference

Previously, the examples in the AST matcher reference, which gets
generated by the doxygen comments in `ASTMatchers.h`, were untested
and best effort.
Some of the matchers had no or wrong examples of how to use the matcher.

This patch introduces a simple DSL around doxygen commands to enable
testing the AST matcher documentation in a way that should be relatively
easy.
In `ASTMatchers.h`, most matchers are documented with a doxygen comment.
Most of these also have a code example that aims to show what the
matcher will match, given a matcher somewhere in the documentation text.
The way that testing the documentation is done, is by using doxygens
alias feature to declare custom aliases. These aliases forward to
`<tt>text</tt>` (which is what doxygens \c does, but for multiple words).
Using the doxygen aliases was the obvious choice, because there are
(now) four consumers:
 - people reading the header/using signature help
 - the doxygen generated documentation
 - the generated html AST matcher reference
 - (new) the generated matcher tests

This patch rewrites/extends the documentation such that all matchers
have a documented example.
The new `generate_ast_matcher_doc_tests.py` script will warn on any
undocumented matchers (but not on matchers without a doxygen comment)
and provides diagnostics and statistics about the matchers.
Below is a file-level comment from the test generation script that
describes how documenting matchers to be tested works on a slightly more
technical level. In general, the new comments can be used as a reference
for how to implement a tested documentation.

The current statistics emitted by the parser are:

```text
Statistics:
        doxygen_blocks                :   519
        missing_tests                 :    10
        skipped_objc                  :    42
        code_snippets                 :   503
        matches                       :   820
        matchers                      :   580
        tested_matchers               :   574
        none_type_matchers            :     6
```

The tests are generated during building and the script will only print
something if it found an issue (compile failure, parsing issues,
the expected and actual number of failures differs).

DSL for generating the tests from documentation.

TLDR:
The order for a single code snippet example is:

  \header{a.h}
  \endheader     <- zero or more header

  \code
    int a = 42;
  \endcode
  \compile_args{-std=c++,c23-or-later} <- optional, supports std ranges and
                                          whole languages

  \matcher{expr()} <- one or more matchers in succession
  \match{42}   <- one ore more matches in succession

  \matcher{varDecl()} <- new matcher resets the context, the above
                         \match will not count for this new
                         matcher(-group)
  \match{int a  = 42} <- only applies to the previous matcher (no the
                         previous case)

The above block can be repeated inside of a doxygen command for multiple
code examples.

Language Grammar:
  [] denotes an optional, and <> denotes user-input

  compile_args j:= \compile_args{[<compile_arg>;]<compile_arg>}
  matcher_tag_key ::= type
  match_tag_key ::= type || std || count
  matcher_tags ::= [matcher_tag_key=<value>;]matcher_tag_key=<value>
  match_tags ::= [match_tag_key=<value>;]match_tag_key=<value>
  matcher ::= \matcher{[matcher_tags$]<matcher>}
  matchers ::= [matcher] matcher
  match ::= \match{[match_tags$]<match>}
  matches ::= [match] match
  case ::= matchers matches
  cases ::= [case] case
  header-block ::= \header{<name>} <code> \endheader
  code-block ::= \code <code> \endcode
  testcase ::= code-block [compile_args] cases

The 'std' tag and '\compile_args' support specifying a specific
language version, a whole language and all of it's versions, and thresholds
(implies ranges). Multiple arguments are passed with a ',' seperator.
For a language and version to execute a tested matcher, it has to match
the specified '\compile_args' for the code, and the 'std' tag for the matcher.
Predicates for the 'std' compiler flag are used with disjunction between
languages (e.g. 'c || c++') and conjunction for all predicates specific
to each language (e.g. 'c++11-or-later && c++23-or-earlier').

Examples:
 - c                                    all available versions of C
 - c++11                                only C++11
 - c++11-or-later                       C++11 or later
 - c++11-or-earlier                     C++11 or earlier
 - c++11-or-later,c++23-or-earlier,c    all of C and C++ between 11 and
                                          23 (inclusive)
 - c++11-23,c                             same as above

Tags:

  Type:
  Match types are used to select where the string that is used to check if
  a node matches comes from.
  Available: code, name, typestr, typeofstr.
  The default is 'code'.

  Matcher types are used to mark matchers as submatchers with 'sub' or as
  deactivated using 'none'. Testing submatchers is not implemented.

  Count:
  Specifying a 'count=n' on a match will result in a test that requires that
  the specified match will be matched n times. Default is 1.

  Std:
  A match allows specifying if it matches only in specific language versions.
  This may be needed when the AST differs between language versions.

Fixes #57607
Fixes #63748

  Commit: 34c151d770045be16eeeb11e847a83909f11a2c9
      https://github.com/llvm/llvm-project/commit/34c151d770045be16eeeb11e847a83909f11a2c9
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-08-24 (Sat, 24 Aug 2024)

  Changed paths:
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  fix cuda tests

  Commit: 49b7206cbe8fd6293b19466937e92af1197c006a
      https://github.com/llvm/llvm-project/commit/49b7206cbe8fd6293b19466937e92af1197c006a
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-08-24 (Sat, 24 Aug 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  fix cuda tests

  Commit: 72cf581cc0cf5c82b29686f39f6bc6e8266062b9
      https://github.com/llvm/llvm-project/commit/72cf581cc0cf5c82b29686f39f6bc6e8266062b9
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-08-24 (Sat, 24 Aug 2024)

  Changed paths:
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  fix f-string containing backslash

  Commit: e9cb878785f2e0804ea6aa6c6141a59479fdf5a7
      https://github.com/llvm/llvm-project/commit/e9cb878785f2e0804ea6aa6c6141a59479fdf5a7
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-08-24 (Sat, 24 Aug 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h
    M clang/unittests/ASTMatchers/ASTMatchersTest.h
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  rm not needed typeofstr match kind

  Commit: 40abc38b3d9e3cda2c2ebbc4885c3233d14cd421
      https://github.com/llvm/llvm-project/commit/40abc38b3d9e3cda2c2ebbc4885c3233d14cd421
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-08-24 (Sat, 24 Aug 2024)

  Changed paths:
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  move some FIXME's to the main function

  Commit: fa50035e9d21f3180f4682e57129bfe52136c987
      https://github.com/llvm/llvm-project/commit/fa50035e9d21f3180f4682e57129bfe52136c987
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-08-24 (Sat, 24 Aug 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h

  Log Message:
  -----------
  replace some type=name matches with explicit code matches to be more expressive

  Commit: 066ea2b1cb2208f8450946d604d3e95b9be6666c
      https://github.com/llvm/llvm-project/commit/066ea2b1cb2208f8450946d604d3e95b9be6666c
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-08-24 (Sat, 24 Aug 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h

  Log Message:
  -----------
  add missing comments on some matchers on the number of matches generated

  Commit: b05a7e090bc31d5a18554a421160719c8b87f105
      https://github.com/llvm/llvm-project/commit/b05a7e090bc31d5a18554a421160719c8b87f105
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-08-24 (Sat, 24 Aug 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h

  Log Message:
  -----------
  add documentation (pr desc) to ASTMatchers.h's file comment

  Commit: 8124411b5b3489425a164b53e4ec9189b8eb293b
      https://github.com/llvm/llvm-project/commit/8124411b5b3489425a164b53e4ec9189b8eb293b
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-08-24 (Sat, 24 Aug 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h

  Log Message:
  -----------
  fix empty enum decl

Compare: https://github.com/llvm/llvm-project/compare/ad11a8972d5d...8124411b5b34

To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications