[all-commits] [llvm/llvm-project] 44b706: [clang][test] add testing for the AST matcher refe...

Fri Sep 27 05:49:55 PDT 2024

  Branch: refs/heads/users/5chmidti/add_testing_for_the_AST_matcher_reference
  Home:   https://github.com/llvm/llvm-project
  Commit: 44b7065d3dec24142553d730255d510408b268c8
      https://github.com/llvm/llvm-project/commit/44b7065d3dec24142553d730255d510408b268c8
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-09-27 (Fri, 27 Sep 2024)

  Changed paths:
    M clang/docs/LibASTMatchersReference.html
    M clang/docs/ReleaseNotes.rst
    M clang/docs/doxygen.cfg.in
    M clang/docs/tools/dump_ast_matchers.py
    M clang/include/clang/ASTMatchers/ASTMatchers.h
    M clang/unittests/ASTMatchers/ASTMatchersTest.h
    M clang/unittests/ASTMatchers/CMakeLists.txt
    A clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  [clang][test] add testing for the AST matcher reference

Previously, the examples in the AST matcher reference, which gets
generated by the doxygen comments in `ASTMatchers.h`, were untested
and best effort.
Some of the matchers had no or wrong examples of how to use the matcher.

This patch introduces a simple DSL around doxygen commands to enable
testing the AST matcher documentation in a way that should be relatively
easy.
In `ASTMatchers.h`, most matchers are documented with a doxygen comment.
Most of these also have a code example that aims to show what the
matcher will match, given a matcher somewhere in the documentation text.
The way that testing the documentation is done, is by using doxygens
alias feature to declare custom aliases. These aliases forward to
`<tt>text</tt>` (which is what doxygens \c does, but for multiple words).
Using the doxygen aliases was the obvious choice, because there are
(now) four consumers:
 - people reading the header/using signature help
 - the doxygen generated documentation
 - the generated html AST matcher reference
 - (new) the generated matcher tests

This patch rewrites/extends the documentation such that all matchers
have a documented example.
The new `generate_ast_matcher_doc_tests.py` script will warn on any
undocumented matchers (but not on matchers without a doxygen comment)
and provides diagnostics and statistics about the matchers.
Below is a file-level comment from the test generation script that
describes how documenting matchers to be tested works on a slightly more
technical level. In general, the new comments can be used as a reference
for how to implement a tested documentation.

The current statistics emitted by the parser are:

```text
Statistics:
        doxygen_blocks                :   519
        missing_tests                 :    10
        skipped_objc                  :    42
        code_snippets                 :   503
        matches                       :   820
        matchers                      :   580
        tested_matchers               :   574
        none_type_matchers            :     6
```

The tests are generated during building and the script will only print
something if it found an issue (compile failure, parsing issues,
the expected and actual number of failures differs).

DSL for generating the tests from documentation.

TLDR:
The order for a single code snippet example is:

  \header{a.h}
  \endheader     <- zero or more header

  \code
    int a = 42;
  \endcode
  \compile_args{-std=c++,c23-or-later} <- optional, supports std ranges and
                                          whole languages

  \matcher{expr()} <- one or more matchers in succession
  \match{42}   <- one ore more matches in succession

  \matcher{varDecl()} <- new matcher resets the context, the above
                         \match will not count for this new
                         matcher(-group)
  \match{int a  = 42} <- only applies to the previous matcher (no the
                         previous case)

The above block can be repeated inside of a doxygen command for multiple
code examples.

Language Grammar:
  [] denotes an optional, and <> denotes user-input

  compile_args j:= \compile_args{[<compile_arg>;]<compile_arg>}
  matcher_tag_key ::= type
  match_tag_key ::= type || std || count
  matcher_tags ::= [matcher_tag_key=<value>;]matcher_tag_key=<value>
  match_tags ::= [match_tag_key=<value>;]match_tag_key=<value>
  matcher ::= \matcher{[matcher_tags$]<matcher>}
  matchers ::= [matcher] matcher
  match ::= \match{[match_tags$]<match>}
  matches ::= [match] match
  case ::= matchers matches
  cases ::= [case] case
  header-block ::= \header{<name>} <code> \endheader
  code-block ::= \code <code> \endcode
  testcase ::= code-block [compile_args] cases

The 'std' tag and '\compile_args' support specifying a specific
language version, a whole language and all of it's versions, and thresholds
(implies ranges). Multiple arguments are passed with a ',' seperator.
For a language and version to execute a tested matcher, it has to match
the specified '\compile_args' for the code, and the 'std' tag for the matcher.
Predicates for the 'std' compiler flag are used with disjunction between
languages (e.g. 'c || c++') and conjunction for all predicates specific
to each language (e.g. 'c++11-or-later && c++23-or-earlier').

Examples:
 - c                                    all available versions of C
 - c++11                                only C++11
 - c++11-or-later                       C++11 or later
 - c++11-or-earlier                     C++11 or earlier
 - c++11-or-later,c++23-or-earlier,c    all of C and C++ between 11 and
                                          23 (inclusive)
 - c++11-23,c                             same as above

Tags:

  Type:
  Match types are used to select where the string that is used to check if
  a node matches comes from.
  Available: code, name, typestr, typeofstr.
  The default is 'code'.

  Matcher types are used to mark matchers as submatchers with 'sub' or as
  deactivated using 'none'. Testing submatchers is not implemented.

  Count:
  Specifying a 'count=n' on a match will result in a test that requires that
  the specified match will be matched n times. Default is 1.

  Std:
  A match allows specifying if it matches only in specific language versions.
  This may be needed when the AST differs between language versions.

Fixes #57607
Fixes #63748

  Commit: fb4ad18c0c9f0591f607cef88731c2290ea3bbe5
      https://github.com/llvm/llvm-project/commit/fb4ad18c0c9f0591f607cef88731c2290ea3bbe5
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-09-27 (Fri, 27 Sep 2024)

  Changed paths:
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  fix cuda tests

  Commit: fe524b79d71e22b4bd700a05f2a714001975886e
      https://github.com/llvm/llvm-project/commit/fe524b79d71e22b4bd700a05f2a714001975886e
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-09-27 (Fri, 27 Sep 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  fix cuda tests

  Commit: 8f091783f7c82fee06dec42c2e07dfc24b886694
      https://github.com/llvm/llvm-project/commit/8f091783f7c82fee06dec42c2e07dfc24b886694
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-09-27 (Fri, 27 Sep 2024)

  Changed paths:
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  fix f-string containing backslash

  Commit: 62e08f124618f6632e587d5e90ff21edb37292e5
      https://github.com/llvm/llvm-project/commit/62e08f124618f6632e587d5e90ff21edb37292e5
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-09-27 (Fri, 27 Sep 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h
    M clang/unittests/ASTMatchers/ASTMatchersTest.h
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  rm not needed typeofstr match kind

  Commit: f3c83e870e3769548277162a8569102d5d2d8660
      https://github.com/llvm/llvm-project/commit/f3c83e870e3769548277162a8569102d5d2d8660
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-09-27 (Fri, 27 Sep 2024)

  Changed paths:
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  move some FIXME's to the main function

  Commit: c42669cc5aa90afea5da65d52ee6e3d1709753d2
      https://github.com/llvm/llvm-project/commit/c42669cc5aa90afea5da65d52ee6e3d1709753d2
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-09-27 (Fri, 27 Sep 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h

  Log Message:
  -----------
  replace some type=name matches with explicit code matches to be more expressive

  Commit: ae7e0c99048250fd2fe2281af1ec2cbaae9c7519
      https://github.com/llvm/llvm-project/commit/ae7e0c99048250fd2fe2281af1ec2cbaae9c7519
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-09-27 (Fri, 27 Sep 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h

  Log Message:
  -----------
  add missing comments on some matchers on the number of matches generated

  Commit: 96e6184c07b133e5a77161e8a0c0f7614b3b53fd
      https://github.com/llvm/llvm-project/commit/96e6184c07b133e5a77161e8a0c0f7614b3b53fd
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-09-27 (Fri, 27 Sep 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h

  Log Message:
  -----------
  add documentation (pr desc) to ASTMatchers.h's file comment

  Commit: 9ef9cb019de5a754257e1e8acb5d45dc93889fbc
      https://github.com/llvm/llvm-project/commit/9ef9cb019de5a754257e1e8acb5d45dc93889fbc
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-09-27 (Fri, 27 Sep 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h

  Log Message:
  -----------
  fix empty enum decl

  Commit: 0e0c24f2293bfff115b9d8cc9fd8405a078b85c1
      https://github.com/llvm/llvm-project/commit/0e0c24f2293bfff115b9d8cc9fd8405a078b85c1
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-09-27 (Fri, 27 Sep 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  fix formatting

  Commit: 9ce62906a3a0a6018ab6cbe22c55865afb846d0f
      https://github.com/llvm/llvm-project/commit/9ce62906a3a0a6018ab6cbe22c55865afb846d0f
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-09-27 (Fri, 27 Sep 2024)

  Changed paths:
    M clang/include/clang/ASTMatchers/ASTMatchers.h

  Log Message:
  -----------
  fix formatting

  Commit: 673f78ffc4e559d64111270289fc90709a503ffa
      https://github.com/llvm/llvm-project/commit/673f78ffc4e559d64111270289fc90709a503ffa
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-09-27 (Fri, 27 Sep 2024)

  Changed paths:
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  [clang][test][NFC] fix for python version requirements

Fix for the buildbot failure due to lower python versions not supporting
some types to be subscripted. Tested with python3.8.

  Commit: dc020ad155623064e512097cf5153bdadc7ac221
      https://github.com/llvm/llvm-project/commit/dc020ad155623064e512097cf5153bdadc7ac221
  Author: Julian Schmidt <git.julian.schmidt at gmail.com>
  Date:   2024-09-27 (Fri, 27 Sep 2024)

  Changed paths:
    M clang/utils/generate_ast_matcher_doc_tests.py

  Log Message:
  -----------
  remove testing if the examples compile in the test gen script

Compare: https://github.com/llvm/llvm-project/compare/44b7065d3dec%5E...dc020ad15562

To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications