[all-commits] [llvm/llvm-project] ac5c99: [clang] Mark keywords that have their own parsing ...

[rsandifo-arm via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: ac5c996d5aaab5f4508531e815e7ee408284e968
      https://github.com/llvm/llvm-project/commit/ac5c996d5aaab5f4508531e815e7ee408284e968
  Author: Richard Sandiford <richard.sandiford at arm.com>
  Date:   2023-05-31 (Wed, 31 May 2023)

  Changed paths:
    M clang/include/clang/Basic/Attr.td

  Log Message:
  -----------
  [clang] Mark keywords that have their own parsing rules

This patch retroactively classifies all existing keyword attributes
as “custom” keyword attributes, in the sense that the keywords have
their own custom parsing rules.  A follow-on patch will add an
alternative type of keyword.

No functional change intended.

Differential Revision: https://reviews.llvm.org/D148699


  Commit: 301eb6b68f30074ee3a90e2dfbd11dfd87076323
      https://github.com/llvm/llvm-project/commit/301eb6b68f30074ee3a90e2dfbd11dfd87076323
  Author: Richard Sandiford <richard.sandiford at arm.com>
  Date:   2023-05-31 (Wed, 31 May 2023)

  Changed paths:
    M clang-tools-extra/pseudo/lib/grammar/CMakeLists.txt
    M clang/docs/InternalsManual.rst
    M clang/include/clang/Basic/Attr.td
    M clang/include/clang/Basic/AttrDocs.td
    M clang/include/clang/Basic/AttributeCommonInfo.h
    M clang/include/clang/Basic/CMakeLists.txt
    M clang/include/clang/Basic/TokenKinds.def
    M clang/include/clang/Basic/TokenKinds.h
    M clang/include/clang/Lex/Token.h
    M clang/lib/AST/TypePrinter.cpp
    M clang/lib/Sema/SemaDeclAttr.cpp
    M clang/lib/Sema/SemaType.cpp
    M clang/lib/Serialization/ASTReaderDecl.cpp
    M clang/lib/Serialization/ASTWriter.cpp
    M clang/unittests/AST/AttrTest.cpp
    M clang/utils/TableGen/ClangAttrEmitter.cpp
    M clang/utils/TableGen/TableGen.cpp
    M clang/utils/TableGen/TableGenBackends.h

  Log Message:
  -----------
  [clang] Add support for “regular” keyword attributes

Platform-specific language extensions often want to provide a way of
indicating that certain functions should be called in a different way,
compiled in a different way, or otherwise treated differently from a
“normal” function.  Honoring these indications is often required for
correctness, rather being than an optimization/QoI thing.

If a function declaration has a property P that matters for correctness,
it will be ODR-incompatible with a function that does not have property P.
If a function type has a property P that affects the calling convention,
it will not be two-way compatible with a function type that does not
have property P.  These properties therefore affect language semantics.
That in turn means that they cannot be treated as standard [[]]
attributes.

Until now, many of these properties have been specified using GNU-style
attributes instead.  GNU attributes have traditionally been more lax
than standard attributes, with many of them having semantic meaning.
Examples include calling conventions and the vector_size attribute.

However, there is a big drawback to using GNU attributes for semantic
information: compilers that don't understand the attributes will
(by default) emit a warning rather than an error.  They will go on to
compile the code as though the attributes weren't present, which will
inevitably lead to wrong code in most cases.  For users who live
dangerously and disable the warning, this wrong code could even be
generated silently.

A more robust approach would be to specify the properties using
keywords, which older compilers would then reject.  Some vendor-specific
extensions have already taken this approach.  But traditionally, each
such keyword has been treated as a language extension in its own right.
This has three major drawbacks:

(1) The parsing rules need to be kept up-to-date as the language evolves.

(2) There are often corner cases that similar extensions handle differently.

(3) Each extension requires more custom code than a standard attribute.

The underlying problem for all three is that, unlike for true attributes,
there is no established template that extensions can reuse.  The purpose
of this patch series is to try to provide such a template.

One option would have been to pick an existing keyword and do whatever
that keyword does.  The problem with that is that most keywords only
apply to specific kinds of types, kinds of decls, etc., and so the
parsing rules are (for good reason) not generally applicable to all
types and decls.

Really, the “only” thing wrong with using standard attributes is that
standard attributes cannot affect semantics.  In all other respects
they provide exactly what we need: a well-defined grammar that evolves
with the language, clear rules about what an attribute appertains to,
and so on.

This series therefore adds keyword “attributes” that can appear
exactly where a standard attribute can appear and that appertain
to exactly what a standard attribute would appertain to.  The link is
mechanical and no opt-outs or variations are allowed.  This should
make the keywords predictable for programmers who are already
familiar with standard attributes.

This does mean that these keywords will be accepted for parsing purposes
in many more places than necessary.  Inappropriate uses will then be
diagnosed during semantic analysis.  However, the compiler would need
to reject the keywords in those positions whatever happens, and treating
them as ostensible attributes shouldn't be any worse than the alternative.
In some cases it might even be better.  For example, SME's
__arm_streaming attribute would make conceptual sense as a statement
attribute, so someone who takes a “try-it-and-see” approach might write:

  __arm_streaming { …block-of-code…; }

In fact, we did consider supporting this originally.  The reason for
rejecting it was that it was too difficult to implement, rather than
because it didn't make conceptual sense.

One slight disadvantage of the keyword-based approach is that it isn't
possible to use #pragma clang attribute with the keywords.  Perhaps we
could add support for that in future, if it turns out to be useful.

For want of a better term, I've called the new attributes "regular"
keyword attributes (in the sense that their parsing is regular wrt
standard attributes), as opposed to "custom" keyword attributes that
have their own parsing rules.

This patch adds the Attr.td support for regular keyword attributes.
Adding an attribute with a RegularKeyword spelling causes tablegen
to define the associated tokens and to record that attributes created
with that syntax are regular keyword attributes rather than custom
keyword attributes.

A follow-on patch contains the main Parse and Sema support,
which is enabled automatically by the Attr.td definition.

Other notes:

* The series does not allow regular keyword attributes to take
arguments, but this could be added in future.

* I wondered about trying to use tablegen for
TypePrinter::printAttributedAfter too, but decided against it.
RegularKeyword is really a spelling-level classification rather
than an attribute-level classification, and in general, an attribute
could have both GNU and RegularKeyword spellings.  In contrast,
printAttributedAfter is only given the attribute kind and the type
that results from applying the attribute.  AFAIK, it doesn't have
access to the original attribute spelling.  This means that some
attribute-specific or type-specific knowledge might be needed
to print the attribute in the best way.

* Generating the tokens automatically from Attr.td means that
pseudo's libgrammar does now depend on tablegen.

* The patch uses the SME __arm_streaming attribute as an example
for testing purposes.  The attribute does not do anything at this
stage.  Later SME-specific patches will add proper semantics for it,
and add other SME-related keyword attributes.

Differential Revision: https://reviews.llvm.org/D148700


  Commit: 33ee5c4663465022ffe288817968e90064d88a09
      https://github.com/llvm/llvm-project/commit/33ee5c4663465022ffe288817968e90064d88a09
  Author: Richard Sandiford <richard.sandiford at arm.com>
  Date:   2023-05-31 (Wed, 31 May 2023)

  Changed paths:
    M clang/examples/Attribute/Attribute.cpp
    M clang/examples/CallSuperAttribute/CallSuperAttrInfo.cpp
    M clang/include/clang/Basic/DiagnosticCommonKinds.td
    M clang/include/clang/Basic/DiagnosticParseKinds.td
    M clang/include/clang/Basic/DiagnosticSemaKinds.td
    M clang/include/clang/Parse/Parser.h
    M clang/include/clang/Sema/DeclSpec.h
    M clang/lib/Parse/ParseDecl.cpp
    M clang/lib/Parse/ParseDeclCXX.cpp
    M clang/lib/Parse/ParseExprCXX.cpp
    M clang/lib/Parse/ParsePragma.cpp
    M clang/lib/Parse/ParseStmt.cpp
    M clang/lib/Parse/ParseTentative.cpp
    M clang/lib/Parse/Parser.cpp
    M clang/lib/Sema/ParsedAttr.cpp
    M clang/lib/Sema/Sema.cpp
    M clang/lib/Sema/SemaDecl.cpp
    M clang/lib/Sema/SemaDeclAttr.cpp
    M clang/lib/Sema/SemaDeclCXX.cpp
    M clang/lib/Sema/SemaStmtAttr.cpp
    M clang/lib/Sema/SemaType.cpp
    A clang/test/Parser/c2x-attribute-keywords.c
    A clang/test/Parser/c2x-attribute-keywords.m
    A clang/test/Parser/cxx0x-keyword-attributes.cpp
    M clang/utils/TableGen/ClangAttrEmitter.cpp

  Log Message:
  -----------
  [clang] Add Parse and Sema support for RegularKeyword attributes

This patch adds the Parse and Sema support for RegularKeyword attributes,
following on from a previous patch that added Attr.td support.

The patch is quite large.  However, nothing outside the tests is
specific to the first RegularKeyword attribute (__arm_streaming).
The patch should therefore be a one-off, up-front cost.  Other
attributes just need an entry in Attr.td and the usual Sema support.

The approach taken in the patch is that the keywords can be used with
any language version.  If standard attributes were added in language
version Y, the keyword rules for version X<Y are the same as they were
for version Y (to the extent possible).  Any extensions beyond Y are
handled in the same way for both keywords and attributes.  This ensures
that existing C++11 successors like C++17 are not treated differently
from versions that have yet to be defined.

Some notes on the implementation:

* The patch emits errors rather than warnings for diagnostics that
relate to keywords.

* Where possible, the patch drops “attribute” from diagnostics
relating to keywords.

* One exception to the previous point is that warnings about C++
extensions do still mention attributes.  The use there seemed OK
since the diagnostics are noting a change in the production rules.

* If a diagnostic string needs to be different for keywords and
attributes, the patch standardizes on passing the attribute/
name/token followed by 0 for attributes and 1 for keywords.

* Although the patch updates warn_attribute_wrong_decl_type_str,
warn_attribute_wrong_decl_type, and warn_attribute_wrong_decl_type,
only the error forms of these strings are used for keywords.

* I couldn't trigger the warnings in checkUnusedDeclAttributes,
even for existing attributes.  An assert on the warnings caused
no failures in the testsuite.  I think in practice all standard
attributes would be diagnosed before this.

* The patch drops a call to standardAttributesAllowed in
ParseFunctionDeclarator.  This is because MaybeParseCXX11Attributes
checks the same thing itself, where appropriate.

* The new tests are based on c2x-attributes.c and
cxx0x-attributes.cpp.  The C++ test also incorporates a version of
cxx11-base-spec-attributes.cpp.  The FIXMEs are carried across from
the originals.

Differential Revision: https://reviews.llvm.org/D148702


Compare: https://github.com/llvm/llvm-project/compare/40c26ec48c8a...33ee5c466346