[clang] [Clang] [Contracts] Design Doc (PR #229978)

[via cfe-commits]

================
@@ -0,0 +1,943 @@
+# Introduction
+
+This is a high level design documentation for Contracts implementation in Clang.
+This summarized experiences from several implementors in seperate branches.
+
+This is also tend to be some developer policy/guide of contracts for how should clang
+developers continue working on this.
+
+## High Level Developing Style
+
+We plan to implement contracts in different small PRs. We don't want to implement
+contracts in a single PR. Smaller PR helps reviewers to understand the patch and give
+higher quality reviewers. Samller PR is also easier for testers to test and find the
+problems.
+
+The implementation contracts is marked as experimental before we announce it is 
+implemented completely.
+
+### ABI
+
+We'd like to keep ABI unstable during the experimental process. We all understand
+ABI's importance. But getting consensus for ABI needs a long time. We prefer to
+move faster instead of waiting the consensus for ABI consensus.
+
+For GCC/libstdc++ interoperability, we need to identify the ABI revision
+which a toolchain targets. We shouldn't assume that one experimental GCC ABI
+will remain unchanged.
+
+### Driver flags
+
+Before we think the support of contracts are minimal usable, users are unable
+to get access to contracts unless users pass `-Xclang` explicitly. 
+
+## AST structure
+
+### Contract Stmt
+
+```C++
+class Stmt {
+
+class ContractStmtBitfields {
+    unsigned Invalid : 1;
+};
+};
+
+class ContractStmtBase : public Stmt {
+  Expr *Predicate = nullptr;
+  SourceLocation KeywordLoc, LParenLoc, RParenLoc;
+  // llvm::ArrayRef<const Attr *> Attributes; // The design allows contracts to have attributes but none
+                                              // contracts attribute are presenting right now.
+};
+
+class ContractAssertStmt : public ContractStmtBase {};
+
+class ContractPreStmt : public ContractStmtBase {};
+
+class ContractPostStmt : public ContractStmtBase {
+  ResultNameDecl *ResultName = nullptr;
+};
+```
+
+Discussion:
+
+We though to use a simple struct to represent `pre` and `post` contracts. But we decided to put all of
+them into Stmt system. The major reason is that now the users can reuse components like 
+`TraverseStmt(...)` more easily.
+
+For attributes, as we don't have any attributes for contracts yet, we can save this on first implemenation.
+
+We split them into 3 Stmt kinds so we can save the bits for ResultName for other contracts types.
+
+For `ResultNameDecl`, see following sections.
+
+### FunctionDecl
+
+```C++
+class FunctionContractSequence final
+    : private llvm::TrailingObjects<FunctionContractSequence, ContractStmtBase *> {
+  friend llvm::TrailingObjects<FunctionContractSequence, ContractStmtBase *>;
+  unsigned NumContracts;
+
+  explicit FunctionContractSequence(unsigned N) : NumContracts(N) {}
+
+public:
+  static FunctionContractSequence *Create(
+      ASTContext &Ctx, llvm::ArrayRef<ContractStmtBase *> Contracts);
+
+  llvm::ArrayRef<ContractStmtBase *> contracts() const {
+    return {getTrailingObjects<ContractStmtBase *>(), NumContracts};
+  }
+};
+
+class FunctionDecl : /*...*/ {
+  /*...*/
+private:
+  // The actual contracts of the function.
+  // Even if it is not explicitly written.
+  FunctionContractSequence *Contracts = nullptr;
+
+public:
+  bool hasContracts() const {
+    return Contracts;
+  }
+
+  void setContracts(FunctionContractSequence *ContractsSeq) {
+    Contracts = ContractsSeq;
+  }
+
+  const FunctionContractSequence *getContracts() const {
+    return Contracts;
+  }
+};
+```
+
+Discussion:
+
+We need to use a single sequence for the contracts of a function. e.g.,
+
+```C++
+int f(int x)
+  pre(x > 0)
+  post(r: r > 0);
+
+int f(int x)
+  post(r: r > 0)
+  pre(x > 0); // Illegal
+```
+
+So that we can't use a sequence for `pre` and another sequence for `post`.
+
+Another problem is that we allow redeclarations without contracts, or
+with semantically equal contracts. So
+
+```C++
+int func(int lhs, int rhs) pre(lhs != rhs); // FD1
+int func(int lhs1, int rhs1) { return lhs1 == rhs1; } // FD2
+```
+
+we should think the function declaration `int func(int lhs1, int rhs1)`
+(FD2) have the same contracts as the first one.
+Then it is a problem that how should we store/access the contracts for
+`FD2`.
+
+Usually, as this is related to redeclarations, we might think we can
+get it by iterating redecls. e.g.,
+
+```C++
+for (auto *RD : this->redecls())
+  if (auto *ContractSeq = RD->getContracts())
+    return ContractSeq;
+```
+
+But if we follow this, the contracts we get for `FD2` will be the contracts
+of `FD1`, `pre(lhs != rhs)`. While this is semantically correct,
+`FD2` doesn't have `lhs` or `rhs`. It is not only they have
+different name but they are actually different AST nodes.
+So it will be problematic for code generation or static analysis
+if we simply store the contracts to the declaration
+which the contracts are written.
+
+The simplest solution is that Sema to put the corresponding
+contracts on every function redeclaration.
+
+This means, **Sema is responsible to construct a new contract
+sequence when it saw a redeclaration if the first function has
+contracts, no matter if the redeclaration has contracts written
+or not**.
+
+#### Possible Later Changes
+
+If needed, we can add `IsWritten` information to `FunctionContractSequence`
+to indicate if this sequence is written in source or not.
+This may be helpful for toolings.
+
+But in the first version, we don't have to do that.
+
+### ResultNameDecl
+
+A post contract can introduce a name to represent the return value of the
+function. e.g.,
+
+```C++
+int f(int x) post(result: result > x);
+```
+
+We introduce a new `ValueDecl` for this name.
+
+```C++
+class ResultNameDecl final : public ValueDecl {
+  ResultNameDecl(DeclContext *DC, SourceLocation Loc,
+                 IdentifierInfo *Name, QualType T);
+
+public:
+  static ResultNameDecl *Create(ASTContext &Ctx, FunctionDecl *FD,
+                                SourceLocation Loc, IdentifierInfo *Name,
+                                QualType T);
+};
+```
+
+We don't use `VarDecl` here because the result name doesn't introduce a new
+variable. It doesn't have its own storage, initializer or lifetime. During
+code generation, references to the `ResultNameDecl` are bound to the return
+value of the function. Sema rejects a result name on a function returning
+`void`.
+
+## Parser
+
+When implementing parser, an important detail is that we need to
+deley parsing for member functions.
+
+```C++
+struct S {
+  bool f() const pre(value > 0 && valid());
+
+  bool valid() const;
+  int value;
+};
+```
+
+A function contract specifier in a member declaration is a complete-class
+context. So `value` and `valid` are both valid names in the predicate, even
+though they are declared after `f`.
+
+When Parser first sees the `pre` specifier, the class is not complete yet. If
+we parse the predicate immediately, name lookup can't find these members. It
+may also see an incomplete overload set when the predicate calls a member
+function. This will reject valid programs.
+
+So for contracts on a member function declared inside a class, Parser only
+collects the tokens of the predicate and keeps their source locations. After
+the class is completely parsed, Parser re-enters the class and function scope,
+puts the parameters and `this` back into scope, and parses the cached tokens
+through the normal Sema path.
+
+Free functions and out-of-class member declarations don't have this problem
+and can parse their predicates immediately. `contract_assert` is parsed in a
+function body and doesn't need this delayed parsing either.
+
+We should reuse Clang's existing late-parsed method declaration mechanism,
+which is already used for other complete-class contexts. We shouldn't add a
+second delayed parsing mechanism only for Contracts.
+
+## Construct Contracts
+
+We should reuse normal expression analysis for contract predicates, including
+the contextual conversion to `bool` and full-expression cleanup handling.
+We still check the predicate when the evaluation semantic is `ignore`.
+
+In the above FunctionDecl section, we've mentioned that, Sema is reponsible
+to rebuild the contracts for every redeclaration to make sure the relationships
+between parameters and contracts are correct.
+
+Except this, there are several other details which need special care.
+
+### contract context
+
+We need a contract context in Sema.We maintain a stack of contract contexts in Sema.
+Before analyzing each pre, post or contract_assert predicate, we push a new context.
+When the analysis finishes, we pop it and restore the previous context. It records
+the scope where the predicate starts and the result name, if there is one. The result
+name is only visible in its own postcondition. The context needs to support nesting,
+since a lambda in a predicate can contain another `contract_assert`.
+
+An important use of this context is implicit const qualification. e.g.,
+
+```C++
+bool check(int &);
+bool check(const int &);
+
+void f(int x) pre(check(x)) { // Calls check(const int &).
+  check(x);                   // Calls check(int &).
+}
+```
+
+Sema should apply implicit const qualification when building references in
+the predicate, before overload resolution. We shouldn't change the type of
+`ParmVarDecl`, since that would also change the function body. Applying const
+after the predicate is built is too late: we may already have selected the
+wrong overload.
+
+Another point we need to care is, when deciding if we're still in the contracts,
+we need to look at the decl contexts too instead of looking at the contract context
+only. Otherwise, we may meet problems if the predicate triggers some instantiations.
+e.g.,
+
+```C++
+template <typename T>
+auto positive_after_increment(T value) {
+  ++value; // legal
+  return value > 0;
+}
+
+void f(int x) pre(positive_after_increment(x));
+```
+
+If we decide if we're in contracts by contract context only, we may think `++value`
+in the instantiation as incorrect due to it didn't meet the bar of implicit
+constness.
+
+### Lambdas
+
+We need to extend lambda capture analysis to handle contract scopes. e.g.,
+
+```C++
+void f(int x) pre([x]() mutable { return ++x > 0; }()); // OK.
+void g(int x) pre([&x] { return ++x > 0; }());          // Illegal
+int h() post(r: [r] { return r > 0; }());              // OK
+```
+
+In `f`, `x` refers to a copy introduced inside the predicate. We shouldn't
+reject as `x` changes. In `g`, it still refers to the parameter outside the
+predicate, so the reference is implicitly const. We need to follow the capture
+chain for nested lambdas to make this distinction. Simply making every expression
+const while a contract is being checked will reject valid programs.
+
+Another problem is that we parse a function's contracts before entering its
+body. e.g.,
+
+```C++
+int check(int x)
+  pre([=] {
+    return [=] { return x > 0; }();
+  }())
+  post(r: [r] { return r > 0; }());
+```
+
+Here `check` is only a declaration. Both lambdas in the precondition need to
+capture `x`, and the lambda in the postcondition needs to capture `r`.
+
+When parsing the precondition, the `FunctionDecl` may not even exist yet,
+and `x` may not have its final `DeclContext`. If capture analysis assumes
+that a parameter without a function `DeclContext` doesn't need capture, it
+will skip the captures of `x`. If it assumes there is already an enclosing
+function body, walking out of the nested lambdas may also leave the
+declaration context and function scope stack inconsistent.
+
+So we should use the recorded contract scope boundary when analyzing these
+captures, and keep the declaration context and function scope stack consistent.
+Each intervening lambda still needs its own capture. For `[r]`, capture
+analysis also needs to accept `ResultNameDecl` as a capturable local entity,
+instead of assuming that every captured name is a `VarDecl`.
+
+The standard allows a capture-default or simple-capture when the enclosing
+scope is a contract-assertion scope, even without a function body.
+[expr.prim.lambda.capture, paragraph 3.3](https://eel.is/c++draft/expr.prim.lambda.capture#3.3)
+states:
+
+> it appears within a contract assertion and its innermost enclosing scope is
+> the corresponding contract-assertion scope ([basic.scope.contract]).
+
+### Template Instantiation
+
+For templates, we also need to instantiate contracts on odr-use, even when
+the function body isn't available. So we shouldn't tie contract instantiation
+only to function body instantiation. e.g.,
+
+```C++
+template <typename T>
+void f(T value) pre(value.size() > 0);
+
+void use_f() {
+  f(1); // Illegal: int has no member named size.
+}
+```
+
+The call odr-uses `f<int>`, so Sema needs to instantiate its predicate. There
+is no function body available here, but we still need to diagnose the invalid
+member access. If we only instantiate contracts when instantiating the body,
+we will miss this error.
+
+## Constract Semantics Checking
----------------
EricWF wrote:

Constant evaluation needs its own section in the doc. It touches the AST, Sema (for ICE contexts), and both constant evaluators.

Under a terminating semantic in a manifestly constant-evaluated context, a violation is ill-formed and makes the expression non-constant, with a note at the assertion ([basic.contract.eval]p7-8). Under `observe`, we emit a diagnostic and the expression stays constant.

A predicate that isn't a core constant expression counts as a violation (p7.3). So `int a[f(1)]`, where `f` hits a non-constant or failing `contract_assert`, is a hard error, not a VLA. Both evaluators have to agree, including on destructor postconditions running after member destruction and reads of the result name.

When checking whether a variable is constant-initialized, assertions are evaluated with `ignore` ([expr.const]p4.1). The initialization itself is manifestly constant-evaluated, so a violation there is ill-formed under `enforce` and diagnosed under `observe`. That means one initializer gets evaluated twice under different semantics, so we can't cache the first result (same thing for constant destruction).

SFINAE and other immediate contexts never observe a violation. The compile-time semantic is implementation-defined and need not match the runtime semantic; how the flags expose that ties into PCH compatibility. See the modules comment.

There are ~20 tests for these cases in `clang/test/Contracts/constexpr-*.cpp` on our branch.


https://github.com/llvm/llvm-project/pull/229978