[all-commits] [llvm/llvm-project] cdce2f: [Syntax] Remove delayed folding from tree building.

[Marcel Hlopko via All-commits]

  Branch: refs/heads/master
  Home:   https://github.com/llvm/llvm-project
  Commit: cdce2fe561eb6e63d77d1d589e521a8e2afb1eef
      https://github.com/llvm/llvm-project/commit/cdce2fe561eb6e63d77d1d589e521a8e2afb1eef
  Author: Marcel Hlopko <hlopko at google.com>
  Date:   2020-03-31 (Tue, 31 Mar 2020)

  Changed paths:
    M clang/lib/Tooling/Syntax/BuildTree.cpp
    M clang/lib/Tooling/Syntax/Tokens.cpp
    M clang/unittests/Tooling/Syntax/TreeTest.cpp

  Log Message:
  -----------
  [Syntax] Remove delayed folding from tree building.

Summary:
This patch removes delayed folding and replaces it with forward peeking.

Delayed folding was previously used as a solution to the problem that
declaration doesn't have a representation in the AST. For example following
code:

```
int a,b;
```

is expressed in the AST as:

```
TranslationUnitDecl
|-...
|-VarDecl `int a`
`-VarDecl `int b`
```

And in the syntax tree we need:

```
*: TranslationUnit
`-SimpleDeclaration
  |-int
  |-SimpleDeclarator
  | `-a
  |-,
  |-SimpleDeclarator
  | `-b
  |-;
```

So in words, we need to create SimpleDeclaration to be a parent of
SimpleDeclarator nodes. Previously we used delayed folding to make sure SimpleDeclarations will be
eventually created. And in case multiple declarators requested declaration
creation, declaration range was extended to cover all declarators.

This design started to be hard to reason about, so we decided to replace it with
forward peeking. The last declarator node in the chain is responsible for creating
SimpleDeclaration for the whole chain. Range of the declaration corresponds to
the source range of the declarator node. Declarator decides whether its the last
one by peeking to the next AST node (see `isResponsibleForCreatingDeclaration`).

This patch does following:
* Removed delayed folding logic
* Tweaks Token.dumpForTests
* Moves getQualifiedNameStart inside BuildTreeVisitor
* Extracts BuildTreeVisitor.ProcessDeclaratorAndDeclaration
* Renames Builder.getDeclRange to Builder.getDeclarationRange and uses the
  method in all places.
* Adds a bunch of tests

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76922