[PATCH] D96033: [clang-repl] Land initial infrastructure for incremental parsing

[Vassil Vassilev via Phabricator via cfe-commits]

v.g.vassilev added a comment.

>> We probably need to talk about it.
>
> +1. Do you use discord/slack/skype?

I will try to summarize the discussion here. @rjmccall , @rsmith please feel free to correct me if I am wrong or add important points that I missed.
The discussion focused on supporting two major REPL-enabling features.

1. Error recovery:

  cpp
  [cling] #include <vector> // #1
  [cling] std::vector<int> v; v[0].error_here; // #2, we need to undo the template instantiation.
  input_line_4:2:26: error: member reference base type 'std::__1::__vector_base<int, std::__1::allocator<int> >::value_type' (aka 'int') is not a structure or union
   std::vector<int> v; v[0].error_here;
                       ~~~~^~~~~~~~~~~

2. Code unloading:

  cpp
  [cling] .L Adder.h // #1, similar to #include "Adder.h"
  [cling] Add(3, 1) // int Add(int a, int b) {return a - b; }
  (int) 2
  [cling] .U Adder.h // reverts the state prior to #1
  [cling] .L Adder.h
  [cling] Add(3, 1) // int Add(int a, int b) {return a + b; }
  (int) 4

The implementation of (1.) requires tracking of state in the clang Frontend and (2.) requires tracking of state in the clang Frontend and Backend. We discussed the current `DeclUnloader` implementation in Cling which tracks the emission of declarations (from Sema & co). And, upon request, cleans up various data structures such as lookup tables, AST, and CodeGen. It does not yet free the bump allocated AST memory but rather makes the AST nodes unreachable. It has a very similar conceptual design to the `ASTReader` which adds declarations to various data structures under the hood. The `DeclUnloader` does the opposite -- removes declarations from various internal data structures. We understood that the approach is not intrusive to the architecture of clang and can be very useful for a number of other tools and IDEs.

In addition, John pointed out that we can allow freeing chunks of memory if `Sema` and `CodeGen` track more rigorously the template instantiations and various other components which should not be a major challenge. What makes the implementation feasible is two assumptions:
a) each incremental input leaves the compiler in a valid state;
b) each state reversal transitions to a previously valid state.

We seem to have reached consensus that incremental compilation is possible to be supported without major changes in clang's architecture or significant changes in its main mission of being a static compiler.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D96033/new/

https://reviews.llvm.org/D96033