[cfe-dev] [RFC] cHash: Integration of AST hashing

[Christian Dietrich via cfe-dev]

Boris Kolpackov <boris at codesynthesis.com> writes:

> We are doing something similar in build2[1] though at the token stream
> level (after preprocessing). One thing that we found to limit the
> applicability of this approach is debug info: if you add a typedef
> (presumably as a new line), then function definitions (if any) after
> this line and in the same translation unit will have different debug
> information (line position in the source code).

The benefit of doing it on the token stream is that you can avoid the
expensive parsing. But I wonder how hashing the token stream can be any
better than doing a textual hash on the preprocessed code before lexing
(as ccache is doing it)?

In the current prototype, we do not include debugging information.
Therefore, the hash does not change if you introduce a newline into the
source. In the future, line numbers/filnames should be included, if
debugging information is requested.

> In build2, when calculating the hash, we include the position information
> for each token. Do you do something similar at the AST level? If so, did
> you find anything to mitigate such "line shift" changes?

However, the AST hash should expose less recompilations compared to
hashing the token stream. On the token stream you have to include all
the tokens and, therefore, all the line information/line shifts. The AST
hash includes only the referenced types, enums, and other declarations.
So, if there is a line shift in a header, but none of the declarations that
comes after that shift is used in this compilation unit, the hash will
not change. However, if any of the declarations after the shift is
referenced, the debug information will change and we cannot call it a
redundant recompilation anymore.

chris
-- 
Christian Dietrich, M.Sc. (Scientific Staff)
Institute for Systems Engineering (Systems and Computerarchitecture)
Leibniz Universität Hannover
Appelstraße 4
30167 Hannover, Germany

Tel:    +49 511 762-19737
Fax:    +49 511 762-19733
eMail:  dietrich at sra.uni-hannover.de
WWW:    https://www.sra.uni-hannover.de