[cfe-commits] r125324 - in /cfe/trunk: include/clang/Serialization/ASTWriter.h lib/Serialization/ASTWriter.cpp test/PCH/reinclude1.h
Douglas Gregor
dgregor at apple.com
Thu Feb 10 16:41:42 PST 2011
On Feb 10, 2011, at 4:26 PM, Douglas Gregor wrote:
> Author: dgregor
> Date: Thu Feb 10 18:26:14 2011
> New Revision: 125324
>
> URL: http://llvm.org/viewvc/llvm-project?rev=125324&view=rev
> Log:
> Implement AST/PCH chaining support for macro definitions. Previously,
> we would deserialize all of the macro definitions we knew about while
> serializing the macro definitions at the end of the AST/PCH file. Even
> though we skipped most of them (since they were unchanged), it's still
> a performance problem.
>
> Now, we do the standard AST/PCH chaining trick: watch what identifiers
> are deserialized as macro names, and consider only those identifiers
> (along with macro definitions that have been deserialized/written in
> the source) when serializing the preprocessor state.
For those of us keeping score, this takes us from:
*** AST File Statistics:
*** AST File Statistics:
2 stat cache hits
6 stat cache misses
1/64463 source location entries read (1.469060%)
15606/16956 types read (92.038216%)
59266/89334 declarations read (66.342041%)
47651/61393 identifiers read (77.616341%)
0/7778 selectors read (0.000000%)
24192/34644 statements read (69.830276%)
8809/8809 macros read (100.000000%)
2095/5189 lexical declcontexts read (40.373867%)
0/4587 visible declcontexts read (0.000000%)
0/7716 method pool entries read (0.000000%)
0 method pool misses
to:
*** AST File Statistics:
2 stat cache hits
6 stat cache misses
1/64463 source location entries read (0.001551%)
15606/16956 types read (92.038216%)
59266/89334 declarations read (66.342041%)
38952/61393 identifiers read (63.446976%)
0/7778 selectors read (0.000000%)
24192/34644 statements read (69.830276%)
388/8809 macros read (4.404586%)
2095/5189 lexical declcontexts read (40.373867%)
0/4587 visible declcontexts read (0.000000%)
0/7716 method pool entries read (0.000000%)
0 method pool misses
when creating a chained PCH file for a file t.h that includes Cocoa.h and adds one simple function declaration. We're still getting 388 macros read because of the huge number of declarations and identifiers being read; those are the subject of my next commit.
- Doug
>
> Modified:
> cfe/trunk/include/clang/Serialization/ASTWriter.h
> cfe/trunk/lib/Serialization/ASTWriter.cpp
> cfe/trunk/test/PCH/reinclude1.h
>
> Modified: cfe/trunk/include/clang/Serialization/ASTWriter.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/ASTWriter.h?rev=125324&r1=125323&r2=125324&view=diff
> ==============================================================================
> --- cfe/trunk/include/clang/Serialization/ASTWriter.h (original)
> +++ cfe/trunk/include/clang/Serialization/ASTWriter.h Thu Feb 10 18:26:14 2011
> @@ -184,6 +184,9 @@
> /// defined.
> llvm::DenseMap<const IdentifierInfo *, uint64_t> MacroOffsets;
>
> + /// \brief The set of identifiers that had macro definitions at some point.
> + std::vector<const IdentifierInfo *> DeserializedMacroNames;
> +
> /// \brief The first ID number we can use for our own macro definitions.
> serialization::MacroID FirstMacroID;
>
>
> Modified: cfe/trunk/lib/Serialization/ASTWriter.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/ASTWriter.cpp?rev=125324&r1=125323&r2=125324&view=diff
> ==============================================================================
> --- cfe/trunk/lib/Serialization/ASTWriter.cpp (original)
> +++ cfe/trunk/lib/Serialization/ASTWriter.cpp Thu Feb 10 18:26:14 2011
> @@ -1536,7 +1536,8 @@
> llvm::SmallVector<std::pair<const IdentifierInfo *, MacroInfo *>, 2>
> MacrosToEmit;
> llvm::SmallPtrSet<const IdentifierInfo*, 4> MacroDefinitionsSeen;
> - for (Preprocessor::macro_iterator I = PP.macro_begin(), E = PP.macro_end();
> + for (Preprocessor::macro_iterator I = PP.macro_begin(Chain == 0),
> + E = PP.macro_end(Chain == 0);
> I != E; ++I) {
> MacroDefinitionsSeen.insert(I->first);
> MacrosToEmit.push_back(std::make_pair(I->first, I->second));
> @@ -1547,10 +1548,21 @@
> llvm::array_pod_sort(MacrosToEmit.begin(), MacrosToEmit.end(),
> &compareMacroDefinitions);
>
> + // Resolve any identifiers that defined macros at the time they were
> + // deserialized, adding them to the list of macros to emit (if appropriate).
> + for (unsigned I = 0, N = DeserializedMacroNames.size(); I != N; ++I) {
> + IdentifierInfo *Name
> + = const_cast<IdentifierInfo *>(DeserializedMacroNames[I]);
> + if (Name->hasMacroDefinition() && MacroDefinitionsSeen.insert(Name))
> + MacrosToEmit.push_back(std::make_pair(Name, PP.getMacroInfo(Name)));
> + }
> +
> for (unsigned I = 0, N = MacrosToEmit.size(); I != N; ++I) {
> const IdentifierInfo *Name = MacrosToEmit[I].first;
> MacroInfo *MI = MacrosToEmit[I].second;
> -
> + if (!MI)
> + continue;
> +
> // Don't emit builtin macros like __LINE__ to the AST file unless they have
> // been redefined by the header (in which case they are not isBuiltinMacro).
> // Also skip macros from a AST file if we're chaining.
> @@ -1603,7 +1615,6 @@
> // FIXME: When reading literal tokens, reconstruct the literal pointer if
> // it is needed.
> AddIdentifierRef(Tok.getIdentifierInfo(), Record);
> -
> // FIXME: Should translate token kind to a stable encoding.
> Record.push_back(Tok.getKind());
> // FIXME: Should translate token flags to a stable encoding.
> @@ -3681,6 +3692,8 @@
>
> void ASTWriter::IdentifierRead(IdentID ID, IdentifierInfo *II) {
> IdentifierIDs[II] = ID;
> + if (II->hasMacroDefinition())
> + DeserializedMacroNames.push_back(II);
> }
>
> void ASTWriter::TypeRead(TypeIdx Idx, QualType T) {
>
> Modified: cfe/trunk/test/PCH/reinclude1.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/PCH/reinclude1.h?rev=125324&r1=125323&r2=125324&view=diff
> ==============================================================================
> --- cfe/trunk/test/PCH/reinclude1.h (original)
> +++ cfe/trunk/test/PCH/reinclude1.h Thu Feb 10 18:26:14 2011
> @@ -2,3 +2,6 @@
> int x;
> int y;
> }
> +
> +int foo;
> +#define foo foo
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20110210/82ab7d0c/attachment.html>
More information about the cfe-commits
mailing list