[cfe-commits] A patch for PPCallbacks::If() & Elif() functions to fix what the FIXME says.

[Douglas Gregor]

Hello Wei,

On Apr 20, 2011, at 7:09 AM, Wei wrote:

> Hi, all,
> 
> I have made a patch to make clang::PPCallbacks::If() & clang::PPCallbacks::Elif() can pass-back a token tree.
> That's what the original FIXME of these 2 functions said: "better to pass in a list (or tree!) of Tokens."
> 
> This patch has passed all the expected tests in the clang-test. (I use Visual C++ 2010 Express as my dev. environment)
> 1>  Testing Time: 738.14s
> 1>    Expected Passes    : 2877
> 1>    Expected Failures  : 26
> 1>    Unsupported Tests  : 5
> 
> I hope this patch can be merged into clang main trunk, and if you have any problems about this patch, please let me know, I'll modify it so that it can be merged.

The preprocessor is an extremely performance-sensitive part of Clang, so we need to be very careful about what computation we perform. Building a tree-like data structure such as

@@ -264,6 +266,57 @@
 
 };
 
+class TokenTreeNode {
+ private:
+  llvm::BumpPtrAllocator *Allocator;
+  Token Self;
+
+  typedef llvm::SmallVector<TokenTreeNode*,3> ChildrenTy;
+  typedef ChildrenTy::iterator iterator;
+  
+  ChildrenTy Children;
+  
+ public:
+  TokenTreeNode(Token &token, llvm::BumpPtrAllocator *allocator) :
+      Allocator(allocator),
+      Self(token) {
+    assert(Allocator != 0);
+  }
+  
+  ~TokenTreeNode() {
+    for (iterator iter = Children.begin(); iter != Children.end(); ++iter) {
+      (*iter)->~TokenTreeNode();
+      Allocator->Deallocate(*iter);
+    }
+  }
+  
+  inline ChildrenTy &getChildren() {
+    return Children;
+  }
+  
+  void addChildren(iterator I, Token &T) {
+    TokenTreeNode *Node = Allocator->Allocate<TokenTreeNode>();
+    new (Node) TokenTreeNode(T, Allocator);
+    Children.insert(I, Node);
+  }
+  
+  void addChildren(iterator I, TokenTreeNode *N) {
+    assert(N != 0);
+    Children.insert(I, N);
+  }
+  
+  inline void addChildren(Token &T) {
+    addChildren(Children.end(), T);
+  }
+  
+  inline void addChildren(TokenTreeNode *N) {
+    assert(N != 0);
+    addChildren(Children.end(), N);
+  }
+  
+  void dump(unsigned IndentLevel);
+};

is likely to incur an unacceptable performance penalty. It's probably reasonable to pass the SourceRange of the expression down to If/ElIf, from which one could re-lex the tokens if desired.

	- Doug