[PATCH] First revision of the dynamic ASTMatcher library

[Samuel Benzaquen]

================
Comment at: lib/ASTMatchers/Dynamic/Parser.cpp:97-98
@@ +96,4 @@
+      size_t TokenLength = 0;
+      while (TokenLength < Code.size() && isAlphanumeric(Code[TokenLength]))
+        ++TokenLength;
+      Result.Kind = TokenInfo::TK_IDENT;
----------------
Dmitri Gribenko wrote:
> What if this loop executes 0 times?  This should be an error.
Fixed. Added an 'invalid' token kind.

================
Comment at: lib/ASTMatchers/Dynamic/Parser.cpp:242
@@ +241,3 @@
+/// parse function.
+bool parseToken(const TokenInfo &Token, CodeTokenizer *Tokenizer,
+                Parser::TokenProcessor *Processor, VariantValue *Value,
----------------
Dmitri Gribenko wrote:
> Samuel Benzaquen wrote:
> > Dmitri Gribenko wrote:
> > > Why doesn't lexer do this?  Every token could have an associated value (optional, of course).
> > It could do the rest of the parsing, except for the matchers. I would have to pass more state to it, though.
> It would be best to associate values with single-token literals in the lexer.  Recursive-descent parsing will be simplified because you will need to parse only high-level constructs in the parser.
Done. Literals are handled by the lexer, and changed TK_STRING to TK_LITERAL that will include other literals too.


http://llvm-reviews.chandlerc.com/D714

BRANCH
  svn

ARCANIST PROJECT
  clang