r198942 - Remove unexpected code completion handling from ConsumeToken()

[Alp Toker]

Author: alp
Date: Fri Jan 10 08:37:02 2014
New Revision: 198942

URL: http://llvm.org/viewvc/llvm-project?rev=198942&view=rev
Log:
Remove unexpected code completion handling from ConsumeToken()

With this change tok::code_completion is finally handled exclusively as a
special token kind like other tokens that need special treatment.

All callers have been updated to use the specific token consumption methods and
the parser has a clear idea the current token isn't special by the time
ConsumeToken() gets called, so this has been unreachable for some time.

ConsumeAnyToken() behaviour is unchanged and will continue to support
unexpected code completion as part of the special token path.

This survived an amount of fuzzing and validation, but please ping the list if
you hit a code path that previously relied on the old unexpected handler and
now asserts.

Modified:
    cfe/trunk/include/clang/Parse/Parser.h

Modified: cfe/trunk/include/clang/Parse/Parser.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Parse/Parser.h?rev=198942&r1=198941&r2=198942&view=diff
==============================================================================
--- cfe/trunk/include/clang/Parse/Parser.h (original)
+++ cfe/trunk/include/clang/Parse/Parser.h Fri Jan 10 08:37:02 2014
@@ -273,16 +273,12 @@ public:
   bool ParseTopLevelDecl(DeclGroupPtrTy &Result);
 
   /// ConsumeToken - Consume the current 'peek token' and lex the next one.
-  /// This does not work with all kinds of tokens: strings and specific other
-  /// tokens must be consumed with custom methods below.  This returns the
-  /// location of the consumed token.
+  /// This does not work with special tokens: string literals, code completion
+  /// and balanced tokens must be handled using the specific consume methods.
+  /// Returns the location of the consumed token.
   SourceLocation ConsumeToken() {
     assert(!isTokenSpecial() &&
            "Should consume special tokens with Consume*Token");
-
-    if (LLVM_UNLIKELY(Tok.is(tok::code_completion)))
-      return handleUnexpectedCodeCompletionToken();
-
     PrevTokLocation = Tok.getLocation();
     PP.Lex(Tok);
     return PrevTokLocation;
@@ -329,7 +325,7 @@ private:
   /// isTokenSpecial - True if this token requires special consumption methods.
   bool isTokenSpecial() const {
     return isTokenStringLiteral() || isTokenParen() || isTokenBracket() ||
-           isTokenBrace();
+           isTokenBrace() || Tok.is(tok::code_completion);
   }
 
   /// \brief Returns true if the current token is '=' or is a type of '='.