r198942 - Remove unexpected code completion handling from ConsumeToken()

[Sean Silva]

On Fri, Jan 10, 2014 at 7:37 AM, Alp Toker <alp at nuanti.com> wrote:

> Author: alp
> Date: Fri Jan 10 08:37:02 2014
> New Revision: 198942
>
> URL: http://llvm.org/viewvc/llvm-project?rev=198942&view=rev
> Log:
> Remove unexpected code completion handling from ConsumeToken()
>
> With this change tok::code_completion is finally handled exclusively as a
> special token kind like other tokens that need special treatment.
>
> All callers have been updated to use the specific token consumption
> methods and
> the parser has a clear idea the current token isn't special by the time
> ConsumeToken() gets called, so this has been unreachable for some time.
>
> ConsumeAnyToken() behaviour is unchanged and will continue to support
> unexpected code completion as part of the special token path.
>
> This survived an amount of fuzzing and validation, but please ping the
> list if
> you hit a code path that previously relied on the old unexpected handler
> and
> now asserts.
>
> Modified:
>     cfe/trunk/include/clang/Parse/Parser.h
>
> Modified: cfe/trunk/include/clang/Parse/Parser.h
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Parse/Parser.h?rev=198942&r1=198941&r2=198942&view=diff
>
> ==============================================================================
> --- cfe/trunk/include/clang/Parse/Parser.h (original)
> +++ cfe/trunk/include/clang/Parse/Parser.h Fri Jan 10 08:37:02 2014
> @@ -273,16 +273,12 @@ public:
>    bool ParseTopLevelDecl(DeclGroupPtrTy &Result);
>
>    /// ConsumeToken - Consume the current 'peek token' and lex the next
> one.
> -  /// This does not work with all kinds of tokens: strings and specific
> other
> -  /// tokens must be consumed with custom methods below.  This returns the
> -  /// location of the consumed token.
> +  /// This does not work with special tokens: string literals, code
> completion
> +  /// and balanced tokens must be handled using the specific consume
> methods.
> +  /// Returns the location of the consumed token.
>    SourceLocation ConsumeToken() {
>      assert(!isTokenSpecial() &&
>             "Should consume special tokens with Consume*Token");
> -
> -    if (LLVM_UNLIKELY(Tok.is(tok::code_completion)))
> -      return handleUnexpectedCodeCompletionToken();
>

Is there some central place where we can mark tok::code_completion as
unlikely?

-- Sean Silva


> -
>      PrevTokLocation = Tok.getLocation();
>      PP.Lex(Tok);
>      return PrevTokLocation;
> @@ -329,7 +325,7 @@ private:
>    /// isTokenSpecial - True if this token requires special consumption
> methods.
>    bool isTokenSpecial() const {
>      return isTokenStringLiteral() || isTokenParen() || isTokenBracket() ||
> -           isTokenBrace();
> +           isTokenBrace() || Tok.is(tok::code_completion);
>    }
>
>    /// \brief Returns true if the current token is '=' or is a type of '='.
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20140110/c1cf37ab/attachment.html>