r225619 - Parse: Get rid of tok::cxx_defaultarg_end, use EOF instead

[David Majnemer]

I actually gave cxx_exceptspec_end a decent go but couldn't think of a
great pointer to attach to the EOF.  The best I could come up with was
Actions.CurContext.

On Sun, Jan 11, 2015 at 11:54 PM, Richard Smith <richard at metafoo.co.uk>
wrote:

> Thanks!
>
> Is it possible to do the same to tok::cxx_exceptspec_end? (I also note we
> don't currently use an end-of-delayed-tokens marker when delay-parsing
> function bodies. It probably takes a bit more work to construct a testcase
> that goes awry, but I suspect that case would benefit from using a
> synthetic tok::eof too.)
>
> On Sun, Jan 11, 2015 at 9:17 PM, David Majnemer <david.majnemer at gmail.com>
> wrote:
>
>> Author: majnemer
>> Date: Sun Jan 11 23:17:40 2015
>> New Revision: 225619
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=225619&view=rev
>> Log:
>> Parse: Get rid of tok::cxx_defaultarg_end, use EOF instead
>>
>> I added setEofData/getEofData to solve this sort of problem back in
>> r224505.  Use the Param's decl to tell us if this is *our* EOF token.
>>
>> Modified:
>>     cfe/trunk/include/clang/Basic/TokenKinds.def
>>     cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp
>>     cfe/trunk/lib/Parse/ParseDecl.cpp
>>     cfe/trunk/lib/Parse/Parser.cpp
>>
>> Modified: cfe/trunk/include/clang/Basic/TokenKinds.def
>> URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/TokenKinds.def?rev=225619&r1=225618&r2=225619&view=diff
>>
>> ==============================================================================
>> --- cfe/trunk/include/clang/Basic/TokenKinds.def (original)
>> +++ cfe/trunk/include/clang/Basic/TokenKinds.def Sun Jan 11 23:17:40 2015
>> @@ -116,7 +116,6 @@ TOK(eof)                 // End of file.
>>  TOK(eod)                 // End of preprocessing directive (end of line
>> inside a
>>                           // directive).
>>  TOK(code_completion)     // Code completion marker
>> -TOK(cxx_defaultarg_end)  // C++ default argument end marker
>>  TOK(cxx_exceptspec_end)  // C++ exception-specification end marker
>>
>>  // C99 6.4.9: Comments.
>>
>> Modified: cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp?rev=225619&r1=225618&r2=225619&view=diff
>>
>> ==============================================================================
>> --- cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp (original)
>> +++ cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp Sun Jan 11 23:17:40 2015
>> @@ -342,7 +342,9 @@ void Parser::ParseLexedMethodDeclaration
>>          Actions.ActOnParamDefaultArgumentError(LM.DefaultArgs[I].Param,
>>                                                 EqualLoc);
>>        else {
>> -        if (!TryConsumeToken(tok::cxx_defaultarg_end)) {
>> +        if (Tok.is(tok::eof) && Tok.getEofData() ==
>> LM.DefaultArgs[I].Param) {
>> +          ConsumeAnyToken();
>> +        } else {
>>            // The last two tokens are the terminator and the saved value
>> of
>>            // Tok; the last token in the default argument is the one
>> before
>>            // those.
>> @@ -360,8 +362,11 @@ void Parser::ParseLexedMethodDeclaration
>>               "ParseAssignmentExpression went over the default arg
>> tokens!");
>>        // There could be leftover tokens (e.g. because of an error).
>>        // Skip through until we reach the original token position.
>> -      while (Tok.getLocation() != origLoc && Tok.isNot(tok::eof))
>> +      while (Tok.getLocation() != origLoc) {
>> +        if (Tok.is(tok::eof) && Tok.getEofData() !=
>> LM.DefaultArgs[I].Param)
>> +          break;
>>          ConsumeAnyToken();
>> +      }
>>
>>        delete Toks;
>>        LM.DefaultArgs[I].Toks = nullptr;
>> @@ -652,7 +657,6 @@ bool Parser::ConsumeAndStoreUntil(tok::T
>>
>>      switch (Tok.getKind()) {
>>      case tok::eof:
>> -    case tok::cxx_defaultarg_end:
>>      case tok::annot_module_begin:
>>      case tok::annot_module_end:
>>      case tok::annot_module_include:
>>
>> Modified: cfe/trunk/lib/Parse/ParseDecl.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/ParseDecl.cpp?rev=225619&r1=225618&r2=225619&view=diff
>>
>> ==============================================================================
>> --- cfe/trunk/lib/Parse/ParseDecl.cpp (original)
>> +++ cfe/trunk/lib/Parse/ParseDecl.cpp Sun Jan 11 23:17:40 2015
>> @@ -5631,8 +5631,9 @@ void Parser::ParseParameterDeclarationCl
>>              // stop when we parse it later on.
>>              Token DefArgEnd;
>>              DefArgEnd.startToken();
>> -            DefArgEnd.setKind(tok::cxx_defaultarg_end);
>> +            DefArgEnd.setKind(tok::eof);
>>              DefArgEnd.setLocation(Tok.getLocation());
>> +            DefArgEnd.setEofData(Param);
>>              DefArgToks->push_back(DefArgEnd);
>>              Actions.ActOnParamUnparsedDefaultArgument(Param, EqualLoc,
>>
>>  (*DefArgToks)[1].getLocation());
>>
>> Modified: cfe/trunk/lib/Parse/Parser.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/Parser.cpp?rev=225619&r1=225618&r2=225619&view=diff
>>
>> ==============================================================================
>> --- cfe/trunk/lib/Parse/Parser.cpp (original)
>> +++ cfe/trunk/lib/Parse/Parser.cpp Sun Jan 11 23:17:40 2015
>> @@ -262,10 +262,6 @@ bool Parser::SkipUntil(ArrayRef<tok::Tok
>>        // Ran out of tokens.
>>        return false;
>>
>> -    case tok::cxx_defaultarg_end:
>> -      // It's never desirable to consume the 'end-of-default-argument'
>> token.
>> -      return false;
>> -
>>      case tok::annot_pragma_openmp_end:
>>        // Stop before an OpenMP pragma boundary.
>>      case tok::annot_module_begin:
>>
>>
>> _______________________________________________
>> cfe-commits mailing list
>> cfe-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20150112/6d8eddcd/attachment.html>