r225619 - Parse: Get rid of tok::cxx_defaultarg_end, use EOF instead
Richard Smith
richard at metafoo.co.uk
Mon Jan 12 00:37:58 PST 2015
On Mon, Jan 12, 2015 at 12:34 AM, David Majnemer <david.majnemer at gmail.com>
wrote:
> I actually gave cxx_exceptspec_end a decent go but couldn't think of a
> great pointer to attach to the EOF. The best I could come up with was
> Actions.CurContext.
>
All we need is an arbitrary but unique marker; maybe a pointer to some
handy object on the stack?
On Sun, Jan 11, 2015 at 11:54 PM, Richard Smith <richard at metafoo.co.uk>
> wrote:
>
>> Thanks!
>>
>> Is it possible to do the same to tok::cxx_exceptspec_end? (I also note
>> we don't currently use an end-of-delayed-tokens marker when delay-parsing
>> function bodies. It probably takes a bit more work to construct a testcase
>> that goes awry, but I suspect that case would benefit from using a
>> synthetic tok::eof too.)
>>
>> On Sun, Jan 11, 2015 at 9:17 PM, David Majnemer <david.majnemer at gmail.com
>> > wrote:
>>
>>> Author: majnemer
>>> Date: Sun Jan 11 23:17:40 2015
>>> New Revision: 225619
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=225619&view=rev
>>> Log:
>>> Parse: Get rid of tok::cxx_defaultarg_end, use EOF instead
>>>
>>> I added setEofData/getEofData to solve this sort of problem back in
>>> r224505. Use the Param's decl to tell us if this is *our* EOF token.
>>>
>>> Modified:
>>> cfe/trunk/include/clang/Basic/TokenKinds.def
>>> cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp
>>> cfe/trunk/lib/Parse/ParseDecl.cpp
>>> cfe/trunk/lib/Parse/Parser.cpp
>>>
>>> Modified: cfe/trunk/include/clang/Basic/TokenKinds.def
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/TokenKinds.def?rev=225619&r1=225618&r2=225619&view=diff
>>>
>>> ==============================================================================
>>> --- cfe/trunk/include/clang/Basic/TokenKinds.def (original)
>>> +++ cfe/trunk/include/clang/Basic/TokenKinds.def Sun Jan 11 23:17:40 2015
>>> @@ -116,7 +116,6 @@ TOK(eof) // End of file.
>>> TOK(eod) // End of preprocessing directive (end of line
>>> inside a
>>> // directive).
>>> TOK(code_completion) // Code completion marker
>>> -TOK(cxx_defaultarg_end) // C++ default argument end marker
>>> TOK(cxx_exceptspec_end) // C++ exception-specification end marker
>>>
>>> // C99 6.4.9: Comments.
>>>
>>> Modified: cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp?rev=225619&r1=225618&r2=225619&view=diff
>>>
>>> ==============================================================================
>>> --- cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp (original)
>>> +++ cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp Sun Jan 11 23:17:40
>>> 2015
>>> @@ -342,7 +342,9 @@ void Parser::ParseLexedMethodDeclaration
>>> Actions.ActOnParamDefaultArgumentError(LM.DefaultArgs[I].Param,
>>> EqualLoc);
>>> else {
>>> - if (!TryConsumeToken(tok::cxx_defaultarg_end)) {
>>> + if (Tok.is(tok::eof) && Tok.getEofData() ==
>>> LM.DefaultArgs[I].Param) {
>>> + ConsumeAnyToken();
>>> + } else {
>>> // The last two tokens are the terminator and the saved value
>>> of
>>> // Tok; the last token in the default argument is the one
>>> before
>>> // those.
>>> @@ -360,8 +362,11 @@ void Parser::ParseLexedMethodDeclaration
>>> "ParseAssignmentExpression went over the default arg
>>> tokens!");
>>> // There could be leftover tokens (e.g. because of an error).
>>> // Skip through until we reach the original token position.
>>> - while (Tok.getLocation() != origLoc && Tok.isNot(tok::eof))
>>> + while (Tok.getLocation() != origLoc) {
>>> + if (Tok.is(tok::eof) && Tok.getEofData() !=
>>> LM.DefaultArgs[I].Param)
>>> + break;
>>> ConsumeAnyToken();
>>> + }
>>>
>>> delete Toks;
>>> LM.DefaultArgs[I].Toks = nullptr;
>>> @@ -652,7 +657,6 @@ bool Parser::ConsumeAndStoreUntil(tok::T
>>>
>>> switch (Tok.getKind()) {
>>> case tok::eof:
>>> - case tok::cxx_defaultarg_end:
>>> case tok::annot_module_begin:
>>> case tok::annot_module_end:
>>> case tok::annot_module_include:
>>>
>>> Modified: cfe/trunk/lib/Parse/ParseDecl.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/ParseDecl.cpp?rev=225619&r1=225618&r2=225619&view=diff
>>>
>>> ==============================================================================
>>> --- cfe/trunk/lib/Parse/ParseDecl.cpp (original)
>>> +++ cfe/trunk/lib/Parse/ParseDecl.cpp Sun Jan 11 23:17:40 2015
>>> @@ -5631,8 +5631,9 @@ void Parser::ParseParameterDeclarationCl
>>> // stop when we parse it later on.
>>> Token DefArgEnd;
>>> DefArgEnd.startToken();
>>> - DefArgEnd.setKind(tok::cxx_defaultarg_end);
>>> + DefArgEnd.setKind(tok::eof);
>>> DefArgEnd.setLocation(Tok.getLocation());
>>> + DefArgEnd.setEofData(Param);
>>> DefArgToks->push_back(DefArgEnd);
>>> Actions.ActOnParamUnparsedDefaultArgument(Param, EqualLoc,
>>>
>>> (*DefArgToks)[1].getLocation());
>>>
>>> Modified: cfe/trunk/lib/Parse/Parser.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/Parser.cpp?rev=225619&r1=225618&r2=225619&view=diff
>>>
>>> ==============================================================================
>>> --- cfe/trunk/lib/Parse/Parser.cpp (original)
>>> +++ cfe/trunk/lib/Parse/Parser.cpp Sun Jan 11 23:17:40 2015
>>> @@ -262,10 +262,6 @@ bool Parser::SkipUntil(ArrayRef<tok::Tok
>>> // Ran out of tokens.
>>> return false;
>>>
>>> - case tok::cxx_defaultarg_end:
>>> - // It's never desirable to consume the 'end-of-default-argument'
>>> token.
>>> - return false;
>>> -
>>> case tok::annot_pragma_openmp_end:
>>> // Stop before an OpenMP pragma boundary.
>>> case tok::annot_module_begin:
>>>
>>>
>>> _______________________________________________
>>> cfe-commits mailing list
>>> cfe-commits at cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20150112/3be0d53b/attachment.html>
More information about the cfe-commits
mailing list