r225619 - Parse: Get rid of tok::cxx_defaultarg_end, use EOF instead
David Majnemer
david.majnemer at gmail.com
Mon Jan 12 01:24:07 PST 2015
On Mon, Jan 12, 2015 at 12:37 AM, Richard Smith <richard at metafoo.co.uk>
wrote:
> On Mon, Jan 12, 2015 at 12:34 AM, David Majnemer <david.majnemer at gmail.com
> > wrote:
>
>> I actually gave cxx_exceptspec_end a decent go but couldn't think of a
>> great pointer to attach to the EOF. The best I could come up with was
>> Actions.CurContext.
>>
>
> All we need is an arbitrary but unique marker; maybe a pointer to some
> handy object on the stack?
>
So the call to Parser::tryParseExceptionSpecification which will invent the
token occurs in a loop and shares a stack frame
with Parser::ParseLexedMethodDeclaration, the intended consumer of the
token, at Parse::ParseCXXMemberSpecification.
I figured that it wasn't worth creating a SmallVector of these markers in
Parse::ParseCXXMemberSpecification and instead have them all share the same
marker: Actions.CurScope. This is all implemented in r225622.
It should be no worse than where we were before with cxx_exceptspec_end.
>
> On Sun, Jan 11, 2015 at 11:54 PM, Richard Smith <richard at metafoo.co.uk>
>> wrote:
>>
>>> Thanks!
>>>
>>> Is it possible to do the same to tok::cxx_exceptspec_end? (I also note
>>> we don't currently use an end-of-delayed-tokens marker when delay-parsing
>>> function bodies. It probably takes a bit more work to construct a testcase
>>> that goes awry, but I suspect that case would benefit from using a
>>> synthetic tok::eof too.)
>>>
>>> On Sun, Jan 11, 2015 at 9:17 PM, David Majnemer <
>>> david.majnemer at gmail.com> wrote:
>>>
>>>> Author: majnemer
>>>> Date: Sun Jan 11 23:17:40 2015
>>>> New Revision: 225619
>>>>
>>>> URL: http://llvm.org/viewvc/llvm-project?rev=225619&view=rev
>>>> Log:
>>>> Parse: Get rid of tok::cxx_defaultarg_end, use EOF instead
>>>>
>>>> I added setEofData/getEofData to solve this sort of problem back in
>>>> r224505. Use the Param's decl to tell us if this is *our* EOF token.
>>>>
>>>> Modified:
>>>> cfe/trunk/include/clang/Basic/TokenKinds.def
>>>> cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp
>>>> cfe/trunk/lib/Parse/ParseDecl.cpp
>>>> cfe/trunk/lib/Parse/Parser.cpp
>>>>
>>>> Modified: cfe/trunk/include/clang/Basic/TokenKinds.def
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/TokenKinds.def?rev=225619&r1=225618&r2=225619&view=diff
>>>>
>>>> ==============================================================================
>>>> --- cfe/trunk/include/clang/Basic/TokenKinds.def (original)
>>>> +++ cfe/trunk/include/clang/Basic/TokenKinds.def Sun Jan 11 23:17:40
>>>> 2015
>>>> @@ -116,7 +116,6 @@ TOK(eof) // End of file.
>>>> TOK(eod) // End of preprocessing directive (end of
>>>> line inside a
>>>> // directive).
>>>> TOK(code_completion) // Code completion marker
>>>> -TOK(cxx_defaultarg_end) // C++ default argument end marker
>>>> TOK(cxx_exceptspec_end) // C++ exception-specification end marker
>>>>
>>>> // C99 6.4.9: Comments.
>>>>
>>>> Modified: cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp?rev=225619&r1=225618&r2=225619&view=diff
>>>>
>>>> ==============================================================================
>>>> --- cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp (original)
>>>> +++ cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp Sun Jan 11 23:17:40
>>>> 2015
>>>> @@ -342,7 +342,9 @@ void Parser::ParseLexedMethodDeclaration
>>>> Actions.ActOnParamDefaultArgumentError(LM.DefaultArgs[I].Param,
>>>> EqualLoc);
>>>> else {
>>>> - if (!TryConsumeToken(tok::cxx_defaultarg_end)) {
>>>> + if (Tok.is(tok::eof) && Tok.getEofData() ==
>>>> LM.DefaultArgs[I].Param) {
>>>> + ConsumeAnyToken();
>>>> + } else {
>>>> // The last two tokens are the terminator and the saved
>>>> value of
>>>> // Tok; the last token in the default argument is the one
>>>> before
>>>> // those.
>>>> @@ -360,8 +362,11 @@ void Parser::ParseLexedMethodDeclaration
>>>> "ParseAssignmentExpression went over the default arg
>>>> tokens!");
>>>> // There could be leftover tokens (e.g. because of an error).
>>>> // Skip through until we reach the original token position.
>>>> - while (Tok.getLocation() != origLoc && Tok.isNot(tok::eof))
>>>> + while (Tok.getLocation() != origLoc) {
>>>> + if (Tok.is(tok::eof) && Tok.getEofData() !=
>>>> LM.DefaultArgs[I].Param)
>>>> + break;
>>>> ConsumeAnyToken();
>>>> + }
>>>>
>>>> delete Toks;
>>>> LM.DefaultArgs[I].Toks = nullptr;
>>>> @@ -652,7 +657,6 @@ bool Parser::ConsumeAndStoreUntil(tok::T
>>>>
>>>> switch (Tok.getKind()) {
>>>> case tok::eof:
>>>> - case tok::cxx_defaultarg_end:
>>>> case tok::annot_module_begin:
>>>> case tok::annot_module_end:
>>>> case tok::annot_module_include:
>>>>
>>>> Modified: cfe/trunk/lib/Parse/ParseDecl.cpp
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/ParseDecl.cpp?rev=225619&r1=225618&r2=225619&view=diff
>>>>
>>>> ==============================================================================
>>>> --- cfe/trunk/lib/Parse/ParseDecl.cpp (original)
>>>> +++ cfe/trunk/lib/Parse/ParseDecl.cpp Sun Jan 11 23:17:40 2015
>>>> @@ -5631,8 +5631,9 @@ void Parser::ParseParameterDeclarationCl
>>>> // stop when we parse it later on.
>>>> Token DefArgEnd;
>>>> DefArgEnd.startToken();
>>>> - DefArgEnd.setKind(tok::cxx_defaultarg_end);
>>>> + DefArgEnd.setKind(tok::eof);
>>>> DefArgEnd.setLocation(Tok.getLocation());
>>>> + DefArgEnd.setEofData(Param);
>>>> DefArgToks->push_back(DefArgEnd);
>>>> Actions.ActOnParamUnparsedDefaultArgument(Param, EqualLoc,
>>>>
>>>> (*DefArgToks)[1].getLocation());
>>>>
>>>> Modified: cfe/trunk/lib/Parse/Parser.cpp
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/Parser.cpp?rev=225619&r1=225618&r2=225619&view=diff
>>>>
>>>> ==============================================================================
>>>> --- cfe/trunk/lib/Parse/Parser.cpp (original)
>>>> +++ cfe/trunk/lib/Parse/Parser.cpp Sun Jan 11 23:17:40 2015
>>>> @@ -262,10 +262,6 @@ bool Parser::SkipUntil(ArrayRef<tok::Tok
>>>> // Ran out of tokens.
>>>> return false;
>>>>
>>>> - case tok::cxx_defaultarg_end:
>>>> - // It's never desirable to consume the 'end-of-default-argument'
>>>> token.
>>>> - return false;
>>>> -
>>>> case tok::annot_pragma_openmp_end:
>>>> // Stop before an OpenMP pragma boundary.
>>>> case tok::annot_module_begin:
>>>>
>>>>
>>>> _______________________________________________
>>>> cfe-commits mailing list
>>>> cfe-commits at cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20150112/dc6ce602/attachment.html>
More information about the cfe-commits
mailing list