[cfe-dev] Bug in template parsing?

[Richard Smith]

On Fri, Mar 8, 2013 at 11:45 AM, John McCall <rjmccall at apple.com> wrote:

> On Mar 8, 2013, at 11:42 AM, Jordan Rose <jordan_rose at apple.com> wrote:
>
> On Mar 8, 2013, at 11:41 , John McCall <rjmccall at apple.com> wrote:
>
> On Mar 8, 2013, at 11:38 AM, Jordan Rose <jordan_rose at apple.com> wrote:
>
> On Mar 8, 2013, at 11:36 , John McCall <rjmccall at apple.com> wrote:
>
> On Mar 8, 2013, at 9:08 AM, Jordan Rose <jordan_rose at apple.com> wrote:
>
> On Mar 8, 2013, at 3:41 , Nicola Gigante <nicola.gigante at gmail.com> wrote:
>
> Hello.
>
> I'm not sure why this piece of code doesn't compile, but I suspect a bug.
> This happens with the clang trunk (r176686) (C++11 mode)
>
> The compilation succeeds if value is made const static OR if I put
> parenthesis around test<1, N>::value.
>
> template<int I, int N>
> struct test
> {
>    bool value = test<1, N>::value;
> };
>
> template<int N>
> struct test<1, N>
> {
>    static const bool value = true;
> };
>
> $ ./llvm/build/bin/clang++ -std=c++11 -fsyntax-only test.cpp
> test.cpp:4:26: error: declaration of 'N' shadows template parameter
>    bool value = test<1, N>::value;
>                         ^
> test.cpp:1:21: note: template parameter is declared here
> template<int I, int N>
>                    ^
> test.cpp:4:27: error: expected ';' at end of declaration list
>    bool value = test<1, N>::value;
>                          ^
>                          ;
> test.cpp:4:24: error: expected '>'
>    bool value = test<1, N>::value;
>                       ^
> 3 errors generated.
>
> Am I missing something (or is it already known)?
>
>
> Clang is parsing this as:
>
> bool value = (test < 1), N > ::value;
>
> The first error is because it thinks you're declaring "bool N", the second
> because "> ::value" isn't a valid initializer for "bool N", and the third
> because "test < 1" isn't a complete template reference.
>
> I can't be sure whether this is correct, and clearly we could have better
> recovery, but it explains what's going on here. I think it's due to the
> fact that we don't know if "test" is a template until *after* it's been
> parsed—it could be an integer that we're trying to compare against 1.
>
>
> Ugh.  I *think* this just a bug and doesn't rise to the level of a
> standards defect.  I did a cursory check and couldn't find an existing
> defect.
>
> The lookup rules for non-static data member initializers are equivalent to
> the rules for being in a member function, i.e. lookup is delayed until the
> class is complete.  However, collecting tokens and delaying their parsing
> is an unambiguous process for member functions, because a member function
> body can be assumed to have balanced delimiters.  That is at least not
> obviously true for non-static data member initializers, because < and > may
> or may not be delimiters depending on whether what came before them was a
> template-id, and we can't decide whether something was a template-id
> without parsing.
>
> So what's apparently currently happening is that we're just collecting
> tokens until we get to a semicolon or comma (but skipping over balanced
> parens, brackets, and braces) and then treating that all as a delayed
> initializer.
>
> Now, I don't know that there's a formal ambiguity here, because a
> template-argument has to be a constant-expression, and a
> constant-expression is just a semantically-constrained
> conditional-expression, which does not include the assignment operator.
>  So, for example, this:
>   bool a = test<1, b=2>::value;
> cannot be parsed as this:
>   bool a = test<1, (b=2)>::value;
> because that's not a grammatically valid template argument.  (It is with
> parens, but then it's almost certainly a *semantically* invalid template
> argument, unless somebody's got a constexpr assignment operator...)
>
> So I think we might be able to tweak our current rule so that we only cut
> off after a comma if what follows looks like a declarator followed by '='.
>
>
> Just for the record, we'd still have a problem with this case, yes?
>
> bool comparison = test<1, *ptr;
>
>
> Ah, right, we'd need to cut off after a comma if what follows looks like a
> declarator followed by '=', or a semicolon, or a comma that we can cut off
> after.
>
> bool templated = test<1, *ptr>::value;
>
>
> This can't possibly be a declaration of 'ptr'; it should be parsed as a
> single initializer.
>
>
> Right, I just included it to demonstrate the common prefix.
>
>
> Right.  Key to my proposed rule is that the prefix is irrelevant;  we
> really don't want to have to do a pseudo-parse of the initializer.
>

This is core issue 325 / PR13657. It is possible to fully disambiguate both
this case and default argument commas, but it's not yet clear whether we
will be required to disambiguate or not. Disambiguation is not trivial in
either case. For default initializers it's possible because we can't have a
'<' in a declarator in a class, and we can't have an unparenthesized '=' in
a template-argument (except in both cases after the token 'operator'). For
default arguments it's possible because any parameter in a class-scope
function declaration *must* have a default argument if the previous
parameter did, and default arguments must start with an '=' (so we can
distinguish a parameter-declaration from a template-argument based on
whether it's followed by an '='). Needless to say, both of these are
*extremely* fragile to tiny changes in the C++ grammar.

I have a patch for disambiguation of these cases, but it's not quite right
in the default argument case -- and it may not match the eventual decision
on CWG 325.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20130308/661a07b1/attachment.html>