Ok. So we'll definitely need to do some work on this. Getting back to this patch: I still think it sort of cannot be wrong, as the CXXConstructExpr currently has the ParenRange and this patch leads to it being correctly populated. Do you mind getting it in or do you think it makes the current inconsistent situation worse?<div class="gmail_extra">
<br><br><div class="gmail_quote">On Wed, Jul 11, 2012 at 7:23 AM, Richard Smith <span dir="ltr"><<a href="mailto:richard@metafoo.co.uk" target="_blank">richard@metafoo.co.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div class="im">On Tue, Jul 10, 2012 at 10:05 PM, Daniel Jasper <span dir="ltr"><<a href="mailto:djasper@google.com" target="_blank">djasper@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_extra"><div class="gmail_quote"><div><div>On Wed, Jul 11, 2012 at 12:33 AM, Richard Smith <span dir="ltr"><<a href="mailto:richard@metafoo.co.uk" target="_blank">richard@metafoo.co.uk</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div>On Tue, Jul 10, 2012 at 3:07 PM, Daniel Jasper <span dir="ltr"><<a href="mailto:djasper@google.com" target="_blank">djasper@google.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I don't think this is fully what I am trying to fix with this patch. This patch addresses the inconsistency that the CXXConstructExpr contains the correct ParenRange for Stmts like:<div><br></div><div>X value(7);</div>
<div><br></div><div>But an incorrect range if the constructor is called like a function, e.g.:</div><div><br></div><div>X(7);</div></blockquote><div><br></div></div><div>Right. My point is we don't have rules for what a correct range would be, and we need such rules before we can say whether your patch is right. My proposed rule means that the source range is wrong in both cases, and shouldn't include the parens (and that we should drop the ParenRange from CXXConstructExpr entirely). The fix for that is completely different from what you're proposing :-)</div>
<div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>As for printing, I don't know whether the SourceRange of the CXXConstructExpr should include the parentheses. However, for refactoring, it is definitely highly beneficial to know it (and in fact I think we'll have to go and add it to other nodes as well). So, as CXXConstructExpr already contains a member for that, we should definitely populate it as best we can.</div>
</blockquote><div><br></div></div><div>That presumes that the parens are somehow logically part of the CXXConstructExpr, which I think is really the relevant question here. CXXConstructExpr is used in various cases where the parens are either part of some other construct, or where there are no parens, so I don't think it makes much sense to include the parens in the source range. I would imagine that for refactoring, what's really desired is a consistent and rational set of rules for what the source range of an expression means, which, as my examples demonstrate, we're *really really far* from having.</div>
<div><br></div><div>Consider these two cases:</div><div><br></div><div>X value(7);</div><div>int value(7);</div><div><br></div><div>In the first case, the source range for the initializer covers 'value' and the parentheses. For the second case, it covers only the '7'. The right fix for the second case seems naturally to be that we should store the source locations of the parentheses on the VarDecl. And that would remove any need for storing them on the CXXConstructExpr in the first case.</div>
<div><br></div><div>Likewise in these expressions:</div><div><br></div><div>X(7)</div><div>int(7)</div><div><br></div><div>Here, the CXXFunctionalCastExpr already contains the source locations of the parentheses. In the 'int' case, we don't store them anywhere else (and nor do we need to). In the 'X' case, we include the 'X' and the '(' in the source range of the CXXConstructExpr. That is inconsistent and unnecessary.</div>
</div></blockquote><div><br></div></div></div><div>Why do we not need the parenthesis for int(7)? Because there are no diagnostics that can trigger on a range including them?</div></div></div></blockquote><div><br></div>
</div><div>Because they can be derived from the CXXFunctionalCastExpr (also, there's no other AST node where they could reasonably go).</div><div class="im"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_extra"><div class="gmail_quote"><div>I agree with everything you say, there should be a consistent way to determine source locations. However, we need pretty much all the source locations (this includes locations for each c/v qualifier, all parenthesis, possibly braces, ...) in order to provide proper refactoring tools. I gather that some of the inconsistencies have occurred because we are trying to save as much space as possible and thus only add locations that are really necessary for diagnostics. So, I see two challenges:</div>
<div>1) Make the current state more consistent. For this we might need to set up some basic rules and then try to fix what can be fixed.</div><div>2) Provide a way to access the currently unavailable source locations. For this, is see two approaches:</div>
<div> - Include way more source locations in the current AST, possibly guarded by a flag to preserve performance if not needed.</div><div> - Provide means to re-parse the required part of the source code.</div><div><br>
</div><div>Thoughts?</div></div></div>
</blockquote></div></div><br><div>I completely agree.</div><div><br></div><div>I think the biggest problem for (2) is that the AST assumes that the location of the next token after a SourceLocation can be determined, and currently there's no easy way to do that. I think that it's possible to extract that information by poking through the SLocEntries in the SourceManager, without storing any extra data and with only minimal re-lexing (and no macro expansion), but if so, we should package that functionality up and make it generally available.</div>
</blockquote></div><br></div>