[PATCH] Unified token breaking logic: support for line comments.

[Manuel Klimek]

On Wed, Apr 17, 2013 at 9:00 AM, Alexander Kornienko <alexfh at google.com>wrote:

>
> On Apr 17, 2013 9:35 AM, "Manuel Klimek" <klimek at google.com> wrote:
> >
> >
> >
> > ================
> > Comment at: lib/Format/UnwrappedLineParser.h:84
> > @@ +83,3 @@
> > +  /// \brief Returns actual token start location without leading escaped
> > +  /// newlines.
> > +  SourceLocation getRealLocation() const {
> > ----------------
> > It's not only escaped newlines, but all whitespace, right?
> >
> >
> > http://llvm-reviews.chandlerc.com/D682
>
> Whitespace is not part of a token anyway.
>

Yes, whitespace is not part of the Token, but it's part of the FormatToken.
Since we refer to both as "token" in speech, this comment (inside the
context of FormatToken) is highly misleading. FormatToken always contained
all preceeding whitespace. I think the comment needs more context, and I'd
rename it to something different. The problem is again that
getTokenLocation() is ambiguous, as it's unclear whether it's the format
token or the lexer's token.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20130417/fadc39f9/attachment.html>