[cfe-commits] r159223 - in /cfe/trunk: include/clang-c/ include/clang/AST/ lib/AST/ test/Index/ tools/c-index-test/ tools/libclang/ unittests/ unittests/AST/

[Jordan Rose]

On Jun 27, 2012, at 9:33 , Dmitri Gribenko <gribozavr at gmail.com> wrote:

> Why not StringRefs?

Per Doug's comment:

> +  /// Contains text value associated with a token.
> +  StringRef TextValue1;
> +
> +  /// Contains text value associated with a token.
> +  StringRef TextValue2;
> 
> It would be nice if Token were a POD type. Could we just store ptr/length for TextValue1 and TextValue2, so that Token remains a POD?

Oops, I forgot that SourceLocation is POD (of course). Never mind.

> 
> I think that it is useful because the extracted text is the semantic
> value of the token, which is used a lot in the parser and sema.  I
> don't think it is a good idea to require them to strip extra
> characters -- that's what the lexer is for.

Oh, I just meant that for things like commands and literal text, getText() and getCommandName() could do the work on the fly, rather than caching it. But I guess since every token takes the cost for those variables, you might as well use them.

(Actually, that's a bit worrisome. Most tokens have an extra pointer-and-size because they don't need the second string argument. But I can't think of a good way around that off the top of my head.)

Jordan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20120627/e8248f8e/attachment.html>