[cfe-dev] More thoughts on user defined literals
public at alisdairm.net
Thu Jul 9 10:58:54 PDT 2009
> -----Original Message-----
> From: Chris Lattner [mailto:clattner at apple.com]
> Sent: 09 July 2009 17:43
> To: Eli Friedman
> Cc: AlisdairM(public); clang-dev Developers
> Subject: Re: [cfe-dev] More thoughts on user defined literals
> On Jul 8, 2009, at 4:59 PM, Eli Friedman wrote:
> > On Wed, Jul 8, 2009 at 4:51 PM,
> > AlisdairM(public)<public at alisdairm.net> wrote:
> >> My inexperienced question is - when we move from lex to parse, is
> >> it easy to identify there is no whitespace between the literal
> >> component and the ud-suffix, or should I be trying to take care of
> >> this in the lexing phase?
> > You can check in the parser with hasLeadingSpace(), but it's probably
> > better to handle it in the lexer; it would massively complicate the
> > parser to check for a string in every single place that expects an
> > identifier.
> This is a really dangerous thing to do. I'd strongly recommend
> returning these as one token in the lexer. If not, you end up with
> wierdness like this:
> #define X ud
> (or whatever the syntax is). Splitting them up will also break token
> pasting and a number of other things in the preprocessor.
Yes, that's the syntax and a good point. I think the clincher here for me though could be string concatentation:
auto x = "foo"X L"bar" "!"X;
In this case, we do regular string concatenation for the strings, ignoring the ud-suffix. If that is well-formed, we proceed to check all the ud-suffix, where present, are the same. If that is true, then we call:
operator "" X(wchar_t *, size_t) with our wide-string (in this case).
To help me understand where I am going with this though, could you explain how we would lex and tokenise:
call("foo" L"bar" "!");
Here we concatenate 3 string literals and pass the resulting literal to the function 'call'. That is analogous to applying a user-defined literal so I would like the end result (by the time we get to AST) to be as similar as possible.
More information about the cfe-dev