[cfe-dev] More thoughts on user defined literals

[AlisdairM(public)]

> -----Original Message-----
> From: Chris Lattner [mailto:clattner at apple.com]
> Sent: 09 July 2009 17:43
> To: Eli Friedman
> Cc: AlisdairM(public); clang-dev Developers
> Subject: Re: [cfe-dev] More thoughts on user defined literals
> 
> 
> On Jul 8, 2009, at 4:59 PM, Eli Friedman wrote:
> 
> > On Wed, Jul 8, 2009 at 4:51 PM,
> > AlisdairM(public)<public at alisdairm.net> wrote:
> >> My inexperienced question is - when we move from lex to parse, is
> >> it easy to identify there is no whitespace between the literal
> >> component and the ud-suffix, or should I be trying to take care of
> >> this in the lexing phase?
> >
> > You can check in the parser with hasLeadingSpace(), but it's probably
> > better to handle it in the lexer; it would massively complicate the
> > parser to check for a string in every single place that expects an
> > identifier.
> 
> This is a really dangerous thing to do.  I'd strongly recommend
> returning these as one token in the lexer.  If not, you end up with
> wierdness like this:
> 
> #define X ud
> 
> "foo"X
> 
> (or whatever the syntax is).  Splitting them up will also break token
> pasting and a number of other things in the preprocessor.

Yes, that's the syntax and a good point.  I think the clincher here for me though could be string concatentation:

  auto x = "foo"X L"bar" "!"X;

In this case, we do regular string concatenation for the strings, ignoring the ud-suffix.  If that is well-formed, we proceed to check all the ud-suffix, where present, are the same.  If that is true, then we call:

  operator "" X(wchar_t *, size_t) with our wide-string (in this case).

To help me understand where I am going with this though, could you explain how we would lex and tokenise:

  call("foo" L"bar" "!");

Here we concatenate 3 string literals and pass the resulting literal to the function 'call'.  That is analogous to applying a user-defined literal so I would like the end result (by the time we get to AST) to be as similar as possible.

AlisdairM