[cfe-dev] Patch for UCNs in string literals

[Chris Lattner]

On Jul 4, 2009, at 7:52 AM, AlisdairM wrote:

> The attached patch applies N2170 for C++0x, which broadens the range  
> of
> valid UCNs in string literals, but not identifiers.
>
> It does not address any of the outstanding issues such as no support  
> for
> UCNs in identifiers or character literals.
>
> It does address an outstanding FIXME to issue an extension warning  
> when
> encountering UCNs in non C++/C99 mode.
>
> Attached are two test cases that should go into the directory:
>
> 	...\llvm\tools\clang\test\CXX\lex\lex.charset
>
> For some reason I can't get my patch generator to add new files :¬(
>
> One test is for C, the other for C++0x.
> Both are XFAIL for now, due to a number of unimplemented features this
> paragraph relies on.  However, I have confirmed that n2170 passes the
> correct subset of tests.

Great, thank you for working on this, some thoughts about the patch:

    // Check UCN constraints (C99 6.4.3p2).
-  if ((UcnVal < 0xa0 &&
+  // C++0x has simpler constraints in literals, but not identifiers.
+  // (C++0x 2.3p2 [lex.charset])
+  if ( (UcnVal >= 0xD800 && UcnVal <= 0xDFFF) ||
+       (UcnVal > 0x10FFFF) /* the maximum legal UTF32 value */ ||
+       (!PP.getLangOptions().CPlusPlus0x && UcnVal < 0xa0 &&
        (UcnVal != 0x24 && UcnVal != 0x40 && UcnVal != 0x60 )) // $,  
@, `
-      || (UcnVal >= 0xD800 && UcnVal <= 0xDFFF)
-      || (UcnVal > 0x10FFFF)) /* the maximum legal UTF32 value */ {
+       ) {


Please split this out into a static helper function that takes UcnVal  
and LangOptions.  That will allow you to use early exits and make the  
comments more clear.

+def warn_ucn_extension_in_c89 : Extension<"UCNs are an extension from  
C99">;

How about "universal character names (UCNs) are supported in c89 mode  
as an extension"?

Do you have commit-after-approval access?  If not, please contact me  
off-list for information.

-Chris