[cfe-dev] [PATCH] C++0x unicode string and character literals now with test cases

[Seth Cantrell]

Okay, so I haven't changed the serialization stuff. I did restrict getString to only char based strings and added a different function for the serializer to use. I also removed the change I'd made to containsNonAsciiOrNull, but since that uses getString() it's still restricted to char based strings now.

results of 'make test' :

Making Clang 'lit.site.cfg' file...
Making Clang 'Unit/lit.site.cfg' file...
--- Running clang tests for x86_64-apple-darwin11.2.0 ---
lit.py: lit.cfg:143: note: using clang: '/Users/********/llvm-make/Release+Asserts/bin/clang'
Testing Time: 57.02s
  Expected Passes    : 3938
  Expected Failures  : 25
  Unsupported Tests  : 1


-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-enable-use-of-ConvertUTF8toUTF32.patch
Type: application/octet-stream
Size: 7147 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20111025/cbfafb6b/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-Convert-string-literal-data-from-source-to-target-en.patch
Type: application/octet-stream
Size: 7403 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20111025/cbfafb6b/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0003-Represent-string-literals-as-arrays-of-elements-of-t.patch
Type: application/octet-stream
Size: 18950 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20111025/cbfafb6b/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0004-create-tests-for-unicode-string-literal-conversions.patch
Type: application/octet-stream
Size: 4853 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20111025/cbfafb6b/attachment-0003.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0005-update-tests-to-account-for-new-string-constant-repr.patch
Type: application/octet-stream
Size: 7743 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20111025/cbfafb6b/attachment-0004.obj>
-------------- next part --------------




On Oct 24, 2011, at 11:16 PM, Eli Friedman wrote:

> On Mon, Oct 24, 2011 at 8:01 PM, Douglas Gregor <dgregor at apple.com> wrote:
>> 
>> On Oct 24, 2011, at 7:13 PM, Eli Friedman wrote:
>> 
>>> On Mon, Oct 24, 2011 at 6:43 PM, Seth Cantrell <seth.cantrell at gmail.com> wrote:
>>>> 
>>>> Consider the literals L"\xD83D\xDC7F" and L"\U0001F47F" (The former being the UTF-16 surrogate pair corresponding to the latter). For a Windows target these two literals are indistinguishable after we do the initial translation in Sema::ActOnStringLiteral. The resulting StringLiteral will store the same data, and have the same Kind and CharByteWidth. But on a platform with sizeof(wchar_t)==4 the two will not be the same.
>>> 
>>> That isn't what I meant; it's okay for the serialized AST to vary
>>> across targets.  It just shouldn't vary across hosts.
>> 
>> 
>> FWIW, serialization does vary across hosts, because we don't take any care whatsoever to deal with endianness issues in the AST reader/writer (and we tend to read integers from mmap'd files).
> 
> Okay; then I guess we don't need to worry about either issue. :)
> 
> -Eli