[cfe-dev] [Review Request] char16_t and char32_t character literals

[Eli Friedman]

On Sun, May 22, 2011 at 2:34 PM, Howard Hinnant <hhinnant at apple.com> wrote:
> On May 22, 2011, at 5:22 PM, Sean Hunt wrote:
>
>> On 11-05-22 06:10 AM, Yusaku Shiga wrote:
>>> * TODO
>>>
>>> (1) No Code Conversion.
>>> At this point, only ascii characters are available in the char16_t and
>>> char32_t constants because
>>> I have not implemented code conversion logic. I plan to fix the problem
>>> in next patch to support
>>> chart16_t and char32_t string literals.
>>
>> Clang needs full UTF-8 support, both inside and outside string-literals.
>> Please bear this in mind when coding support for it, otherwise it will
>> be just as much work to put full UTF-8 support in as it already is, and
>> you'll have wasted effort.
>>
>> The intended design is to convert universal-character-names to UTF-8
>> internally, which we do not currently do.
>
> If it helps, libc++ has conversion among all of UTF-8, UTF-16 and UTF-32 (well UCS4 actually).  It is in locale.cpp (http://llvm.org/svn/llvm-project/libcxx/trunk/src/locale.cpp).  Look for:

We already have routines for that; see include/clang/Basic/ConvertUTF.h.

-Eli