[PATCH] UTF-8 support for clang-format.

[Alexander Kornienko]

Hi klimek, djasper,

Detect if the file is valid UTF-8, and if this is the case, count code
points instead of just using number of bytes in all (hopefully) places, where
number of columns is needed. In particular, use the new
FormatToken.CodePointCount instead of TokenLength where appropriate.
Changed BreakableToken implementations to respect utf-8 character boundaries
when in utf-8 mode.

http://llvm-reviews.chandlerc.com/D918

Files:
  lib/Format/BreakableToken.cpp
  lib/Format/BreakableToken.h
  lib/Format/Format.cpp
  lib/Format/FormatToken.h
  lib/Format/Utils.h
  unittests/Format/FormatTest.cpp
-------------- next part --------------
A non-text attachment was scrubbed...
Name: D918.1.patch
Type: text/x-patch
Size: 29402 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20130604/82112dcd/attachment.bin>