[libcxx-commits] [PATCH] D150031: [libc++][format] Adds a UTF transcoder.

[Mark de Wever via Phabricator via libcxx-commits]

Mordante created this revision.
Mordante added reviewers: ldionne, tahonermann.
Herald added a project: All.
Mordante updated this revision to Diff 520068.
Mordante added a comment.
Herald added a subscriber: arichardson.
Mordante updated this revision to Diff 520079.
Mordante updated this revision to Diff 520084.
Mordante published this revision for review.
Herald added a project: libc++.
Herald added a subscriber: libcxx-commits.
Herald added a reviewer: libc++.

CI fixes.


Mordante added a comment.

CI fixes.


Mordante added a comment.

CI fixes.


This is a preparation for

  P2093R14 Formatted output

When the output of print is to the terminal it needs to use the native
API. This means transcoding UTF-8 to UTF-16 on Windows. The encoder's
interface is modeled after

P2728 <https://reviews.llvm.org/P2728> Unicode in the Library, Part 1: UTF Transcoding

But only the required part for P2093R14 is implemented.

On Windows wchar_t is 16 bits, in order to test on platforms where
wchar_t is 32 bits the transcoder has support for char16_t. It also adds
and UTF-8 to UTF-32 encoder which is useful for other tests.

Note it is possible to use <codecvt> for transcoding, but that header is
deprecated. So rather write new code that is not deprecated; the hard
part, decoding, has already been done. The <codecvt> header also
requires locale support while the new code works without including
<locale>.

Note the current transcoder implementation can be optimized since it
basically does UTF-8 -> UTF-32 -> UTF-16. The first goal is to have a
working implementation. Since it's not part of the ABI it's possible to
do the optimization later.

Depends on D149672 <https://reviews.llvm.org/D149672>


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D150031

Files:
  libcxx/include/CMakeLists.txt
  libcxx/include/module.modulemap.in
  libcxx/include/print
  libcxx/test/libcxx/assertions/headers_declare_verbose_abort.sh.cpp
  libcxx/test/libcxx/clang_tidy.sh.cpp
  libcxx/test/libcxx/double_include.sh.cpp
  libcxx/test/libcxx/input.output/iostream.format/print.fun/transcoding.pass.cpp
  libcxx/test/libcxx/min_max_macros.compile.pass.cpp
  libcxx/test/libcxx/modules_include.sh.cpp
  libcxx/test/libcxx/nasty_macros.compile.pass.cpp
  libcxx/test/libcxx/no_assert_include.compile.pass.cpp
  libcxx/test/libcxx/transitive_includes.sh.cpp
  libcxx/test/libcxx/transitive_includes/cxx03.csv
  libcxx/test/libcxx/transitive_includes/cxx11.csv
  libcxx/test/libcxx/transitive_includes/cxx14.csv
  libcxx/test/libcxx/transitive_includes/cxx17.csv
  libcxx/test/libcxx/transitive_includes/cxx20.csv
  libcxx/test/libcxx/transitive_includes/cxx2b.csv
  libcxx/utils/ci/run-buildbot

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D150031.520084.patch
Type: text/x-patch
Size: 62444 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/libcxx-commits/attachments/20230506/64a60b61/attachment-0001.bin>