[all-commits] [llvm/llvm-project] 59e66c: [libc++][format] Switches to Unicode 15.1. (#86543)

[Mark de Wever via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 59e66c515a475bc53db011f3ccca0d2831314443
      https://github.com/llvm/llvm-project/commit/59e66c515a475bc53db011f3ccca0d2831314443
  Author: Mark de Wever <koraq at xs4all.nl>
  Date:   2024-04-09 (Tue, 09 Apr 2024)

  Changed paths:
    M libcxx/docs/ReleaseNotes/19.rst
    M libcxx/include/CMakeLists.txt
    M libcxx/include/__format/escaped_output_table.h
    A libcxx/include/__format/indic_conjunct_break_table.h
    M libcxx/include/__format/unicode.h
    M libcxx/include/__format/width_estimation_table.h
    M libcxx/include/libcxx.imp
    M libcxx/include/module.modulemap
    M libcxx/test/libcxx/utilities/format/format.string/format.string.std/extended_grapheme_cluster.h
    M libcxx/test/libcxx/utilities/format/format.string/format.string.std/extended_grapheme_cluster.pass.cpp
    M libcxx/utils/CMakeLists.txt
    M libcxx/utils/data/unicode/DerivedCoreProperties.txt
    M libcxx/utils/data/unicode/DerivedGeneralCategory.txt
    M libcxx/utils/data/unicode/EastAsianWidth.txt
    M libcxx/utils/data/unicode/GraphemeBreakProperty.txt
    M libcxx/utils/data/unicode/GraphemeBreakTest.txt
    M libcxx/utils/data/unicode/emoji-data.txt
    M libcxx/utils/generate_extended_grapheme_cluster_table.py
    A libcxx/utils/generate_indic_conjunct_break_table.py

  Log Message:
  -----------
  [libc++][format] Switches to Unicode 15.1. (#86543)

In addition to changes in the tables the extended grapheme clustering
algorithm has been overhauled. Before I considered a separate state
machine to implement the rules. With the new rule GB9c this became more
attractive and the design has changed.

This change initially had quite an impact on the performance. By making
the state machine persistent the performance was improved greatly. Note
it is still slower than before due to the larger Unicode tables.

Before
--------------------------------------------------------------------
Benchmark                          Time             CPU   Iterations
--------------------------------------------------------------------
BM_ascii_text<char>             1891 ns         1889 ns       369504
BM_unicode_text<char>         106642 ns       106397 ns         6576
BM_cyrillic_text<char>         73420 ns        73277 ns         9445
BM_japanese_text<char>         62485 ns        62387 ns        11153
BM_emoji_text<char>             1895 ns         1893 ns       369525
BM_ascii_text<wchar_t>          2015 ns         2013 ns       346887
BM_unicode_text<wchar_t>       92119 ns        92017 ns         7598
BM_cyrillic_text<wchar_t>      62637 ns        62568 ns        11117
BM_japanese_text<wchar_t>      53850 ns        53785 ns        12803
BM_emoji_text<wchar_t>          2016 ns         2014 ns       347325

After
--------------------------------------------------------------------
Benchmark                          Time             CPU   Iterations
--------------------------------------------------------------------
BM_ascii_text<char>             1906 ns         1904 ns       369409
BM_unicode_text<char>         265462 ns       265175 ns         2628
BM_cyrillic_text<char>        181063 ns       180865 ns         3871
BM_japanese_text<char>        130927 ns       130789 ns         5324
BM_emoji_text<char>             1892 ns         1890 ns       370537
BM_ascii_text<wchar_t>          2038 ns         2035 ns       343689
BM_unicode_text<wchar_t>      277603 ns       277282 ns         2526
BM_cyrillic_text<wchar_t>     188558 ns       188339 ns         3727
BM_japanese_text<wchar_t>     133084 ns       132943 ns         5262
BM_emoji_text<wchar_t>          2012 ns         2010 ns       348015

Persistent
--------------------------------------------------------------------
Benchmark                          Time             CPU   Iterations
--------------------------------------------------------------------
BM_ascii_text<char>             1904 ns         1899 ns       367472
BM_unicode_text<char>         133609 ns       133287 ns         5246
BM_cyrillic_text<char>         90185 ns        89941 ns         7796
BM_japanese_text<char>         75137 ns        74946 ns         9316
BM_emoji_text<char>             1906 ns         1901 ns       368081
BM_ascii_text<wchar_t>          2703 ns         2696 ns       259153
BM_unicode_text<wchar_t>      131497 ns       131168 ns         5341
BM_cyrillic_text<wchar_t>      87071 ns        86840 ns         8076
BM_japanese_text<wchar_t>      72279 ns        72099 ns         9682
BM_emoji_text<wchar_t>          2021 ns         2016 ns       346767



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications