<html>
<head>
<base href="https://bugs.llvm.org/">
</head>
<body><table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Bug ID</th>
<td><a class="bz_bug_link
bz_status_NEW "
title="NEW - codecvt<wchar_t, char, std::mbstate_t>::do_out() does not convert a leading NUL wide character"
href="https://bugs.llvm.org/show_bug.cgi?id=44911">44911</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>codecvt<wchar_t, char, std::mbstate_t>::do_out() does not convert a leading NUL wide character
</td>
</tr>
<tr>
<th>Product</th>
<td>libc++
</td>
</tr>
<tr>
<th>Version</th>
<td>9.0
</td>
</tr>
<tr>
<th>Hardware</th>
<td>PC
</td>
</tr>
<tr>
<th>OS</th>
<td>Linux
</td>
</tr>
<tr>
<th>Status</th>
<td>NEW
</td>
</tr>
<tr>
<th>Severity</th>
<td>normal
</td>
</tr>
<tr>
<th>Priority</th>
<td>P
</td>
</tr>
<tr>
<th>Component</th>
<td>All Bugs
</td>
</tr>
<tr>
<th>Assignee</th>
<td>unassignedclangbugs@nondot.org
</td>
</tr>
<tr>
<th>Reporter</th>
<td>philipp.kutin@dcam.de
</td>
</tr>
<tr>
<th>CC</th>
<td>llvm-bugs@lists.llvm.org, mclow.lists@gmail.com
</td>
</tr></table>
<p>
<div>
<pre>Created <span class=""><a href="attachment.cgi?id=23128" name="attach_23128" title="Test case using C++17">attachment 23128</a> <a href="attachment.cgi?id=23128&action=edit" title="Test case using C++17">[details]</a></span>
Test case using C++17
The mentioned function (found in src/locale.cpp) converts the input wide
character sequence in chunks of wide-NUL-terminated subsequences (exception:
the last such subsequence may not contain a wide NUL).
If one such subsequence is empty (that is, it consists only of the wide NUL)
before then end of the input sequence has been seen, then do_out() prematurely
returns with codecvt<>::partial when it still could have converted the wide NUL
and proceeded with the next subsequence.
This is easiest demonstrated by attempting to convert a single wide NUL. A test
source file is attached to this bug report. To build:
$ clang++-9 -O0 -g -std=c++17 -Wall -Wextra -Wpedantic
codecvt_wchar_t_char_wNUL.cpp -o test_codecvt_libstdcxx
$ clang++-9 -O0 -g -std=c++17 -stdlib=libc++ -Wall -Wextra -Wpedantic
codecvt_wchar_t_char_wNUL.cpp -o test_codecvt_libcxx
(The environment I am testing this in is an Ubuntu Linux 18.04 with a libc++
whose locale.cpp matches git tag 'llvmorg-9.0.1', but I initially observed the
behavior on a machine running macOS Catalina and having XCode installed there
just recently. I also expect the behavior to be present in master because the
source for the function in question is identical there.)
static const auto *ws = L"\0";
const auto res = codeConvert.out(state, ws, ws + 1, fromNext, buf, buf
+ 1, toNext);
assert(res == CodeConvert::ok);
For comparison, the test program compiled against libstdc++ exhibits the
behavior I expect.
The behavior is easily explained when looking at the code: in case the
subsequence passed to __libcpp_wcsnrtombs_l() is empty (fend - frm == 0), there
is nothing for the C function wrapper to do, and zero as return value seems
like the resonable result. However, further below do_out() returns with with
'partial' in that case, even though the return of zero was just because there
was nothing to convert in the first place!
So, I'm wondering: should the block from
// save state in case it is needed to recover to_nxt on error
to and excluding
if (fend != frm_end) // set up next null terminated sequence
rather not be conditional on "if (fend-frm > 0)"? A quick GDB session using a
'jump' in the (hopefully) right place made my test case pass.
I'm assuming that the observed behavior is unintentional. Looking at the C++
Standard to answer the question whether such behavior is allowed -- I'm reading
n4659 25.4.1.4.2 "codecvt virtual functions" [locale.codecvt.virtuals] -- it
looks to me like it may still be OK (as in no contracts broken -- "not all
source characters converted" is just what happened, and 'from_next' and
'to_next' would be consistent) but it seems odd that the ability to convert a
wide NUL character should be context-dependent / behave like C string functions
-- do_out() does take the length of its input sequence explicitly, after all.</pre>
</div>
</p>
<hr>
<span>You are receiving this mail because:</span>
<ul>
<li>You are on the CC list for the bug.</li>
</ul>
</body>
</html>