<html>
<head>
<base href="http://llvm.org/bugs/" />
</head>
<body><table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Bug ID</th>
<td><a class="bz_bug_link
bz_status_NEW "
title="NEW --- - codecvt_utf16 facet with utf-16LE-bom file..."
href="http://llvm.org/bugs/show_bug.cgi?id=20481">20481</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>codecvt_utf16 facet with utf-16LE-bom file...
</td>
</tr>
<tr>
<th>Product</th>
<td>libraries
</td>
</tr>
<tr>
<th>Version</th>
<td>3.4
</td>
</tr>
<tr>
<th>Hardware</th>
<td>PC
</td>
</tr>
<tr>
<th>OS</th>
<td>Linux
</td>
</tr>
<tr>
<th>Status</th>
<td>NEW
</td>
</tr>
<tr>
<th>Severity</th>
<td>normal
</td>
</tr>
<tr>
<th>Priority</th>
<td>P
</td>
</tr>
<tr>
<th>Component</th>
<td>Core LLVM classes
</td>
</tr>
<tr>
<th>Assignee</th>
<td>unassignedbugs@nondot.org
</td>
</tr>
<tr>
<th>Reporter</th>
<td>frederic.metrich@free.fr
</td>
</tr>
<tr>
<th>CC</th>
<td>llvmbugs@cs.uiuc.edu
</td>
</tr>
<tr>
<th>Classification</th>
<td>Unclassified
</td>
</tr></table>
<p>
<div>
<pre>Hi,
It seems, the facet codecvt_utf16 is buggy.
I've a file which is UTF-16LE encoded with BOM.
I define 2 facets like this:
std::locale utf16_locale( std::locale(), new std::codecvt_utf16< wchar_t,
0x10ffff, std::consume_header > );
std::locale utf16_localeLEBOM( std::locale(), new std::codecvt_utf16<
wchar_t, 0x10ffff, static_cast< std::codecvt_mode >( 5 ) /*std::little_endian |
std::consume_header*/ > );
I then define 2 streams like this:
std::wifstream xmlfile1;
std::wifstream xmlfile2;
And read them like this:
{
std::istreambuf_iterator< wchar_t > eos;
xmlfile1.imbue( utf16_locale );
xmlfile1.open( szFilename, std::ios::in | std::ios::binary );
std::wstring strXML( std::istreambuf_iterator< wchar_t >( xmlfile1 ), eos );
}
{
std::istreambuf_iterator< wchar_t > eos;
xmlfile2.imbue( utf16_localeLEBOM );
xmlfile2.open( szFilename, std::ios::in | std::ios::binary );
std::wstring strXML( std::istreambuf_iterator< wchar_t >( xmlfile2 ), eos );
}
With the first one, the BOM isn't recognized/analyzed, and I end up with a
strXML containing the "raw" content of the file, including the BOM!
With the second one, I end up with the correct string, as I can read it with a
text editor.
If need, I can provide a complete package with test-file.
Thx
Fred</pre>
</div>
</p>
<hr>
<span>You are receiving this mail because:</span>
<ul>
<li>You are on the CC list for the bug.</li>
</ul>
</body>
</html>