[libcxx-dev] std::ctype, std::numpunct base templates

[James Skene via libcxx-dev]

On Wed, 5 Aug 2020 at 02:41, Marshall Clow <mclow.lists at gmail.com> wrote:

>
>
> On Aug 4, 2020, at 12:57 AM, James Skene <j.skene at gmail.com> wrote:
>
> Hi Marshall,
>
> Thanks for your prompt response.  I have some follow-ups below.  I hope
> you don't object to me pursuing this a little bit.
>
> On Tue, 4 Aug 2020 at 17:30, Marshall Clow <mclow.lists at gmail.com> wrote:
>
>> On Aug 3, 2020, at 8:25 PM, James Skene via libcxx-dev <
>> libcxx-dev at lists.llvm.org> wrote:
>> >
>> > Hi,
>> >
>> > I'm considering submitting a bug report against libc++, but thought I
>> would check here first.
>> >
>> > The standard gives a synopsis for the locale facet std::ctype<CharT> in
>> Section 25.4.1 (C++17, but it is also in C++11), and std::numpunct<CharT>
>> in Section 25.4.3.  Under libc++ specialisations of these templates are
>> defined for char and wchar_t (in header '__locale') but the base template
>> definition is omitted.  This means in particular that I cannot define:
>> >
>> > class my_ctype : public std::ctype<char16_t> { ... };
>> >
>> > class my_numpunct : public std::numpunct<char16_t> { ... };
>>
>> Yes, I believe that this is correct.
>>
>> > This seems to go against the intent of these classes, which include
>> abstract virtual function declarations.  Although clause 30.2.2 restricts
>> the support for stream-based I/O that an implementation must provide to the
>> types char and wchar_t, it is not clear to me that this means the
>> definitions of the above templates can be omitted.
>> >
>> > I would appreciate advice as to: a. whether this is actually an issue
>> or my misreading of the spec; and b. if so, if it is a known issue.
>>
>> https://wg21.link/locale.category lists the required specializations for
>> numpunct and ctype.
>>
>> It doesn’t say anything about the general template.
>>
>> So, I would say this is not a bug.
>>
>
> Does it, anywhere in the specification, state that where required
> specialisations are listed, the need to implement the general template is
> removed?
>
> There seems to be no such statement in Section 20.5.5.2 Conforming
> Implementations.  On the other hand 20.5.2.2.1 states:
>
> A C++ header shall provide the declarations and definitions that appear in
> its synopsis.
>
>
>> From an implementation point of view, the implementations of ctype<char>
>> and type<wchar_t> are fairly different, and have a lot of system -specific
>> knowledge baked into them. I don’t think I really know enough about
>> charXX_t (8/16/32) on various systems to provide high-quality
>> implementations (and in the case of char32_t, in a reasonable size of
>> code/data). ICU does much of this, but it’s quite large.
>>
>>
> I think there is a problem with the way the specification is written in
> this instance.  The virtual functions in the base class should really be
> pure virtual and the descriptions given of their (general) behaviour should
> be requirements on subclasses or specialisations.  GCC and Visual Studio
> both implement these functions by casting (or trying to narrow) their
> character arguments to char.  GCCs inline documentation states
> 'implementations are provided for all the protected virtual functions, but
> will likely not be useful'.  That said, bearing in mind that the ctype
> functionality only applies to characters in the basic execution character
> set, which are usually 7-bit ASCII (invariant) characters, this is probably
> a reasonable choice.  Another choice, and perhaps no more nonstandard than
> simply omitting the base class, would be to implement these functions as
> assertion failures.
>
> I agree that, notwithstanding the way the specification is written, it is
> unreasonable to expect implementers to provide useful implementations for
> more than the required specialisations.  But actually I do not want a
> useful implementation, just the opportunity to override a virtual function.
> As it happens my goal is specifically to extend ctype, numpunct, and the
> other facets to implement the functionality using ICU.  I would prefer this
> work to be portable to clang.
>
>
> When  you try to do this:
> class my_numpunct : public std::numpunct<char16_t> { ... };
>
> You are depending on the existence of `numpunct<char16_t>`. (Whether as an
> explicit class, or via the general template).
> And that’s not something that the standard insists upon.
>
> So I don’t see how you can do this portably.
>
> — Marshall
>

The general templates would be declared as per the spec.  The member
functions would be defined something like this:

template<class charT>

charT

ctype<charT>::

do_widen(char c) const {

    return c <= 0x7f ?

        static_cast<charT>(c) :

        static_cast<charT>('?');

}


template<class charT>

char*

ctype<charT>::

do_widen(const char* low, const char* high, charT* dest) const {

    for(; low < high; low++)

        dest++ = widen(*low);

    return high;

}


template<class charT>

typename numpunct<charT>::char_type

numpunct<charT>::

do_decimal_point() const {

    return use_facet<ctype<typename char_type>>(locale()).widen('.');

}

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/libcxx-dev/attachments/20200805/1fd63630/attachment.html>