[PATCH] Add mask values for the default rune table.

[Jonathan Roelofs]

I dug through my notes some more, found the bug with Newlib/Bionic's non-masks. This testcase demonstrates the issue:

  #include <locale>
  #include <assert.h>
  
  class CheckUpper : public std::ctype_byname<wchar_t> {
  public:
    CheckUpper() : std::ctype_byname<wchar_t>("C",0) { }
    void check() {
      assert(false == do_is(ctype_base::upper, (L"a")[0]));
    }
  };
  
  int main() {
    CheckUpper().check();
  }

The mask for alpha/upper/lower is defined as this in the Newlib section of the ctype_base mask definitions in `<__locale>`:

  static const mask upper = _U;
  static const mask lower = _L;
  static const mask alpha = _U | _L;

Here's the definition of `do_is` in src/locale.cpp:

  1260 bool
  1261 ctype_byname<wchar_t>::do_is(mask m, char_type c) const
  1262 {
  1263 #ifdef _LIBCPP_WCTYPE_IS_MASK
  1264     return static_cast<bool>(iswctype_l(c, m, __l));
  1265 #else
  1266     bool result = false;
  1267     wint_t ch = static_cast<wint_t>(c);
  1268     if (m & space) result |= (iswspace_l(ch, __l) != 0);
  1269     if (m & print) result |= (iswprint_l(ch, __l) != 0);
  1270     if (m & cntrl) result |= (iswcntrl_l(ch, __l) != 0);
  1271     if (m & upper) result |= (iswupper_l(ch, __l) != 0);
  1272     if (m & lower) result |= (iswlower_l(ch, __l) != 0);
  1273     if (m & alpha) result |= (iswalpha_l(ch, __l) != 0);
  1274     if (m & digit) result |= (iswdigit_l(ch, __l) != 0);
  1275     if (m & punct) result |= (iswpunct_l(ch, __l) != 0);
  1276     if (m & xdigit) result |= (iswxdigit_l(ch, __l) != 0);
  1277     if (m & blank) result |= (iswblank_l(ch, __l) != 0);
  1278     return result;
  1279 #endif
  1280 }

In this example, line `1273` sets the result to true, but lower_a is not an upper-case character, so § 22.3.3.1.1 doesn't hold true for isupper:

> ...

> template <class charT> bool isupper (charT c, const locale& loc);

> template <class charT> bool islower (charT c, const locale& loc);

> template <class charT> bool isalpha (charT c, const locale& loc);

> ...

> 1 Each of these functions isF returns the result of the expression:

> `use_facet< ctype<charT> >(loc).is(ctype_base::F, c)`

> where F is the ctype_base::mask value corresponding to that function (22.4.1)


A few months ago, I put in a patch that changes all of the lines like `if (m & alpha) ...` to `if ((m & alpha) == alpha)` to avoid this problem, and shortly thereafter reverted the patch thinking that it was non-conforming. Now I'm not so sure I should have reverted it.


REPOSITORY
  rL LLVM

http://reviews.llvm.org/D8129

EMAIL PREFERENCES
  http://reviews.llvm.org/settings/panel/emailpreferences/