[lldb-dev] UnicodeDecodeError for serialize SBValue description

[Jeffrey Tan via lldb-dev]

Thanks Siva. All the *DW_TAG_member *related errors seems to go away after
patching with your fix. The current problem is handling the decoding.

Here is the correct decoding from gdb whic might be useful:
(gdb) p corpus
$3 = (const std::string &) @0x7fd133cfb888: {
  static npos = 18446744073709551615, store_ = {
    static kIsLittleEndian = <optimized out>,
    static kIsBigEndian = <optimized out>, {
      small_ = "www", '\000' <repeats 20 times>, "\024", ml_ = {
        data_ = 0x777777 <std::_Any_data::_M_access<void
folly::fibers::Baton::waitFiber<folly::fibers::FirstArgOf<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::{lambda(folly::fibers::Promise<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::SelectionResult>)#1},
void>::type::value_type
folly::fibers::await<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::{lambda(folly::fibers::Promise<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::SelectionResult>)#1}>(folly::fibers::FirstArgOf&&)::{lambda()#1}>(folly::fibers::FiberManager&,
folly::fibers::FirstArgOf<folly::fibers::FirstArgOf<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::{lambda(folly::fibers::Promise<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::SelectionResult>)#1},
void>::type::value_type
folly::fibers::await<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::{lambda(folly::fibers::Promise<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::SelectionResult>)#1}>(folly::fibers::FirstArgOf&&)::{lambda()#1},
void>::type::value_type)::{lambda(folly::fibers::Fiber&)#1}*>() const+25>
"\311\303UH\211\345H\211}\370H\213E\370]ÐUH\211\345H\203\354\020H\211}\370H\213E\370H\211\307\350~\264\312\377\220\311\303UH\211\345SH\203\354\030H\211}\350H\211u\340H\213E\340H\211\307\350\236\377\377\377H\213\030H\213E\350H\211\307\350O\264\312\377H\211ƿ\b",
size_ = 0,
        capacity_ = 1441151880758558720}}}}

Utf-16 does not seem to decode it, while 'latin-1' does:
>>> '\xc9'.decode('utf-16')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File
"/mnt/gvfs/third-party2/python/55c1fd79d91c77c95932db31a4769919611c12bb/2.7.8/centos6-native/da39a3e/lib/python2.7/encodings/utf_16.py",
line 16, in decode
    return codecs.utf_16_decode(input, errors, True)
UnicodeDecodeError: 'utf16' codec can't decode byte 0xc9 in position 0:
truncated data
>>> '\xc9'.decode('latin-1')
u'\xc9'

Instead of guessing what kind of decoding I should use, I would use
'ensure_ascii=False' to prevent the crash for now.

I tried to reproduce this crash, but it seems that the crash might be
related with some internal stl implementation we are using. I will see if I
can narrow down to a small repro later.

Thanks
Jeffrey

On Sun, Mar 27, 2016 at 2:49 PM, Siva Chandra <sivachandra at gmail.com> wrote:

> On Sat, Mar 26, 2016 at 11:58 PM, Jeffrey Tan <jeffrey.fudan at gmail.com>
> wrote:
> > Btw: after patching with Siva's fix http://reviews.llvm.org/D18008, the
> > first field 'small_' is fixed, however the second field 'ml_' still emits
> > garbage:
> >
> > (lldb) fr v corpus
> > (const string &const) corpus = error: summary string parsing error: {
> >   store_ = {
> >      = {
> >       small_ = "www"
> >       ml_ = (data_ =
> >
> "��UH\x89�H�}�H\x8bE�]ÐUH\x89�H��H\x89}�H\x8bE�H\x89��~\xb4��\x90��UH\x89�SH\x83�H\x89}�H�u�H�E�H���\x9e���H\x8b\x18H\x8bE�H���O\xb4��H\x89ƿ\b",
> > size_ = 0, capacity_ = 1441151880758558720)
> >     }
> >   }
> > }
>
> Do you still see the DW_TAG_member related error?
>
> A wild (and really wild at that) guess: Is it utf16 data that is being
> decoded as utf8?
>
> As David Blaikie mentioned on the other thread, it would really help
> if you provide us with a minimal example to repro this. Atleast, repro
> instructions.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20160327/c8c9668a/attachment.html>