[lldb-dev] something just toasted the test suite on OS X

Zachary Turner via lldb-dev lldb-dev at lists.llvm.org
Tue Jan 26 19:52:34 PST 2016 

Humm..  it's a unicode literal, not sure why it's complaining.  I guess
I'll have to crack open my linux machine and see what's going on tomorrow.

On Tue, Jan 26, 2016 at 6:12 PM Todd Fiala <todd.fiala at gmail.com> wrote:

> I'm still getting a lot of these:
>
>
> Traceback (most recent call last):
>   File "test/dotest.py", line 7, in <module>
>     lldbsuite.test.run_suite()
>   File
> "/Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test/dotest.py",
> line 1089, in run_suite
>     resultclass=test_result.LLDBTestResult).run(configuration.suite)
>   File
> "/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/runner.py",
> line 162, in run
>     test(result)
>   File
> "/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py",
> line 65, in __call__
>     return self.run(*args, **kwds)
>   File
> "/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py",
> line 85, in run
>     self._wrapped_run(result)
>   File
> "/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py",
> line 115, in _wrapped_run
>     test._wrapped_run(result, debug)
>   File
> "/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py",
> line 117, in _wrapped_run
>     test(result)
>   File
> "/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/case.py",
> line 433, in __call__
>     return self.run(*args, **kwds)
>   File
> "/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/case.py",
> line 369, in run
>     self.dumpSessionInfo()
>   File
> "/Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test/lldbtest.py",
> line 1810, in dumpSessionInfo
>     print(u"Session info generated @", datetime.datetime.now().ctime(),
> file=self.session)
> TypeError: must be unicode, not str
>
> On Tue, Jan 26, 2016 at 5:42 PM, Todd Fiala <todd.fiala at gmail.com> wrote:
>
>> Oh missed this.  I'll give it a shot, hang on.
>>
>> On Tue, Jan 26, 2016 at 5:14 PM, Zachary Turner <zturner at google.com>
>> wrote:
>>
>>> Bump.  Can I re-submit this?
>>>
>>> On Tue, Jan 26, 2016 at 10:54 AM Zachary Turner <zturner at google.com>
>>> wrote:
>>>
>>>> Can one of you guys try out this patch and see if it works?  If so I'll
>>>> commit it.
>>>>
>>>> I don't know of a way to make this "elegant".  i.e. a single syntax /
>>>> paradigm that works in both versions without introducing any helper
>>>> functions.
>>>>
>>>>
>>>> On Tue, Jan 26, 2016 at 12:26 AM Zachary Turner <zturner at google.com>
>>>> wrote:
>>>>
>>>>> No worries, worst case scenario a sledgehammer solution is to change
>>>>> all the places where we write to the session file to convert to unicode
>>>>> first (which would be a trivial conversion, since everything is going to be
>>>>> ascii, which is already valid utf 8).
>>>>>
>>>>> The reason a problem arose at all is because TestCxxWcharT.py tried to
>>>>> write actual unicode characters to the session file.  I don't think there's
>>>>> any way to prevent that because the characters could appear in a backtrace,
>>>>> in a variable name, or in a test that is specifically testing unicode.  So
>>>>> we can't just have the one place that needs to write unicode encode it as
>>>>> bytes because there's no one place.
>>>>>
>>>>> I'll sleep on it and try to see if there's a better solution.  Maybe
>>>>> we can just make a function called write_session_file() that takes either a
>>>>> string or a unicode, converts it to a unicode if it's not already one, and
>>>>> then writes.
>>>>>
>>>>> On Mon, Jan 25, 2016 at 9:45 PM Todd Fiala <todd.fiala at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Okay we're back to green here:
>>>>>> http://lab.llvm.org:8080/green/job/lldb_build_test/16173/
>>>>>>
>>>>>> Thanks, Enrico!
>>>>>>
>>>>>> Zachary, I may let this rest until the morning.  If you want to try
>>>>>> something else, shoot me a patch and I'll gladly try it.
>>>>>>
>>>>>> -Todd
>>>>>>
>>>>>> On Mon, Jan 25, 2016 at 9:16 PM, Todd Fiala <todd.fiala at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> It's in item 3 from Effective Python, by Brett Slatkin, which goes
>>>>>>> over having methods that always go to unicode or to byte streams taking
>>>>>>> either unicode or byte style strings, for both Python 2 and Python 3.
>>>>>>> Essentially you figure out what you want it to be in, and you write a
>>>>>>> couple helper routes to go in either the "to unicode" or the "to bytes"
>>>>>>> direction.  It basically looks at the type of the string/bytes you give it,
>>>>>>> and makes sure it becomes what you need.  It's going to assume an encoding
>>>>>>> like utf-8.
>>>>>>>
>>>>>>> On Mon, Jan 25, 2016 at 9:09 PM, Zachary Turner <zturner at google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I'm also not sure why Linux isn't failing.  Looking at the
>>>>>>>> documentation for io.write object, i see this:
>>>>>>>>
>>>>>>>> write(*s*)
>>>>>>>> <https://docs.python.org/2/library/io.html#io.TextIOBase.write>
>>>>>>>>
>>>>>>>> Write the unicode
>>>>>>>> <https://docs.python.org/2/library/functions.html#unicode> string
>>>>>>>> *s* to the stream and return the number of characters written.
>>>>>>>> So clearly it does have to be a unicode object, and saying
>>>>>>>> print(self.getvalue(), file=self.session) is clearly NOT printing a unicode
>>>>>>>> string to the file.
>>>>>>>>
>>>>>>>> What's the pattern you're referring to?  You can't convert a string
>>>>>>>> to a unicode without specifying an encoding, and it seems annoying to have
>>>>>>>> to do that on every single call to print.
>>>>>>>>
>>>>>>>> On Mon, Jan 25, 2016 at 8:54 PM Zachary Turner <zturner at google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> sorry, yea I stuck around for a while after that patch waiting for
>>>>>>>>> emails, but nothing came through.  Please revert in the meantime, I'll work
>>>>>>>>> on a fix tomorrow.
>>>>>>>>>
>>>>>>>>> On Mon, Jan 25, 2016 at 8:52 PM Todd Fiala via lldb-dev <
>>>>>>>>> lldb-dev at lists.llvm.org> wrote:
>>>>>>>>>
>>>>>>>>>> I think I see what happened w/r/t why no emails when out when the
>>>>>>>>>> build went heavy red.  (Well they went out internally, but not
>>>>>>>>>> externally).  When I made the change on Friday to improve the workflow for
>>>>>>>>>> the Green Dragon OS X builder and test output, I switched email over to the
>>>>>>>>>> builder step, which doesn't know anything about who made which changes.  So
>>>>>>>>>> it didn't know who to put on the blame list for the broken build.  Drats,
>>>>>>>>>> I'll have to figure that out.
>>>>>>>>>>
>>>>>>>>>> I'd really prefer to have all those stages happening in one build
>>>>>>>>>> step to keep it clear what's going on.
>>>>>>>>>>
>>>>>>>>>> On Mon, Jan 25, 2016 at 8:25 PM, Todd Fiala <todd.fiala at gmail.com
>>>>>>>>>> > wrote:
>>>>>>>>>>
>>>>>>>>>>> Well our whole test suite just stopped running, so yes.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jan 25, 2016 at 6:58 PM, Enrico Granata <
>>>>>>>>>>> egranata at apple.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Jan 25, 2016, at 6:48 PM, Todd Fiala via lldb-dev <
>>>>>>>>>>>> lldb-dev at lists.llvm.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Not sure exactly what it is, but all the tests are failing due
>>>>>>>>>>>> to some bad assumptions of unicode vs. str on Python 2 vs. 3 if I had to
>>>>>>>>>>>> guess.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Author: zturner
>>>>>>>>>>>> Date: Mon Jan 25 18:59:42 2016
>>>>>>>>>>>> New Revision: 258759
>>>>>>>>>>>>
>>>>>>>>>>>> URL: http://llvm.org/viewvc/llvm-project?rev=258759&view=rev
>>>>>>>>>>>> Log:
>>>>>>>>>>>> Write the session log file in UTF-8.
>>>>>>>>>>>>
>>>>>>>>>>>> Previously we were writing in the default encoding, which
>>>>>>>>>>>> depends
>>>>>>>>>>>> on the operating system and is not guaranteed to be unicode
>>>>>>>>>>>> aware.
>>>>>>>>>>>> On Python 3, this would lead to a situation where writing
>>>>>>>>>>>> unicode
>>>>>>>>>>>> text to the log file generates an exception.  The fix here is to
>>>>>>>>>>>> write session logs using the proper encoding, which incidentally
>>>>>>>>>>>> fixes another test, so xfail is removed from that.
>>>>>>>>>>>>
>>>>>>>>>>>> sounds like a likely culprit from what you’re saying
>>>>>>>>>>>>
>>>>>>>>>>>> I am not going to be able to look at details on that, but
>>>>>>>>>>>> here's a link to the log on the OS X builder:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Do you want me to revert?
>>>>>>>>>>>>
>>>>>>>>>>>> http://lab.llvm.org:8080/green/job/lldb_build_test/16166/console
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> -Todd
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> lldb-dev mailing list
>>>>>>>>>>>> lldb-dev at lists.llvm.org
>>>>>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> *- Enrico*
>>>>>>>>>>>> 📩 egranata@.com ☎️ 27683
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> -Todd
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> -Todd
>>>>>>>>>> _______________________________________________
>>>>>>>>>> lldb-dev mailing list
>>>>>>>>>> lldb-dev at lists.llvm.org
>>>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> -Todd
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> -Todd
>>>>>>
>>>>>
>>
>>
>> --
>> -Todd
>>
>
>
>
> --
> -Todd
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20160127/6f629c1a/attachment.html>

More information about the lldb-dev mailing list