[lldb-dev] something just toasted the test suite on OS X

Todd Fiala via lldb-dev lldb-dev at lists.llvm.org
Wed Jan 27 08:03:22 PST 2016 

Okay.  I'm stuck fixing the OS X build, but if you have something else to
check, feel free to send it over my way and I can run it on another machine.

On Tue, Jan 26, 2016 at 7:52 PM, Zachary Turner <zturner at google.com> wrote:

> Humm..  it's a unicode literal, not sure why it's complaining.  I guess
> I'll have to crack open my linux machine and see what's going on tomorrow.
>
> On Tue, Jan 26, 2016 at 6:12 PM Todd Fiala <todd.fiala at gmail.com> wrote:
>
>> I'm still getting a lot of these:
>>
>>
>> Traceback (most recent call last):
>>   File "test/dotest.py", line 7, in <module>
>>     lldbsuite.test.run_suite()
>>   File
>> "/Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test/dotest.py",
>> line 1089, in run_suite
>>     resultclass=test_result.LLDBTestResult).run(configuration.suite)
>>   File
>> "/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/runner.py",
>> line 162, in run
>>     test(result)
>>   File
>> "/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py",
>> line 65, in __call__
>>     return self.run(*args, **kwds)
>>   File
>> "/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py",
>> line 85, in run
>>     self._wrapped_run(result)
>>   File
>> "/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py",
>> line 115, in _wrapped_run
>>     test._wrapped_run(result, debug)
>>   File
>> "/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py",
>> line 117, in _wrapped_run
>>     test(result)
>>   File
>> "/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/case.py",
>> line 433, in __call__
>>     return self.run(*args, **kwds)
>>   File
>> "/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/case.py",
>> line 369, in run
>>     self.dumpSessionInfo()
>>   File
>> "/Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test/lldbtest.py",
>> line 1810, in dumpSessionInfo
>>     print(u"Session info generated @", datetime.datetime.now().ctime(),
>> file=self.session)
>> TypeError: must be unicode, not str
>>
>> On Tue, Jan 26, 2016 at 5:42 PM, Todd Fiala <todd.fiala at gmail.com> wrote:
>>
>>> Oh missed this.  I'll give it a shot, hang on.
>>>
>>> On Tue, Jan 26, 2016 at 5:14 PM, Zachary Turner <zturner at google.com>
>>> wrote:
>>>
>>>> Bump.  Can I re-submit this?
>>>>
>>>> On Tue, Jan 26, 2016 at 10:54 AM Zachary Turner <zturner at google.com>
>>>> wrote:
>>>>
>>>>> Can one of you guys try out this patch and see if it works?  If so
>>>>> I'll commit it.
>>>>>
>>>>> I don't know of a way to make this "elegant".  i.e. a single syntax /
>>>>> paradigm that works in both versions without introducing any helper
>>>>> functions.
>>>>>
>>>>>
>>>>> On Tue, Jan 26, 2016 at 12:26 AM Zachary Turner <zturner at google.com>
>>>>> wrote:
>>>>>
>>>>>> No worries, worst case scenario a sledgehammer solution is to change
>>>>>> all the places where we write to the session file to convert to unicode
>>>>>> first (which would be a trivial conversion, since everything is going to be
>>>>>> ascii, which is already valid utf 8).
>>>>>>
>>>>>> The reason a problem arose at all is because TestCxxWcharT.py tried
>>>>>> to write actual unicode characters to the session file.  I don't think
>>>>>> there's any way to prevent that because the characters could appear in a
>>>>>> backtrace, in a variable name, or in a test that is specifically testing
>>>>>> unicode.  So we can't just have the one place that needs to write unicode
>>>>>> encode it as bytes because there's no one place.
>>>>>>
>>>>>> I'll sleep on it and try to see if there's a better solution.  Maybe
>>>>>> we can just make a function called write_session_file() that takes either a
>>>>>> string or a unicode, converts it to a unicode if it's not already one, and
>>>>>> then writes.
>>>>>>
>>>>>> On Mon, Jan 25, 2016 at 9:45 PM Todd Fiala <todd.fiala at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Okay we're back to green here:
>>>>>>> http://lab.llvm.org:8080/green/job/lldb_build_test/16173/
>>>>>>>
>>>>>>> Thanks, Enrico!
>>>>>>>
>>>>>>> Zachary, I may let this rest until the morning.  If you want to try
>>>>>>> something else, shoot me a patch and I'll gladly try it.
>>>>>>>
>>>>>>> -Todd
>>>>>>>
>>>>>>> On Mon, Jan 25, 2016 at 9:16 PM, Todd Fiala <todd.fiala at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> It's in item 3 from Effective Python, by Brett Slatkin, which goes
>>>>>>>> over having methods that always go to unicode or to byte streams taking
>>>>>>>> either unicode or byte style strings, for both Python 2 and Python 3.
>>>>>>>> Essentially you figure out what you want it to be in, and you write a
>>>>>>>> couple helper routes to go in either the "to unicode" or the "to bytes"
>>>>>>>> direction.  It basically looks at the type of the string/bytes you give it,
>>>>>>>> and makes sure it becomes what you need.  It's going to assume an encoding
>>>>>>>> like utf-8.
>>>>>>>>
>>>>>>>> On Mon, Jan 25, 2016 at 9:09 PM, Zachary Turner <zturner at google.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> I'm also not sure why Linux isn't failing.  Looking at the
>>>>>>>>> documentation for io.write object, i see this:
>>>>>>>>>
>>>>>>>>> write(*s*)
>>>>>>>>> <https://docs.python.org/2/library/io.html#io.TextIOBase.write>
>>>>>>>>>
>>>>>>>>> Write the unicode
>>>>>>>>> <https://docs.python.org/2/library/functions.html#unicode> string
>>>>>>>>> *s* to the stream and return the number of characters written.
>>>>>>>>> So clearly it does have to be a unicode object, and saying
>>>>>>>>> print(self.getvalue(), file=self.session) is clearly NOT printing a unicode
>>>>>>>>> string to the file.
>>>>>>>>>
>>>>>>>>> What's the pattern you're referring to?  You can't convert a
>>>>>>>>> string to a unicode without specifying an encoding, and it seems annoying
>>>>>>>>> to have to do that on every single call to print.
>>>>>>>>>
>>>>>>>>> On Mon, Jan 25, 2016 at 8:54 PM Zachary Turner <zturner at google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> sorry, yea I stuck around for a while after that patch waiting
>>>>>>>>>> for emails, but nothing came through.  Please revert in the meantime, I'll
>>>>>>>>>> work on a fix tomorrow.
>>>>>>>>>>
>>>>>>>>>> On Mon, Jan 25, 2016 at 8:52 PM Todd Fiala via lldb-dev <
>>>>>>>>>> lldb-dev at lists.llvm.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> I think I see what happened w/r/t why no emails when out when
>>>>>>>>>>> the build went heavy red.  (Well they went out internally, but not
>>>>>>>>>>> externally).  When I made the change on Friday to improve the workflow for
>>>>>>>>>>> the Green Dragon OS X builder and test output, I switched email over to the
>>>>>>>>>>> builder step, which doesn't know anything about who made which changes.  So
>>>>>>>>>>> it didn't know who to put on the blame list for the broken build.  Drats,
>>>>>>>>>>> I'll have to figure that out.
>>>>>>>>>>>
>>>>>>>>>>> I'd really prefer to have all those stages happening in one
>>>>>>>>>>> build step to keep it clear what's going on.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jan 25, 2016 at 8:25 PM, Todd Fiala <
>>>>>>>>>>> todd.fiala at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Well our whole test suite just stopped running, so yes.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jan 25, 2016 at 6:58 PM, Enrico Granata <
>>>>>>>>>>>> egranata at apple.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Jan 25, 2016, at 6:48 PM, Todd Fiala via lldb-dev <
>>>>>>>>>>>>> lldb-dev at lists.llvm.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Not sure exactly what it is, but all the tests are failing due
>>>>>>>>>>>>> to some bad assumptions of unicode vs. str on Python 2 vs. 3 if I had to
>>>>>>>>>>>>> guess.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Author: zturner
>>>>>>>>>>>>> Date: Mon Jan 25 18:59:42 2016
>>>>>>>>>>>>> New Revision: 258759
>>>>>>>>>>>>>
>>>>>>>>>>>>> URL: http://llvm.org/viewvc/llvm-project?rev=258759&view=rev
>>>>>>>>>>>>> Log:
>>>>>>>>>>>>> Write the session log file in UTF-8.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Previously we were writing in the default encoding, which
>>>>>>>>>>>>> depends
>>>>>>>>>>>>> on the operating system and is not guaranteed to be unicode
>>>>>>>>>>>>> aware.
>>>>>>>>>>>>> On Python 3, this would lead to a situation where writing
>>>>>>>>>>>>> unicode
>>>>>>>>>>>>> text to the log file generates an exception.  The fix here is
>>>>>>>>>>>>> to
>>>>>>>>>>>>> write session logs using the proper encoding, which
>>>>>>>>>>>>> incidentally
>>>>>>>>>>>>> fixes another test, so xfail is removed from that.
>>>>>>>>>>>>>
>>>>>>>>>>>>> sounds like a likely culprit from what you’re saying
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am not going to be able to look at details on that, but
>>>>>>>>>>>>> here's a link to the log on the OS X builder:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Do you want me to revert?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://lab.llvm.org:8080/green/job/lldb_build_test/16166/console
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> -Todd
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> lldb-dev mailing list
>>>>>>>>>>>>> lldb-dev at lists.llvm.org
>>>>>>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> *- Enrico*
>>>>>>>>>>>>> 📩 egranata@.com ☎️ 27683
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> -Todd
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> -Todd
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> lldb-dev mailing list
>>>>>>>>>>> lldb-dev at lists.llvm.org
>>>>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> -Todd
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> -Todd
>>>>>>>
>>>>>>
>>>
>>>
>>> --
>>> -Todd
>>>
>>
>>
>>
>> --
>> -Todd
>>
>


-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20160127/c0e1e356/attachment-0001.html>

More information about the lldb-dev mailing list