[lldb-dev] something just toasted the test suite on OS X

Todd Fiala via lldb-dev lldb-dev at lists.llvm.org
Tue Jan 26 18:12:14 PST 2016


I'm still getting a lot of these:


Traceback (most recent call last):
  File "test/dotest.py", line 7, in <module>
    lldbsuite.test.run_suite()
  File
"/Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test/dotest.py",
line 1089, in run_suite
    resultclass=test_result.LLDBTestResult).run(configuration.suite)
  File
"/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/runner.py",
line 162, in run
    test(result)
  File
"/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py",
line 65, in __call__
    return self.run(*args, **kwds)
  File
"/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py",
line 85, in run
    self._wrapped_run(result)
  File
"/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py",
line 115, in _wrapped_run
    test._wrapped_run(result, debug)
  File
"/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py",
line 117, in _wrapped_run
    test(result)
  File
"/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/case.py",
line 433, in __call__
    return self.run(*args, **kwds)
  File
"/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/case.py",
line 369, in run
    self.dumpSessionInfo()
  File
"/Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test/lldbtest.py",
line 1810, in dumpSessionInfo
    print(u"Session info generated @", datetime.datetime.now().ctime(),
file=self.session)
TypeError: must be unicode, not str


On Tue, Jan 26, 2016 at 5:42 PM, Todd Fiala <todd.fiala at gmail.com> wrote:

> Oh missed this.  I'll give it a shot, hang on.
>
> On Tue, Jan 26, 2016 at 5:14 PM, Zachary Turner <zturner at google.com>
> wrote:
>
>> Bump.  Can I re-submit this?
>>
>> On Tue, Jan 26, 2016 at 10:54 AM Zachary Turner <zturner at google.com>
>> wrote:
>>
>>> Can one of you guys try out this patch and see if it works?  If so I'll
>>> commit it.
>>>
>>> I don't know of a way to make this "elegant".  i.e. a single syntax /
>>> paradigm that works in both versions without introducing any helper
>>> functions.
>>>
>>>
>>> On Tue, Jan 26, 2016 at 12:26 AM Zachary Turner <zturner at google.com>
>>> wrote:
>>>
>>>> No worries, worst case scenario a sledgehammer solution is to change
>>>> all the places where we write to the session file to convert to unicode
>>>> first (which would be a trivial conversion, since everything is going to be
>>>> ascii, which is already valid utf 8).
>>>>
>>>> The reason a problem arose at all is because TestCxxWcharT.py tried to
>>>> write actual unicode characters to the session file.  I don't think there's
>>>> any way to prevent that because the characters could appear in a backtrace,
>>>> in a variable name, or in a test that is specifically testing unicode.  So
>>>> we can't just have the one place that needs to write unicode encode it as
>>>> bytes because there's no one place.
>>>>
>>>> I'll sleep on it and try to see if there's a better solution.  Maybe we
>>>> can just make a function called write_session_file() that takes either a
>>>> string or a unicode, converts it to a unicode if it's not already one, and
>>>> then writes.
>>>>
>>>> On Mon, Jan 25, 2016 at 9:45 PM Todd Fiala <todd.fiala at gmail.com>
>>>> wrote:
>>>>
>>>>> Okay we're back to green here:
>>>>> http://lab.llvm.org:8080/green/job/lldb_build_test/16173/
>>>>>
>>>>> Thanks, Enrico!
>>>>>
>>>>> Zachary, I may let this rest until the morning.  If you want to try
>>>>> something else, shoot me a patch and I'll gladly try it.
>>>>>
>>>>> -Todd
>>>>>
>>>>> On Mon, Jan 25, 2016 at 9:16 PM, Todd Fiala <todd.fiala at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> It's in item 3 from Effective Python, by Brett Slatkin, which goes
>>>>>> over having methods that always go to unicode or to byte streams taking
>>>>>> either unicode or byte style strings, for both Python 2 and Python 3.
>>>>>> Essentially you figure out what you want it to be in, and you write a
>>>>>> couple helper routes to go in either the "to unicode" or the "to bytes"
>>>>>> direction.  It basically looks at the type of the string/bytes you give it,
>>>>>> and makes sure it becomes what you need.  It's going to assume an encoding
>>>>>> like utf-8.
>>>>>>
>>>>>> On Mon, Jan 25, 2016 at 9:09 PM, Zachary Turner <zturner at google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I'm also not sure why Linux isn't failing.  Looking at the
>>>>>>> documentation for io.write object, i see this:
>>>>>>>
>>>>>>> write(*s*)
>>>>>>> <https://docs.python.org/2/library/io.html#io.TextIOBase.write>
>>>>>>>
>>>>>>> Write the unicode
>>>>>>> <https://docs.python.org/2/library/functions.html#unicode> string
>>>>>>> *s* to the stream and return the number of characters written.
>>>>>>> So clearly it does have to be a unicode object, and saying
>>>>>>> print(self.getvalue(), file=self.session) is clearly NOT printing a unicode
>>>>>>> string to the file.
>>>>>>>
>>>>>>> What's the pattern you're referring to?  You can't convert a string
>>>>>>> to a unicode without specifying an encoding, and it seems annoying to have
>>>>>>> to do that on every single call to print.
>>>>>>>
>>>>>>> On Mon, Jan 25, 2016 at 8:54 PM Zachary Turner <zturner at google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> sorry, yea I stuck around for a while after that patch waiting for
>>>>>>>> emails, but nothing came through.  Please revert in the meantime, I'll work
>>>>>>>> on a fix tomorrow.
>>>>>>>>
>>>>>>>> On Mon, Jan 25, 2016 at 8:52 PM Todd Fiala via lldb-dev <
>>>>>>>> lldb-dev at lists.llvm.org> wrote:
>>>>>>>>
>>>>>>>>> I think I see what happened w/r/t why no emails when out when the
>>>>>>>>> build went heavy red.  (Well they went out internally, but not
>>>>>>>>> externally).  When I made the change on Friday to improve the workflow for
>>>>>>>>> the Green Dragon OS X builder and test output, I switched email over to the
>>>>>>>>> builder step, which doesn't know anything about who made which changes.  So
>>>>>>>>> it didn't know who to put on the blame list for the broken build.  Drats,
>>>>>>>>> I'll have to figure that out.
>>>>>>>>>
>>>>>>>>> I'd really prefer to have all those stages happening in one build
>>>>>>>>> step to keep it clear what's going on.
>>>>>>>>>
>>>>>>>>> On Mon, Jan 25, 2016 at 8:25 PM, Todd Fiala <todd.fiala at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Well our whole test suite just stopped running, so yes.
>>>>>>>>>>
>>>>>>>>>> On Mon, Jan 25, 2016 at 6:58 PM, Enrico Granata <
>>>>>>>>>> egranata at apple.com> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Jan 25, 2016, at 6:48 PM, Todd Fiala via lldb-dev <
>>>>>>>>>>> lldb-dev at lists.llvm.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Not sure exactly what it is, but all the tests are failing due
>>>>>>>>>>> to some bad assumptions of unicode vs. str on Python 2 vs. 3 if I had to
>>>>>>>>>>> guess.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Author: zturner
>>>>>>>>>>> Date: Mon Jan 25 18:59:42 2016
>>>>>>>>>>> New Revision: 258759
>>>>>>>>>>>
>>>>>>>>>>> URL: http://llvm.org/viewvc/llvm-project?rev=258759&view=rev
>>>>>>>>>>> Log:
>>>>>>>>>>> Write the session log file in UTF-8.
>>>>>>>>>>>
>>>>>>>>>>> Previously we were writing in the default encoding, which depends
>>>>>>>>>>> on the operating system and is not guaranteed to be unicode
>>>>>>>>>>> aware.
>>>>>>>>>>> On Python 3, this would lead to a situation where writing unicode
>>>>>>>>>>> text to the log file generates an exception.  The fix here is to
>>>>>>>>>>> write session logs using the proper encoding, which incidentally
>>>>>>>>>>> fixes another test, so xfail is removed from that.
>>>>>>>>>>>
>>>>>>>>>>> sounds like a likely culprit from what you’re saying
>>>>>>>>>>>
>>>>>>>>>>> I am not going to be able to look at details on that, but here's
>>>>>>>>>>> a link to the log on the OS X builder:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Do you want me to revert?
>>>>>>>>>>>
>>>>>>>>>>> http://lab.llvm.org:8080/green/job/lldb_build_test/16166/console
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> -Todd
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> lldb-dev mailing list
>>>>>>>>>>> lldb-dev at lists.llvm.org
>>>>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> *- Enrico*
>>>>>>>>>>> 📩 egranata@.com ☎️ 27683
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> -Todd
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> -Todd
>>>>>>>>> _______________________________________________
>>>>>>>>> lldb-dev mailing list
>>>>>>>>> lldb-dev at lists.llvm.org
>>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> -Todd
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> -Todd
>>>>>
>>>>
>
>
> --
> -Todd
>



-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20160126/1429f08c/attachment-0001.html>


More information about the lldb-dev mailing list