[lldb-dev] something just toasted the test suite on OS X

Tue Jan 26 00:27:05 PST 2016

No worries, worst case scenario a sledgehammer solution is to change all
the places where we write to the session file to convert to unicode first
(which would be a trivial conversion, since everything is going to be
ascii, which is already valid utf 8).

The reason a problem arose at all is because TestCxxWcharT.py tried to
write actual unicode characters to the session file.  I don't think there's
any way to prevent that because the characters could appear in a backtrace,
in a variable name, or in a test that is specifically testing unicode.  So
we can't just have the one place that needs to write unicode encode it as
bytes because there's no one place.

I'll sleep on it and try to see if there's a better solution.  Maybe we can
just make a function called write_session_file() that takes either a string
or a unicode, converts it to a unicode if it's not already one, and then
writes.

On Mon, Jan 25, 2016 at 9:45 PM Todd Fiala <todd.fiala at gmail.com> wrote:

> Okay we're back to green here:
> http://lab.llvm.org:8080/green/job/lldb_build_test/16173/
>
> Thanks, Enrico!
>
> Zachary, I may let this rest until the morning.  If you want to try
> something else, shoot me a patch and I'll gladly try it.
>
> -Todd
>
> On Mon, Jan 25, 2016 at 9:16 PM, Todd Fiala <todd.fiala at gmail.com> wrote:
>
>> It's in item 3 from Effective Python, by Brett Slatkin, which goes over
>> having methods that always go to unicode or to byte streams taking either
>> unicode or byte style strings, for both Python 2 and Python 3.  Essentially
>> you figure out what you want it to be in, and you write a couple helper
>> routes to go in either the "to unicode" or the "to bytes" direction.  It
>> basically looks at the type of the string/bytes you give it, and makes sure
>> it becomes what you need.  It's going to assume an encoding like utf-8.
>>
>> On Mon, Jan 25, 2016 at 9:09 PM, Zachary Turner <zturner at google.com>
>> wrote:
>>
>>> I'm also not sure why Linux isn't failing.  Looking at the documentation
>>> for io.write object, i see this:
>>>
>>> write(*s*)
>>> <https://docs.python.org/2/library/io.html#io.TextIOBase.write>
>>>
>>> Write the unicode
>>> <https://docs.python.org/2/library/functions.html#unicode> string *s* to
>>> the stream and return the number of characters written.
>>> So clearly it does have to be a unicode object, and saying
>>> print(self.getvalue(), file=self.session) is clearly NOT printing a unicode
>>> string to the file.
>>>
>>> What's the pattern you're referring to?  You can't convert a string to a
>>> unicode without specifying an encoding, and it seems annoying to have to do
>>> that on every single call to print.
>>>
>>> On Mon, Jan 25, 2016 at 8:54 PM Zachary Turner <zturner at google.com>
>>> wrote:
>>>
>>>> sorry, yea I stuck around for a while after that patch waiting for
>>>> emails, but nothing came through.  Please revert in the meantime, I'll work
>>>> on a fix tomorrow.
>>>>
>>>> On Mon, Jan 25, 2016 at 8:52 PM Todd Fiala via lldb-dev <
>>>> lldb-dev at lists.llvm.org> wrote:
>>>>
>>>>> I think I see what happened w/r/t why no emails when out when the
>>>>> build went heavy red.  (Well they went out internally, but not
>>>>> externally).  When I made the change on Friday to improve the workflow for
>>>>> the Green Dragon OS X builder and test output, I switched email over to the
>>>>> builder step, which doesn't know anything about who made which changes.  So
>>>>> it didn't know who to put on the blame list for the broken build.  Drats,
>>>>> I'll have to figure that out.
>>>>>
>>>>> I'd really prefer to have all those stages happening in one build step
>>>>> to keep it clear what's going on.
>>>>>
>>>>> On Mon, Jan 25, 2016 at 8:25 PM, Todd Fiala <todd.fiala at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Well our whole test suite just stopped running, so yes.
>>>>>>
>>>>>> On Mon, Jan 25, 2016 at 6:58 PM, Enrico Granata <egranata at apple.com>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>> On Jan 25, 2016, at 6:48 PM, Todd Fiala via lldb-dev <
>>>>>>> lldb-dev at lists.llvm.org> wrote:
>>>>>>>
>>>>>>> Not sure exactly what it is, but all the tests are failing due to
>>>>>>> some bad assumptions of unicode vs. str on Python 2 vs. 3 if I had to guess.
>>>>>>>
>>>>>>>
>>>>>>> Author: zturner
>>>>>>> Date: Mon Jan 25 18:59:42 2016
>>>>>>> New Revision: 258759
>>>>>>>
>>>>>>> URL: http://llvm.org/viewvc/llvm-project?rev=258759&view=rev
>>>>>>> Log:
>>>>>>> Write the session log file in UTF-8.
>>>>>>>
>>>>>>> Previously we were writing in the default encoding, which depends
>>>>>>> on the operating system and is not guaranteed to be unicode aware.
>>>>>>> On Python 3, this would lead to a situation where writing unicode
>>>>>>> text to the log file generates an exception.  The fix here is to
>>>>>>> write session logs using the proper encoding, which incidentally
>>>>>>> fixes another test, so xfail is removed from that.
>>>>>>>
>>>>>>> sounds like a likely culprit from what you’re saying
>>>>>>>
>>>>>>> I am not going to be able to look at details on that, but here's a
>>>>>>> link to the log on the OS X builder:
>>>>>>>
>>>>>>>
>>>>>>> Do you want me to revert?
>>>>>>>
>>>>>>> http://lab.llvm.org:8080/green/job/lldb_build_test/16166/console
>>>>>>>
>>>>>>> --
>>>>>>> -Todd
>>>>>>> _______________________________________________
>>>>>>> lldb-dev mailing list
>>>>>>> lldb-dev at lists.llvm.org
>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> *- Enrico*
>>>>>>> 📩 egranata@.com ☎️ 27683
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> -Todd
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> -Todd
>>>>> _______________________________________________
>>>>> lldb-dev mailing list
>>>>> lldb-dev at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>
>>>>
>>
>>
>> --
>> -Todd
>>
>
>
>
> --
> -Todd
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20160126/be6434fe/attachment.html>