[lldb-dev] something just toasted the test suite on OS X
Zachary Turner via lldb-dev
lldb-dev at lists.llvm.org
Tue Jan 26 10:54:00 PST 2016
Can one of you guys try out this patch and see if it works? If so I'll
commit it.
I don't know of a way to make this "elegant". i.e. a single syntax /
paradigm that works in both versions without introducing any helper
functions.
On Tue, Jan 26, 2016 at 12:26 AM Zachary Turner <zturner at google.com> wrote:
> No worries, worst case scenario a sledgehammer solution is to change all
> the places where we write to the session file to convert to unicode first
> (which would be a trivial conversion, since everything is going to be
> ascii, which is already valid utf 8).
>
> The reason a problem arose at all is because TestCxxWcharT.py tried to
> write actual unicode characters to the session file. I don't think there's
> any way to prevent that because the characters could appear in a backtrace,
> in a variable name, or in a test that is specifically testing unicode. So
> we can't just have the one place that needs to write unicode encode it as
> bytes because there's no one place.
>
> I'll sleep on it and try to see if there's a better solution. Maybe we
> can just make a function called write_session_file() that takes either a
> string or a unicode, converts it to a unicode if it's not already one, and
> then writes.
>
> On Mon, Jan 25, 2016 at 9:45 PM Todd Fiala <todd.fiala at gmail.com> wrote:
>
>> Okay we're back to green here:
>> http://lab.llvm.org:8080/green/job/lldb_build_test/16173/
>>
>> Thanks, Enrico!
>>
>> Zachary, I may let this rest until the morning. If you want to try
>> something else, shoot me a patch and I'll gladly try it.
>>
>> -Todd
>>
>> On Mon, Jan 25, 2016 at 9:16 PM, Todd Fiala <todd.fiala at gmail.com> wrote:
>>
>>> It's in item 3 from Effective Python, by Brett Slatkin, which goes over
>>> having methods that always go to unicode or to byte streams taking either
>>> unicode or byte style strings, for both Python 2 and Python 3. Essentially
>>> you figure out what you want it to be in, and you write a couple helper
>>> routes to go in either the "to unicode" or the "to bytes" direction. It
>>> basically looks at the type of the string/bytes you give it, and makes sure
>>> it becomes what you need. It's going to assume an encoding like utf-8.
>>>
>>> On Mon, Jan 25, 2016 at 9:09 PM, Zachary Turner <zturner at google.com>
>>> wrote:
>>>
>>>> I'm also not sure why Linux isn't failing. Looking at the
>>>> documentation for io.write object, i see this:
>>>>
>>>> write(*s*)
>>>> <https://docs.python.org/2/library/io.html#io.TextIOBase.write>
>>>>
>>>> Write the unicode
>>>> <https://docs.python.org/2/library/functions.html#unicode> string *s* to
>>>> the stream and return the number of characters written.
>>>> So clearly it does have to be a unicode object, and saying
>>>> print(self.getvalue(), file=self.session) is clearly NOT printing a unicode
>>>> string to the file.
>>>>
>>>> What's the pattern you're referring to? You can't convert a string to
>>>> a unicode without specifying an encoding, and it seems annoying to have to
>>>> do that on every single call to print.
>>>>
>>>> On Mon, Jan 25, 2016 at 8:54 PM Zachary Turner <zturner at google.com>
>>>> wrote:
>>>>
>>>>> sorry, yea I stuck around for a while after that patch waiting for
>>>>> emails, but nothing came through. Please revert in the meantime, I'll work
>>>>> on a fix tomorrow.
>>>>>
>>>>> On Mon, Jan 25, 2016 at 8:52 PM Todd Fiala via lldb-dev <
>>>>> lldb-dev at lists.llvm.org> wrote:
>>>>>
>>>>>> I think I see what happened w/r/t why no emails when out when the
>>>>>> build went heavy red. (Well they went out internally, but not
>>>>>> externally). When I made the change on Friday to improve the workflow for
>>>>>> the Green Dragon OS X builder and test output, I switched email over to the
>>>>>> builder step, which doesn't know anything about who made which changes. So
>>>>>> it didn't know who to put on the blame list for the broken build. Drats,
>>>>>> I'll have to figure that out.
>>>>>>
>>>>>> I'd really prefer to have all those stages happening in one build
>>>>>> step to keep it clear what's going on.
>>>>>>
>>>>>> On Mon, Jan 25, 2016 at 8:25 PM, Todd Fiala <todd.fiala at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Well our whole test suite just stopped running, so yes.
>>>>>>>
>>>>>>> On Mon, Jan 25, 2016 at 6:58 PM, Enrico Granata <egranata at apple.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> On Jan 25, 2016, at 6:48 PM, Todd Fiala via lldb-dev <
>>>>>>>> lldb-dev at lists.llvm.org> wrote:
>>>>>>>>
>>>>>>>> Not sure exactly what it is, but all the tests are failing due to
>>>>>>>> some bad assumptions of unicode vs. str on Python 2 vs. 3 if I had to guess.
>>>>>>>>
>>>>>>>>
>>>>>>>> Author: zturner
>>>>>>>> Date: Mon Jan 25 18:59:42 2016
>>>>>>>> New Revision: 258759
>>>>>>>>
>>>>>>>> URL: http://llvm.org/viewvc/llvm-project?rev=258759&view=rev
>>>>>>>> Log:
>>>>>>>> Write the session log file in UTF-8.
>>>>>>>>
>>>>>>>> Previously we were writing in the default encoding, which depends
>>>>>>>> on the operating system and is not guaranteed to be unicode aware.
>>>>>>>> On Python 3, this would lead to a situation where writing unicode
>>>>>>>> text to the log file generates an exception. The fix here is to
>>>>>>>> write session logs using the proper encoding, which incidentally
>>>>>>>> fixes another test, so xfail is removed from that.
>>>>>>>>
>>>>>>>> sounds like a likely culprit from what you’re saying
>>>>>>>>
>>>>>>>> I am not going to be able to look at details on that, but here's a
>>>>>>>> link to the log on the OS X builder:
>>>>>>>>
>>>>>>>>
>>>>>>>> Do you want me to revert?
>>>>>>>>
>>>>>>>> http://lab.llvm.org:8080/green/job/lldb_build_test/16166/console
>>>>>>>>
>>>>>>>> --
>>>>>>>> -Todd
>>>>>>>> _______________________________________________
>>>>>>>> lldb-dev mailing list
>>>>>>>> lldb-dev at lists.llvm.org
>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> *- Enrico*
>>>>>>>> 📩 egranata@.com ☎️ 27683
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> -Todd
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> -Todd
>>>>>> _______________________________________________
>>>>>> lldb-dev mailing list
>>>>>> lldb-dev at lists.llvm.org
>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>>
>>>>>
>>>
>>>
>>> --
>>> -Todd
>>>
>>
>>
>>
>> --
>> -Todd
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20160126/a4ffbdff/attachment-0001.html>
-------------- next part --------------
diff --git a/packages/Python/lldbsuite/test/lang/cpp/char1632_t/TestChar1632T.py b/packages/Python/lldbsuite/test/lang/cpp/char1632_t/TestChar1632T.py
index 9bb1faf..fbb5551 100644
--- a/packages/Python/lldbsuite/test/lang/cpp/char1632_t/TestChar1632T.py
+++ b/packages/Python/lldbsuite/test/lang/cpp/char1632_t/TestChar1632T.py
@@ -25,7 +25,6 @@ class Char1632TestCase(TestBase):
line_number(self.source, '// breakpoint2') ]
@expectedFailureIcc # ICC (13.1) does not emit the DW_TAG_base_type for char16_t and char32_t.
- @expectedFailureWindows("llvm.org/pr24489: Name lookup not working correctly on Windows")
def test(self):
"""Test that the C++11 support for char16_t and char32_t works correctly."""
self.build()
diff --git a/packages/Python/lldbsuite/test/lldbtest.py b/packages/Python/lldbsuite/test/lldbtest.py
index 7d9ba75..6448be2 100644
--- a/packages/Python/lldbsuite/test/lldbtest.py
+++ b/packages/Python/lldbsuite/test/lldbtest.py
@@ -262,7 +262,10 @@ class recording(SixStringIO):
if self.trace:
print(self.getvalue(), file=sys.stderr)
if self.session:
- print(self.getvalue(), file=self.session)
+ value = self.getvalue()
+ if six.PY2:
+ value = unicode(self.getvalue(), "utf-8")
+ print(value, file=self.session)
self.close()
@add_metaclass(abc.ABCMeta)
@@ -1440,7 +1443,7 @@ class Base(unittest2.TestCase):
session_file = "{}.log".format(self.log_basename)
# Python 3 doesn't support unbuffered I/O in text mode. Open buffered.
- self.session = open(session_file, "w")
+ self.session = io.open(session_file, mode="w", encoding="utf-8")
# Optimistically set __errored__, __failed__, __expected__ to False
# initially. If the test errored/failed, the session info
@@ -1794,6 +1797,10 @@ class Base(unittest2.TestCase):
if not self.__unexpected__ and not self.__skipped__:
for test, traceback in pairs:
if test is self:
+ if six.PY2:
+ # Not sure of a better way to do this, but on Py3 traceback
+ # is already a unicode, and so executing this would fail.
+ traceback = unicode(traceback, "utf-8")
print(traceback, file=self.session)
# put footer (timestamp/rerun instructions) into session
@@ -1804,9 +1811,9 @@ class Base(unittest2.TestCase):
benchmarks = False
import datetime
- print("Session info generated @", datetime.datetime.now().ctime(), file=self.session)
- print("To rerun this test, issue the following command from the 'test' directory:\n", file=self.session)
- print("./dotest.py %s -v %s %s" % (self.getRunOptions(),
+ print(u"Session info generated @", datetime.datetime.now().ctime(), file=self.session)
+ print(u"To rerun this test, issue the following command from the 'test' directory:\n", file=self.session)
+ print(u"./dotest.py %s -v %s %s" % (self.getRunOptions(),
('+b' if benchmarks else '-t'),
self.getRerunArgs()), file=self.session)
self.session.close()
More information about the lldb-dev
mailing list