[Lldb-commits] [PATCH] D16736: Always write the session log file in UTF-8

[Zachary Turner via lldb-commits]

zturner created this revision.
zturner added reviewers: tfiala, labath, tberghammer.
zturner added a subscriber: lldb-commits.
Herald added a subscriber: aemerson.

Hopefully the 10th time's a charm on this one.  

This patch attempts to solve the Python 2 / Python 3 incompatibilities by introducing a new `encoded_file` abstraction that we use instead of `io.open()`.  The problem with the builtin implementation of `io.open` is that `read` and `write` accept and return `unicode` objects, which are not always convenient to work with in Python 2.  We solve this by making `encoded_file.open()` return the same object returned by `io.open()` but with hooked `read()` and `write()` methods.  These hooked methods will accept binary or text data, and conditionally convert what it gets to a `unicode` object using the correct encoding.  When calling `read()` it also does any conversion necessary to convert the output back into the native `string` type of the running python version.

I actually tested this one on both Windows and Linux and it works fine.  So hopefully this is closer to a working solution!

http://reviews.llvm.org/D16736

Files:
  packages/Python/lldbsuite/support/encoded_file.py
  packages/Python/lldbsuite/test/lang/cpp/char1632_t/TestChar1632T.py
  packages/Python/lldbsuite/test/lldbtest.py

Index: packages/Python/lldbsuite/test/lldbtest.py
===================================================================
--- packages/Python/lldbsuite/test/lldbtest.py
+++ packages/Python/lldbsuite/test/lldbtest.py
@@ -58,11 +58,13 @@
 import six
 
 # LLDB modules
+import use_lldb_suite
 import lldb
 from . import configuration
 from . import lldbtest_config
 from . import lldbutil
 from . import test_categories
+from lldbsuite.support import encoded_file
 
 from .result_formatter import EventBuilder
 
@@ -1489,7 +1491,7 @@
 
         session_file = "{}.log".format(self.log_basename)
         # Python 3 doesn't support unbuffered I/O in text mode.  Open buffered.
-        self.session = open(session_file, "w")
+        self.session = encoded_file.open(session_file, "utf-8", mode="w")
 
         # Optimistically set __errored__, __failed__, __expected__ to False
         # initially.  If the test errored/failed, the session info
Index: packages/Python/lldbsuite/test/lang/cpp/char1632_t/TestChar1632T.py
===================================================================
--- packages/Python/lldbsuite/test/lang/cpp/char1632_t/TestChar1632T.py
+++ packages/Python/lldbsuite/test/lang/cpp/char1632_t/TestChar1632T.py
@@ -25,7 +25,6 @@
                        line_number(self.source, '// breakpoint2') ]
 
     @expectedFailureIcc # ICC (13.1) does not emit the DW_TAG_base_type for char16_t and char32_t.
-    @expectedFailureWindows("llvm.org/pr24489: Name lookup not working correctly on Windows")
     def test(self):
         """Test that the C++11 support for char16_t and char32_t works correctly."""
         self.build()
Index: packages/Python/lldbsuite/support/encoded_file.py
===================================================================
--- /dev/null
+++ packages/Python/lldbsuite/support/encoded_file.py
@@ -0,0 +1,48 @@
+"""
+                     The LLVM Compiler Infrastructure
+
+This file is distributed under the University of Illinois Open Source
+License. See LICENSE.TXT for details.
+
+Prepares language bindings for LLDB build process.  Run with --help
+to see a description of the supported command line arguments.
+"""
+
+# Python modules:
+import io
+
+# Third party modules
+import six
+
+def _encoded_read(old_read, encoding):
+    def impl(size):
+        result = old_read(size)
+        # If this is Python 2 then we need to convert the resulting `unicode` back
+        # into a `str` before returning
+        if six.PY2:
+            result = result.encode(encoding)
+        return result
+    return impl
+
+def _encoded_write(old_write, encoding):
+    def impl(s):
+        # If we were asked to write a `str` (in Py2) or a `bytes` (in Py3) decode it
+        # as unicode before attempting to write.
+        if isinstance(s, six.binary_type):
+            s = s.decode(encoding)
+        return old_write(s)
+    return impl
+
+'''
+Create a Text I/O file object that can be written to with either unicode strings or byte strings
+under Python 2 and Python 3, and automatically encodes and decodes as necessary to return the
+native string type for the current Python version
+'''
+def open(file, encoding, mode='r', buffering=-1, errors=None, newline=None, closefd=True):
+    wrapped_file = io.open(file, mode=mode, buffering=buffering, encoding=encoding,
+                           errors=errors, newline=newline, closefd=closefd)
+    new_read = _encoded_read(getattr(wrapped_file, 'read'), encoding)
+    new_write = _encoded_write(getattr(wrapped_file, 'write'), encoding)
+    setattr(wrapped_file, 'read', new_read)
+    setattr(wrapped_file, 'write', new_write)
+    return wrapped_file


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D16736.46426.patch
Type: text/x-patch
Size: 3653 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20160129/72fc0233/attachment-0001.bin>