r289308 - [clang-format] Another attempt at python 3 compatibility
Vedant Kumar via cfe-commits
cfe-commits at lists.llvm.org
Mon Dec 12 11:47:21 PST 2016
Thanks for double-checking this. I realized that there's a problem with my
patch: skipping the encoding step when using Python 3 could cause problems when
we use difflib.SequenceMatcher.
I don't know how to fix this, since the strings in vim.current.buffer appear to
be 'raw'. They are not encoded in any valid way. This happens with both the
python 2/3 vim interfaces. You can test this by creating a buffer with an emoji
in it ('😀') and then running:
:python import vim
That makes us responsible for picking an encoding and using it consistently [*].
However, I don't know how to properly encode an invalid string in Python 3. Is
there some way to 'reinterpret_cast' it to a bytes object?
[*] I'd rather get rid of the encoding/decoding steps entirely, but I don't think
this is possible, because that would prevent us from using the JSON module.
> On Dec 10, 2016, at 6:16 AM, Nico Weber <thakis at chromium.org> wrote:
> Shouldn't this at least use >=? (But it feels like there is probably a way that doesn't have to do UA sniffing, so to speak)
> On Dec 9, 2016 8:04 PM, "Vedant Kumar via cfe-commits" <cfe-commits at lists.llvm.org> wrote:
> Author: vedantk
> Date: Fri Dec 9 18:54:13 2016
> New Revision: 289308
> URL: http://llvm.org/viewvc/llvm-project?rev=289308&view=rev
> [clang-format] Another attempt at python 3 compatibility
> The entries in vim.current.buffer appear to be decoded strings, which
> means that python3 won't allow invoking 'decode' on them. Keep the old
> behavior when running under python2, but skip the error-inducing decode
> step with python3..
> Modified: cfe/trunk/tools/clang-format/clang-format.py
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-format/clang-format.py?rev=289308&r1=289307&r2=289308&view=diff
> --- cfe/trunk/tools/clang-format/clang-format.py (original)
> +++ cfe/trunk/tools/clang-format/clang-format.py Fri Dec 9 18:54:13 2016
> @@ -29,6 +29,7 @@ from __future__ import print_function
> import difflib
> import json
> +import platform
> import subprocess
> import sys
> import vim
> @@ -48,10 +49,15 @@ fallback_style = None
> if vim.eval('exists("g:clang_format_fallback_style")') == "1":
> fallback_style = vim.eval('g:clang_format_fallback_style')
> +def get_buffer(encoding):
> + if platform.python_version_tuple() == '3':
> + return vim.current.buffer
> + return [ line.decode(encoding) for line in vim.current.buffer ]
> def main():
> # Get the current text.
> encoding = vim.eval("&encoding")
> - buf = [ line.decode(encoding) for line in vim.current.buffer ]
> + buf = get_buffer(encoding)
> text = '\n'.join(buf)
> # Determine range to format.
> cfe-commits mailing list
> cfe-commits at lists.llvm.org
More information about the cfe-commits