[lldb-dev] Editline Rewrite : issues surround wide character handling on different platforms

[Zachary Turner]

I haven't had time to look into the libedit stuff on Windows, but my
understanding is that lldb on Windows has never been using libedit to begin
with.  If that's the case, then we can probably continue not using it as
that will eliminate the risk of any breaks, and python's builtin console
control handler on Windows is not that bad.  It might be worth trying to
port this implementation to Windows at some point, but if we're not using
it currently (and I don't think we are), I don't see any reason to block
this change because of Windows, as long as the change keeps it disabled for
Windows.

On the Linux front, the times where I have had to work on Linux (testing
patches etc) when you go into the embedded interpreter by typing "script",
everything is totally broken, and you can't use anything that is not a
simple ascii character.  No arrows, no control codes, nothing.  Do you know
if this patch fixes that on Linux (or at least assuming the issues you
pointed out are addressed, will it fix that)?

On Tue Oct 28 2014 at 12:45:54 PM Shawn Best <sbest at blueshiftinc.com> wrote:

> There is a significant Editline rewrite, adding a bunch of
> improvements.  It has been well tested on OSx, but not yet upstreamed.
> I have spent some time reviewing the proposed patch and working through
> issues to get it running on linux.  To see the patch and accompanying
> discussion, refer to: http://reviews.llvm.org/D5835    The main issues
> that came up are related to handling wide characters and differences
> between platforms.
>
> Internally, lldb uses std::string which is an array of 8 bit chars, that
> can either be 7 bit ascii, or utf8 encoded wide characters. Libedit uses
> either char, or wchar_t which is a 32 bit char on linux.
>
> <codecvt> : the patch uses a c++ 11 class std::codecvt_utf8, this is a
> facet implementation that will do utf8 to wchar convervsion.  It is part
> of c++ 11 standard, but not yet supported in gcc.  I can use #ifdef to
> temporarily write equivalent functionality in that case while we wait
> for gcc to catch up.
>
> libedit : libedit is a prerequisite that a new linux/lldb user installs
> ( sudo apt-get install libedit-dev ).  A few years ago, libedit added
> versions of its functions that work on wchar_t. Unfortunately, this
> option is not built by default, and not present in the Ubuntu
> distribution.  To get around this, I see a few options:
>
> - take libedit source files (or subset) and add to the lldb project.  We
> could either build a .so file, or just statically link the .cpp files.
>
> - rework the Editline rewrite, so it either uses standard 8 bit chars,
> or wchar_t/utf8 depending on the platform.  This would be conditionally
> built depending on the platform.
>
> - modify Ubuntu, so 'sudo apt-get install dev-libedit' installs a
> version that has been built with wide character support enabled.
>
> - introduce custom step for new linux lldb users, where they download
> libedit source and build and install a wchar version
>
> The last 2 options don't seem that great.
>
> I expect there will be problems on Windows, which I think uses utf16
> coding.  The file EditLineWin.cpp, contains prototypes for most of the
> structures and functions needed, but they look stubbed out.
>
> Any thoughts?
>
> Shawn.
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20141028/7f8ec81c/attachment.html>