[lldb-dev] Editline Rewrite : issues surround wide character handling on different platforms

[Shawn Best]

Vince - I don't think dropping gcc on Linux is a great option, as other 
platforms (FreeBSD) use gcc. The issue is also more related libc++ vs 
libstdc++

Zachary - I believe the patch is meant to address problems like that, 
but I have only ever used lldb command line in a simplistic way.  I will 
test something more complex and verify it works same on linux as OSX.

The wchar_t is used by internally in libedit, then translated to utf8 
when to interface to lldb.  I agree, utf8 std::string seems the cleanest 
way to go.

Shawn.

On 10/28/2014 1:45 PM, Vince Harron wrote:
> > codecvt> : the patch uses a c++ 11 class std::codecvt_utf8, this is a 
> facet implementation that will do utf8 to wchar convervsion.  It is 
> part of c++ 11 standard, but not yet supported in gcc.
>
> Should we drop support for building with gcc on Linux?
>
> > - take libedit source files (or subset) and add to the lldb 
> project.  We could either build a .so file, or just statically link 
> the .cpp files.
>
> Is it a problem to drop these "berkeley stype license" files into the 
> project?
>
> > - rework the Editline rewrite, so it either uses standard 8 bit 
> chars, or wchar_t/utf8 depending on the platform.  This would be 
> conditionally built depending on the platform.
>
> This would be my favorite option if possible.  wchar_t never really 
> took roots in Linux AFAIK.
>
> > - introduce custom step for new linux lldb users, where they 
> download libedit source and build and install a wchar version
>
> Not as good as options above but we can work with this.
>
>
>
>
> On Tue, Oct 28, 2014 at 12:42 PM, Shawn Best <sbest at blueshiftinc.com 
> <mailto:sbest at blueshiftinc.com>> wrote:
>
>     There is a significant Editline rewrite, adding a bunch of
>     improvements.  It has been well tested on OSx, but not yet
>     upstreamed.  I have spent some time reviewing the proposed patch
>     and working through issues to get it running on linux.  To see the
>     patch and accompanying discussion, refer to:
>     http://reviews.llvm.org/D5835   The main issues that came up are
>     related to handling wide characters and differences between platforms.
>
>     Internally, lldb uses std::string which is an array of 8 bit
>     chars, that can either be 7 bit ascii, or utf8 encoded wide
>     characters. Libedit uses either char, or wchar_t which is a 32 bit
>     char on linux.
>
>     <codecvt> : the patch uses a c++ 11 class std::codecvt_utf8, this
>     is a facet implementation that will do utf8 to wchar convervsion. 
>     It is part of c++ 11 standard, but not yet supported in gcc.  I
>     can use #ifdef to temporarily write equivalent functionality in
>     that case while we wait for gcc to catch up.
>
>     libedit : libedit is a prerequisite that a new linux/lldb user
>     installs ( sudo apt-get install libedit-dev ).  A few years ago,
>     libedit added versions of its functions that work on wchar_t.
>     Unfortunately, this option is not built by default, and not
>     present in the Ubuntu distribution.  To get around this, I see a
>     few options:
>
>     - take libedit source files (or subset) and add to the lldb
>     project.  We could either build a .so file, or just statically
>     link the .cpp files.
>
>     - rework the Editline rewrite, so it either uses standard 8 bit
>     chars, or wchar_t/utf8 depending on the platform.  This would be
>     conditionally built depending on the platform.
>
>     - modify Ubuntu, so 'sudo apt-get install dev-libedit' installs a
>     version that has been built with wide character support enabled.
>
>     - introduce custom step for new linux lldb users, where they
>     download libedit source and build and install a wchar version
>
>     The last 2 options don't seem that great.
>
>     I expect there will be problems on Windows, which I think uses
>     utf16 coding.  The file EditLineWin.cpp, contains prototypes for
>     most of the structures and functions needed, but they look stubbed
>     out.
>
>     Any thoughts?
>
>     Shawn.
>     _______________________________________________
>     lldb-dev mailing list
>     lldb-dev at cs.uiuc.edu <mailto:lldb-dev at cs.uiuc.edu>
>     http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>
>
>
>
> -- 
>
> Vince Harron | 	 Technical Lead Manager | 	vharron at google.com 
> <mailto:vharron at google.com> | 	 858-442-0868
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20141028/64a166d3/attachment.html>