[lldb-dev] Python 3 and dotest

[Zachary Turner via lldb-dev]

On Tue, Oct 13, 2015 at 9:08 PM Todd Fiala <todd.fiala at gmail.com> wrote:

> On Tue, Oct 13, 2015 at 9:45 AM, Zachary Turner <zturner at google.com>
> wrote:
>
>> On Mon, Oct 12, 2015 at 7:31 PM Zachary Turner <zturner at google.com>
>> wrote:
>>
>>> Moving this to the public list because it seems useful to see what other
>>> members of the community have to say as well.
>>>
>>
>> BTW , I realized I didn't give any context here and it's hard to follow a
>> long quote thread.  Mostly this is just to give a heads up that support for
>> Python 3 in LLDB is -- at a minimum -- coming, so I want to find out who
>> (if anyone) is interested in this and how people plan to use it (or not use
>> it).
>>
>> I'm sure the first question is probably "Why are you doing this?".  The
>> short version is that it's quite literally the only path forward on
>> Windows.  There is no way to transition to the next version of the compiler
>> without moving to Python 3.5 or greater.  If you want the long version, let
>> me know.
>>
>> The biggest challenge (which is what most of the quote thread is about)
>> is going to be having dotest support both Python 2 and Python 3
>> simultaneously.  LLDB supporting both at the native level shouldn't be a
>> problem, it's mostly about writing Python code inside of dotest that works
>> in both 2 and 3.
>>
>> This will be the first time I've tried porting a large Python codebase to
>> Python 3, so I'm open for suggestions on how to minimize the impact of
>> this.  Currently I'm of the mind that the onus should be on the person
>> checking in a test (or a change to the test suite) to make sure it works in
>> both versions of Python.
>>
>
> I'd probably throw out here that as a change is being introduced for the
> benefit primarily of getting things running on the Windows side,
>

I actually disagree with this.  Having Python 3 support in LLDB is a big
win for the entire project.  We all know Python 2.x is effectively dead,
and the only reason anyone is still using it is for legacy support.  This
means that we are hindering the adoption of LLDB for anyone who doesn't
have a large legacy codebase to maintain.  Which, for all practical
purposes, means anyone starting a new project and considering using LLDB.
That's a huge negative from an open source perspective IMO.

But that's just a lower bound.  There are other people who are ready and
willing to move to Python 3, but can't because certain tools and libraries
require them to use Python 2.x.  Unfortunately, LLDB is one of those
tools.  When changes started going into lit on the LLVM side to make it
Python 3 compatible, more than one person spoke up to say that they were
pleased they could finally remove Python 2 from their systems.  We're also
hindering those people from making contributions from LLDB.

Python 3 is the future, Python 2 is the past, so supporting it is a good
long term strategy.  The only reason Windows should be part of the
discussion at all is because it's the catalyst that makes *me* want to do
the work now, instead of someone else doing the work later.

Believe it or not, there are also benefits for LLDB developers not on
Windows, even if they are stuck on 2.7.  By moving a newer version of the
compiler, we get almost 100% C++11 and C99 conformance.  This opens the
door for LLDB to use a lot of new language features that people have been
either intentionally holding back on or forced to remove because of this
one compiler.  Off the top of my head, a wildly incomplete list is:
constexpr, shared_mutex, complete implementation of C++ standard library,
rvalue references, initializer lists, thread_local, thread-safe function
local statics, compiler-agnostic __attribute__ syntax, and more.  Everyone
benefits from that, ironically, by supporting Python 3


> I'm not sure it's a great idea to put the onus on everyone making sure
> they run in both places, at least up front.
>

TBH I can't think of a reasonable alternative.  There's going to have to be
a buildbot testing LLDB against Python 3, and that buildbot has to stay
green, like all other buildbots.  So any policies which apply normally to
buildbots should apply to this one.  We do have a special exception in LLDB
specifically as it relates to OS specific differences (for example someone
Linux breaks the Windows build and doesn't know how to fix it), but that
case is different for two reasons.

First, not everyone has access to or is an expert on every piece of
hardware supported by LLDB, so it's impossible to expect they would be able
to fix every change.

Second, when it is not obvious from the buildbot logs what the problem is
on a specific platform, a test can simply be xfailed or skipped to give the
platform owner a chance to look into the cause of the failure.

Neither of these holds if we're talking about the choice of scripting
environment.  For the first point, it *is* reasonable to expect that
someone can install Python 3 on their machine.  This is of course
contingent on your findings about what difficulties arise with side-by-side
installs on MacOSX, but at the same time OSX is kind of a popular platform,
and if there's really a widespread problem, I have to imagine that the
Python developers have thought of this and have a solution.  Still though,
I'll reserve judgement until we know more.  For the second point, if
someone checks in something that prevents the test suite from even running
in the first place, there's nothing you can do to xfail it or skip it
because it's in infrastructure code.  Even if it were in a test, not being
able to write code that runs in Python 2 and Python 3 is not a reasonable
excuse to xfail or skip a test IMO.

Maybe saying "the onus is on the developer to fix it" is too harsh, but in
that case the only reasonable alternative I can think of is "the onus is on
the developer to either fix it or revert it".  Again, this is just the
normal overarching LLVM policy with no special exceptions applied.


> (Particularly if we find that, say, the system python co-existing with
> another python causes trouble on some systems).  I think we want to make
> sure it is realistic for people to be able to test on two pythons without
> undue difficulty on that front.  This one we can probably figure out over
> time, though, as I'm not expecting everyone will rewrite existing tests in
> python 2/3 incompatible ways for the fun of it.  (Of course, if this makes
> people more interested in writing more tests, than that's great, too ;-) )
>
As mentioned earlier, I agree this is a potential unknown.  So we can
reserve judgement on this until we know more.

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151014/a6bfb389/attachment.html>