[lldb-dev] swig generation

[Todd Fiala via lldb-dev]

I will probably tackle this as two phases:

Phase 1:
* Python script modernization (the python swig wrapper generation)
* Move Xcode onto it.

Phase 2:
* The maintainer-mode, static Python binding generation changes for cmake
and Xcode.

I want to make sure we have proven, still-functional Python from the new
Python script before we move forward.  I will also probably turn on the
Green Dragon OS X public builder's test running at this stage so we can
also see that.

And I want to work out any issues with the current script logic and what OS
X was doing before shifting any of this around.

On Fri, Nov 13, 2015 at 9:02 AM, Todd Fiala <todd.fiala at gmail.com> wrote:

> Hi all,
>
> I'd like to do a few things with our swig generation and handling:
>
> * Create a maintainer-mode style setup where the swig Python bindings are
> generated and checked into the repo (I'll call it the static python
> binding).
>
> This will be used by default, removing the need for most people and all
> builders/bots from having swig on their system just for lldb.  In the event
> that Windows needs a different version (e.g. for, say, Python 3.5 support
> that is incompatible with the swig-1.3.40-generated bindings), we can
> support multiple source bindings, or we can post process the swig-1.3.x
> generated bindings.
>
> We'll keep them by swig-{swig-major}-{swig-minor}.  Internally over at
> Apple, we will stick with the swig-1.x bindings.  I don't think we will
> care about the dot release (1.3.40 vs. 1.3.39, for example).  We'll just
> make sure to use the latest for a {swig-major}.{swig-minor}.  As always, SB
> API changes generated by swig need to remain swig-1.3 compatible (i.e.
> swig-1.3 must be capable of generating usable Python wrappers).
>
> Ideally we don't need more than one set of bindings so we can avoid
> needing to generate multiple ones when we do it.
>
>
> * Add an explicit Python binding generation step.
>
> This would only be used by those who are touching the bindings (adding SB
> API or adjusting the documentation of existing SB API).  In the event that
> we need two bindings, we may just need a handshake that says "okay, we'll
> take care of the swig 1.3 bindings, and {somebody who needs the other
> binding} generates it for the other."  As SBAPI is additive only, this
> should generally be fine as the worst case I can think of that would happen
> would be failure to see new SBAPI in Python for a very brief time.  Since
> maintainers will be writing Python tests to check their new SBAPIs, failure
> to do the explicit generation step will be immediately obvious as a test
> failure.  (The challenge here will likely be the swig 1.3.x requirement).
>
> If generating and testing new binding content with the right swig (i.e.
> swig 1.3.x) becomes a real issue, we may be able to support a "please use
> my local swig, whatever it is, but don't check in the static binding), with
> a handshake that says "somebody with swig 1.3, please generate the modified
> binding and check in).  We'll see if this becomes an issue.  I don't see
> this as insurmountable.
>
>
> * Clean up the swig generation scripts
>
> These look to have been implemented as direct ports of the older OS X
> style bash scripts.  This Python script is very much the essence of using
> the paradigms of one language in another, and being the worse for it.  It
> implements features that are already present in the standard Python lib,
> and is more verbose than it needs to be.
>
> Also, it is likely the script needs to be broken out into a few parts.
> The scripts don't just generate the python binding using swig, they also
> setup (for OS X) some packaging and other bits.  Right now none of that is
> clearly broken out.  When we move to an explicit binding generation mode,
> this does need to be broken out better.
>
>
> * Get OS X Xcode build using the same bindings machinery as others.
>
> I tried this a while back (having Xcode adopt the Python-based swig
> wrapper handling code), but gave up after hitting a few bugs where the
> translated behavior was incorrect for Xcode.  I will move over to adopting
> this on Xcode if possible while going through these changes.
>
> The primary goal here is to remove the requirement of having swig on the
> system (e.g. for builders and most developers), shifting that burden a bit
> more to those who actually modify the Python binding.
>
> As a potential added benefit, this opens us up to easier modification of
> that binding generation step, which may prove to be useful later.
>
> Let me know if you have any feedback before I dive into this.
>
> -Todd
>



-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151113/9b04347c/attachment.html>