[lldb-dev] swig generation

[Todd Fiala via lldb-dev]

On Fri, Nov 13, 2015 at 9:02 AM, Todd Fiala <todd.fiala at gmail.com> wrote:

> Hi all,
>
> I'd like to do a few things with our swig generation and handling:
>
> * Create a maintainer-mode style setup where the swig Python bindings are
> generated and checked into the repo (I'll call it the static python
> binding).
>
> This will be used by default, removing the need for most people and all
> builders/bots from having swig on their system just for lldb.  In the event
> that Windows needs a different version (e.g. for, say, Python 3.5 support
> that is incompatible with the swig-1.3.40-generated bindings), we can
> support multiple source bindings, or we can post process the swig-1.3.x
> generated bindings.
>
> We'll keep them by swig-{swig-major}-{swig-minor}.  Internally over at
> Apple, we will stick with the swig-1.x bindings.  I don't think we will
> care about the dot release (1.3.40 vs. 1.3.39, for example).  We'll just
> make sure to use the latest for a {swig-major}.{swig-minor}.  As always, SB
> API changes generated by swig need to remain swig-1.3 compatible (i.e.
> swig-1.3 must be capable of generating usable Python wrappers).
>
> Ideally we don't need more than one set of bindings so we can avoid
> needing to generate multiple ones when we do it.
>
>
> * Add an explicit Python binding generation step.
>
> This would only be used by those who are touching the bindings (adding SB
> API or adjusting the documentation of existing SB API).  In the event that
> we need two bindings, we may just need a handshake that says "okay, we'll
> take care of the swig 1.3 bindings, and {somebody who needs the other
> binding} generates it for the other."  As SBAPI is additive only, this
> should generally be fine as the worst case I can think of that would happen
> would be failure to see new SBAPI in Python for a very brief time.  Since
> maintainers will be writing Python tests to check their new SBAPIs, failure
> to do the explicit generation step will be immediately obvious as a test
> failure.  (The challenge here will likely be the swig 1.3.x requirement).
>
> If generating and testing new binding content with the right swig (i.e.
> swig 1.3.x) becomes a real issue, we may be able to support a "please use
> my local swig, whatever it is, but don't check in the static binding), with
> a handshake that says "somebody with swig 1.3, please generate the modified
> binding and check in).  We'll see if this becomes an issue.  I don't see
> this as insurmountable.
>
>
> * Clean up the swig generation scripts
>
> These look to have been implemented as direct ports of the older OS X
> style bash scripts.  This Python script is very much the essence of using
> the paradigms of one language in another, and being the worse for it.  It
> implements features that are already present in the standard Python lib,
> and is more verbose than it needs to be.
>
> Also, it is likely the script needs to be broken out into a few parts.
> The scripts don't just generate the python binding using swig, they also
> setup (for OS X) some packaging and other bits.  Right now none of that is
> clearly broken out.  When we move to an explicit binding generation mode,
> this does need to be broken out better.
>
>
> * Get OS X Xcode build using the same bindings machinery as others.
>
> I tried this a while back (having Xcode adopt the Python-based swig
> wrapper handling code), but gave up after hitting a few bugs where the
> translated behavior was incorrect for Xcode.  I will move over to adopting
> this on Xcode if possible while going through these changes.
>

I've begun this process.  I've got Xcode build (and the Xcode build only)
wired up to scripts/prepare_bindings.py.  This, and the Python-specific
binding preparation script, are considerably shorter than the older
implementation.  It also addresses the bugs that prevented the former from
working with the Xcode build.  The new scripts are pylint clean  (with
stock configuration) except for a small handful of places that I intend to
revisit when I have a few cycles to really tidy things up.  (Right now I'm
just going for the 90% wins).

Tomorrow I will fix up the tail end (the finalization) in a similar manner
and plug it into the Xcode build.  I'll also try it locally on Linux and
work out any issues there.

I'll be tackling the static binding area (and will put up for review) only
after I know the cleaned up binding preparation scripts are working
properly everywhere.  I just didn't want to tackle that until I really
understood what the scripts were already doing and it was in a state I felt
I could maintain.


>
> The primary goal here is to remove the requirement of having swig on the
> system (e.g. for builders and most developers), shifting that burden a bit
> more to those who actually modify the Python binding.
>
> As a potential added benefit, this opens us up to easier modification of
> that binding generation step, which may prove to be useful later.
>
> Let me know if you have any feedback before I dive into this.
>
> -Todd
>



-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151116/61e22147/attachment.html>