[lldb-dev] swig generation

[Todd Fiala via lldb-dev]

On Mon, Nov 16, 2015 at 11:28 PM, Todd Fiala <todd.fiala at gmail.com> wrote:

>
>
> On Fri, Nov 13, 2015 at 9:02 AM, Todd Fiala <todd.fiala at gmail.com> wrote:
>
>> Hi all,
>>
>> I'd like to do a few things with our swig generation and handling:
>>
>> * Create a maintainer-mode style setup where the swig Python bindings are
>> generated and checked into the repo (I'll call it the static python
>> binding).
>>
>> This will be used by default, removing the need for most people and all
>> builders/bots from having swig on their system just for lldb.  In the event
>> that Windows needs a different version (e.g. for, say, Python 3.5 support
>> that is incompatible with the swig-1.3.40-generated bindings), we can
>> support multiple source bindings, or we can post process the swig-1.3.x
>> generated bindings.
>>
>> We'll keep them by swig-{swig-major}-{swig-minor}.  Internally over at
>> Apple, we will stick with the swig-1.x bindings.  I don't think we will
>> care about the dot release (1.3.40 vs. 1.3.39, for example).  We'll just
>> make sure to use the latest for a {swig-major}.{swig-minor}.  As always, SB
>> API changes generated by swig need to remain swig-1.3 compatible (i.e.
>> swig-1.3 must be capable of generating usable Python wrappers).
>>
>> Ideally we don't need more than one set of bindings so we can avoid
>> needing to generate multiple ones when we do it.
>>
>>
>> * Add an explicit Python binding generation step.
>>
>> This would only be used by those who are touching the bindings (adding SB
>> API or adjusting the documentation of existing SB API).  In the event that
>> we need two bindings, we may just need a handshake that says "okay, we'll
>> take care of the swig 1.3 bindings, and {somebody who needs the other
>> binding} generates it for the other."  As SBAPI is additive only, this
>> should generally be fine as the worst case I can think of that would happen
>> would be failure to see new SBAPI in Python for a very brief time.  Since
>> maintainers will be writing Python tests to check their new SBAPIs, failure
>> to do the explicit generation step will be immediately obvious as a test
>> failure.  (The challenge here will likely be the swig 1.3.x requirement).
>>
>> If generating and testing new binding content with the right swig (i.e.
>> swig 1.3.x) becomes a real issue, we may be able to support a "please use
>> my local swig, whatever it is, but don't check in the static binding), with
>> a handshake that says "somebody with swig 1.3, please generate the modified
>> binding and check in).  We'll see if this becomes an issue.  I don't see
>> this as insurmountable.
>>
>>
>> * Clean up the swig generation scripts
>>
>> These look to have been implemented as direct ports of the older OS X
>> style bash scripts.  This Python script is very much the essence of using
>> the paradigms of one language in another, and being the worse for it.  It
>> implements features that are already present in the standard Python lib,
>> and is more verbose than it needs to be.
>>
>> Also, it is likely the script needs to be broken out into a few parts.
>> The scripts don't just generate the python binding using swig, they also
>> setup (for OS X) some packaging and other bits.  Right now none of that is
>> clearly broken out.  When we move to an explicit binding generation mode,
>> this does need to be broken out better.
>>
>>
>> * Get OS X Xcode build using the same bindings machinery as others.
>>
>> I tried this a while back (having Xcode adopt the Python-based swig
>> wrapper handling code), but gave up after hitting a few bugs where the
>> translated behavior was incorrect for Xcode.  I will move over to adopting
>> this on Xcode if possible while going through these changes.
>>
>
> I've begun this process.
>

With r253317, that is.


> I've got Xcode build (and the Xcode build only) wired up to
> scripts/prepare_bindings.py.  This, and the Python-specific binding
> preparation script, are considerably shorter than the older
> implementation.  It also addresses the bugs that prevented the former from
> working with the Xcode build.  The new scripts are pylint clean  (with
> stock configuration) except for a small handful of places that I intend to
> revisit when I have a few cycles to really tidy things up.  (Right now I'm
> just going for the 90% wins).
>
> Tomorrow I will fix up the tail end (the finalization) in a similar manner
> and plug it into the Xcode build.  I'll also try it locally on Linux and
> work out any issues there.
>
> I'll be tackling the static binding area (and will put up for review) only
> after I know the cleaned up binding preparation scripts are working
> properly everywhere.  I just didn't want to tackle that until I really
> understood what the scripts were already doing and it was in a state I felt
> I could maintain.
>
>
>> The primary goal here is to remove the requirement of having swig on the
>> system (e.g. for builders and most developers), shifting that burden a bit
>> more to those who actually modify the Python binding.
>>
>> As a potential added benefit, this opens us up to easier modification of
>> that binding generation step, which may prove to be useful later.
>>
>> Let me know if you have any feedback before I dive into this.
>>
>> -Todd
>>
>
>
>
> --
> -Todd
>



-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151116/6bc209c9/attachment.html>