[LLVMdev] Re: LLVM and PyPy

[Chris Lattner]

On Fri, 31 Oct 2003, Armin Rigo wrote:

> Hello Chris,
>
> We have been investigating your project and the good documentation
> and are very impressed. If we understood your goals correctly
> this seems like a good match for our ongoing and active PyPy project,
> a reimplementation of the Python language in Python.

Cool.  We are all big fans of Python here.  :)

> We'll definitely try using llvm as our low-level backend.
> But actually we contact you now to ask you if you'd be interested
> in some bidirectional collaboration.

I've read up a bit on PyPy, and it looks like LLVM could be a nice way to
get the JIT type interface that you would like.  Also, making use of the
LLVM optimizer can make your staticly generated code nice and fast.  :)

> Maybe a bit more background what we did or are:
>
> - we are an open international group of individuals collaborating on
>   our free time mostly. We are very involved with research, open source
>   communities and especially the Python communities.
>
> - during the course of four one-week meetings (which we call development
>   "sprints") we have done a rather complete interpreter and can translate
>   parts of it to C or Lisp code already (using a control-flow representation
>   which is actually very similar to LLVM code, hence our enthusiasm!)
>
> - we very recently submitted a funding proposal to the European Union:
>
>     http://codespeak.net/svn/pypy/trunk/doc/funding/proposal/part_b.pdf
>
>   and you may find these two chapters particularly interesting:
>
>     http://codespeak.net/pypy/index.cgi?doc/funding/B1.0_objectives
>     http://codespeak.net/pypy/index.cgi?doc/funding/B6.0_detailed

It sounds like LLVM could be a good implementation strategy for your
goals.  You're right that developing a JIT from stratch is a lot of work
:)

> On the technical level we are interested to know about and maybe collaborate
> with efforts to support very-high-level language features in LLVM (e.g.
> walking the stack, for garbage collection), fine-grained runtime code generation
> (generating code only one basic block at a time),

These are definitely features that we plan to add, but just haven't gotten
to yet.  In particular, Alkis is working on a Java front-end, which will
require similar features.  In the beginning, we will probably just use a
conservative collector, eventually adding support for precise GC.

We already have the capability of doing function-at-a-time code
generation: what is basic-block at a time generation used for?  How do you
do global optimizations like register allocation?

> and possibly also contribute
> a PowerPC back-end, and Python bindings for LLVM.

That would be great!  We've tossed around the idea of creating C bindings
for LLVM, which would make interfacing from other languages easier than
going directly to the C++ API, but we just haven't had a chance to yet.
Maybe you guys would be interested in helping with that project?

> On another level, some of the PyPy core developers are actually also
> involved with the 'codespeak' site which aims at connecting interesting
> open source projects and provide new collaborative development services.
> The PyPy project is extensively using subversion which is a very interesting
> (and stable) alternative to cvs.  So if you need any help with setting
> up some publically accessible infrastructure the codespeak guys will
> certainly welcome you.

At this point, we're working like crazy to get important features
implemented in LLVM.  We certainly acknowledge that CVS has severe
deficiencies, but in the near future we'll probably stay with it.
Perhaps after SVN 1.0 comes out... :)

> Feel free to forward this mail to the LLVM mailing list, btw.  We are
> just interested in getting some first contact and enter a productive
> discussion and - who knows - some interesting collaboration!

Done.  In general, that's a good place to discuss all kinds of issues like
this.  Please let us know what your plans are and what the next step is.
Perhaps C bindings would be the most logical starting place?

-Chris

-- 
http://llvm.cs.uiuc.edu/
http://www.nondot.org/~sabre/Projects/