[llvm-dev] [RFC] Move py-mlir-release to new top-level repo in the LLVM org
Stella Laurenzo via llvm-dev
llvm-dev at lists.llvm.org
Thu Jan 7 10:55:51 PST 2021
Hi folks, I would like to propose that we create a new top-level repo in
the LLVM organization for organizing the Python MLIR Releases (both daily
and official numbered releases, whenever we are ready for such a thing) and
corresponding pushes to package repositories, etc.
I have prototyped such a release process in a personal repo:
Additional development on that release process is currently blocked on more
work on the shared library organization in LLVM (discussed here
being worked on independently) but it is useful as is and a reasonable
starting point for further work.
I would propose that we just fork my current repo into a new one in the
LLVM organization and then take the necessary steps to get
credentials/permissions/secrets set up in the new context.
Some answers to questions that may come up:
- *Why should this be a repo separate from llvm-project? *These kind of
automation repos tend to have a lot of "garbage" commits that I think is
best if they do not pollute the main repo (and also don't face contention
on automatic jobs bumping things, etc). They also tend to require special
permissions and secrets that we will want to more tightly control. They
also make use of other GitHub features that it seems like we would like not
polluting the main development flow ("Releases" tab, Actions, etc). Also,
this is the kind of thing that tends to get revised en-masse periodically,
and again, it would be good to not pollute the monorepo.
- *Why not land this in llvm-zorg? *llvm-zorg claims to be for "LLVM
Testing Infrastructure" and seems well scoped to that statement. What I am
managing above is periodic, automated release tooling based on open-source
CI systems (currently GitHub Actions), which are fairly standardized across
the Python releasing community, easy to set up, etc.
- *What ultimately will the code in this repo do?*
- Have periodic GitHub actions to select new LLVM revisions and
schedule daily/snapshot releases.
- Have manual actions for triggering official, numbered releases.
- Facilities for building Python wheels for PyPi and house any
additional metadata/automation needed for anaconda.
- Builds releases for all supported operating systems (currently
Linux/CentOS7/manylinux2014, MacOS, and Windows) and supported Python
versions (currently 3.6, 3.7, 3.8, 3.9).
- Publish release artifacts on the Releases tab for daily/snapshot
- Provide a stable reference point for downstream projects that
extend MLIR-Python and need to produce version-matched artifacts of their
- *Could this graduate to be more than "MLIR" python?* Maybe. I chose
the name because that is what I am focused on and didn't want to grab too
much land. But there is nothing stopping this from becoming automation for
general LLVM monorepo+incubator Python releasing.
- *What if we don't do this?*
- *Option A:* We keep running this in a private repo with the
disclaimer that is currently at the top: "Note that this is a
a real MLIR release process being run by a member of the community. These
are not official releases of the LLVM Foundation in any way, and they are
likely only going to be useful to people actually working on LLVM/MLIR
until we get things productionized." We would miss opportunities for
convergence with other projects and would cause things to fragment.
- *Option B: *We only publish Python bindings in official LLVM
release packages, and only for the Python version they are built with. We
don't release Python binaries through normal package management channels.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the llvm-dev