[llvm-dev] RFC: Enzyme, Automatic Differentiation for LLVM as an LLVM Incubator Project

[William Moses via llvm-dev]

Hi all,

Since it seems like all of the feedback here is positive, what would be the
next steps (migrate Enzyme mailing list to LLVM, create discord/discourse,
etc)?

@Renato

The split support sounds like a good solution to me.

Regarding Enzyme/MLIR, the idea there isn't necessarily to use Enzyme to
differentiate MLIR directly, but lowering MLIR to LLVM then running Enzyme
could be an interesting (though not necessarily ideal) way to provide
differentiable programming in MLIR. We're also considering extending Enzyme
to work directly on MLIR as well and while indeed many parts of the
analysis are specific to LLVM instructions, other differentiation specific
analyses likely will have components which can be shared (e.g. Activity
Analysis which determines whether there exists a path through the program
that enables differentiable information to flow from input to output).

Cheers,
Billy

On Fri, Jan 15, 2021 at 12:54 PM Renato Golin <rengolin at gmail.com> wrote:

> Hi William,
>
> I think this is a really cool project and worthy of being in LLVM. For now
> it's a plugin pass, which goes well with incubator projects, but it could
> very well be a standard IR pass that is enabled by flags, etc. We have
> similar examples for OpenCL, OpenMP, etc. which need integration on both
> Clang and LLVM. Shouldn't be too messy.
>
> On Tue, 12 Jan 2021 at 22:14, William Moses via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Obviously integration into the monorepo itself would fix this for
>> subsequent versions so it's somewhat of a chicken-and-the egg issue. I'd
>> love to hear any thoughts from folks on how something like this might be
>> eventually handled. I'd also like to see upstream users (for example to
>> differentiate MLIR -- ideally with nice integration for reductions, see
>> comment below regarding parallelism).
>>
>
> I think long term you should split support for older versions with the
> trunk version. Once (and if) it gets merged into the monorepo, we can keep
> the incubation repo with the previous versions only, and only to official
> versions, mostly static and for historical value. People should be
> encouraged to use the newer versions that have native support.
>
> Differentiating MLIR would probably be a separate infrastructure, not sure
> using the same libraries? I haven't dug too much, but I expect some of the
> analysis to be tailored to LLVM's operations and types, which have fixed
> semantics and are very different from MLIR, with custom dialects,
> operations and types. But it would be really nice if we had that at the
> MLIR level, at least for the standard upstream dialects, as it would be a
> major boost for ML compilers to start using MLIR more aggressively.
>
> cheers,
> --renato
>
> --
> You received this message because you are subscribed to the Google Groups
> "Enzyme AD" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to enzyme-dev+unsubscribe at googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/enzyme-dev/CAPH-gfeYP1ix4qa4jbGCSMoqY19cce_CV-q-NkjwiuEw2aMpCA%40mail.gmail.com
> <https://groups.google.com/d/msgid/enzyme-dev/CAPH-gfeYP1ix4qa4jbGCSMoqY19cce_CV-q-NkjwiuEw2aMpCA%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210126/c66384eb/attachment.html>