[llvm-dev] Fwd: FW: [GSoC 2021] Some ideas & questions about the Enzyme GSoC project

Tue Mar 30 19:53:27 PDT 2021

---------- Forwarded message ---------
De: William Moses via llvm-dev <llvm-dev at lists.llvm.org>
Date: mar., 30 de mar. de 2021 20:51
Subject: Re: [llvm-dev] FW: [GSoC 2021] Some ideas & questions about the
Enzyme GSoC project
To: Chuyang Chen <chuyangchen at foxmail.com>
Cc: llvm-dev <llvm-dev at lists.llvm.org>, Enzyme AD <
enzyme-dev at googlegroups.com>

There's certainly utility to be gained by hooking into the rust compiler.
At minimum this allows us to effectively integrate high level functions for
calling into Enzyme as an API without going through FFI.

As you say, one of the biggest issues here is essentially parsing Rust
types. This actually may not require a modification to rustc as I believe
this is available in LLVM as LLVM metadata (see
https://github.com/tiberiusferreira/oxide-enzyme/issues/1#issuecomment-735634144
for an earlier discussion on this topic specifically).

It turns out that one doesn't need to piggyback off LLD if rustc itself can
call the custom LLVM pass which runs Enzyme. This could be done by say
loading a plugin to rust, or otherwise calling the API. Of course in
general enabling better integration of LLVM internals/pass plugins into
rust directly is potentially desirable. I recently proposed a patch to
rustc that would allow importing LLVM plugins that may be worth a look (see
https://github.com/rust-lang/compiler-team/issues/419
https://github.com/rust-lang/rust/pull/82734). That said such support isn't
a blocker, as much as we need to start parsing this metadata. In theory we
can even fork the "nice interface" and "parsing metadata" into two
concurrent projects/pieces.

On Tue, Mar 30, 2021 at 12:04 PM Chuyang Chen <chuyangchen at foxmail.com>
wrote:

> A thing I forgot to mention is that a Rust feature named “compiler plugin”
> do satisfy the requirement of “emitting extra debuginfo to the generated
> IR code”, but it’s unstable and won’t be stabilized in the foreseeable
> future. Moreover, as to “using lld as its linker and add the Enzyme pass
> to lld”, compiler plugins help nothing.
>
>
>
> *From:* Chuyang Chen <chuyangchen at foxmail.com>
> *Sent:* Tuesday, March 30, 2021 11:44 PM
> *To:* wmoses at mit.edu
> *Cc:* llvm-dev at lists.llvm.org; 'Enzyme AD' <enzyme-dev at googlegroups.com>
> *Subject:* [GSoC 2021] Some ideas & questions about the Enzyme GSoC
> project
>
>
>
> Hello, I’m working on the GSoC project “Integrate Enzyme into Rust to
> provide high-performance differentiation in Rust”. I have skimmed over
> the codebase and now I have some preliminary ideas about the project.
>
>
>
> As you have said, the project can be divided to two main parts:
>
> 1.      passing and parsing Rust type metadata into LLVM / Enzyme type
> analysis, and
>
> 2.      integrating the Enzyme API/pass into rust.
>
>
>
> Integrating Enzyme API to rust is trivial by using proc_macros in Rust.
> However, passing Rust type metadata to LLVM and integrating the Enzyme pass
> into rust is difficult if we don’t modify the Rust compiler. The Rust
> compiler seems to lack ability to emit metadata to LLVM, and if we want to
> add the Enzyme pass without modifying the compiler, the only way is, I
> think, using a command like
>
> [image: cid:image003.png at 01D725BE.92A91F10]
>
> which is not user-friendly. I sought hard for a nonintrusive way to
> implement them but got nothing, so it seems that the only option is
> modifying the Rust compiler to let it
>
> 1.      emit extra debuginfo to the generated IR code, and
>
> 2.      use lld as its linker and add the Enzyme pass to lld
>
> with a specific option, say “--enzyme”
>
>
>
> However, *is it appropriate to modify the compiler to just port a plugin
> to it, though the modification won**’t be big or impact the rest of the
> compiler?*
>
>
>
> PS: Another question is about differentiating closures in Rust, but that
> can be postponed to the following discussions.
>
>
>
> Chuyang Chen
>
> MSc Student
>
> At Department of Computer Science and Technology, Nanjing University
>
>
>
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210330/88fdfcb1/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 30328 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210330/88fdfcb1/attachment-0001.png>