[llvm-dev] FW: [GSoC 2021] Some ideas & questions about the Enzyme GSoC project

Sun Apr 4 19:49:24 PDT 2021

Seems reasonable to me.

On Sat, Apr 3, 2021 at 10:50 AM <chuyangchen at foxmail.com> wrote:

> I have considered user-custom derivatives. I think we can define
> proc-macros that allows users to use their own defined derivatives (instead
> of ones generated by Enzyme) in this project, then the registration of
> frequently-used functions’ derivatives can be implemented in a separated
> projects.
>
>
>
> So, if not misunderstood, the outline of the project is thus:
>
> 1.       a Rust metadata parser and some modification to Enzyme to use
> the parsed metadata
>
> 2.       a Rust-specific optimization pass or extending the Rust compiler
>
> 3.       some utilities to provide handlers to Rust to implement the
> fore-mentioned functions, like user-defined derivatives and indicating
> markers
>
> As to the Rust API, it can be implemented in Manuel Drehwald’s project.
> The 3rd part may seems to overlap with that, but essentially, in my idea,
> the 3rd part of the GSoC project will only contains the minimal set to
> make Enzyme function well in Rust, and the rest will be left to Manuel
> Drehwald’s project.
>
>
>
> If there’s no problem, I’ll further dive into the 1st, 2nd, and 3rd part
> sequentially, and write a draft of the proposal soon.
>
>
>
> Chuyang Chen
>
> MSc Student
>
> At Department of Computer Science and Technology, Nanjing University
>
>
>
>
>
> *From:* William Moses <wmoses at mit.edu>
> *Sent:* Friday, April 2, 2021 12:01 PM
> *To:* Chuyang Chen <chuyangchen at foxmail.com>
> *Cc:* William Moses <wmoses at mit.edu>; llvm-dev <llvm-dev at lists.llvm.org>;
> Enzyme AD <enzyme-dev at googlegroups.com>
> *Subject:* Re: Re: FW: [GSoC 2021] Some ideas & questions about the
> Enzyme GSoC project
>
>
>
> Regarding debug information, yeah it's not necessarily the best mechanism
> (obviously that would be direct Rust compiler integration), but I believe
> it should be sufficient at least for now. Presumably, we can always run a
> pass that removes debug info after Enzyme if the extra information becomes
> a concern.
>
> I think your understanding of the project components is mostly correct.
> The one thing that I might add (and of course there's a lot here, not all
> of which needs completion during GSOC) is potential registration of custom
> derivatives for various rust internal routines/allocation mechanisms, and
> possibly marking explicitly the activity analysis markers of routines such
> as print to be inactive. The second bit is to clarify that Enzyme, by
> virtue of being an LLVM pass natively has support for LLVM Metadata. The
> specific issue here is parsing the specific layout of Rust types in the
> metadata into something more easily digestible to LLVM/Enzyme type analysis.
>
> Manuel Drehwald (trying to find email and will add to cc) has recently
> been looking into exporting the Enzyme C API as a crate to be called from
> Rust. Once that exists, we can extend the Rust compiler or perhaps create a
> special Rust-specific optimization pass that calls the Enzyme API with the
> Type metadata parsing to perform end-to-end tests.
>
> If you're looking to dive right in, I'd probably suggest pushing either
> the metadata parsing or Rust plugin (or custom codegen pass) as next steps.
>
>
>
> On Wed, Mar 31, 2021 at 7:21 AM Chuyang Chen <chuyangchen at foxmail.com>
> wrote:
>
> The metadata parser in the github discussion
> <https://github.com/tiberiusferreira/oxide-enzyme/issues/1#issuecomment-735634144>
> seems to work on debuginfo. It means that each time we want Enzyme to work,
> we should add a “-g” option to the compiling command “rustc”. I’m a bit
> in doubt whether this will be OK though it doesn’t make much difference.
> But will there be some cases, like running on a device whose memory is
> limited, where adding debuginfo do have some negative effect? Anyway, this
> may be the most fitting way.
>
>
>
> Now, AIUI, there are 4 things need to be done in the project:
>
> 1.      writing a parser parsing LLVM metadata and passing it to Enzyme
>
> 2.      adjusting Enzyme to leverage the metadata
>
> 3.      writing a Rust crate to provide APIs to Rust user
>
> 4.      writing a Rust compiler patch to allow importing LLVM plugins to
> the Rust compiler
>
> Is that right?
>
>
>
> *From:* William Moses <wmoses at mit.edu>
> *Sent:* Wednesday, March 31, 2021 10:51 AM
> *To:* Chuyang Chen <chuyangchen at foxmail.com>
> *Cc:* Billy Moses <wmoses at mit.edu>; llvm-dev <llvm-dev at lists.llvm.org>;
> Enzyme AD <enzyme-dev at googlegroups.com>
> *Subject:* Re: FW: [GSoC 2021] Some ideas & questions about the Enzyme
> GSoC project
>
>
>
> There's certainly utility to be gained by hooking into the rust compiler.
> At minimum this allows us to effectively integrate high level functions for
> calling into Enzyme as an API without going through FFI.
>
> As you say, one of the biggest issues here is essentially parsing Rust
> types. This actually may not require a modification to rustc as I believe
> this is available in LLVM as LLVM metadata (see
> https://github.com/tiberiusferreira/oxide-enzyme/issues/1#issuecomment-735634144
> for an earlier discussion on this topic specifically).
>
> It turns out that one doesn't need to piggyback off LLD if rustc itself
> can call the custom LLVM pass which runs Enzyme. This could be done by say
> loading a plugin to rust, or otherwise calling the API. Of course in
> general enabling better integration of LLVM internals/pass plugins into
> rust directly is potentially desirable. I recently proposed a patch to
> rustc that would allow importing LLVM plugins that may be worth a look (see
> https://github.com/rust-lang/compiler-team/issues/419
> https://github.com/rust-lang/rust/pull/82734). That said such support
> isn't a blocker, as much as we need to start parsing this metadata. In
> theory we can even fork the "nice interface" and "parsing metadata" into
> two concurrent projects/pieces.
>
>
>
> On Tue, Mar 30, 2021 at 12:04 PM Chuyang Chen <chuyangchen at foxmail.com>
> wrote:
>
> A thing I forgot to mention is that a Rust feature named “compiler plugin”
> do satisfy the requirement of “emitting extra debuginfo to the generated
> IR code”, but it’s unstable and won’t be stabilized in the foreseeable
> future. Moreover, as to “using lld as its linker and add the Enzyme pass
> to lld”, compiler plugins help nothing.
>
>
>
> *From:* Chuyang Chen <chuyangchen at foxmail.com>
> *Sent:* Tuesday, March 30, 2021 11:44 PM
> *To:* wmoses at mit.edu
> *Cc:* llvm-dev at lists.llvm.org; 'Enzyme AD' <enzyme-dev at googlegroups.com>
> *Subject:* [GSoC 2021] Some ideas & questions about the Enzyme GSoC
> project
>
>
>
> Hello, I’m working on the GSoC project “Integrate Enzyme into Rust to
> provide high-performance differentiation in Rust”. I have skimmed over
> the codebase and now I have some preliminary ideas about the project.
>
>
>
> As you have said, the project can be divided to two main parts:
>
> 1.      passing and parsing Rust type metadata into LLVM / Enzyme type
> analysis, and
>
> 2.      integrating the Enzyme API/pass into rust.
>
>
>
> Integrating Enzyme API to rust is trivial by using proc_macros in Rust.
> However, passing Rust type metadata to LLVM and integrating the Enzyme pass
> into rust is difficult if we don’t modify the Rust compiler. The Rust
> compiler seems to lack ability to emit metadata to LLVM, and if we want to
> add the Enzyme pass without modifying the compiler, the only way is, I
> think, using a command like
>
> [image: cid:image003.png at 01D725BE.92A91F10]
>
> which is not user-friendly. I sought hard for a nonintrusive way to
> implement them but got nothing, so it seems that the only option is
> modifying the Rust compiler to let it
>
> 1.      emit extra debuginfo to the generated IR code, and
>
> 2.      use lld as its linker and add the Enzyme pass to lld
>
> with a specific option, say “--enzyme”
>
>
>
> However, *is it appropriate to modify the compiler to just port a plugin
> to it, though the modification won**’t be big or impact the rest of the
> compiler?*
>
>
>
> PS: Another question is about differentiating closures in Rust, but that
> can be postponed to the following discussions.
>
>
>
> Chuyang Chen
>
> MSc Student
>
> At Department of Computer Science and Technology, Nanjing University
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210404/2ed7862c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 34386 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210404/2ed7862c/attachment-0001.png>