[llvm-dev] RFC: Revisiting LLD-as-a-library design

[David Blaikie via llvm-dev]

On Thu, Jun 10, 2021 at 3:16 PM Tom Stellard via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> On 6/10/21 11:14 AM, Reid Kleckner via llvm-dev wrote:
> > Hey all,
> >
> > Long ago, the LLD project contributors decided that they weren't going
> to design LLD as a library, which stands in opposition to the way that the
> rest of LLVM strives to be a reusable library. Part of the reasoning was
> that, at the time, LLD wasn't done yet, and the top priority was to finish
> making LLD a fast, useful, usable product. If sacrificing reusability
> helped LLD achieve its project goals, the contributors at the time felt
> that was the right tradeoff, and that carried the day.
> >
>
> I think it would be great to move in this directions.  It's a little bit
> unclear
> at the moment whether or not the library use case is supported, because we
> do include headers and libraries in the install targets.
>
> As a package maintainer my wish list for an LLD library is:
>
> 1. Single shared object: https://reviews.llvm.org/D85278
> 2. All symbols are assumed private unless explicitly given library
> visibility.
>

Is this ^ consistent with how other parts of LLVM are handled? My
understanding was generally LLVM's API is wide/unbounded and not stable.
I'd hesitate to restrict future libraries in some way due to some benefits
that provides - ease of refactoring (LLVM's ability to be changed
frequently is very valuable).


> 3. Some subset of the API that is stable across major releases.
>

A limited stable C API seems plausible to me, if there's need.

- Dave


>
> -Tom
>
>
> > However, it is now ${YEAR} 2021, and I think we ought to reconsider this
> design decision. LLD was a great success: it works, it is fast, it is
> simple, many users have adopted it, it has many ports
> (COFF/ELF/mingw/wasm/new MachO). Today, we have actual users who want to
> run the linker as a library, and they aren't satisfied with the option of
> launching a child process. Some users are interested in process reuse as a
> performance optimization, some are including the linker in the frontend.
> Who knows. I try not to pre-judge any of these efforts, I think we should
> do what we can to enable experimentation.
> >
> > So, concretely, what could change? The main points of reusability are:
> > - Fatal errors and warnings exit the process without returning control
> to the caller
> > - Conflicts over global variables between threads
> >
> > Error recovery is the big imposition here. To avoid a giant rewrite of
> all error handling code in LLD, I think we should *avoid* returning failure
> via the llvm::Error class or std::error_code. We should instead use an
> approach more like clang, where diagnostics are delivered to a diagnostic
> consumer on the side. The success of the link is determined by whether any
> errors were reported. Functions may return a simple success boolean in
> cases where higher level functions need to exit early. This has worked
> reasonably well for clang. The main failure mode here is that we miss an
> error check, and crash or report useless follow-on errors after an error
> that would normally have been fatal.
> >
> > Another motivation for all of this is increasing the use of parallelism
> in LLD. Emitting errors in parallel from threads and then exiting the
> process is risky business. A new diagnostic context or consumer could make
> this more reliable. MLIR has this issue as well, and I believe they use
> this pattern. They use some kind of thread shard index to order the
> diagnostics, LLD could do the same.
> >
> > Finally, we'd work to eliminate globals. I think this is mainly a small
> matter of programming (SMOP) and doesn't need much discussion, although the
> `make` template presents interesting challenges.
> >
> > Thoughts? Tomatoes? Flowers? I apologize for the lack of context links
> to the original discussions. It takes more time than I have to dig those up.
> >
> > Reid
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210610/e2e3949a/attachment.html>