[llvm-dev] RFC: Revisiting LLD-as-a-library design

Reid Kleckner via llvm-dev llvm-dev at lists.llvm.org
Tue Jun 15 10:35:14 PDT 2021

Thanks for all the responses! I read through all the emails to try to
summarize the points that were brought up. I apologize if I missed yours.

I think the majority of responses were from LLD users, and they are clearly
in favor of this. Maintainers seem open to change with some concerns about
things like performance and merge conflicts.

I think the important thing is that we've established a direction. If you
review LLD code, consider reviewing and accepting patches that move in this
direction. I will try to put together a patch for COFF and we can
discuss the details of the code structure there.

As for the shared library discussion, I think the refactoring of error
handling and global state should come first. Then we can add a narrow API
and a shared library around it. I think fine grained APIs and extension
points should come later, if at all. I think it's important for LLD to stay
focused on the performance of its primary use case, and I wouldn't want
abstractions and extension points to get in the way of that. For these use
cases, Orc may be more appropriate. In the long run, though, anything can
happen. :)

On Thu, Jun 10, 2021 at 11:14 AM Reid Kleckner <rnk at google.com> wrote:

> Hey all,
> Long ago, the LLD project contributors decided that they weren't going to
> design LLD as a library, which stands in opposition to the way that the
> rest of LLVM strives to be a reusable library. Part of the reasoning was
> that, at the time, LLD wasn't done yet, and the top priority was to finish
> making LLD a fast, useful, usable product. If sacrificing reusability
> helped LLD achieve its project goals, the contributors at the time felt
> that was the right tradeoff, and that carried the day.
> However, it is now ${YEAR} 2021, and I think we ought to reconsider this
> design decision. LLD was a great success: it works, it is fast, it is
> simple, many users have adopted it, it has many ports
> (COFF/ELF/mingw/wasm/new MachO). Today, we have actual users who want to
> run the linker as a library, and they aren't satisfied with the option of
> launching a child process. Some users are interested in process reuse as a
> performance optimization, some are including the linker in the frontend.
> Who knows. I try not to pre-judge any of these efforts, I think we should
> do what we can to enable experimentation.
> So, concretely, what could change? The main points of reusability are:
> - Fatal errors and warnings exit the process without returning control to
> the caller
> - Conflicts over global variables between threads
> Error recovery is the big imposition here. To avoid a giant rewrite of all
> error handling code in LLD, I think we should *avoid* returning failure via
> the llvm::Error class or std::error_code. We should instead use an approach
> more like clang, where diagnostics are delivered to a diagnostic consumer
> on the side. The success of the link is determined by whether any errors
> were reported. Functions may return a simple success boolean in cases where
> higher level functions need to exit early. This has worked reasonably well
> for clang. The main failure mode here is that we miss an error check, and
> crash or report useless follow-on errors after an error that would normally
> have been fatal.
> Another motivation for all of this is increasing the use of parallelism in
> LLD. Emitting errors in parallel from threads and then exiting the process
> is risky business. A new diagnostic context or consumer could make this
> more reliable. MLIR has this issue as well, and I believe they use this
> pattern. They use some kind of thread shard index to order the diagnostics,
> LLD could do the same.
> Finally, we'd work to eliminate globals. I think this is mainly a small
> matter of programming (SMOP) and doesn't need much discussion, although the
> `make` template presents interesting challenges.
> Thoughts? Tomatoes? Flowers? I apologize for the lack of context links to
> the original discussions. It takes more time than I have to dig those up.
> Reid
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210615/4c37bc96/attachment.html>

More information about the llvm-dev mailing list