[llvm-dev] A libc in LLVM

[Siva Chandra via llvm-dev]

On Tue, Jun 25, 2019 at 2:53 AM Peter Smith <peter.smith at linaro.org> wrote:
>
> On Mon, 24 Jun 2019 at 23:23, Siva Chandra via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> >
> > Hello LLVM Developers,
> >
> >
> > Within Google, we have a growing range of needs that existing libc implementations don't quite address. This is pushing us to start working on a new libc implementation.
> >
>
> Are you able to share what some of these needs are? My reason for
> asking is to see if there is a particular niche where existing libc
> designs are not working, or if there is an approach that will handle
> many use cases better than existing libc implementations.

There have been a lot of questions about our reasons for opting to
build a new libc and why an existing libc implementation does not meet
our needs. I will try to address these questions in a general fashion
in this email. I will answer individual concerns separately. Before I
start, I also want to apologize if I am being late to answer, or
appearing to be ignoring some of the emails. I am not trying to ignore
or avoid any one or any question - it is just that I need time to
process your questions and compose meaningful answers.

So, we have a bunch of reasons for a new libc and why we prefer it to
be a part of the LLVM project:

1. Static linking without the complexity of dynamic linking - Most
libc implementations end up being complicated because they support
dynamic loading/linking. This is not bad by itself, but we want to be
able to take out dynamic linking capability where possible and get the
benefits of the much simpler system. We believe that building
everything in a “as a library fashion” would facilitate this.

2. As somebody else has pointed out in the list, we want to have a
libc with as much fine grained modularity as possible. This not only
helps one to pick and choose what they want, but also makes it easy to
adapt to different build systems. Moreover, such a modular system will
also facilitate deploying chunks of functionality during the
transition from another libc to this new libc.

3. Sanitizer supported testing and fuzz testing from the start - Doing
this from the start will impact few design choices non-trivially. For
example, sanitizers need that a target be rebuilt with sanitizer
specific specialized options. We want to develop the new libc in such
a fashion that it will work with these specialized options as well.

4. ABI independent implementation as far as possible - There will be
places where it would not be possible to implement in an ABI
independent fashion. However, wherever possible, we want to use normal
source code so that compiler-based changes to the ABI are easy. Our
reasons for ABI independent implementations fall into two categories:

a) Long term changes to the ABI for security like SCADS, and for
performance tuning like caller/callee register ratios to better match
software and hardware.
b) Rapid deployment of specific ABI changes as part of security
mitigation strategies such as those for Spectre. For example,
speculative load hardening would have vastly benefitted from being
able to change the calling convention.

5. Avoid assembly language as far as possible - Again, there will be
places where one cannot avoid assembly level implementations. But,
wherever possible, we want to avoid assembly level implementations.
There are a few reasons here as well:

a) We want to leverage the compiler for performance wherever possible,
and as part of the LLVM project, fix compiler bugs rather than use
assembly.
b) Enable sanitizers and coverage-based fuzzing to work well across
the implementation of libc.
c) Allow deploying compiler-based security mitigations such as those
we needed for Spectre.

6. Having the support of the LLVM community, project, and
infrastructure - From access to the broad platform expertise in the
community to the strong license and project structure, we think the
project will be significantly more successful as part of LLVM than
elsewhere.

All this does not mean we want to implement everything from scratch.
If someone has implementations for parts of the libc ready, and would
like to contribute to this project under the LLVM license, we will
certainly welcome it.