[llvm-dev] A libc in LLVM

[Peter Smith via llvm-dev]

On Mon, 24 Jun 2019 at 23:23, Siva Chandra via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> Hello LLVM Developers,
>
>
> Within Google, we have a growing range of needs that existing libc implementations don't quite address. This is pushing us to start working on a new libc implementation.
>

Are you able to share what some of these needs are? My reason for
asking is to see if there is a particular niche where existing libc
designs are not working, or if there is an approach that will handle
many use cases better than existing libc implementations.

>
> Informal conversations with others within the LLVM community has told us that a libc in LLVM is actually a broader need, and we are increasingly consolidating our toolchains around LLVM. Hence, we wanted to see if the LLVM project would be interested in us developing this upstream as part of the project.
>
>
> To be very clear: we don't expect our needs to exactly match everyone else's -- part of our impetus is to simplify things wherever we can, and that may not quite match what others want in a libc. That said, we do believe that the effort will still be directly beneficial and usable for the broader LLVM community, and may serve as a starting point for others in the community to flesh out an increasingly complete set of libc functionality.
>

I'm definitely interested in hearing more. Assembling an LLVM based
toolchain when there isn't an obvious native platform C library that
can be used could in theory benefit greatly from something like this.
As you point out, this might not be in your set of needs though.

>
> We are still in the early stages, but we do have some high-level goals and guiding principles of the initial scope we are interested in pursuing:
>
>
> The project should mesh with the "as a library" philosophy of the LLVM project: even though "the C Standard Library" is nominally "a library," most implementations are, in practice, quite monolithic.
>

There can be good reasons for designs to be monolithic though, for
example https://wiki.musl-libc.org/design-concepts.html . I'm not
enough of a C-library expert to say that this is always true, but it
does at least highlight that there is a risk that a toolkit suitable
for many libraries becomes too cumbersome to use in practice.

> The libc should support static non-PIE and static-PIE linking. This means, providing the CRT (the C runtime) and a PIE loader for static non-PIE and static-PIE linked executables.
>

Interesting. I've seen an embedded static-PIE loader embedded into an
image so that it could relocate itself. As all the dependencies were
statically linked there were only simple relative relocations to
resolve. Are you thinking of something along those lines or an
external loader program?

> If there is a specification, we should follow it. The scope that we need includes most of the C Standard Library; POSIX additions; and some necessary, system-specific extensions. This does not mean we should (or can) follow the entire specification -- there will be some parts which simply aren't worth implementing, and some parts which cannot be safely used in modern coding practice.
>

I'm interested in what sort of platform that the libc could run on and
what would be needed to be provided externally? In particular I'm
interested in whether a platform OS is required? I'm also interested
in where the boundaries of the libc, for example I'm thinking of
something like the separation of newlib and libgloss here?

> Vendor extensions must be considered very carefully, and only admitted when necessary. Similar to Clang and libc++, it does seem inevitable that we will need to provide some level of compatibility with other vendors' extensions.
>
> The project should be an exemplar of developing with LLVM tooling. Two examples are fuzz testing from the start, and sanitizer-supported testing.
>
>
> There are also few areas which we do not intend to invest in at this point:
>
>
> Implement dynamic loading and linking support.
>
> Support for more architectures (we'll start with just x86-64 for simplicity).
>

I strongly recommend you choose at least one other architecture and
build cross platform support in from the beginning. I suspect that
trying to put this in retroactively will put huge stress on the design
and the supporting infrastructure such as the build system. There is
also a danger of baking design decisions favouring one architecture
into the system, 32-bit vs 64-bit support is one obvious case. I'm
thinking that this is one area where the community could contribute.

>
> For these areas, the community is of course free to contribute. Our hope is that, preserving the "as a library" design philosophy will make such extensions easy, and allow retaining the simplicity when these features aren't needed.
>
>
> We intend to build the new libc in a gradual manner. To begin with,  the new libc will be a layer sitting between the application and the system libc. Eventually, when the implementation is sufficiently complete, it will be able to replace the system libc at least for some use cases and contexts.
>

I'm interested to see which system libc and existing platforms you
intend to support? Does this go as low as embedded system where the
platform is more like a board support package, or is this purely a
libc for platforms?

>
> So, what do you think about incorporating this new libc under the LLVM project?
>

Personally I think that if it can satisfy the needs of a sufficiently
broad segment of the community then I'm in favour. I'm looking forward
to seeing more.

Peter

>
> Thank you,
>
> Siva Chandra and the rest of the Google LLVM contributors
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev