[llvm-dev] A libc in LLVM

Fri Jul 12 13:19:32 PDT 2019

* Siva Chandra via llvm-dev <llvm-dev at lists.llvm.org> [2019-07-12 08:15:40 -0700]:
> Below is a write up which I think would qualify as the "charter" for
> the new libc. It is also answering questions like, "where we’ll get
> with this libc?", "what's this libc actually going to be?" and similar
> ones. I have used libcxx.llvm.org landing page as a template to write
> it down.

sorry, but this charter does not make much sense.

i think you should get some c runtime developers involved.

> 
> ###############################################
> 
> "llvm-libc" C Standard Library
> ========================
> 
> llvm-libc is an implementation of the C standard library targeting C11
> and above. It also provides platform specific extensions as relevant.
> For example, on Linux it also provides pthreads, librt and other POSIX
> extension libraries.
> 
> Documentation
> ============
> 
> The llvm-libc project is still in the planning phase. Stay tuned for
> updates soon.
> 
> Features and Goals
> ================
> 
> * C11 and upwards conformant.
> * A modular libc with individual pieces implemented in the "as a
> library" philosophy of the LLVM project.

this is wrong on many levels:

LLVM is a bad example of library design, as it has library
safety and interface stability issues.

libc is necessarily a library so of course it is implemented
"as a library".

"modular libc" does not make much sense: with satic linking
you only link what you use (libc internal dependencies have
to be minimized if you static link anyway), with dynamic
linking, multiple modules is a huge mistake for other reasons
(it creates internal abi between the components that is
difficult to manage)

if modularity means configurability, then you have a much
bigger problem: maintenance and testing becomes harder
and if components are actually interchangable then you need
stable libc internal interfaces. (this is easy to do in
existing libcs, just not done because it's a disaster,
..except in uclibc ..which is a disaster)

> * Ability to layer this libc over the system libc.
> * Provide C symbols as specified by the standards, but take advantage
> and use C++ language facilities for the core implementation.

this mistake was already made in bionic, why ppl want to
rely on underspecified freestanding c++ language semantics
and making assumptions about c++ implementation internals
as well as dealing with subtle language incompatibilities
when writing a libc is a mistery.

> * Provides POSIX extensions on POSIX compliant platforms.
> * Provides system-specific extensions as appropriate. For example,
> provides the Linux API on Linux.
> * Vendor extensions if and only if necessary.
> * Designed and developed from the start to work with LLVM tooling and
> testing like fuzz testing and sanitizer-supported testing.

the difficulty of this is not in the libc, but various
issues in the sanitizer libraries.. e.g. you don't want to
fuzz test a libc with a fuzz runtime that depends on a c++
runtime (or any other external component that can call
back to libc outside of the control of the fuzz runtime).

> * ABI independent implementation as far as possible.

if you mean call abi, then you get this for free, just use
a portable language (such as c), which is what existing
libc implementations do anyway. (glibc has a bit more
target specifc asm than usual, but most of that has generic
fallback code so easy to drop the asm)

if you mean other abis then clarify.

> * Use source based implementations as far possible rather than
> assembly. Will try to “fix” the compiler rather than use assembly
> language workarounds.

same as above, all libcs do this already.

> 
> Why a new C Standard Library?
> =========================
> 
> Implementing a libc is no small task and is not be taken lightly. A
> natural question to ask is, "why a new implementation of the C
> standard library?" There is no single answer to this question, but
> some of the major reasons are as follows:
> 
> * Most libc implementations are monolithic. It is a non-trivial
> porting task to pick and choose only the pieces relevant to one's
> platform. The new libc will be developed with sufficient modularity to
> make picking and choosing a straightforward task.

this does not make sense (see above).

> * Most libc implementations break when built with sanitizer specific
> compiler options. The new libc will be developed from the start to
> work with those specialized compiler options.

this does not make sense.

(new libc does not make this easier: e.g. adding asan support
to musl is about a week work and most issues are sanitizer
related problems where you have to give up correctness or
reliability for sanitizers to work)

> * The new libc will be developed to support and employ fuzz testing
> from the start.

this does not make sense.

(new libc does not make this easier)

> * Most libc implementations use a good amount of assembly language,
> and assume specific ABIs (may be platform dependent). With the new
> libc implementation, we want to use normal source code as much as
> possible so that compiler-based changes to the ABI are easy. Moreover,
> as part of the LLVM project, we want to use this opportunity to fix
> performance related compiler bugs rather than using assembly
> workarounds.

citation needed.

(removing all unnecessary asm from musl and replacing it with
intrinsics is a weekend project)

> * A large hole in the llvm toolchain will be plugged with this new
> libc. With the broad platform expertise in the LLVM community, and the
> strong license and project structure, we think that the new libc will
> be more tunable and robust, without sacrificing the simplicity and
> accessibility typical of the LLVM project.

for this hole to be filled in on linux you need distros to be able
to rebuild everything against the new libc, otherwise you just
have a new libc that nobody can use because applications have
dependencies that have to be built against the same libc.

in some contexts it is enough to have a libc with build scripts to
build the dependencies.. but existing build scripts often don't
work out of the box with a new libc.

creating a software platform around a libc is a much bigger task
than the libc itself, and without that the libc has limited value.

it seems to me that none of the answers make much sense as is.

> 
> Platform Support
> ==============
> 
> llvm-libc development is still in the planning phase. However, we
> envision that it will support a variety of platforms in the coming
> years. Interested parties are encouraged to participate in the design
> and implementation, and add support for their favorite platforms.

the problem is not porting the new libc to a platfrom but
porting the platform to a new libc.

i.e. you need to worry a lot more about what's above the libc
(all userspace software) than what's below it (tiny os kernel
interface).

> 
> Current Status
> ============
> 
> llvm-libc development is still in the planning phase.
> 
> Build Bots
> =========
> 
> Coming soon.
> 
> Get involved!
> ===========
> 
> First please review our Developer's Policy. Stay tuned for llvm-libc
> specific information.
> 
> Design Documents
> ===============
> 
> Coming soon.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev