[LLVMdev] Sanitizers libs in Compiler-RT

[Iain Sandoe]

Hi Nick,

A few of my colleagues and I have been giving these general issues some serious thought - with a view to floating an abstract for a BOF proposal for Edinburgh - seems like a topic with a fair few interests to be resolved.

I think it is probably clear that we can't achieve everything that everyone wants in a single step - but also that having an agreed "end goal" and a possible route to get there is essential if we want to avoid chaos.

On 1 Feb 2014, at 00:44, Nick Kledzik wrote:

> On Jan 31, 2014, at 12:29 AM, Chandler Carruth <chandlerc at google.com> wrote:
>> I thought some of it still need libunwind (whichever of the various versions you like)? It would be good to get a nice, clean implementation of that functionality (whether based on one with an MIT license if there is such or not) if we don't already have it.
> The unwinder in libcxxabi implements both the _Unwind_* functions needed by libcxxabi and the unw_* functions that are the “libunwind” API.  There is nothing more needed.

So that component *could* comprise a stand-alone unwind dylib/so/a?

> On Jan 31, 2014, at 12:23 AM, David Chisnall <David.Chisnall at cl.cam.ac.uk> wrote
>> On 31 Jan 2014, at 08:12, Chandler Carruth <chandlerc at google.com> wrote:
>> 
>>> - There is the core runtime library. Historically this was called 'compiler-rt' informally, but perhaps better called 'libclang_rt', which provides the core necessary runtime library facilities to compile C or C++ applications. It's analogous to libgcc but without some of the unwinding code (as I understand it, there may be details I'm wrong about here or glossing over, but it's not relevant to the organization of things).
>> 
>> For some reason, the (generic, language-agnostic) unwind code is in libcxxabi.  There was some discussion about moving it into the compiler-rt repository, where it would make sense.  No one objected, but I'd rather not move it without a 'yes' from someone who is actually working on the code currently (I'd like to start factoring out the #ifdefs into a cleaner platform / architecture layer).
> 
> The logic was that libcxxabi is the biggest client of the unwinder, and well, compiler-rt was already complicated enough ;-)  That said, if we had a nice clean, scalable model for organizing all the runtime support libraries, I’d be happy to migrate the unwinder there.
> 
> Also, to help explain my bias, at Apple, the unwinder (and libc++abi) are dylibs that ship as part of the OS.  They have nothing to do with the compiler. 

Thus, a comment about language runtimes.

So, [in respect of language runtime libraries] a useful model moving forward should be that the compiler does not assume that target runtimes are installed as part of the compiler.  Essentially, the target system (even if it is the same as the host) might not have the compiler installed.

However, the compiler itself needs runtimes (or at least stub libraries) to link against for each supported target - so ideally (IMO) a set of sysroots for these supported targets would be installed relative to compiler - such that the whole package is completely relocatable.  In my ideal world the sysroots would be installed in a "SDK/sysroots" directory parallel to the bin, lib etc. so that they remain distinct from the compiler itself.  It might actually be tidier to consider the host as "just another cross target" - so that the entire process is uniform.

====

A comment about the unwind library:

Assume:
 (1) that there is only sensibly one unwinder [a design that allows multiple unwinders would solve some problems]
 (2) that there might be other clients for the unwind library in addition to c++ (e.g. an Ada, Fortran, other ObjC dialects .. etc.)
     It might be that new FEs would choose interoperability with c++ (even quite likely) - but it seems a little unreasonable for the compiler to impose that as a pre-constraint.

 (3) categories of system

    (a) ones where the entire stack is clang-based like modern OSX or freeBSD10.
    (b) ones where the clang components must interface with an existing unwinder (e.g. older OSX or Linux)
    (c) bare metal.

We have found over the course of darwin's history that binding the unwinder into a library (although that was not a Darwin-specific decision) made it difficult to maintain things in proper isolation - and made it very difficult to have tidy implementations of "3rd party" compiler ports.

Given that clang is a '3rd party' port to all the systems in categories (b) and (c) above - it would be a good idea to make maintenance and installation easier.

In fact, what this argues for is probably an unwinder.dylib / unwinder.so / unwinder.a since this component can be corrected or replaced without needing to install dependent libs (ABI constraints notwithstanding).

> A couple of other random thoughts about compiler-rt:
> 
> * One of the makefile dimensions of complexity is the ability to build optimized, profile, and debug copies of everything.  This was once needed at Apple, but no longer is necessary.
> 
> * One of the interesting things about compiler-rt is the static library to dynamic library migration (e.g. libgcc.a vs libgcc_s.so, or on Darwin libclang_*.a vs libSystem.dylib).  If the shared library ships independently from the compiler, then the compiler may need a .a file that can ship with it that contains any support functions not available in a shared library on the target.  Currently, it is a very manual process to figure out which functions are needed where.
> 
> * It would be nice if the clang build system could output a list of all possible support functions it might need for compiler being built.  That list could drive what parts of compiler-rt need to be built.
> 
> So, to me an ideal build system for compiler-rt would not just compile the snippets of code, it would figure out which snippets to build based on what the compiler needs and what the OS needs/provides.

This "library of fragments" would be probably the only runtime candidate for installation inside the actual compiler tree since it's essentially bound to a compiler version.

cheers
Iain