[cfe-dev] Clang builds a working Linux Kernel (Boots to RL5 with SMP, networking and X, self hosts)

[Bryce Lelbach]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Clang can now compile a functional Linux Kernel (version 2.6.36, SMP).

General Details
===============

  * Development and testing environment is a Macbook 5.1 (Intel C2D, x86_64)
    running Debian GNU/Linux.

  * The kernel can successfully boot to runlevel 5 (aka X + networking) on the
    Macbook, both on bare metal and in Qemu.

  * The kernel can successfully boot to runlevel 3 on a secondary test machine,
    a microATX desktop box (Intel Atom). I haven't tried to start X on this box
    yet.

  * The kernel can self-host; I am currently running a "fourth generation" self-
    hosted Linux kernel built by a "fourth generation" Clang.

Major Subsystems Successfully Built
===================================

  I have likely left out a few things here.

  * Core kernel stuff, Filesystems, Bus, PCI jazz, ACPI - No problems here that
    I've run into. I'm sure there are issues under the hood here, just haven't
    gotten to stress testing these yet.

  * SMP, SMT, SysV, pthreads, and POSIX IPC - I have tested these fairly
    extensively, (using the tiobench and rt-tests Debian packages, as well as
    the proposed Boost.Process library and the Boost.Thread). Multithreading and
    Multiprocessing in the compiled kernel seems to be functioning fine, but
    multitasking has a couple of intermittent issues when used in user space
    code (see below). The kernel -IS- doing a concurrent boot in runlevel 2,
    though. 

  * NUMA, swap, mm, slab allocator - Memory seems to be fine. I've compiled both
    LLVM + Clang and Linux multiple times on the clang built kernel, and I threw
    some "eat your memoryez" programs from the Debian repositories at it;
    doesn't look like there's trouble here.

  * Network stack (IPv4) - The IPv4 stack is fine, except for IPSec (see the
    notes on crypto support below). Netfilter and IPv6 both (separately) caused
    ICEs when I initially tried to build the network stack. I think I fixed
    this issue elsewhere (getting harddrive drivers to compile), so this may
    not be in bad shape at all (harddrive issue was an ICE caused by some
    seemingly legal conditional operators semantics in the kernel. The problem
    is specifically in CodeGen, which is the part of the clang that I've hacked
    the most, so it's likely this is a bug I introduced on my local fork).

  * Drivers and Firmware Stuff - Drivers have all been relatively well behaved
    thus far. However, I've had to be very picky, as any driver that depends on
    crypto stuff (even the basic kernel crc routines) breaks things. Mostly,
    I've just compiled what my laptop has needed and the generic drivers needed
    for my microATX desktop box/running the kernel in Qemu. I have added some
    of less obscure drivers to the build configuration, though. Here's a short
    list of things that work on my laptop (all the open source drivers are
    clang-compiled):

      * Graphics and sound. I installed Flash and listened to some BoostCon
        videos from last year to confirm this.
      * Keyboard. No backlight, though, but I never had that working in the
        first place.
      * DVD/CDROM.
      * Touchpad.
      * Various USB stuff.
      * iSight.
      * Speaker
 
Non-Essential Subsystems Failing at Compile Time
================================================

  Again, I'm possibly forgetting things here. I should also note that it's very
  possible that my hacks to either clang, Linux or the Linux build setup caused
  this.

  * SELinux, Posix ACLs, IPSec, eCrypt, anything that uses the crypto API - None
    of these will compile, due to either an ICE or variable-length arrays in
    structures (don't remember which, it's in my notes somewhere). If it's
    variable-length arrays or another intentionally unsupported GNUtension, I'm
    hoping it's just used in some isolated implementation detail (or details),
    and not a fundamental part of the crypto API (honestly just haven't had a
    chance to dive into the crypto source yet). I'm really hoping it's an issue
    in Clang, though, as it's easier for me to hack Clang and I'm trying to
    avoid kernel patches as much as possible.

  * IPv6 and Netfilters/Router stuff - Some of this is tied to the above issues
    with the crypto API, but IPv6 and Netfilters each have their own fatal
    errors. 
  * Virtualization - Virtualization is the only thing I haven't done any work on
    yet. I tried compiling minimal Xen support, ran into a fatal error, and put
    it on my "Get the basics functional first". 

Fatal Subsystem Failures Temporarily Averted by Compiling with GCC
==================================================================

  These two issues were already known when I started working on this, and
  conveniently, people had patched the Linux build system to build these systems
  with GCC, but were still unable to build a usable kernel. As I'm lazy, I took
  this work and reused it.

  * VDSO - VDSO breaks in strange ways with clang, at least, it did a week ago
    when I put some time into investigating this. ATM, building VDSO with GCC
    works, but I believe that this is still causing issues. I think the issue
    here is similiar in nature to the issue with LKMs. 

  * Boot - The very early kernel boot code breaks with clang, because of obscure
    inline assembly GNUtensions (.code16gcc stuff). I have no clue what needs to
    be done to fix this, but as I actually know where this problem is, it should
    be (relatively) easy to fix.

Crippling Problems
==================

  * Modules - Module loading is totally broken. I'm pretty sure I just figured
    out why, though (I'd elaborate, but this is pretty lengthy, and I might as
    well just go implement it).

Brief closing notes - I'm adapting a slightly aged POSIX test suite developed by
some Intel hackers for use in stress testing/functionality testing/unit testing
clang compiled kernels. This is something of a side project for me (I have school
and what not), but my intention is to maintain a Linux git repo with the source
patches/build set up needed to compile with a matching Clang git repo. Hopefully
in a few weeks/couple of months, with intensive automated testing and fixes to
the above issues, Clang + Linux will be feasible for producing production quality
Linux kernels.

I'm going to try to clean up some of my modifications to Clang (mostly hacks in
CodeGen stuff, local labels (not 100% done yet), explicit register variables, 
a more complete implementation of GNU inline assembly constraints). Some of my
changes implement the sort of cryptic GNUtensions that I sense most Clang devs
would find distasteful (I haven't added support for anything explicitly stated
as unsupported on the clang website).

P.S. boot logs:

http://gist.github.com/644488
http://gist.github.com/644490

I saved build times somewhere. Depending on how the configuration is tweaked/the
SMP support of the kernel that I'm building on top of, Clang builds Linux in about
13-15 minutes. 

Back to hacking :)

- -- 
Bryce Lelbach aka wash
http://groups.google.com/group/ariel_devel
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkzFI6IACgkQO/fqqIuE2t7RRQCaAzpFjAC7T9qTHunR69NNZzfk
Sj8AnR28NvtaRQv0fqM8S8gA+1UieqLj
=3ZwN
-----END PGP SIGNATURE-----