[LLVMdev] how to track down a kernel miscompilation?

Mon Nov 24 09:01:55 PST 2008

One other note I forgot...

sig_sysreturn (arch/x86/kernel/signal_32.c) has a magic fudge factor
to fix up stack layout differences between llvm and gcc.  If when you
hit userspace you get signal handling errors, look at the frame
addresses printed out and adjust the fudge factor to make the frame
addresses match.  (the code automatically enables the debug output
after the first signal frame address error).  The fudge factor depends
on the optimizations and register allocator used, but is otherwise
seems stable for a given configuration.

Andrew

On Mon, Nov 24, 2008 at 10:53 AM, Andrew Lenharth <andrewl at lenharth.org> wrote:
> For people who want to try hacking on the linux kernel with llvm, here
> is a head start:
>
> http://llvm.org/~alenhar2/k.tbz
>
> This is a 2.6.27.5 kernel with a .config file for qemu/kvm with virtio
> devices (I also think it will work with the default devices).  The
> build process uses llvm-gcc -> opt -> llc so you can add your own
> (per-file) passes to the build process or debug specific passes.
>
> Directories arch/x86/boot and arch/x86/boot/compressed specify NOLLVM
> in their makefile which causes the make system to use gcc to build the
> directory.  options to opt are controlled by OPT_OPTIONS in the top
> level makefile.
>
> Some caviots and notes:
> * This has only been tested in kvm and qemu and mostly only with
> virtio net and block devices.  Other configurations are untested
> * The kernel is compiled at -O0 with some optimizations in the opt
> step.  This is because...
> * instcombine breaks the kernel
>
> Andrew
>
> On Mon, Nov 24, 2008 at 5:49 AM, Török Edwin <edwintorok at gmail.com> wrote:
>> Hi,
>>
>> I am trying to build the Linux kernel with LLVM.
>> 'ARCH=um' appears to work, now I am trying to get 'ARCH=x86' to work.
>>
>> So far it seems there is something wrong with the boot vga code (it
>> finds no video modes), the acpi code, and the serial console code.
>>
>> I am now trying to compile drivers/ with llvm-gcc and the rest with
>> gcc-4.2 (I have a wrapper script), I am compiling to native code, not
>> .bc files.
>>
>> When compiling drivers/serial/serial_core.c with gcc-4.2 it boots fine,
>> but when compiled with llvm-gcc, I get a kernel panic.
>>
>> How should I proceed further to find where the miscompilation is?
>> Can bugpoint help me here? (it takes ~10 seconds to compile and boot a
>> failing kernel with kvm/qemu)
>>
>> This is 2.6.28-rc6-tip, and LLVM SVN r59914.
>>
>> [    0.000000] Detected 2832.689 MHz processor.
>> [    0.004000] Console: colour VGA+ 80x25
>> [    0.004000] BUG: unable to handle kernel NULL pointer dereference at
>> 00000000000000e8
>> [    0.004000] IP: [<ffffffff802b3680>] kmem_cache_alloc+0x30/0xa0
>> [    0.004000] PGD 0
>> [    0.004000] Thread overran stack, or stack corrupted
>> [    0.004000] Oops: 0000 [#1] PREEMPT SMP
>> [    0.004000] last sysfs file:
>> [    0.004000] CPU 0
>> [    0.004000] Modules linked in:
>> [    0.004000] Pid: 0, comm: swapper Not tainted 2.6.28-rc6-tip #165
>> [    0.004000] RIP: 0010:[<ffffffff802b3680>]  [<ffffffff802b3680>]
>> kmem_cache_alloc+0x30/0xa0
>> [    0.004000] RSP: 0018:ffffffff8075dd48  EFLAGS: 00010086
>> [    0.004000] RAX: 0000000000000000 RBX: ffffffff806e2320 RCX:
>> ffffffff80213546
>> [    0.004000] RDX: 0000000000002580 RSI: 00000000000000d0 RDI:
>> 0000000000000000
>> [    0.004000] RBP: ffffffff8075dd68 R08: 0000000000000008 R09:
>> 000000000000006e
>> [    0.004000] R10: 0000000000000000 R11: 0000000000000000 R12:
>> 0000000000000286
>> [    0.004000] R13: 00000000000000d0 R14: ffffffff80847500 R15:
>> 0000000000000000
>> [    0.004000] FS:  0000000000000000(0000) GS:ffffffff80757ec0(0000)
>> knlGS:0000000000000000
>> [    0.004000] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
>> [    0.004000] CR2: 00000000000000e8 CR3: 0000000000201000 CR4:
>> 00000000000006a0
>> [    0.004000] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
>> 0000000000000000
>> [    0.004000] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
>> 0000000000000400
>> [    0.004000] Process swapper (pid: 0, threadinfo ffffffff8075c000,
>> task ffffffff806e2320)
>> [    0.004000] Stack:
>> [    0.004000]  ffffffff806e2320 ffffffff806e2320 0000000000000000
>> ffffffff80847500
>> [    0.004000]  ffffffff8075dd88 ffffffff80213546 ffffffff806e2320
>> ffffffff8075c000
>> [    0.004000]  ffffffff8075dda8 ffffffff8020d6b9 0000000000000001
>> 0000000000000000
>> [    0.004000] Call Trace:
>> [    0.004000]  [<ffffffff80213546>] init_fpu+0x106/0x130
>> [    0.004000]  [<ffffffff8020d6b9>] math_state_restore+0x89/0xc0
>> [    0.004000]  [<ffffffff80588739>] do_device_not_available+0x9/0x10
>> [    0.004000]  [<ffffffff8020d2a5>] device_not_available+0x15/0x20
>> [    0.004000]  [<ffffffff8045a5df>] ? uart_set_options+0xf/0xf0
>> [    0.004000]  [<ffffffff8045ca5f>] ? uart_parse_options+0x2f/0x90
>> [    0.004000]  [<ffffffff807896a8>] serial8250_console_setup+0xa8/0xc0
>> [    0.004000]  [<ffffffff8023801e>] register_console+0x28e/0x2f0
>> [    0.004000]  [<ffffffff807899c5>] serial8250_console_init+0x155/0x160
>> [    0.004000]  [<ffffffff80788542>] console_init+0x32/0x50
>> [    0.004000]  [<ffffffff80765d30>] start_kernel+0x230/0x3e0
>> [    0.004000]  [<ffffffff8076527c>] x86_64_start_reservations+0x7c/0xc0
>> [    0.004000]  [<ffffffff807653bb>] x86_64_start_kernel+0xcb/0xf0
>> [    0.004000] Code: 83 ec 20 4c 89 6c 24 10 48 89 1c 24 41 89 f5 4c 89
>> 64 24 08 4c 89 74 24 18 48 8b 4d 08 9c 41 5c fa 65 8b 04 25 24 00 00 00
>> 48 98 <48> 8b 94 c7 e8 00 00 00 48 8b 1a 44 8b 72 18 48 85 db 74 44 8b
>> [    0.004000] RIP  [<ffffffff802b3680>] kmem_cache_alloc+0x30/0xa0
>> [    0.004000]  RSP <ffffffff8075dd48>
>> [    0.004000] CR2: 00000000000000e8
>>
>> Best regards,
>> --Edwin
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>