[LLVMdev] LLVM JIT on a Baremetal x86 Machine !!!

Tue Nov 8 07:59:02 PST 2011

Hi,

First question: "/module" is mapped to a special file that reads a kernel module passed in by the bootloader. Much like GRUB, kiwi's bootloader loads a kernel and can load one or more extra files into memory. These are passed to the kernel.

The horizon kernel expects one file, which it makes accessible at "/module". This should be set up to be whatever you set KERNEL_HBC to be.

Second question: The program is statically compiled on the host system against the host C and C++ libraries. Then, it is patched slightly to change linux syscall instructions to "int 0xff" instructions, which get trapped at runtime.

At build time it reads your linux kernel headers to find what syscall numbers map to what syscalls, and emulates a subset of the linux kernel. This is how the C standard library etc works.

Cheers,

James

-----Original Message-----
From: Mian M. Hamayun [mailto:mian-muhammad.hamayun at imag.fr]
Sent: 08 November 2011 15:41
To: James Molloy
Cc: llvmdev at cs.uiuc.edu
Subject: Re: [LLVMdev] LLVM JIT on a Baremetal x86 Machine !!!

Hi James,

I have two questions for you.

Firstly, what is the role of 'module' in init.cc? I can see that its
being treated like it is a 'bytecode' file, as we open it and then pass
it to the ByteCoder and eventually construct llvm module from it.

Like In file init.cc, line:121
     FILE *stream  = fopen("/module", "rb");
     ...
     fread(c, 1, sz, stream);
     ...
     Bytecoder bc(std::cout, c, sz, &errors);
     ...

The Open SysCall returns a pointer to the 'Special File' that was
created in 'InitialiseSpecialFiles':
In file SpecialFile.cc, line:204, its defined as a special file:
     static SpecialModule modl("/module");
     s_files[3] = &modl;

Is this 'module' somehow related to 'KERNEL_HBC' that we specified earlier ?
If yes then how I am getting the size of this special file to be zero,
whereas the size of my 'hello_world.hbc' is 225 bytes.
If no, then what is it ?

Secondly You have mentioned on the Horizon wiki:
"The build process will produce a version of horizon statically linked
with code that provides enough of a UNIX environment for LLVM and
libstdc++ to function"

Could you please shed some more light on this aspect (Some links, if
possible), on how you have accomplished this ?
I am not an expert in these aspects, your help in this regard will be
highly appreciated.
--
Hamayun

P.S. I am also attaching files with a basic lseek (SyscallLSeek)
implementation and an execution log showing the current failure
("Invalid magic number reading bytecode file.").

On 11/03/2011 05:37 PM, James Molloy wrote:
> Hi Mian,
>
> Looking at the runlog, everything seems fine until LLVM attempts to use lseek() on a file.
>
> You see the PANIC because Horizon hasn't implemented lseek yet.
>
> Obviously the version of GlibC I was using does not use lseek in that circumstance, but yours does. You just need to implement lseek :)
>
> Cheers,
>
> James
>
> -----Original Message-----
> From: Mian M. Hamayun [mailto:mian-muhammad.hamayun at imag.fr]
> Sent: 03 November 2011 15:59
> To: James Molloy
> Cc: llvmdev at cs.uiuc.edu
> Subject: Re: [LLVMdev] LLVM JIT on a Baremetal x86 Machine !!!
>
> Hello James,
>
> I hope I will not be disturbing you too much. As I mentioned earlier, I
> am interested in unhosted (baremetal) x86 environment for LLVM JIT.
> Let me come to the point straight.
>
> I have set the KERNEL_HBC as:
> set(KERNEL_HBC
> "/altamaha/home3/hamayun/workspace/horizon/code-samples/hello_world.hbc")
>
> And Compiled the Horizon Sources. Compilation works fine and it creates
> the cd.img as you described on:
> http://www.quokforge.org/projects/horizon/wiki/Building
>
> See the attached "Horizon_Build.Log" file for more details.
>
> Now when I load this 'image' using qemu; It gives me the following error:
> horizon: PANIC: syscall 0x8 (lseek) @ 0xc0f530
>
> See the attached "Horizon_Qemu_Run.log" file for more details.
>
> I guess I am not setting the KERNEL_HBC variable properly.
> Could you give me an idea of whats going wrong here ?
>
> Thanks a lot,
> Hamayun
>
>
> On 10/28/2011 02:46 PM, Mian M. Hamayun wrote:
>> Hi James,
>>
>> Thanks A lot. I have downloaded the source code and I am currently
>> looking into it.
>> I will get back to you if I need some clarifications.
>>
>> Thanks Again,
>> --
>> Hamayun
>>
>> On 10/26/2011 03:20 PM, James Molloy wrote:
>>> Hi Mian,
>>>
>>> I have actually done this. I wrote a safe bytecode which compiles down to
>>> LLVM IR, and had baremetal applications running using it.
>>>
>>> Code here:
>>> http://www.quokforge.org/projects/horizon/repository/revisions/master/show/h
>>>
>>> orizon/Baremetal
>>>
>>> My method was to compile for hosted linux, then modify the resulting
>>> code to
>>> run baremetal. For me, this involved rewriting "syscall" instructions to
>>> "int $X" as everything was running in supervisor mode and "syscall" only
>>> works from user mode.
>>>
>>> Apart from that there's just a load of stubs to implement. Pretty easy
>>> really.
>>>
>>> Cheers,
>>>
>>> James
>>>
>>>
>>> -----Original Message-----
>>> From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On
>>> Behalf Of Mian M. Hamayun
>>> Sent: 26 October 2011 14:12
>>> To: llvmdev at cs.uiuc.edu
>>> Subject: [LLVMdev] LLVM JIT on a Baremetal x86 Machine !!!
>>>
>>> Dear All,
>>>
>>> I have tested a few examples of LLVM-JIT Framework on Linux x86 Machine.
>>> So generating functions on the fly and then executing them is OK on
>>> linux i.e. i686-pc-linux-gnu
>>>
>>> My question is:
>>>
>>> Can we use the LLVM-JIT on a baremetal x86 machine ? Actually my target
>>> is a virtual machine, and I need some dynamic code generation support. I
>>> intend to use LLVM-JIT (if possible) for this purpose.
>>>
>>> Furthermore if it is possible, then how much effort would be required ?
>>> And where should I start looking for doing this kind of port.
>>>
>>> Thanks for your help.
>>>
>>> --
>>> Hamayun
>>> Grenoble, France.
>>>
>>>
>>>
>>
> -- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.
>

-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.