Thanks, Evan. We've finally got a working version of python.bc which runs most of non-multithreaded python scripts (those that do not use the 'threadmodule') with lli. It needed a few more workarounds (apart from the one to make sure that functions whose addresses are taken are compiled before their addresses are taken):<br>
<br>(a) When any shared library (.so) is loaded using dlopen() by the program currently being JIT'ed by lli, if the initialization code in the library makes a call back to a function in the main process, the program crashes. The statically compiled code (llc + gcc) works however using the -disable-internalize and -rdynamic flags (to llc and gcc respectively). The -rdynamic flag makes sure that the function being called (in case of python, a single function Py_InitModule4() is always called by any module loaded as a .so in response to 'import' statements in python) is exported to the dynamic symbol table of the main process. I do not know a way to achieve this using lli. I changed python code to statically link all the modules that are required for some the scripts that we are running. <br>
<br>(b) Since the LLVM JIT does not support inline asm, I had to redefine some _byteswap() macros in C rather than inline asm. <br><br>With these changes, python.bc runs smoothly with lli for most of the python scripts that I have tested with :).<br>
<br>regards,<br>Prakash<br><br><div class="gmail_quote">On Tue, Nov 11, 2008 at 11:40 AM, Evan Cheng <span dir="ltr"><<a href="mailto:evan.cheng@apple.com">evan.cheng@apple.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div style="">I've filed PR3043 for this.<div><br></div><font color="#888888"><div>Evan</div></font><div><div></div><div class="Wj3C7c"><div><br><div><div>On Nov 3, 2008, at 4:00 PM, Prakash Prabhu wrote:</div><br><blockquote type="cite">
Hi Evan,<br><br>Thanks for the pointers. We found a simple test case that causes the problem (thanks to Tom in my group):<br><br>#include<stdio.h><br>#include<stdlib.h><br><br>void test();<br>void (*funcPtr)();<br>
<br>int main(int argc, char **argv) {<br> funcPtr = test;<br> test();<br>}<br><br>void test() {<br> if(funcPtr == test) {<br> printf("OK!\n");<br> } else {<br> fprintf(stderr, "Bad!\n");<br>
exit(1);<br> }<br>}<br><br>$ llvm-gcc -emit-llvm -o FPtrEqTest.bc -c FPtrEqTest.c<br>$ llc -f FPtrEqTest.bc<br>$ gcc -o FPtrEqTest FPtrEqTest.s<br>$ ./FPtrEqTest<br>OK!<br><br>$ lli FPtrEqTest.bc<br>Bad!<br><br>The above test case is just a smaller version of the one in Python's subtype_traverse which also tests a function pointer and calls itself. It seems the problem arises due comparison with the stub's address when a comparison with the actual address of the compiled function is intended. <br>
<br>thanks,<br>Prakash<br><br><div class="gmail_quote">On Mon, Nov 3, 2008 at 2:07 AM, Evan Cheng <span dir="ltr"><<a href="mailto:evan.cheng@apple.com" target="_blank">evan.cheng@apple.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>Hi Prakash,<div><br></div><div>Unfortunately it looks like you need to do quite a bit of investigation into this. However, I hope I can provide some useful tips. </div><div><br></div><div>1. In general, lli and llc generate exact the same code except lli default to static codegen while llc defaults to dynamic-no-pic codegen. So try passing -relocation-model=dynamic-no-pic to lli. If this works, that means there are issues with static codegen.</div>
<div>2. It could be a JIT encoding bug. If you can identify a problematic function, it's possible examine the generated code in gdb and compare it with llc generated assembly.</div><div>3. It could be a bug in the app and it's exposed when running under the JIT. You can try enabling additional debugging output.</div>
<div><br></div><div>Hope this helps.</div><div><br></div><font color="#888888"><div>Evan</div></font><div><div></div><div><div><br><div><div>On Nov 2, 2008, at 2:55 PM, Prakash Prabhu wrote:</div><br><blockquote type="cite">
Hi Eli,<br><br>Thanks for the reply. I tried with -Xlinker="-ldl ". However it does not seem to make a difference. It seems that when bugpoint is run with --run-jit, the linker args are not passed to gcc (from tools/bugpoint/ExecutionDriver.cpp) :<br>
<br>if (InterpreterSel == RunLLC || InterpreterSel == RunCBE ||<br> InterpreterSel == CBE_bug || InterpreterSel == LLC_Safe)<br> <br> RetVal = AI->ExecuteProgram(BitcodeFile, InputArgv, InputFile,<br> OutputFile, AdditionalLinkerArgs, SharedObjs,<br>
Timeout, MemoryLimit);<br> <br> else<br> <br> <br> RetVal = AI->ExecuteProgram(BitcodeFile, InputArgv, InputFile,<br> OutputFile, std::vector<std::string>(),<br>
SharedObjs, Timeout, MemoryLimit);<br> <br><br>I tried the following after this:<br><br>(1) Firstly instead of running Gap (<a href="http://www.gap-system.org/Download/UNIXInst.html" target="_blank">http://www.gap-system.org/Download/UNIXInst.html</a>), I am now trying to run python with lli (<a href="http://www.python.org/download/releases/2.5.2/" target="_blank">http://www.python.org/download/releases/2.5.2/</a>). I managed to compile python.bc and here again I face the same problem:<br>
<br>llc and gcc can get python.exe to run (which is great :)!) :<br><br>$ llc -f python.bc<br>$ gcc -o python.exe python.s -ldl -lutil -lm -lrt<br>$ ./python.exe<br>Python 2.5.2 (r252:60911, Oct 31 2008, 14:41:11)<br>[GCC 4.2.1 (Based on Apple Inc. build 5623) (LLVM build)] on linux2<br>
Type "help", "copyright", "credits" or "license" for more information.<br>>>><br><br>however, when I try to run python.bc using lli it crashes with a segmentation fault:<br>
<br>$ lli -load=/usr/lib/libdl.so -load=/usr/lib/libutil.so -load=/usr/lib/libm.so -load=/usr/lib/librt.so python.bc<br><br>When i try it with gdb, it seems that the crash is somewhere inside python code (since bt only shows ?? ). Before the crash, I could see that the memory consumption(VM) reaches somewhere near 80% of my 2GB RAM (seen via top, and that too a sudden increase from around when it was previously occupying around 2-3% of VM). I tried to run this on a 64-bit machine which has 8GB RAM and still have the same issue wrt memory. <br>
<br>(2) Finally I wrote a pass (and loaded it through opt) to instrument each function's (in python code ) entry and exit and then ran the instrumented program with both [llc ; gcc] combination and lli. In the lli version a single method (subtype_traverse) is recursively called (about 2 million times) until the program runs out of memory while the statically compiled code (llc + gcc) calls this method (I am comparing the calls in the same context in both cases) only once:<br>
<br>python with llc + gcc :<br>...<br>(tupletraverse (visit_decref (type_is_gc))) <br>(subtype_traverse (visit_decref (type_is_gc))<br>(type_traverse (visit_decref) (visit_decref) ...<br><br>python with lli:<br><br>(tupletraverse (visit_decref (type_is_gc)))<br>
(subtype_traverse (visit_decref (type_is_gc)) (subtype_traverse (visit_decref (type_is_gc))(subtype_traverse (visit_decref (type_is_gc)) ..... about 2 million times<br><br>Looking at the code (Objects/typeobject.c: <a href="http://google.com/codesearch?hl=en&q=show:VK_wUSuAZto:jHKC99mjNVM:4z02hQcYQRY&sa=N&ct=rd&cs_p=http://gentoo.osuosl.org/distfiles/Python-2.5.tar.bz2&cs_f=Python-2.5/Objects/typeobject.c" target="_blank">http://google.com/codesearch?hl=en&q=show:VK_wUSuAZto:jHKC99mjNVM:4z02hQcYQRY&sa=N&ct=rd&cs_p=http://gentoo.osuosl.org/distfiles/Python-2.5.tar.bz2&cs_f=Python-2.5/Objects/typeobject.c</a>)<br>
<br> it seems the last call (through a function pointer) in subtype_traverse results in this never-ending recursive call.<br><br>Has anyone tried compiling python to bit code and running it the LLVM JIT before ? <br><br>
Thanks for your time. <br><br>- Prakash<br><br><br><br><br><br><div class="gmail_quote">On Tue, Oct 28, 2008 at 3:02 PM, Eli Friedman <span dir="ltr"><<a href="mailto:eli.friedman@gmail.com" target="_blank">eli.friedman@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div>On Tue, Oct 28, 2008 at 12:17 PM, Prakash Prabhu<br> <<a href="mailto:prakash.prabhu@gmail.com" target="_blank">prakash.prabhu@gmail.com</a>> wrote:<br>
> Generating reference output from raw program: <cbe><gcc><br> > Error running tool:<br> </div>[snip]<br> <div>> /tmp/cc08IpX8.o: In function `SyLoadModule':<br> > bugpoint-test-program.bc.cbe.c:(.text+0x25705): undefined reference to<br>
> `dlopen'<br> </div>[snip]<br> <br> This is saying that compilation with CBE is failing. Try something<br> like -Xlinker -ldl?<br> <br> -Eli<br> _______________________________________________<br> LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu" target="_blank">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br> <a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
</blockquote></div><br> _______________________________________________<br>LLVM Developers mailing list<br><a href="mailto:LLVMdev@cs.uiuc.edu" target="_blank">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br></blockquote></div><br></div></div></div></div><br>_______________________________________________<br>
LLVM Developers mailing list<br> <a href="mailto:LLVMdev@cs.uiuc.edu" target="_blank">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br> <a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
<br></blockquote></div><br> _______________________________________________<br>LLVM Developers mailing list<br><a href="mailto:LLVMdev@cs.uiuc.edu" target="_blank">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br></blockquote></div><br></div></div></div></div><br>_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
<br></blockquote></div><br>