<div dir="ltr">Here's what I'm finding on our setup:<div><br></div><div><div style="font-family:arial,sans-serif;font-size:13px">>> FAIL: LLDB (suite) :: TestConvenienceVariables.py (Linux<a href="http://tfiala2.mtv.corp.google.com/" target="_blank">tfiala2.mtv.corp.google.com</a> 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT 2013 x86_64 x86_64)</div>
</div><div><br></div><div>This test is failing most of the time. There appears to be some kind of race on the readline/libedit functionality. The embedded python that is being tested by the script is using the system readline library, which is different than the new libedit that we're using in the lldb executable. The embedded python is core dumping (seg fault) around reading a string. I'm getting a back trace like so from the lldb test process that seg faulted:</div>
<div><br></div><div><font face="courier new, monospace" size="1"><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#0 0x00007f8749507e87 in ?? () from /usr/lib/python2.7/lib-dynload</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">/readline.so</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#1 0x00007f874b6c3610 in PyOS_Readline () from /usr/lib/libpython2.7.so.1.0</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)"><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#2 0x00007f874b640fad in ?? () from /usr/lib/libpython2.7.so.1.0</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#3 0x00007f874b60c5d5 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)"><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#4 0x00007f874b5cc6b5 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#5 0x00007f874b60c650 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)"><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#6 0x00007f874b5cc6b5 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#7 0x00007f874b60c650 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)"><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#8 0x00007f874b5cc6b5 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#9 0x00007f874b60c650 in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)"><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#10 0x00007f874b60d37b in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#11 0x00007f874b5cc6b5 in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)"><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#12 0x00007f874b5cc9e2 in PyEval_EvalCode () from /usr/lib/libpython2.7.so.1.0</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#13 0x00007f874b5cca7c in PyRun_StringFlags () from /usr/lib/libpython2.7.so.1.0</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)"><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#14 0x00007f874b5cd6cb in PyRun_SimpleStringFlags () from /usr/lib/libpython2.7.so.1.0</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#15 0x00007f874e71cbf0 in IOHandlerPythonInterpreter:</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">:Run (this=0x19e79c0)</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)"> at /mnt/ssd/work/svn/lgs/llvm</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">/tools/lldb/source/Interpreter</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">/ScriptInterpreterPython.cpp</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">:747</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#16 0x00007f874e4d36bc in lldb_private::Debugger:</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">:ExecuteIOHanders (this=0x17cc400)</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)"> at /mnt/ssd/work/svn/lgs/llvm</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">/tools/lldb/source/Core</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">/Debugger.cpp:865</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#17 0x00007f874e647c16 in lldb_private::CommandInterpret</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">er::RunCommandInterpreter (this=0x17e03f0,</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)"> auto_handle_events=true, spawn_thread=false)</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)"><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)"> at /mnt/ssd/work/svn/lgs/llvm</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">/tools/lldb/source/Interpreter</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">/CommandInterpreter.cpp:3002</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#18 0x00007f874e3c93f5 in lldb::SBDebugger::RunCommandIn</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">terpreter (this=0x7fff9fb8c2a0,</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)"> auto_handle_events=true, spawn_thread=false)</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)"><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)"> at /mnt/ssd/work/svn/lgs/llvm</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">/tools/lldb/source/API/SBDebug</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">ger.cpp:961</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#19 0x000000000040f5ae in Driver::MainLoop (this=0x7fff9fb8c280)</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)"><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)"> at /mnt/ssd/work/svn/lgs/llvm</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">/tools/lldb/tools/driver</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">/Driver.cpp:946</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)">
<span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">#20 0x000000000040f8f4 in main (argc=3, argv=0x7fff9fb8c468, envp=0x7fff9fb8c488)</span><br style="color:rgb(0,0,0);background-color:rgb(233,246,255)"><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)"> at /mnt/ssd/work/svn/lgs/llvm</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">/tools/lldb/tools/driver</span><span style="color:rgb(0,0,0);background-color:rgb(233,246,255)">/Driver.cpp:1039</span></font><br>
</div><div><br></div><div>On Ubuntu 12.04 LTS x86_64, the /usr/lib/python2.7/lib-dynload/readline.so links against GNU readline:</div><div><br></div><div><div><font size="1" face="courier new, monospace">tfiala@tfiala2:/usr/lib/python2.7/lib-dynload$ ldd readline.so</font></div>
<div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>linux-vdso.so.1 => (0x00007fff137f1000)</font></div><div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span><b>libreadline.so.6 => /lib/x86_64-linux-gnu/libreadline.so.6 (0x00007fdd7d66c000)</b></font></div>
<div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fdd7d44f000)</font></div><div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fdd7d08e000)</font></div>
<div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libgcc_s.so.1 => /usr/local/gcc/gcc-current/lib64/libgcc_s.so.1 (0x00007fdd7ce78000)</font></div><div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007fdd7cc51000)</font></div>
<div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>/lib64/ld-linux-x86-64.so.2 (0x00007fdd7dad6000)</font></div></div><div><font size="1" face="courier new, monospace"><br></font></div>
<div>That makes me think that the embedded python interpreter is likely fighting with libedit.so in some way. My lldb is built with the latest config-enabled <a href="http://thrysoee.dk/editline/">libedit</a>, as we found the Ubuntu 12.04 LTS libedit-dev package was too old for some of the newer iohandler features:</div>
<div><br></div><div><div><font size="1" face="courier new, monospace">tfiala@tfiala2:~/lldb/svn/lgs/build/Debug+Asserts/bin$ ldd lldb</font></div><div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>linux-vdso.so.1 => (0x00007fff8c579000)</font></div>
<div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span><b>libedit.so.0 => /usr/local/google/home/tfiala/lldb/tools/libedit/linux-x86_64/lib/libedit.so.0 (0x00007f00fdac3000)</b></font></div>
<div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>liblldb.so => /mnt/ssd/work/svn/lgs/build/Debug+Asserts/bin/./../lib/liblldb.so (0x00007f00f87c3000)</font></div><div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f00f858d000)</font></div>
<div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f00f8370000)</font></div><div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f00f8148000)</font></div>
<div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f00f7f40000)</font></div><div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f00f7d3c000)</font></div>
<div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libstdc++.so.6 => /usr/local/gcc/gcc-current/lib64/libstdc++.so.6 (0x00007f00f7a38000)</font></div><div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f00f773c000)</font></div>
<div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libgcc_s.so.1 => /usr/local/gcc/gcc-current/lib64/libgcc_s.so.1 (0x00007f00f7526000)</font></div><div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f00f7165000)</font></div>
<div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f00f6f62000)</font></div><div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libpython2.7.so.1.0 => /usr/lib/libpython2.7.so.1.0 (0x00007f00f6a65000)</font></div>
<div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libncurses.so.5 => /lib/x86_64-linux-gnu/libncurses.so.5 (0x00007f00f6843000)</font></div><div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libpanel.so.5 => /usr/lib/x86_64-linux-gnu/libpanel.so.5 (0x00007f00f663f000)</font></div>
<div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>/lib64/ld-linux-x86-64.so.2 (0x00007f00fdcf6000)</font></div><div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f00f63e0000)</font></div>
<div><font size="1" face="courier new, monospace"><span class="" style="white-space:pre"> </span>libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f00f6005000)</font></div></div><div><font size="1" face="courier new, monospace"><br>
</font></div><div><font face="arial, helvetica, sans-serif">I'm not yet sure if this is an inherent issue with two pieces of low level code (libedit & readline) both thinking they have sole access to the input/output file descriptors (which may require them both using readline or libedit, but not both), or just a bug in assumptions on our new iohandler input handling. I'm going to see if I can get lldb to build with readline first and see if that changes the behavior. It may convert a seg fault into a failing test, which may be easier to diagnose, or it might outright solve the problem. Back after I check out the readline route.</font></div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Jan 27, 2014 at 4:38 PM, Todd Fiala <span dir="ltr"><<a href="mailto:tfiala@google.com" target="_blank">tfiala@google.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">FYI - these are the tests that are failing for me on Ubuntu 12.04 x86_64 with top of tree after the iohandler merge sync:<div>
<br></div><div><div>Failing Tests (6)</div><div>FAIL: LLDB (suite) :: TestAliases.py (Linux <a href="http://tfiala2.mtv.corp.google.com" target="_blank">tfiala2.mtv.corp.google.com</a> 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT 2013 x86_64 x86_64)</div>
<div>FAIL: LLDB (suite) :: TestCommandScript.py (Linux <a href="http://tfiala2.mtv.corp.google.com" target="_blank">tfiala2.mtv.corp.google.com</a> 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT 2013 x86_64 x86_64)</div><div>
FAIL: LLDB (suite) :: TestConvenienceVariables.py (Linux <a href="http://tfiala2.mtv.corp.google.com" target="_blank">tfiala2.mtv.corp.google.com</a> 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT 2013 x86_64 x86_64)</div>
<div>FAIL: LLDB (suite) :: TestConditionalBreak.py (Linux <a href="http://tfiala2.mtv.corp.google.com" target="_blank">tfiala2.mtv.corp.google.com</a> 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT 2013 x86_64 x86_64)</div>
<div>FAIL: LLDB (suite) :: TestDataFormatterCategories.py (Linux <a href="http://tfiala2.mtv.corp.google.com" target="_blank">tfiala2.mtv.corp.google.com</a> 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT 2013 x86_64 x86_64)</div>
<div>FAIL: LLDB (suite) :: TestDataFormatterGlobals.py (Linux <a href="http://tfiala2.mtv.corp.google.com" target="_blank">tfiala2.mtv.corp.google.com</a> 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT 2013 x86_64 x86_64)</div>
</div><div><br>
</div><div>I'll start looking at those tomorrow if somebody doesn't beat me to them.</div></div><div class="gmail_extra"><div><div class="h5"><br><br><div class="gmail_quote">On Mon, Jan 27, 2014 at 3:50 PM, Greg Clayton <span dir="ltr"><<a href="mailto:gclayton@apple.com" target="_blank">gclayton@apple.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">We will need everyone to make sure their buildbots are ok and get any issues fixed. This is the one and only time the input/output and error streams will be changed in this drastic of a way. The benefits are many:<br>
<br>
1 - Input/Output/Error streams are now handled as real streams not a push style input<br>
2 - auto completion in python embedded interpreter<br>
3 - multi-line input for "script" and "expression" commands now allow you to edit previous/next lines using up and down arrow keys and this makes multi-line input actually a viable thing to use<br>
4 - it is now possible to use curses to drive LLDB (please try the "gui" command)<br>
<br>
We will need to deal with and fix any buildbot failures and tests and arise now that input/output and error are correctly hooked up in all cases.<br>
<br>
Greg Clayton<br>
<br>
_______________________________________________<br>
lldb-dev mailing list<br>
<a href="mailto:lldb-dev@cs.uiuc.edu" target="_blank">lldb-dev@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev</a><br>
</blockquote></div><br><br clear="all"><div><br></div></div></div><span class="HOEnZb"><font color="#888888">-- <br><div dir="ltr"><table cellspacing="0" cellpadding="0" style="color:rgb(136,136,136);font-family:'Times New Roman'">
<tbody><tr style="color:rgb(85,85,85);font-family:sans-serif;font-size:small">
<td nowrap style="border-top-style:solid;border-top-color:rgb(213,15,37);border-top-width:2px">Todd Fiala |</td><td nowrap style="border-top-style:solid;border-top-color:rgb(51,105,232);border-top-width:2px"> Software Engineer |</td>
<td nowrap style="border-top-style:solid;border-top-color:rgb(0,153,57);border-top-width:2px"> <a href="mailto:tfiala@google.com" style="color:rgb(17,85,204)" target="_blank"><span style="background-color:rgb(255,255,204);color:rgb(34,34,34);background-repeat:initial initial">tfiala@google.com</span></a> |</td>
<td nowrap style="border-top-style:solid;border-top-color:rgb(238,178,17);border-top-width:2px"><font color="#1155cc"> <a>650-943-3180</a></font></td></tr></tbody></table><br></div>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><table cellspacing="0" cellpadding="0" style="color:rgb(136,136,136);font-family:'Times New Roman'"><tbody><tr style="color:rgb(85,85,85);font-family:sans-serif;font-size:small">
<td nowrap style="border-top-style:solid;border-top-color:rgb(213,15,37);border-top-width:2px">Todd Fiala |</td><td nowrap style="border-top-style:solid;border-top-color:rgb(51,105,232);border-top-width:2px"> Software Engineer |</td>
<td nowrap style="border-top-style:solid;border-top-color:rgb(0,153,57);border-top-width:2px"> <a href="mailto:tfiala@google.com" style="color:rgb(17,85,204)" target="_blank"><span style="background-color:rgb(255,255,204);color:rgb(34,34,34);background-repeat:initial initial">tfiala@google.com</span></a> |</td>
<td nowrap style="border-top-style:solid;border-top-color:rgb(238,178,17);border-top-width:2px"><font color="#1155cc"> <a>650-943-3180</a></font></td></tr></tbody></table><br></div>
</div>