[lldb-dev] IOHandler branch has been merged into top of tree
Todd Fiala
tfiala at google.com
Tue Jan 28 16:27:12 PST 2014
Yeah I figured that wouldn't be the final solution. But I'm guessing this
is exposing some kind of bad interaction that is going to (potentially)
happen anywhere that python links with libreadline, which is everywhere
that __APPLE__ isn't defined (so maybe wider than Linux).
> So we still need to figure out what is going wrong on linux with readline.
Right. At this point it might even be a manifestation of how test
input/output is interacting with it. I haven't caught it happening outside
of the test environment yet (although I've only tried a handful of times),
although nothing in the nature of the crash screams test-only since this is
crashing in a python module using readline.
> You could import "platform" in python and and set
"readfunc=readfunc_stdio" for now for non darwin platforms (or anything
that is readline based and causing the problem) just to get things working,
but I am guessing people will really want the auto completion in the
embedded python interpreter...
Ok - I may also try to work it in so that it only disables auto completion
in the test environment. If we started getting bug reports around it
outside of the test environment, we would know this is a bigger issue.
-Todd
On Tue, Jan 28, 2014 at 4:11 PM, Greg Clayton <gclayton at apple.com> wrote:
>
> On Jan 28, 2014, at 3:52 PM, Todd Fiala <tfiala at google.com> wrote:
>
> > I was able to address the TestConvenienceVariables.py break with this
> patch (thanks for some clues on that, Greg!):
> >
> > Index: test/dotest.py
> > ===================================================================
> > --- test/dotest.py (revision 200343)
> > +++ test/dotest.py (working copy)
> > @@ -873,10 +873,13 @@
> > pluginPath = os.path.join(scriptPath, 'plugins')
> > pexpectPath = os.path.join(scriptPath, 'pexpect-2.4')
> >
> > - # Append script dir, plugin dir, and pexpect dir to the sys.path.
> > + # Put embedded pexpect at front of the load path so we ensure we
> > + # use that version.
> > + sys.path.insert(0, pexpectPath)
> > +
> > + # Append script dir and plugin dir to the sys.path.
> > sys.path.append(scriptPath)
> > sys.path.append(pluginPath)
> > - sys.path.append(pexpectPath)
> >
> > # This is our base name component.
> > base = os.path.abspath(os.path.join(scriptPath, os.pardir))
> > Index: source/Interpreter/embedded_interpreter.py
> > ===================================================================
> > --- source/Interpreter/embedded_interpreter.py (revision 200343)
> > +++ source/Interpreter/embedded_interpreter.py (working copy)
> > @@ -86,7 +86,7 @@
> > code.interact(banner="Python Interactive Interpreter.
> To exit, type 'quit()', 'exit()' or Ctrl-D.", readfunc=readfunc_stdio,
> local=local_dict)
> > else:
> > # We have a real interactive terminal
> > - code.interact(banner="Python Interactive Interpreter. To
> exit, type 'quit()', 'exit()' or Ctrl-D.", local=local_dict)
> > + code.interact(banner="Python Interactive Interpreter. To
> exit, type 'quit()', 'exit()' or Ctrl-D.", readfunc=readfunc_stdio,
> local=local_dict)
> > except SystemExit as e:
> > global g_builtin_override_called
> > if not g_builtin_override_called:
> >
>
>
>
> So we can't checkin the code above where we set "readfunc=readfunc_stdio"
> as this disables all auto completion as it won't use readline + rlcompleter
> if this is enabled. This simply identifies that the problem has something
> to do with readline...
>
> So we still need to figure out what is going wrong on linux with readline.
> You could import "platform" in python and and set "readfunc=readfunc_stdio"
> for now for non darwin platforms (or anything that is readline based and
> causing the problem) just to get things working, but I am guessing people
> will really want the auto completion in the embedded python interpreter...
>
>
> > The other 5 still remain.
> >
> > Looking at TestAliases.py:
> >
> > python ../llvm/tools/lldb/test/dotest.py -C gcc --executable
> `pwd`/Debug+Asserts/bin/lldb -p TestAliases.py
> >
> > Yields:
> >
> > Ran 2 tests in 1.613s
> >
> > OK (skipped=1)
> > Session logs for test failures/errors/unexpected successes can be found
> in directory '2014-01-28-15_39_58'
> > Fatal Python error: PyEval_SaveThread: NULL tstate
> > Aborted (core dumped)
> >
> > Python is dumping core at this point, it looks to be in cleanup code
> since the test harness already reported an OK test result. The rest of the
> errors seem to be either identical or nearly identical to the trace below.
> I'll have a look at the cleanup code to see what's wrong here.
> >
> > #0 0x00007f7073b52425 in __GI_raise (sig=<optimized out>) at
> ../nptl/sysdeps/unix/sysv/linux/raise.c:64
> > 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
> > (gdb) bt
> > #0 0x00007f7073b52425 in __GI_raise (sig=<optimized out>) at
> ../nptl/sysdeps/unix/sysv/linux/raise.c:64
> > #1 0x00007f7073b55b8b in __GI_abort () at abort.c:91
> > #2 0x000000000053ed5d in Py_FatalError ()
> > #3 0x00000000004925cf in ?? ()
> > #4 0x000000000047f797 in ?? ()
> > #5 0x00007f706e4f9059 in lldb_private::PythonObject::Reset
> (this=0x2899a20, py_obj=0x0)
> > at
> /mnt/ssd/work/svn/lgs/llvm/tools/lldb/source/Interpreter/../../include/lldb/Interpreter/PythonDataObjects.h:65
> > #6 0x00007f706e4f8fb5 in lldb_private::PythonObject::~PythonObject
> (this=0x2899a20, __in_chrg=<optimized out>)
> > at
> /mnt/ssd/work/svn/lgs/llvm/tools/lldb/source/Interpreter/../../include/lldb/Interpreter/PythonDataObjects.h:51
> > #7 0x00007f706e4fbad0 in
> lldb_private::ScriptInterpreterPython::~ScriptInterpreterPython
> (this=0x28999e0,
> > __in_chrg=<optimized out>)
> > at
> /mnt/ssd/work/svn/lgs/llvm/tools/lldb/source/Interpreter/ScriptInterpreterPython.cpp:205
> > #8 0x00007f706e4fbb34 in
> lldb_private::ScriptInterpreterPython::~ScriptInterpreterPython
> (this=0x28999e0,
> > __in_chrg=<optimized out>)
> > at
> /mnt/ssd/work/svn/lgs/llvm/tools/lldb/source/Interpreter/ScriptInterpreterPython.cpp:207
> > #9 0x00007f706e429754 in
> std::default_delete<lldb_private::ScriptInterpreter>::operator()
> (this=0x256a108,
> > __ptr=0x28999e0) at
> /usr/local/gcc/gcc-4.8.2/include/c++/4.8.2/bits/unique_ptr.h:67
> > #10 0x00007f706e428959 in
> std::unique_ptr<lldb_private::ScriptInterpreter,
> std::default_delete<lldb_private::ScriptInterpreter> >::~unique_ptr
> (this=0x256a108, __in_chrg=<optimized out>)
> > at /usr/local/gcc/gcc-4.8.2/include/c++/4.8.2/bits/unique_ptr.h:184
> > #11 0x00007f706e425b2b in
> lldb_private::CommandInterpreter::~CommandInterpreter (this=0x2569ec0,
> > __in_chrg=<optimized out>)
> > at
> /mnt/ssd/work/svn/lgs/llvm/tools/lldb/source/Interpreter/CommandInterpreter.cpp:2097
> > #12 0x00007f706e425c20 in
> lldb_private::CommandInterpreter::~CommandInterpreter (this=0x2569ec0,
> > __in_chrg=<optimized out>)
> > at
> /mnt/ssd/work/svn/lgs/llvm/tools/lldb/source/Interpreter/CommandInterpreter.cpp:2099
> > #13 0x00007f706e2bd49a in
> std::default_delete<lldb_private::CommandInterpreter>::operator()
> (this=0x2569da8,
> > __ptr=0x2569ec0) at
> /usr/local/gcc/gcc-4.8.2/include/c++/4.8.2/bits/unique_ptr.h:67
> > #14 0x00007f706e2bb9ed in
> std::unique_ptr<lldb_private::CommandInterpreter,
> std::default_delete<lldb_private::CommandInterpreter> >::~unique_ptr
> (this=0x2569da8, __in_chrg=<optimized out>)
> > at /usr/local/gcc/gcc-4.8.2/include/c++/4.8.2/bits/unique_ptr.h:184
> > #15 0x00007f706e2b3580 in lldb_private::Debugger::~Debugger
> (this=0x2569a10, __in_chrg=<optimized out>)
> > at /mnt/ssd/work/svn/lgs/llvm/tools/lldb/source/Core/Debugger.cpp:668
> > #16 0x00007f706e2b368e in lldb_private::Debugger::~Debugger
> (this=0x2569a10, __in_chrg=<optimized out>)
> > at /mnt/ssd/work/svn/lgs/llvm/tools/lldb/source/Core/Debugger.cpp:671
> > #17 0x00007f706e2c1cba in std::_Sp_counted_ptr<lldb_private::Debugger*,
> (__gnu_cxx::_Lock_policy)2>::_M_dispose
> > (this=0x28727f0) at
> /usr/local/gcc/gcc-4.8.2/include/c++/4.8.2/bits/shared_ptr_base.h:290
> > #18 0x00007f706e198334 in
> std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release
> (this=0x28727f0)
> > at
> /usr/local/gcc/gcc-4.8.2/include/c++/4.8.2/bits/shared_ptr_base.h:144
> > #19 0x00007f706e197de7 in
> std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count
> (this=0x2939e78,
> > __in_chrg=<optimized out>) at
> /usr/local/gcc/gcc-4.8.2/include/c++/4.8.2/bits/shared_ptr_base.h:546
> > #20 0x00007f706e19dd7c in std::__shared_ptr<lldb_private::Debugger,
> (__gnu_cxx::_Lock_policy)2>::~__shared_ptr
> > (this=0x2939e70, __in_chrg=<optimized out>)
> > at
> /usr/local/gcc/gcc-4.8.2/include/c++/4.8.2/bits/shared_ptr_base.h:781
> > #21 0x00007f706e19dd96 in
> std::shared_ptr<lldb_private::Debugger>::~shared_ptr (this=0x2939e70,
> > __in_chrg=<optimized out>) at
> /usr/local/gcc/gcc-4.8.2/include/c++/4.8.2/bits/shared_ptr.h:93
> > #22 0x00007f706e1a7da6 in lldb::SBDebugger::~SBDebugger (this=0x2939e70,
> __in_chrg=<optimized out>)
> > at
> /mnt/ssd/work/svn/lgs/llvm/tools/lldb/source/API/SBDebugger.cpp:247
> > ---Type <return> to continue, or q <return> to quit---
> > #23 0x00007f706e4543df in _wrap_delete_SBDebugger (args=0x24df4d0) at
> LLDBWrapPython.cpp:14646
> > #24 0x00000000004f7496 in PyObject_Call ()
> > #25 0x00000000004f87cb in PyObject_CallFunctionObjArgs ()
> > #26 0x00007f706e4337bd in SwigPyObject_dealloc (v=0x2a7a5d0) at
> LLDBWrapPython.cpp:1697
> > #27 0x000000000054878d in ?? ()
> > #28 0x000000000045d198 in ?? ()
> > #29 0x0000000000555d65 in ?? ()
> > #30 0x00000000004bef09 in PyDict_SetItem ()
> > #31 0x000000000044c9b9 in _PyModule_Clear ()
> > #32 0x00000000004369c8 in PyImport_Cleanup ()
> > #33 0x000000000047ec4c in Py_Finalize ()
> > #34 0x000000000047f148 in Py_Exit ()
> > #35 0x000000000047f27c in ?? ()
> > #36 0x0000000000424a8c in PyRun_SimpleFileExFlags ()
> > #37 0x0000000000425cb6 in Py_Main ()
> > #38 0x00007f7073b3d76d in __libc_start_main (main=0x41bb00 <main>,
> argc=8, ubp_av=0x7fffced281e8,
> > init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized
> out>, stack_end=0x7fffced281d8)
> > at libc-start.c:226
> > #39 0x000000000041bb31 in _start ()
> >
> > I'll track this one down and will get a fix in for all of them
> (including the patch above) in one shot.
> >
> > -Todd
> >
> >
> >
> > On Tue, Jan 28, 2014 at 1:42 PM, Todd Fiala <tfiala at google.com> wrote:
> > Digging through the python readline.c code, the libedit support appears
> to be hardcoded to working with __APPLE__ #ifdefs --- I am not seeing an
> easy way to build python with libedit support under Linux.
> >
> > Any ideas here? It's possible our issue is not so much a
> libedit/libreadline issue and more just faulty assumptions (possibly in
> Python's readline.so) about atomic operations that are causing the crash.
> I was just trying to get both Python and LLDB to be using the same one.
> >
> > -Todd
> >
> >
> > On Tue, Jan 28, 2014 at 11:33 AM, Todd Fiala <tfiala at google.com> wrote:
> > Building with libreadline-dev on Ubuntu 12.04 LTS x86_64 isn't going to
> work. We're using editline-specific functionality.
> >
> > I'm going to try the other approach now, which is building and using a
> python that is built against libedit.
> >
> >
> > On Tue, Jan 28, 2014 at 11:02 AM, Todd Fiala <tfiala at google.com> wrote:
> > Here's what I'm finding on our setup:
> >
> > >> FAIL: LLDB (suite) :: TestConvenienceVariables.py (
> Linuxtfiala2.mtv.corp.google.com 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18
> PDT 2013 x86_64 x86_64)
> >
> > This test is failing most of the time. There appears to be some kind of
> race on the readline/libedit functionality. The embedded python that is
> being tested by the script is using the system readline library, which is
> different than the new libedit that we're using in the lldb executable.
> The embedded python is core dumping (seg fault) around reading a string.
> I'm getting a back trace like so from the lldb test process that seg
> faulted:
> >
> > #0 0x00007f8749507e87 in ?? () from
> /usr/lib/python2.7/lib-dynload/readline.so
> > #1 0x00007f874b6c3610 in PyOS_Readline () from
> /usr/lib/libpython2.7.so.1.0
> > #2 0x00007f874b640fad in ?? () from /usr/lib/libpython2.7.so.1.0
> > #3 0x00007f874b60c5d5 in PyEval_EvalFrameEx () from
> /usr/lib/libpython2.7.so.1.0
> > #4 0x00007f874b5cc6b5 in PyEval_EvalCodeEx () from
> /usr/lib/libpython2.7.so.1.0
> > #5 0x00007f874b60c650 in PyEval_EvalFrameEx () from
> /usr/lib/libpython2.7.so.1.0
> > #6 0x00007f874b5cc6b5 in PyEval_EvalCodeEx () from
> /usr/lib/libpython2.7.so.1.0
> > #7 0x00007f874b60c650 in PyEval_EvalFrameEx () from
> /usr/lib/libpython2.7.so.1.0
> > #8 0x00007f874b5cc6b5 in PyEval_EvalCodeEx () from
> /usr/lib/libpython2.7.so.1.0
> > #9 0x00007f874b60c650 in PyEval_EvalFrameEx () from
> /usr/lib/libpython2.7.so.1.0
> > #10 0x00007f874b60d37b in PyEval_EvalFrameEx () from
> /usr/lib/libpython2.7.so.1.0
> > #11 0x00007f874b5cc6b5 in PyEval_EvalCodeEx () from
> /usr/lib/libpython2.7.so.1.0
> > #12 0x00007f874b5cc9e2 in PyEval_EvalCode () from
> /usr/lib/libpython2.7.so.1.0
> > #13 0x00007f874b5cca7c in PyRun_StringFlags () from
> /usr/lib/libpython2.7.so.1.0
> > #14 0x00007f874b5cd6cb in PyRun_SimpleStringFlags () from
> /usr/lib/libpython2.7.so.1.0
> > #15 0x00007f874e71cbf0 in IOHandlerPythonInterpreter::Run
> (this=0x19e79c0)
> > at
> /mnt/ssd/work/svn/lgs/llvm/tools/lldb/source/Interpreter/ScriptInterpreterPython.cpp:747
> > #16 0x00007f874e4d36bc in lldb_private::Debugger::ExecuteIOHanders
> (this=0x17cc400)
> > at /mnt/ssd/work/svn/lgs/llvm/tools/lldb/source/Core/Debugger.cpp:865
> > #17 0x00007f874e647c16 in
> lldb_private::CommandInterpreter::RunCommandInterpreter (this=0x17e03f0,
> > auto_handle_events=true, spawn_thread=false)
> > at
> /mnt/ssd/work/svn/lgs/llvm/tools/lldb/source/Interpreter/CommandInterpreter.cpp:3002
> > #18 0x00007f874e3c93f5 in lldb::SBDebugger::RunCommandInterpreter
> (this=0x7fff9fb8c2a0,
> > auto_handle_events=true, spawn_thread=false)
> > at /mnt/ssd/work/svn/lgs/llvm/tools/lldb/source/API/SBDebugger.cpp:961
> > #19 0x000000000040f5ae in Driver::MainLoop (this=0x7fff9fb8c280)
> > at /mnt/ssd/work/svn/lgs/llvm/tools/lldb/tools/driver/Driver.cpp:946
> > #20 0x000000000040f8f4 in main (argc=3, argv=0x7fff9fb8c468,
> envp=0x7fff9fb8c488)
> > at /mnt/ssd/work/svn/lgs/llvm/tools/lldb/tools/driver/Driver.cpp:1039
> >
> > On Ubuntu 12.04 LTS x86_64, the
> /usr/lib/python2.7/lib-dynload/readline.so links against GNU readline:
> >
> > tfiala at tfiala2:/usr/lib/python2.7/lib-dynload$ ldd readline.so
> > linux-vdso.so.1 => (0x00007fff137f1000)
> > libreadline.so.6 => /lib/x86_64-linux-gnu/libreadline.so.6
> (0x00007fdd7d66c000)
> > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
> (0x00007fdd7d44f000)
> > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fdd7d08e000)
> > libgcc_s.so.1 => /usr/local/gcc/gcc-current/lib64/libgcc_s.so.1
> (0x00007fdd7ce78000)
> > libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5
> (0x00007fdd7cc51000)
> > /lib64/ld-linux-x86-64.so.2 (0x00007fdd7dad6000)
> >
> > That makes me think that the embedded python interpreter is likely
> fighting with libedit.so in some way. My lldb is built with the latest
> config-enabled libedit, as we found the Ubuntu 12.04 LTS libedit-dev
> package was too old for some of the newer iohandler features:
> >
> > tfiala at tfiala2:~/lldb/svn/lgs/build/Debug+Asserts/bin$ ldd lldb
> > linux-vdso.so.1 => (0x00007fff8c579000)
> > libedit.so.0 =>
> /usr/local/google/home/tfiala/lldb/tools/libedit/linux-x86_64/lib/libedit.so.0
> (0x00007f00fdac3000)
> > liblldb.so =>
> /mnt/ssd/work/svn/lgs/build/Debug+Asserts/bin/./../lib/liblldb.so
> (0x00007f00f87c3000)
> > libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f00f858d000)
> > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
> (0x00007f00f8370000)
> > libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5
> (0x00007f00f8148000)
> > librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f00f7f40000)
> > libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f00f7d3c000)
> > libstdc++.so.6 => /usr/local/gcc/gcc-current/lib64/libstdc++.so.6
> (0x00007f00f7a38000)
> > libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f00f773c000)
> > libgcc_s.so.1 => /usr/local/gcc/gcc-current/lib64/libgcc_s.so.1
> (0x00007f00f7526000)
> > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f00f7165000)
> > libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1
> (0x00007f00f6f62000)
> > libpython2.7.so.1.0 => /usr/lib/libpython2.7.so.1.0
> (0x00007f00f6a65000)
> > libncurses.so.5 => /lib/x86_64-linux-gnu/libncurses.so.5
> (0x00007f00f6843000)
> > libpanel.so.5 => /usr/lib/x86_64-linux-gnu/libpanel.so.5
> (0x00007f00f663f000)
> > /lib64/ld-linux-x86-64.so.2 (0x00007f00fdcf6000)
> > libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0
> (0x00007f00f63e0000)
> > libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
> (0x00007f00f6005000)
> >
> > I'm not yet sure if this is an inherent issue with two pieces of low
> level code (libedit & readline) both thinking they have sole access to the
> input/output file descriptors (which may require them both using readline
> or libedit, but not both), or just a bug in assumptions on our new
> iohandler input handling. I'm going to see if I can get lldb to build with
> readline first and see if that changes the behavior. It may convert a seg
> fault into a failing test, which may be easier to diagnose, or it might
> outright solve the problem. Back after I check out the readline route.
> >
> >
> > On Mon, Jan 27, 2014 at 4:38 PM, Todd Fiala <tfiala at google.com> wrote:
> > FYI - these are the tests that are failing for me on Ubuntu 12.04 x86_64
> with top of tree after the iohandler merge sync:
> >
> > Failing Tests (6)
> > FAIL: LLDB (suite) :: TestAliases.py (Linux tfiala2.mtv.corp.google.com3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT 2013 x86_64 x86_64)
> > FAIL: LLDB (suite) :: TestCommandScript.py (Linux
> tfiala2.mtv.corp.google.com 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT
> 2013 x86_64 x86_64)
> > FAIL: LLDB (suite) :: TestConvenienceVariables.py (Linux
> tfiala2.mtv.corp.google.com 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT
> 2013 x86_64 x86_64)
> > FAIL: LLDB (suite) :: TestConditionalBreak.py (Linux
> tfiala2.mtv.corp.google.com 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT
> 2013 x86_64 x86_64)
> > FAIL: LLDB (suite) :: TestDataFormatterCategories.py (Linux
> tfiala2.mtv.corp.google.com 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT
> 2013 x86_64 x86_64)
> > FAIL: LLDB (suite) :: TestDataFormatterGlobals.py (Linux
> tfiala2.mtv.corp.google.com 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT
> 2013 x86_64 x86_64)
> >
> > I'll start looking at those tomorrow if somebody doesn't beat me to them.
> >
> >
> > On Mon, Jan 27, 2014 at 3:50 PM, Greg Clayton <gclayton at apple.com>
> wrote:
> > We will need everyone to make sure their buildbots are ok and get any
> issues fixed. This is the one and only time the input/output and error
> streams will be changed in this drastic of a way. The benefits are many:
> >
> > 1 - Input/Output/Error streams are now handled as real streams not a
> push style input
> > 2 - auto completion in python embedded interpreter
> > 3 - multi-line input for "script" and "expression" commands now allow
> you to edit previous/next lines using up and down arrow keys and this makes
> multi-line input actually a viable thing to use
> > 4 - it is now possible to use curses to drive LLDB (please try the "gui"
> command)
> >
> > We will need to deal with and fix any buildbot failures and tests and
> arise now that input/output and error are correctly hooked up in all cases.
> >
> > Greg Clayton
> >
> > _______________________________________________
> > lldb-dev mailing list
> > lldb-dev at cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
> >
> >
> >
> > --
> > Todd Fiala | Software Engineer | tfiala at google.com |
> 650-943-3180
> >
> >
> >
> >
> > --
> > Todd Fiala | Software Engineer | tfiala at google.com |
> 650-943-3180
> >
> >
> >
> >
> > --
> > Todd Fiala | Software Engineer | tfiala at google.com |
> 650-943-3180
> >
> >
> >
> >
> > --
> > Todd Fiala | Software Engineer | tfiala at google.com |
> 650-943-3180
> >
> >
> >
> >
> > --
> > Todd Fiala | Software Engineer | tfiala at google.com |
> 650-943-3180
> >
>
>
--
Todd Fiala | Software Engineer | tfiala at google.com | 650-943-3180
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20140128/7649422a/attachment.html>
More information about the lldb-dev
mailing list