[lldb-dev] How to redirect stdin/out/err to different pty?

Eran Ifrah eran.ifrah at gmail.com
Sat Mar 29 09:24:03 PDT 2014


Hi,

Testing further seems like that it does fixes the redirection thing when
using lldb as a library.
However, the more common case of using lldb from the command line fails
miserably :(

There are 2 options here that I have looked into:
- Use the above hack with some more conditions that will ensure that the
hack will only kicks when redirection is actually requested (e.g. the
terminal is different from the current process terminal)
- Try and fix PseudoTerminal::Fork() to accept the requested terminal (so
it will not _always_ call to 'OpenFirstAvailableMaster')

What do you think?
Eran





On Fri, Mar 28, 2014 at 8:16 PM, Eran Ifrah <eran.ifrah at gmail.com> wrote:

> Hi all,
>
> I finally had some time to build lldb in debug mode and debug through the
> problem.
> It seems that the problem is caused in this function:
>
> ProcessPOSIX::GetFilePath(...)
>
> This check:
>
> // By default the stdio paths passed in will be pseudo-terminal
> // (/dev/pts). If so, convert to using a different default path
> // instead to redirect I/O to the debugger console. This should
> //  also handle user overrides to /dev/null or a different file.
> if (!path || ::strncmp(path, pts_name, ::strlen(pts_name)) == 0)
>     path = default_path;
>
> Note that when redirecting to a terminal, "path" will most likely be in
> the form of "/dev/pts/XX"
> and since "pts_name" is set to "/dev/pts/" (its hardcoded at the top of
> this function) the above condition is always true
>
> So "path = default_path" will return a NULL pointer instead of the desired
> pts
>
> commenting this part of the "if" statement fixes the problem:
> ::strncmp(path, pts_name, ::strlen(pts_name)) == 0
>
> Can anyone explain what was the rational behind this check?
>
> Thanks,
> Eran
>
>
> On Wed, Mar 26, 2014 at 11:09 PM, Piotr Rak <piotr.rak at gmail.com> wrote:
>
>> Sorry, we do have O_CREAT and even when open failed we'd exit with
>> different exit code earlier.
>> I'll rather go sleep - enough for today. :)
>>
>>
>> 2014-03-26 22:01 GMT+01:00 Piotr Rak <piotr.rak at gmail.com>:
>>
>> Also other option is that we don't have O_CREAT in flags and we get
>>> ENOENT from open().... but again can not check that today.
>>>
>>>
>>> 2014-03-26 21:52 GMT+01:00 Piotr Rak <piotr.rak at gmail.com>:
>>>
>>> No, closing before dup2 is not required, it should close fd and it
>>>> should be also atomic. I am wondering now if LLDB sets FileActions like it
>>>> was expecting that we use posix_spawn.
>>>>
>>>> I can not check it today... but will look at it tomorrow or during
>>>> weekend.
>>>>
>>>>
>>>> 2014-03-26 21:33 GMT+01:00 Greg Clayton <gclayton at apple.com>:
>>>>
>>>> Looks like you might need to look at:
>>>>>
>>>>> ProcessMonitor::Launch(LaunchArgs *args)
>>>>>
>>>>> It is what does the fork + exec.
>>>>>
>>>>> It also looks like no matter what is sent to
>>>>> ProcessMonitor::Launch(LaunchArgs *args) for stdin, stdout, stderr, Linux
>>>>> _always_ launches using a pseudo terminal by doing:
>>>>>
>>>>>     lldb_utility::PseudoTerminal terminal;
>>>>>     if ((pid = terminal.Fork(err_str, err_len)) == -1)
>>>>>
>>>>>
>>>>> If you look at the "terminal.Fork()" code you will see that it
>>>>> actually sets stdin/out/err to the slave slide of the pseudo terminal, so
>>>>> there is no telling if the code in ProcessMonitor::Launch():
>>>>>
>>>>>         if (stdin_path != NULL && stdin_path[0])
>>>>>             if (!DupDescriptor(stdin_path, STDIN_FILENO, O_RDONLY))
>>>>>                 exit(eDupStdinFailed);
>>>>>
>>>>>         if (stdout_path != NULL && stdout_path[0])
>>>>>             if (!DupDescriptor(stdout_path, STDOUT_FILENO, O_WRONLY |
>>>>> O_CREAT))
>>>>>                 exit(eDupStdoutFailed);
>>>>>
>>>>>         if (stderr_path != NULL && stderr_path[0])
>>>>>             if (!DupDescriptor(stderr_path, STDERR_FILENO, O_WRONLY |
>>>>> O_CREAT))
>>>>>                 exit(eDupStderrFailed);
>>>>>
>>>>>
>>>>> Will work? ProcessMonitor::DupDescriptor() does:
>>>>>
>>>>>
>>>>> bool
>>>>> ProcessMonitor::DupDescriptor(const char *path, int fd, int flags)
>>>>> {
>>>>>     int target_fd = open(path, flags, 0666);
>>>>>
>>>>>     if (target_fd == -1)
>>>>>         return false;
>>>>>
>>>>>     return (dup2(target_fd, fd) == -1) ? false : true;
>>>>> }
>>>>>
>>>>> I would assume you would need to call close() on stdin/out/err first?
>>>>> Again the PseudoTerminal::Fork() has always already setup stdin/out/err to
>>>>> the slave side and the DupDescriptor calls are probably failing.
>>>>>
>>>>> Greg
>>>>>
>>>>>
>>>>> On Mar 26, 2014, at 12:26 PM, Piotr Rak <piotr.rak at gmail.com> wrote:
>>>>>
>>>>> > Hi,
>>>>> >
>>>>> > That got me curious and now I am bit confused how it works, and it
>>>>> should be really simple.
>>>>> >
>>>>> > I've checked Greg's example, it will just exit before main doing
>>>>> nothing.
>>>>> > I've no idea why yet, however I attached my perfect tracee:
>>>>> >
>>>>> > extern "C" void _start()
>>>>> > {
>>>>> >       __asm__ volatile (
>>>>> >                       "again:;"
>>>>> >                       "int $0x03;"
>>>>> >                       "jmp again;"
>>>>> >                       "movl $1,%eax;"
>>>>> >                       "xorl %ebx, %ebx;"
>>>>> >                       "int $0x80;"
>>>>> >    );
>>>>> > }
>>>>> > ls -al /proc/`pidof hello`/fd
>>>>> >
>>>>> > lrwx------ 1 prak prak 64 03-26 19:42 0 -> /dev/pts/19
>>>>> > lrwx------ 1 prak prak 64 03-26 19:42 1 -> /dev/pts/19
>>>>> > lrwx------ 1 prak prak 64 03-26 19:42 2 -> /dev/pts/19
>>>>> > lrwx------ 1 prak prak 64 03-26 19:42 3 -> /dev/ptmx
>>>>> > lrwx------ 1 prak prak 64 03-26 19:42 5 -> /dev/pts/19
>>>>> >
>>>>> > ls -al /proc/`pidof lldb`/fd
>>>>> >
>>>>> > lrwx------ 1 prak prak 64 03-26 19:43 0 -> /dev/pts/16
>>>>> > lrwx------ 1 prak prak 64 03-26 19:43 1 -> /dev/pts/16
>>>>> > lrwx------ 1 prak prak 64 03-26 19:43 2 -> /dev/pts/16
>>>>> > lrwx------ 1 prak prak 64 03-26 19:43 3 -> /dev/ptmx
>>>>> > lrwx------ 1 prak prak 64 03-26 19:43 4 -> /dev/ptmx
>>>>> > lr-x------ 1 prak prak 64 03-26 19:43 5 -> pipe:[1301667]
>>>>> > l-wx------ 1 prak prak 64 03-26 19:43 6 -> pipe:[1301667]
>>>>> > lr-x------ 1 prak prak 64 03-26 19:43 7 -> pipe:[1299830]
>>>>> > l-wx------ 1 prak prak 64 03-26 19:43 8 -> pipe:[1299830]
>>>>> >
>>>>> > So it was spawned as usual using fork() by Linux/ProcessMonitor.cpp
>>>>> > And it was given slave pts, lldb however has no '/tmp/out.txt' file
>>>>> anywhere.
>>>>> >
>>>>> > Should this for this case?:
>>>>> > a) be passed opened to inferior before exec, or
>>>>> > b) lldb should read master pty and write to /tmp/out.txt
>>>>> >
>>>>> > I would guess that should be a:
>>>>> >
>>>>> > But probably then:
>>>>> >
>>>>> > SetSTDIOFileDescriptor(m_monitor->GetTerminalFD());
>>>>> >
>>>>> > from ProcessPosix::DoLaunch() ProcessPosix.cpp:253 after creating
>>>>> process spoils the fun.
>>>>> > Also std{in,err,path)_path had to be empty here.
>>>>> >
>>>>> > Cheers,
>>>>> > /Piotr
>>>>> >
>>>>> >
>>>>> > 2014-03-26 19:08 GMT+01:00 Greg Clayton <gclayton at apple.com>:
>>>>> > Seems like we are having some problem re-directing to terminals and
>>>>> files that exist. I would try debugging through the launch process and see
>>>>> who is doing what with file re-direction. It works on MacOSX just fine, so
>>>>> this is probably a linux only issue. Linux does fork() + exec() so some
>>>>> code in there isn't doing the right things.
>>>>> >
>>>>> > % lldb
>>>>> > (lldb) settings set target.output-path /tmp/out.txt
>>>>> > (lldb) file /bin/ls
>>>>> > Current executable set to '/bin/ls' (x86_64).
>>>>> > (lldb) run /tmp/
>>>>> > (lldb) Process 65933 launched: '/bin/ls' (x86_64)
>>>>> > Process 65933 exited with status = 0 (0x00000000)
>>>>> > (lldb) q
>>>>> >
>>>>> > % cat /tmp/out.txt
>>>>> > launch-B6FwKk
>>>>> > launch-OEyacj
>>>>> > launchd-142.5fRyOk
>>>>> > launchd-175.RBU3HO
>>>>> > launchd-193.Asuh1k
>>>>> > launchd-2701.dSHLJu
>>>>> > launchd-738.U2ACnW
>>>>> > out.txt
>>>>> >
>>>>> > % xcrun lldb
>>>>> > (lldb) settings set target.output-path /tmp/out.txt
>>>>> > (lldb) file /bin/ls
>>>>> > Current executable set to '/bin/ls' (x86_64).
>>>>> > (lldb) run /
>>>>> > (lldb) Process 65940 launched: '/bin/ls' (x86_64)
>>>>> > Process 65940 exited with status = 0 (0x00000000)
>>>>> > (lldb) q
>>>>> > lldb:/tmp % cat /tmp/out.txt
>>>>> > AppleInternal
>>>>> > Applications
>>>>> > Library
>>>>> > Network
>>>>> > SWE
>>>>> > System
>>>>> > Users
>>>>> > Volumes
>>>>> > bin
>>>>> > cores
>>>>> > dev
>>>>> > etc
>>>>> > home
>>>>> > mach_kernel
>>>>> > net
>>>>> >
>>>>> >
>>>>> > So this works on Darwin and needs to be fixed on Linux.
>>>>> >
>>>>> > Greg
>>>>> >
>>>>> > On Mar 25, 2014, at 7:15 AM, Eran Ifrah <eran.ifrah at gmail.com>
>>>>> wrote:
>>>>> >
>>>>> > > Hello Greg,
>>>>> > > Thanks for the input. It still does not work ( I rewrote my
>>>>> terminal code to look similar to lldb's PseudoTerminal, and it is still not
>>>>> working)
>>>>> > >
>>>>> > > To simplify things, I tried some basic things with the command
>>>>> line tool 'lldb':
>>>>> > >
>>>>> > > I created a file ~/.lldbinit with the following content:
>>>>> > >
>>>>> > > eran at eran-linux: ~/llvm/build/bin $ cat ~/.lldbinit
>>>>> > > settings set target.output-path /tmp/dbg.out
>>>>> > > eran at eran-linux: ~/llvm/build/bin $
>>>>> > >
>>>>> > > I then ran lldb while having tail -f /tmp/dbg.out& in another
>>>>> terminal to see if the stdout is being redirected
>>>>> > >
>>>>> > > Now, this is the interesting part:
>>>>> > > In the first run when the file /tmp/dbg.out was empty - the
>>>>> redirection worked (tail showed the debuggee stdout)
>>>>> > > In the second run (and later) - nothing was written to the file
>>>>> > >
>>>>> > > However, if I truncate the file using the below command:
>>>>> > >
>>>>> > > $ > /tmp/dbg.out
>>>>> > >
>>>>> > > and run lldb again - I see the stdout again - but same as before
>>>>> only for the first time (i.e. as long as the file is empty the stdout was
>>>>> redirected)
>>>>> > >
>>>>> > > The next thing I tried was to use a terminal name for redirection:
>>>>> > >
>>>>> > > - Open a new terminal and type `tty` (in my case it gave
>>>>> /dev/pts/19 )
>>>>> > > - Edit the ~/.lldbinit: settings set target.output-path /dev/pts/19
>>>>> > > - Start lldb and verify that the setting is set properly by
>>>>> running: settings show target.output-path
>>>>> > > - Run the program under lldb - the output is not redirected (i.e.
>>>>> it is show in the same console where I ran lldb)
>>>>> > >
>>>>> > > Any ideas?
>>>>> > >
>>>>> > > P.S.
>>>>> > > Sorry if this looks like a voodoo, but this is what I am getting
>>>>> here... ;)
>>>>> > >
>>>>> > > Eran
>>>>> > >
>>>>> > >
>>>>> > >
>>>>> > >
>>>>> > >
>>>>> > > On Mon, Mar 24, 2014 at 6:46 PM, Greg Clayton <gclayton at apple.com>
>>>>> wrote:
>>>>> > > Check out the PseudoTerminal class in
>>>>> trunk/source/Utility/PseudoTerminal.cpp.
>>>>> > >
>>>>> > > See the function named
>>>>> PseudoTerminal::OpenFirstAvailableMaster(...). You must call posix_openpt,
>>>>> grantpt, and unlockpt. I am guessing that because you aren't calling
>>>>> grantpt and granting access to the slave you are failing to be able to use
>>>>> the slave in your child process.
>>>>> > >
>>>>> > > Greg Clayton
>>>>> > >
>>>>> > > On Mar 23, 2014, at 11:32 AM, Eran Ifrah <eran.ifrah at gmail.com>
>>>>> wrote:
>>>>> > >
>>>>> > > > Sure, thanks for the help so far
>>>>> > > > Eran
>>>>> > > >
>>>>> > > >
>>>>> > > > On Sun, Mar 23, 2014 at 8:31 PM, Piotr Rak <piotr.rak at gmail.com>
>>>>> wrote:
>>>>> > > > I am sorry, but nothing obvious comes to me right now, probably
>>>>> you'll need to wait for Monday, when people more familiar with lldb will be
>>>>> able to help you debug this problem.
>>>>> > > >
>>>>> > > >
>>>>> > > > 2014-03-23 16:07 GMT+01:00 Eran Ifrah <eran.ifrah at gmail.com>:
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > > On Sun, Mar 23, 2014 at 4:29 PM, Piotr Rak <piotr.rak at gmail.com>
>>>>> wrote:
>>>>> > > > Sorry I misinformed you about posix_spawn - it is not true for
>>>>> Linux and FreeBSD at least, it will use ordinary fork.
>>>>> > > >
>>>>> > > >
>>>>> > > > 2014-03-23 15:23 GMT+01:00 Piotr Rak <piotr.rak at gmail.com>:
>>>>> > > >
>>>>> > > >
>>>>> > > > Hi,
>>>>> > > >
>>>>> > > > 2014-03-23 14:04 GMT+01:00 Eran Ifrah <eran.ifrah at gmail.com>:
>>>>> > > >
>>>>> > > > Thanks for your pointer Piotr. Here is the code I am using to
>>>>> open a pseudo-terminal (there is the UI part, which I left out):
>>>>> > > >
>>>>> > > >     char __name[128];
>>>>> > > >     memset(__name, 0, sizeof(__name));
>>>>> > > >
>>>>> > > >     int master(-1);
>>>>> > > >     m_slave = -1;
>>>>> > > >     if(openpty(&master, &m_slave, __name, NULL, NULL) != 0)
>>>>> > > >         return wxT("");
>>>>> > > >
>>>>> > > >     // disable ECHO
>>>>> > > >     struct termios termio;
>>>>> > > >     tcgetattr(master, &termio);
>>>>> > > >     termio.c_lflag = ICANON;
>>>>> > > >     termio.c_oflag = ONOCR | ONLRET;
>>>>> > > >     tcsetattr(master, TCSANOW, &termio);
>>>>> > > >
>>>>> > > >     m_tty = wxString(__name, wxConvUTF8);
>>>>> > > >
>>>>> > > > At the end, m_tty contains a string name (e.g. /dev/pts/19 ).
>>>>> > > > Note that the above code works flawlessly when using it with gdb
>>>>> (i.e. if I pass this "/dev/pts/19" to gdb's switch -tty=/dev/pts/19 I will
>>>>> get all the inferior output/err/input to my internal terminal)
>>>>> > > >
>>>>> > > > However, doing the same with LLDB (using C++ API not the command
>>>>> line , i.e. passing "/dev/pts/19" as an argument to SBTarget::Launch(...))
>>>>> I get nothing as output...
>>>>> > > >
>>>>> > > > Looks sane to me.
>>>>> > > >
>>>>> > > > Also, I am not sure I am following the idea behind replacing the
>>>>> "Launch" function with my own fork(), looking at the code of Launch()
>>>>> suggests that it does more than a simple fork...
>>>>> > > >
>>>>> > > >
>>>>> > > > That was my idea to debug issue if nothing else helps.
>>>>> > > > Or rather bisect on which side it really is, sorry if I did not
>>>>> make it clear...
>>>>> > > > So I was trying suggest replacing SBTarget::Launch with fork,
>>>>>  write to child stdout/err, and see if that works alone....
>>>>> > > >
>>>>> > > > SBTarget::Launch is usually actually posix_spawn right now, it
>>>>> uses posix_spawnattr_addopen to open descriptors for your specified paths,
>>>>> and should open it 3 times - given current implementation - even it is just
>>>>> one file.
>>>>> > > >
>>>>> > > > Have you inspected SBProcess and SBError returned by
>>>>> SBTarget::Launch?
>>>>> > > >
>>>>> > > > I checked IsValid() on both and its OK for both. I can actually
>>>>> run "next" Continue etc and seems to be working. Its just that I can't seem
>>>>> to redirect the stdout/err to my own console.
>>>>> > > >
>>>>> > > >
>>>>> > > > Do you see your inferior process is indeed launching, just not
>>>>> displaying anything?
>>>>> > > > Yes, ps -ef shows the debugee
>>>>> > > >
>>>>> > > > Do you have an option to check if those terminals are actually
>>>>> being opened (like examining /proc/<pid>/fd for linux)?
>>>>> > > > The terminal is opened. Like I mentioned in my previous email,
>>>>> using the _same_ code with gdb works
>>>>> > > > I also have a standalone terminal application which I wrote
>>>>> which is also using the same set of classes all of the are working for
>>>>> couple of years now without any problems
>>>>> > > >
>>>>> > > > I also tried this:
>>>>> > > > I typed in my konsole 'tty' and used that as the input for
>>>>> Launch - it also seems to have no effect
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > > Good luck,
>>>>> > > > /Piotr
>>>>> > > >
>>>>> > > > Any more hints?
>>>>> > > > Eran
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > > On Sat, Mar 22, 2014 at 9:36 PM, Piotr Rak <piotr.rak at gmail.com>
>>>>> wrote:
>>>>> > > > Hi,
>>>>> > > >
>>>>> > > > It should.
>>>>> > > > Have you opened master pseudoterminal like?:
>>>>> > > >
>>>>> > > > int fd = posix_openpt(flags); // open("/dev/ptmx") might work
>>>>> here too but less portable;
>>>>> > > > grantpt(fd);
>>>>> > > > unlockpt(fd);
>>>>> > > >
>>>>> > > > Depending on target you might need some bizarre ioctls here, but
>>>>> assuming you are using Linux/FreeBSD/MacOSX
>>>>> > > > you should be fine.
>>>>> > > >
>>>>> > > > If you had already master pseudo-terminal file descriptor you
>>>>> can skip steps above.
>>>>> > > >
>>>>> > > > You can use ptsname for master file descriptor it will return
>>>>> you name of slave pseudo-terminal for your master.
>>>>> > > > Later you can pass name returned by ptsname(fd) as Launch
>>>>> arguments.
>>>>> > > >
>>>>> > > > If above won't work you can try replacing Launch() call with
>>>>> ordinary fork, and in child process:
>>>>> > > >
>>>>> > > > slavefd = open(slavename, O_RDWR);
>>>>> > > >
>>>>> > > > dup2(0, slavefd);
>>>>> > > > dup2(1, slavefd);
>>>>> > > > dup2(2, slavefd);
>>>>> > > >
>>>>> > > > And see if that works alone for you...
>>>>> > > >
>>>>> > > > Good luck,
>>>>> > > > /Piotr
>>>>> > > >
>>>>> > > >
>>>>> > > > 2014-03-22 19:29 GMT+01:00 Eran Ifrah <eran.ifrah at gmail.com>:
>>>>> > > > Hello,
>>>>> > > >
>>>>> > > > I am trying to use the C++ API with good success so far.
>>>>> > > > I am now at a point where I want to redirect stdin/out/err of
>>>>> the inferior to my application (my application creates a separate pseudo
>>>>> terminal window)
>>>>> > > >
>>>>> > > > Looking at the SBTarget::Launch, I thought that simply passing
>>>>> "/dev/pts/<some-number>" as the 3rd, 4th and 5th argument will do the trick
>>>>> .. well, it did not.
>>>>> > > > I am missing something basic here, can anyone shed some light
>>>>> please? or give an example (better) of how to achieve this?
>>>>> > > >
>>>>> > > > Thanks!
>>>>> > > >
>>>>> > > > --
>>>>> > > > Eran Ifrah
>>>>> > > > Author of codelite, a cross platform open source C/C++ IDE:
>>>>> http://www.codelite.org
>>>>> > > > wxCrafter, a wxWidgets RAD: http://wxcrafter.codelite.org
>>>>> > > >
>>>>> > > > _______________________________________________
>>>>> > > > lldb-dev mailing list
>>>>> > > > lldb-dev at cs.uiuc.edu
>>>>> > > > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > > --
>>>>> > > > Eran Ifrah
>>>>> > > > Author of codelite, a cross platform open source C/C++ IDE:
>>>>> http://www.codelite.org
>>>>> > > > wxCrafter, a wxWidgets RAD: http://wxcrafter.codelite.org
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > > --
>>>>> > > > Eran Ifrah
>>>>> > > > Author of codelite, a cross platform open source C/C++ IDE:
>>>>> http://www.codelite.org
>>>>> > > > wxCrafter, a wxWidgets RAD: http://wxcrafter.codelite.org
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > > --
>>>>> > > > Eran Ifrah
>>>>> > > > Author of codelite, a cross platform open source C/C++ IDE:
>>>>> http://www.codelite.org
>>>>> > > > wxCrafter, a wxWidgets RAD: http://wxcrafter.codelite.org
>>>>> > > > _______________________________________________
>>>>> > > > lldb-dev mailing list
>>>>> > > > lldb-dev at cs.uiuc.edu
>>>>> > > > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>>>>> > >
>>>>> > >
>>>>> > >
>>>>> > >
>>>>> > > --
>>>>> > > Eran Ifrah
>>>>> > > Author of codelite, a cross platform open source C/C++ IDE:
>>>>> http://www.codelite.org
>>>>> > > wxCrafter, a wxWidgets RAD: http://wxcrafter.codelite.org
>>>>> >
>>>>> >
>>>>>
>>>>>
>>>>
>>>
>>
>
>
> --
> Eran Ifrah
> Author of codelite, a cross platform open source C/C++ IDE:
> http://www.codelite.org
> wxCrafter, a wxWidgets RAD: http://wxcrafter.codelite.org
>



-- 
Eran Ifrah
Author of codelite, a cross platform open source C/C++ IDE:
http://www.codelite.org
wxCrafter, a wxWidgets RAD: http://wxcrafter.codelite.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20140329/e17c8974/attachment.html>


More information about the lldb-dev mailing list