[lldb-dev] Process::GetSTDOUT and PutSTDIN

[Greg Clayton]

> On Dec 3, 2014, at 11:22 AM, Zachary Turner <zturner at google.com> wrote:
> 
> AFAICT, it's just not a thing Windows can do.  If it's doable then it seems very difficult, because I haven't figured it out yet.  I guess in Linux / MacOSX you support this via the pseudoterminal, which from my weak understanding appears to be an fd that you can use to write into the terminal.  We don't have anything like that.  You can only write into a terminal or read from it from within a process running in the terminal.
> 
> Normally in Windows you can duplicate handles across process boundaries, so one idea I had was to get the terminal's stdin and stdout handles and duplicate them with the inferior as the source process and LLDB as the target process.  But for console handles specifically, this isn't supported.  http://msdn.microsoft.com/en-us/library/windows/desktop/ms724251%28v=vs.85%29.aspx
> 
> "Console input	The handle is returned by the CreateFile function when CONIN$ is specified, or by the GetStdHandle function when STD_INPUT_HANDLE is specified. Console handles can be duplicated for use only in the same process."
> 
> One idea I had was to inject some code into the inferior and have LLDB communicate with it via IPC.  But that's admittedly a little bit insane.
> 
> Just so I understand the use case, what is the actual need for allowing both LLDB and a user to write into (or read from) the same terminal?

Think of the master pty as read/write file descriptor that we hand to read write from and the slave as a single descriptor/handle we use for stdin/out/err on the debuggee side. You will probably need to make individual handles for each in/out/err.

>  In the case of a stdout for example, LLDB would consume the stdout, and the user would never see it in the terminal.  If that's the case, then it seems like LLDB would want to consume everything, and you might as well achieve the same thing via a pipe.

Can't you just specify hStdInput, hStdOutput and hStdError in STARTUPINFO as the second to last parameter to:

BOOL WINAPI CreateProcess(
  _In_opt_     LPCTSTR lpApplicationName,
  _Inout_opt_  LPTSTR lpCommandLine,
  _In_opt_     LPSECURITY_ATTRIBUTES lpProcessAttributes,
  _In_opt_     LPSECURITY_ATTRIBUTES lpThreadAttributes,
  _In_         BOOL bInheritHandles,
  _In_         DWORD dwCreationFlags,
  _In_opt_     LPVOID lpEnvironment,
  _In_opt_     LPCTSTR lpCurrentDirectory,
  _In_         LPSTARTUPINFO lpStartupInfo,
  _Out_        LPPROCESS_INFORMATION lpProcessInformation
);


typedef struct
 _STARTUPINFO {
  DWORD  cb;
  LPTSTR lpReserved;
  LPTSTR lpDesktop;
  LPTSTR lpTitle;
  DWORD  dwX;
  DWORD  dwY;
  DWORD  dwXSize;
  DWORD  dwYSize;
  DWORD  dwXCountChars;
  DWORD  dwYCountChars;
  DWORD  dwFillAttribute;
  DWORD  dwFlags;
  WORD   wShowWindow;
  WORD   cbReserved2;
  LPBYTE lpReserved2;
  HANDLE hStdInput;
  HANDLE hStdOutput;
  HANDLE hStdError;
} STARTUPINFO, *LPSTARTUPINFO;

So when redirecting stdout you would make a writable file handle for the file "/tmp/stdout.txt" and then pass the read/write end of the pipe for stdin and stderr?

> On Wed Dec 03 2014 at 11:11:50 AM Greg Clayton <gclayton at apple.com> wrote:
> Feel free to do what you need to to make this happen. Not sure why both can't coexist. What is the hinderance?
> 
> > On Dec 3, 2014, at 11:04 AM, Zachary Turner <zturner at google.com> wrote:
> >
> > Redirecting to a file is easy, and creating a pipe is easy.  The problem is that both of these will result in stdio to/from the actual terminal not happening anymore.  In other words, I can only easily have one or the other.  Either LLDB has access to the streams, or the terminal has access to the streams.  Not both.
> >
> > On Wed Dec 03 2014 at 10:59:38 AM Greg Clayton <gclayton at apple.com> wrote:
> >
> > > On Dec 2, 2014, at 11:53 PM, Zachary Turner <zturner at google.com> wrote:
> > >
> > > On Fri Nov 21 2014 at 5:46:46 PM Zachary Turner <zturner at google.com> wrote:
> > > I only see these functions being called from a couple of tests so I wasn't able to figure out from the documentation all the preconditions and assumptions that these functions make.    So I have a few questions:
> > >
> > > 1) Does the use of these functions require or assume that the process is stopped at a breakpoint, or should they work even if the process is running?
> >
> > they should work when the process is running. On Unix variants we use a pseudo terminal where we hold the master side and the debugged process gets the slave side of the pseudo terminal. When the debuggee writes to the slave side, we can immediately read it from the master side. Any files that you use to implement this need to be unbuffered and the stdio needs to be able to be delivered immediately down to the debuggee and also LLDB needs to be able to read the data as soon as it comes in.
> >
> > > 2) When you call GetSTDOUT or GetSTDERR and they return successfully, does it matter whether this eats the output from the actual STDOUT / STDERR?
> >
> > It should be consuming it from the actual file you are using to receive it (we consume the stdout/stderr by reading from the master_fd from the master side of the pseudo terminal). It probably isn't a good idea to not consume the data from stdout/stderr because you might fill up the stdio buffers and and subsequent writes to stdout/stderr can cause your program to deadlock with a full stdio buffer. The current implementation starts up a stdio read thread where it reads from stdout/stderr continuously and places anything it receives into Process::m_stdout_data or Process::m_stderr_data. These are protected by the m_stdio_communication_mutex so it is thread safe.
> >
> > >
> > > For example, let's say we are using the public API and we launch a process with STDOUT re-directed to a file.  Later, we call Process.GetSTDOUT using the public API, and the value is returned to our script.  Does the output that we read still need to go to the file?
> >
> > You shouldn't get any stdout in lldb_private::Process if it was re-directed to a file. For example in unix, if we redirect only stdout to a file (not stderr or stdin), we would end up launching a child process with:
> >
> > stdin:  /dev/slave-pty-01
> > stdout: /tmp/foo.txt
> > stderr: /dev/slave-pty-01
> >
> > So the debugged process would be writing directly to /tmp/foo.txt and nothing should show up in the LLDB console.
> >
> > We would still startup a read thread for stderr (see Process::SetSTDIOFileDescriptor (int fd)) and it currently has a bug where it doesn't separate stdout and stderr correctly (most people redirect both stdout/stderr or don't do it), so we should get that. But when redirecting stdout and stderr, we should never see anything in the LLDB console and if we redirect both stdout/stderr, we shouldn't be starting up a read thread via "Process::SetSTDIOFileDescriptor (int fd)".
> >
> >
> > > Looks like I sent this to the wrong mailing list, and during the holiday season to boot.  Posting it to the right mailing list this time.
> > >
> > > To add a little bit to my original post, I did end up finding a few more places where this is called.  In particular, in addition to the aforementioned tests, it also seems to be called in order to get the process's as a way to embed the application's input/output in the LLDB terminal.
> > >
> > > With that said, I also want to add a 3rd question to my original list.
> > >
> > > 3) Can we satisfy every use case of GetSTDOUT, GetSTDERR, and PutSTDIN with a different mechanism?  For example, stdio redirection of the target process.  Unless there is some use case of these functions that require LLDB to be able to manipulate a target's stdio *without* re-directing them (so that they are still hooked up to the standard terminal), then it seems like we should be ok in principle with just repurposing stdio redirection for this.
> > >
> > > The reason I ask is that it is almost impossible (at the very least, it is extremely difficult) to implement this on Windows in such a way that LLDB can read / write to these streams without creating a pipe between LLDB and the target.  If there's a way to satisfy every platform that would be preferable.  If not, what feature set am I missing out on?
> >
> > It is very easy for unix and unix doesn't need to change. You will need to make this happen for windows. What is the problem with creating a pipe? Is there a reason you can't do this? What part is hard? Redirecting to a file? Or using live streams between LLDB and the inferior when stdio isn't redirected. IO redirection is a host layer thing that should be implemented in your Host::LaunchProcess(). Did you do any of the file redirection stuff in there? Or are you using a separate launching mechanism for debugging?