[Lldb-commits] [PATCH] D96623: [lldb-vscode] Fix lldb init file stdout issue

[Greg Clayton via Phabricator via lldb-commits]

clayborg added a comment.

After thinking about this a bit more, it seems a mistake that we can't change the file handles _prior_ to sourcing the init files. I have proposed one solution in the inline comments to create a new SBDebugger::Create() method that takes the input output and error file handles as arguments. The other way to do this is to make it so that we _can_ run the init files later. Currently if we say "false" to "bool source_init_files", then the command interpreter marks the init files as not being able to be run again later if you call the SBCommandInterpreter::SourceInitFileInHomeDirectory(...). So we need to either be able to either specify the file handles at debugger creation time, or run them one time later. The current "run them once" means "you can only do this at debugger creation time, but never again if you specify false for source_init_files. We can change this to be "only run the init files once. so if we already run them in SBDebugger::Create(), then don't run them again if you call SBCommandInterpreter::SourceInitFileInHomeDirectory()



================
Comment at: lldb/tools/lldb-vscode/lldb-vscode.cpp:1444
+  // call SourceInitFile() below after dev_null redirection.
+  g_vsc.debugger = lldb::SBDebugger::Create(false /*source_init_files*/);
+
----------------
So after thinking about this some more, it might be better to add a new lldb::SBDebugger::Create method that takes the input, output and error handles as SBFile objects. This will allow us to not have to duplicate the entire .lldbinit loading stuff in this patch. My proposal would be to add a new create method:

```
lldb::SBDebugger lldb::SBDebugger::Create(bool source_init_files, SBFile input, SBFile output, SBFile error);
```
SBFile has multiple constructors that will work for us:
```
SBFile()
SBFile(FILE *file, bool transfer_ownership);
SBFile(int fd, const char *mode, bool transfer_ownership);
```
If we default construct a SBFile, it will contain no valid shared pointer in the private "FileSP m_opaque_sp;" member. Any of the SBFile objects that are not valid, won't end up being set in the debugger. All valid file handles will be set _prior_ to sourcing the init files, and then we can avoid any problems.

in the SBDebugger.cpp you would do something like:
```
lldb::SBDebugger lldb::SBDebugger::Create(bool source_init_files, SBFile input, SBFile output, SBFile error) {

  static std::recursive_mutex g_mutex;
  std::lock_guard<std::recursive_mutex> guard(g_mutex);
  debugger.reset(Debugger::CreateInstance(callback, baton));
  if (input.IsValid())
    debugger.SetInputFile(input);
  if (output.IsValid())
    debugger.SetOutputFile(output):
  if (error.IsValid())
    debugger.SetErrorFile(error);
  SBCommandInterpreter interp = debugger.GetCommandInterpreter();
  if (source_init_files) {
    interp.get()->SkipLLDBInitFiles(false);
    interp.get()->SkipAppInitFiles(false);
    SBCommandReturnObject result;
    interp.SourceInitFileInHomeDirectory(result, false);
  } else {
    interp.get()->SkipLLDBInitFiles(true);
    interp.get()->SkipAppInitFiles(true);
  }
  return debugger;
}
```


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D96623/new/

https://reviews.llvm.org/D96623