[Lldb-commits] [PATCH] D95165: Implement workaround for issue that shows between libedit and low-level terminal routines on Linux

[Augusto Noronha via Phabricator via lldb-commits]

augusto2112 created this revision.
augusto2112 added reviewers: teemperor, aprantl, jingham, mib.
Herald added a subscriber: krytarowski.
augusto2112 requested review of this revision.
Herald added a project: LLDB.
Herald added a subscriber: lldb-commits.

On some Linux distributions, after setting up the EditLine object, the eventual call to set_curterm, which happens when calculating terminal properties, breaks the libedit configuration, causing characters that have functions bound to them not to show up. As a workaround, we calculate these terminal properties eagerly, before initializing the EditLine object.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D95165

Files:
  lldb/include/lldb/Host/File.h
  lldb/source/Core/IOHandler.cpp
  lldb/source/Host/common/File.cpp


Index: lldb/source/Host/common/File.cpp
===================================================================
--- lldb/source/Host/common/File.cpp
+++ lldb/source/Host/common/File.cpp
@@ -184,6 +184,11 @@
 #endif
 }
 
+void File::EagerlyCalculateInteractiveAndTerminalProperties() {
+  if (m_supports_colors == eLazyBoolCalculate)
+    CalculateInteractiveAndTerminal();
+}
+
 bool File::GetIsInteractive() {
   if (m_is_interactive == eLazyBoolCalculate)
     CalculateInteractiveAndTerminal();
Index: lldb/source/Core/IOHandler.cpp
===================================================================
--- lldb/source/Core/IOHandler.cpp
+++ lldb/source/Core/IOHandler.cpp
@@ -262,6 +262,18 @@
                  m_input_sp && m_input_sp->GetIsRealTerminal();
 
   if (use_editline) {
+#if defined(__linux__)
+    // On some Linux distributions, after setting up the EditLine object,
+    // the eventual call to set_curterm,
+    // which happens when calculating terminal properties,
+    // breaks the libedit configuration, causing characters that have
+    // functions bound to them not to show up. As a workaround, we calculate
+    // these terminal properties eagerly, before initializing the EditLine
+    // object.
+    m_output_sp->GetFile().EagerlyCalculateInteractiveAndTerminalProperties();
+    m_error_sp->GetFile().EagerlyCalculateInteractiveAndTerminalProperties();
+#endif
+
     m_editline_up = std::make_unique<Editline>(editline_name, GetInputFILE(),
                                                GetOutputFILE(), GetErrorFILE(),
                                                m_color_prompts);
Index: lldb/include/lldb/Host/File.h
===================================================================
--- lldb/include/lldb/Host/File.h
+++ lldb/include/lldb/Host/File.h
@@ -327,6 +327,11 @@
   ///     in lldb_private::File::Permissions.
   uint32_t GetPermissions(Status &error) const;
 
+  /// Ensures that the properties which represent if this file is interactive,
+  /// a real terminal, and a terminal with colors are calculated eagerly.
+  void EagerlyCalculateInteractiveAndTerminalProperties();
+
+
   /// Return true if this file is interactive.
   ///
   /// \return


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D95165.318296.patch
Type: text/x-patch
Size: 2199 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20210121/0394bf06/attachment.bin>