[Lldb-commits] [lldb] r312270 - lldb-mi: -var-update can hang when traversing complex types with pointers

[Ted Woodward via lldb-commits]

Author: ted
Date: Thu Aug 31 12:22:33 2017
New Revision: 312270

URL: http://llvm.org/viewvc/llvm-project?rev=312270&view=rev
Log:
lldb-mi: -var-update can hang when traversing complex types with pointers

Summary:
-var-update calls CMICmdCmdVarUpdate::ExamineSBValueForChange to check if a varObj has been updated. It checks that the varObj is updated, then recurses on all of its children. If a child is a pointer pointing back to a parent node, this will result in an infinite loop, and lldb-mi hanging.

The problem is exposed by packages/Python/lldbsuite/test/tools/lldb-mi/variable/TestMiVar.py, but this test is skipped everywhere.

This patch changes ExamineSBValueForChange to not traverse children of varObjs that are pointers.

Reviewers: ki.stfu, zturner, clayborg, abidh

Reviewed By: clayborg

Subscribers: lldb-commits

Differential Revision: https://reviews.llvm.org/D37154

Modified:
    lldb/trunk/tools/lldb-mi/MICmdCmdVar.cpp

Modified: lldb/trunk/tools/lldb-mi/MICmdCmdVar.cpp
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/tools/lldb-mi/MICmdCmdVar.cpp?rev=312270&r1=312269&r2=312270&view=diff
==============================================================================
--- lldb/trunk/tools/lldb-mi/MICmdCmdVar.cpp (original)
+++ lldb/trunk/tools/lldb-mi/MICmdCmdVar.cpp Thu Aug 31 12:22:33 2017
@@ -509,19 +509,19 @@ bool CMICmdCmdVarUpdate::ExamineSBValueF
     return MIstatus::success;
   }
 
-  lldb::SBType valueType = vrwValue.GetType();
-
   const MIuint nChildren = vrwValue.GetNumChildren();
   for (MIuint i = 0; i < nChildren; ++i) {
     lldb::SBValue member = vrwValue.GetChildAtIndex(i);
     if (!member.IsValid())
       continue;
 
-    if (member.GetValueDidChange()) {
-      vrwbChanged = true;
-      return MIstatus::success;
-    } else if (ExamineSBValueForChange(member, vrwbChanged) && vrwbChanged)
-      // Handle composite types (i.e. struct or arrays)
+    // skip pointers and references to avoid infinite loop
+    if (member.GetType().GetTypeFlags() &
+        (lldb::eTypeIsPointer | lldb::eTypeIsReference))
+      continue;
+
+    // Handle composite types (i.e. struct or arrays)
+    if (ExamineSBValueForChange(member, vrwbChanged) && vrwbChanged)
       return MIstatus::success;
   }
   vrwbChanged = false;