[lldb-dev] [Bug 42271] New: Hanging lldb-vscode test: TestVSCode_setBreakpoints

[via lldb-dev]

https://bugs.llvm.org/show_bug.cgi?id=42271

            Bug ID: 42271
           Summary: Hanging lldb-vscode test: TestVSCode_setBreakpoints
           Product: lldb
           Version: unspecified
          Hardware: PC
                OS: Linux
            Status: NEW
          Keywords: code-quality
          Severity: normal
          Priority: P
         Component: All Bugs
          Assignee: lldb-dev at lists.llvm.org
          Reporter: kkleine at redhat.com
                CC: clayborg at gmail.com, jan.kratochvil at redhat.com,
                    jdevlieghere at apple.com, llvm-bugs at lists.llvm.org

Created attachment 22096
  --> https://bugs.llvm.org/attachment.cgi?id=22096&action=edit
Patch to have test timeout after 1 second instead of hanging forever.

Summary
=======

On one of our build bots and with local builds a test related to the
lldb-vscode binary is sporadically hanging: TestVSCode_setBreakpoints.py. When
I run it 100 times it eventually hangs after an arbitrary number of times.

How to reproduce
================

In order to produce some load on my developer machine it is enough to run
"ninja check-lldb" in a separate terminal. Then in another terminal you can run
the hanging test manually in a loop:

cd ~/llvm/lldb/test/
for i in {1..100}; do
  echo "=============== TEST $i ================"; 
  
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/llvm-builds/relwithdebinfo-ninja-clang-gold-ccache-distcc/lib
python dotest.py \
    -v \
    --executable
~/llvm-builds/relwithdebinfo-ninja-clang-gold-ccache-distcc/bin/lldb \
    -p TestVSCode_setBreakpoints.py \
    ../../lldb/packages/Python/lldbsuite/test/tools/lldb-vscode/breakpoint; 
  if [ $? -ne 0 ]; then
    echo "ERROR IN TEST $i";
    break;
  fi
done

Locally I have my LLVM monorepo checked out in ~/llvm and my build directory is
~/llvm-builds/relwithdebinfo-ninja-clang-gold-ccache-distcc . It doesn't matter
how you've compiled lldb.

Once you've reproduced that you sometimes (with a pretty high chance though)
get a hanging test, you might want to apply the attached patch and retry to
reproduce problem. You'll notice that it is gone. I'm not sure this really is
the proper patch to address the problem which is why I haven't created a
revision phabricator yet.


First analysis
==============

I've augmented the Python test scripts with print()'s to show what's going on.
That's when I found out that there's this a simplified call graph (at the
bottom is the hanging part): 


class VSCodeTestCaseBase: def verify_breakpoint_hit(self, breakpoint_ids):

  stopped_events = self.vscode.wait_for_stopped()

class DebugCommunication: def wait_for_stopped(self, timeout=None):

  stopped_event = self.wait_for_event(filter=['stopped', 'exited'],
timeout=timeout)

  def wait_for_event(self, filter=None, timeout=None):

    self.recv_packet(filter_type='event', filter_event=filter, timeout=timeout)

  def recv_packet(self, filter_type=None, filter_event=None, timeout=None):

    self.recv_condition.wait(timeout)

  threading.Condition.wait(timeout=None)


This is where our test hangs. One thing to note here is that from the call
side, there's no timeout passed along, so it evaluates to None in
threading.Condition.wait(timeout). The documentation for this function
(https://docs.python.org/3/library/threading.html#threading.Condition.wait)
says "Wait until notified or until a timeout occurs." Somehow it waits forever
because it is not notified?

My question is why there's no timeout passed along to avoid those hanging
problems altogether? Just do a simple grep for "wait_for_stopped",
"wait_for_event", or "recv_packet" to find out how many times those functions
are called without a proper timeout.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20190613/6cd9c34f/attachment.html>