[PATCH] D91737: [Dexter] Avoid potentially infinite loop in dbgeng driver

Tue Jan 5 09:32:38 PST 2021

TWeaver added a comment.

> The "go" method of Debugger classes in Dexter used to be called once, to launch the process. However with the new DebuggerController classes it's now used to set the target process free running until we hit a relevant source line. Alas, I'd baked the former assumption into the DbgEng driver and had the "go" method do nothing. This can lead to an infinite loop where the DebuggerController repeatedly calls "go".

This assertion is false. The launch method is still called once at the start of a 'debug' session **[1]** (once all other initialisation has finished) followed by subsequent calls to the 'go' method when 'breaked' in a source file that isn't part of the test (i.e. library headers etc). If the debugger has paused within a test file the step method is called instead. **[2]**

This is only the case for the DefaultController and not the ConditionalController, which calls 'go' no matter what. But, and it's a big but, dbgeng doesn't have the necessary breakpoint setting methods for use with ConditionalController, so this should never happen.

The confusion here may have arose from a previous change that switched the VisualStudio debugger implementation's launch method from the step function to the go function. But again, this change shouldn't affect dbgeng's operation. **[3]**

It could be that the infinite looping/strange behaviour we're seeing is a result of being 'stuck' in library files and dexter repeatedly calling the empty 'go' method of dbgeng?

[1]  https://github.com/llvm/llvm-project/blob/main/debuginfo-tests/dexter/dex/debugger/DebuggerControllers/DefaultController.py#L38

[2] https://github.com/llvm/llvm-project/blob/main/debuginfo-tests/dexter/dex/debugger/DebuggerControllers/DefaultController.py#L61

[3] https://github.com/llvm/llvm-project/commit/c6aa829644f30d5590451b892918298f8117c985#diff-1cda01223f50d4b2b748ef665d422cd9d4972b1843da218ce79e9c21c5e048ddR130

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D91737/new/

https://reviews.llvm.org/D91737