[lldb-dev] RFC: Support for unwinding on windows

[Pavel Labath via lldb-dev]

Hello everyone,

after a short recess, I have started to resume work on the breakpad 
symbols project. The project is "nearly" finished, but there is one more 
thing remaining to be done. This is to support stack unwinding for 32 
bit windows targets.

The unwinding info in breakpad is represented differently in breakpad, 
because unwinding on 32bit windows is... different. For example, an 
"unwind plan" for a typical win32 function would be described by a 
string like "$T0 .raSearch = $eip $TO ^ = $esp $TO 4 + =". Now, this 
string is basically a program in a special postfix language, and if you 
look closely, you can recognise fragments like "$eip = deref(T0)", and 
"$esp = $T0 + 4", which look fairly typical. However, there's this 
".raSearch" bit that is unlike anything we have in lldb at the moment.

The way that .raSearch works is that it asks the debugger to look around 
the stack for something that looks like a plausible return address, and 
then return the address of that return address. Now, the reason that 
this works (mostly) reliably is that on windows, the debug info contains 
information about the size of functions' stack frames -- the number of 
bytes taken up by local variables, arguments, etc. Armed with this 
knowledge, the debugger can skip portions of the stack that definitely 
do not hold the return address and can e.g. avoid confusing function 
pointer arguments for the return addresses.

The way I propose to implement this in lldb is:
- SymbolFile: add two new APIs to get the number of stack bytes used by 
a function for locals and the size of function arguments. Two functions 
are needed because (as usual) the function arguments are considered a 
part of the callers stack frame, and so the number of arguments for a 
function is relevant not when unwinding this function, but for the 
unwinding of its caller. SymbolFileBreakpad (and later SymbolFilePDB -- 
Adrian is working on that) will implement these function to return the 
appropriate values.
- UnwindPlan: add a new value (isFoundHeuristically) to the CFA kind 
enumeration. When encountering this value, the unwinder will search the 
stack for a value that looks like a pointer to a known code section. It 
will use the above SymbolFile APIs to skip over uninteresting parts of 
the stack. SymbolFileBreakpad will search for the known usage patterns 
of the .raSearch keyword and generate appropriate unwind plans (i.e. set 
CFA to "isFoundHeuristically", and appropriate rules for the other 
registers)

I've created a couple of patches which demonstrate how this could be 
implemented. The most interesting one is 
<https://reviews.llvm.org/D66638>, which is WIP, but I should be 
sufficient for demonstration purposes. 
<https://reviews.llvm.org/D66634>, and <https://reviews.llvm.org/D66633> 
are also needed for this patch to work, but they are largely 
uninteresting from an general unwind perspective.

Let me know if you have any questions, concerns, suggestions, etc.

Pavel