[Lldb-commits] [lldb] [lldb] Add a compiler/interpreter of LLDB data formatter bytecode to lldb/examples (PR #113398)

[David Spickett via lldb-commits]

================
@@ -0,0 +1,165 @@
+# A bytecode for (LLDB) data formatters
+
+## Background
+
+LLDB provides very rich customization options to display data types (see https://lldb.llvm.org/use/variable.html ). To use custom data formatters, developers typically need to edit the global `~/.lldbinit` file to make sure they are found and loaded. An example for this workflow is the `llvm/utils/lldbDataFormatters.py` script. Because of the manual configuration that is involved, this workflow doesn't scale very well. What would be nice is if developers or library authors could ship ship data formatters with their code and LLDB automatically finds them.
+
+In Swift we added the `DebugDescription` macro (see https://www.swift.org/blog/announcing-swift-6/#debugging ) that translates Swift string interpolation into LLDB summary strings, and puts them into a `.lldbsummaries` section, where LLDB can find them. This works well for simple summaries, but doesn't scale to synthetic child providers or summaries that need to perform some kind of conditional logic or computation. The logical next step would be to store full Python formatters instead of summary strings, but Python code is larger and more importantly it is potentially dangerous to just load an execute untrusted Python code in LLDB.
+
+This document describes a minimal bytecode tailored to running LLDB formatters. It defines a human-readable assembler representation for the language, an efficient binary encoding, a virtual machine for evaluating it, and format for embedding formatters into binary containers.
+
+### Goals
+
+Provide an efficient and secure encoding for data formatters that can be used as a compilation target from user-friendly representations (such as DIL, Swift DebugDescription, or NatVis).
+
+### Non-goals
+
+While humans could write the assembler syntax, making it user-friendly is not a goal.
+
+## Design of the virtual machine
+
+The LLDB formatter virtual machine uses a stack-based bytecode, comparable with DWARF expressions, but with higher-level data types and functions.
+
+The virtual machine has two stacks, a data and a control stack. The control stack is kept separate to make it easier to reason about the security aspects of the VM.
+
+### Data types
+These data types are "host" data types, in LLDB parlance.
+- _String_ (UTF-8)
+- _Int_ (64 bit)
+- _UInt_ (64 bit)
+- _Object_ (Basically an `SBValue`)
+- _Type_ (Basically an `SBType`)
+- _Selector_ (One of the predefine functions)
+
+_Object_ and _Type_ are opaque, they can only be used as a parameters of `call`.
+
+## Instruction set
+
+### Stack operations
+
+These manipulate the data stack directly.
+
+- `dup  (x -> x x)`
+- `drop (x y -> x)`
+- `pick (x ... UInt -> x ... x)`
+- `over (x y -> y)`
+- `swap (x y -> y x)`
+- `rot (x y z -> z x y)`
+
+### Control flow
+
+- `{` pushes a code block address onto the control stack
+- `}` (technically not an opcode) denotes the end of a code block
+- `if` pops a block from the control stack, if the top of the data stack is nonzero, executes it
+- `ifelse` pops two blocks from the control stack, if the top of the data stack is nonzero, executes the first, otherwise the second.
+
+### Literals for basic types
+
+- `123u ( -> UInt)` an unsigned 64-bit host integer.
+- `123 ( -> Int)` a signed 64-bit host integer.
+- `"abc" ( -> String)` a UTF-8 host string.
+- `@strlen ( -> Selector)` one of the predefined functions supported by the VM.
+
+### Arithmetic, logic, and comparison operations
+- `+ (x y -> [x+y])`
+- `-` etc ...
+- `*`
+- `/`
+- `%`
+- `<<`
+- `>>`
+- `shra` (arithmetic shift right)
+- `~`
+- `|`
+- `^`
+- `=`
+- `!=`
+- `<`
+- `>`
+- `=<`
+- `>=`
+
+### Function calls
+
+For security reasons the list of functions callable with `call` is predefined. The supported functions are either existing methods on `SBValue`, or string formatting operations.
+
+- `call (Object arg0 ... Selector -> retval)`
+
+Method is one of a predefined set of _Selectors_
+- `(Object @summary -> String)`
+- `(Object @type_summary -> String)`
+
+- `(Object @get_num_children -> UInt)`
+- `(Object UInt @get_child_at_index -> Object)`
+- `(Object String @get_child_index -> UInt)`
+- `(Object @get_type -> Type)`
+- `(Object UInt @get_template_argument_type -> Type)`
+- `(Object @get_value -> Object)`
+- `(Object @get_value_as_unsigned -> UInt)`
+- `(Object @get_value_as_signed -> Int)`
+- `(Object @get_value_as_address -> UInt)`
+- `(Object Type @cast -> Object)`
+
+- `(UInt @read_memory_byte -> UInt)`
+- `(UInt @read_memory_uint32 -> UInt)`
+- `(UInt @read_memory_int32 -> Int)`
+- `(UInt @read_memory_unsigned -> UInt)`
+- `(UInt @read_memory_signed -> Int)`
+- `(UInt @read_memory_address -> UInt)`
+- `(UInt Type @read_memory -> Object)`
+ 
+- `(String arg0 ... fmt -> String)`
+- `(String arg0 ... sprintf -> String)`
+- `(String strlen -> String)`
+
+## Byte Code
+
+Most instructions are just a single byte opcode. The only exceptions are the literals:
+
+- String: Length in bytes encoded as ULEB128, followed length bytes
+- Int: LEB128
+- UInt: ULEB128
+- Selector: ULEB128
+
+### Embedding
+
+Expression programs are embedded into an `.lldbformatters` section (an evolution of the Swift `.lldbsummaries` section) that is a dictionary of type names/regexes and descriptions. It consists of a list of records. Each record starts with the following header:
+
+- version number (ULEB128)
+- remaining size of the record (minus the header) (ULEB128)
+
+Space between two records may be padded with NULL bytes.
----------------
DavidSpickett wrote:

Does the size of the first record include those null bytes?

Or do we say that the version number will never begin with 0x00 and so you can:
* Read the size of record 1
* Read that many bytes
* Check the next byte
* If it's 0 keep reading forward until it isn't
* If it's non zero treat this as the start of the next record.

I guess that this accounts for linkers preferring to align sections in certain ways. So when those sections get merged later you might have a gap.

https://github.com/llvm/llvm-project/pull/113398