[Lldb-commits] [lldb] [LLDB] Add user documentation for DIL. (PR #228275)

[via lldb-commits]

================
@@ -0,0 +1,315 @@
+# DIL: The Data Inspection Language
+
+This page explains the Data Inspection Language (DIL) in LLDB. Most of this
+document is intended for LLDB users, from both the command line and through IDEs
+(Integrated Development Environments), Data Formatters, and other scripting
+affordances (via the SB APIs). At the end there is some additional information
+for LLDB developers.
+
+## Background: What is DIL and why did we implement it?
+
+In the context of LLDB, an **expression path** is the structured syntax used to
+navigate and access specific fields, members, or elements within a data
+structure starting from a root variable. A **path expression** is an expression
+consisting entirely of such structured syntax (and starting from a root
+variable).
+
+LLDB has always had two modes for accessing values in your program: path
+expressions, that commands like `frame variable` could understand and interpret;
+and "other" expressions, which could contain path expressions, but usually also
+included other pieces written in a source language, and which could be passed to
+the `expression` (aka `expr`) command. These "other" expressions were evaluated
+using a language-accurate parser for that language, and the results were
+obtained by running code in the target program.
+
+LLDB's path expressions, however, are not necessarily a direct representation of
+the type layout of structures in your program. Instead they grew from the
+observation that very often the most useful representation of a value is not the
+underlying layout of the object (particularly for container classes). In fact,
+users who are not data type maintainers are more likely to be confused if shown
+the underlying layout of the object. Users generally want to see the semantic
+meaning of the object, not its implementation.
+
+LLDB solves this problem by using Data Formatters that take the types in the
+type system and produce an alternate layout for the types that correspond to how
+the class is used (what the user really wants to see), not how it is
+implemented. These re-formatted representations include [Synthetic
+Children](https://lldb.llvm.org/use/variable.html#synthetic-children), new
+constructs that are not actually part of the original data type, but which
+faciliate showing users what they expect to see. The path expressions give you
+access to these re-formatted representations. In addition, these re-formatted
+representations allow LLDB to display the dynamic type of an object, not just
+the static type. But these re-formatted representations mean nothing to the
+underlying source language, so you cannot use them in the expression evaluator
+(which is based on the source language). This severely limited the utility of
+the reformatted values, since there was no way to perform logic operations (or
+other simple expression evaluation) on them.
+
+The Data Inspection Language (DIL) was designed to solve this problem. As the
+name implies, the DIL is a language that was created specifically to _improve
+the performance_ and _expand the capabilities_ of program data introspection in
+LLDB, particularly when that introspection requires evaluating simple
+expressions (which is often the case when data formatters and synthetic children
+are involved).
+
+The full expression evaluator uses the compiler's language-accurate parser to
+fully parse source-language expressions, building up ASTs to represent the
+expressions and then JIT'ing code for these ASTs and executing it within the
+current LLDB context, by running code in the target. This is a very flexible
+mechanism, and can have true source language fidelity, but it can also be a bit
+slow (and can also affect the program state).
+
+DIL was explicitly designed for speed. It addresses these issues by avoiding the
+compiler altogether. Instead, it defines its own simple grammar, AST
+representation, lexer, parser and interpreter. By working on the reformatted
+values instead of the raw types, the DIL also allows you to not just view but
+write tests and other simple expressions using the values as they are shown to
+the user.
+
+## The DIL Language
+
+First and foremost, it is important to understand that DIL is **its own
+language**.
+
+While it is very similar to C-based languages, DIL is NOT C, nor C++, nor any
+other specific programming language. Deciding to make DIL its own language, for
+the specific purpose of helping introspect program data in LLDB, allows us to
+both simplify the language (for example, we do not need to support all the
+complexities of C++), and to easily support multiple programming languages -- we
+are free to add features outside of standard C/C++ that might facilitate
+supporting such languages as Rust, Swift or Fortran.
+
+The actual formal definition of DIL can be found in an EBNF file in the LLDB
+source code repository in
+[dil-expr-lang.ebnf](../../dil-expr-lang.ebnf).
+
+
+Here is a quick summary of the types of expressions DIL can support:
+
+- Identifier names, boolean values, numbers, register names, `nullptr`
+- Logical-and, logical-or, logical-not: `&&`, `||`, `!`
+- Bitwise-and, exclusive-or, inclusive-or: `&`, `^`, `|`
+- Relational and equality expressions: `<`, `>`, `>=`, `<=`, `==`, `!=`
+- Shift expressions: `>>`, `<<`
+- Basic arithmetic expressions: `+`, `-`, `*`, `/`, `%`
+- Array/Vector indexing expression: `[]`
+- Member-of, both via `.` and `->` (as appropriate)
+- Pointer dereferencing: `->`, `*`
+- Address-of: `&`
+- Bitwise Not: `~`
+- Type casting, using C-Style syntax, for builtin types, class names, enum names and typedef names
+- Assignment and Composite Assignment: `=`, `+=`, `-=`, `*=`, `/=`, `%=`, `&=`, `|=`, `^=`, `<<=`, `>>=`
+- Parenthesized expressions: `(`, `)`
+- Ternary conditional operator: `?` `:`
+
+NOTE: DIL only allows ONE assignment of any kind in any given expression. This
+is because with multiple assignments it becomes impossible to guarantee a
+deterministic evaluation order, which could have unexpected consequences (think
+about `a=0; a++ + a++ + a++` -- what should that return?).
+
+## CAUTION: DIL vs. Expression Evaluator
+
+As we took pains to point out above, DIL is NOT identical to C/C++. Therefore,
+you cannot count on it behaving identically to C/C++.
+
+DIL, and those places in LLDB where it is used to evaluate expressions, can now
+handle many expressions that only the full expression evaluator could handle in
+the past.
+
+Most of the time, this is not a problem, as the two different execution paths
+will usually return the same value for the same expression. HOWEVER, there are
+cases where they will both return apparently valid but DIFFERENT RESULTS. One
+example of this is in the case of operator overloading. If DIL is used in a
+situation where an operator has an overloaded definition, DIL will
+ignore the overloaded definition, and therefore return a result different from
+what the full expression evaluator would return.
+
+
+Because of this potential for DIL returning a different value than the full
+expression evaluator might, we have been cautious about using the full DIL
+capabilities in those places in LLDB where we now call DIL before falling back
+on the full expression evaluator. For more information on this, see the section
+'Programmer settings and changes for controlling or using DIL' below.
+
+## Where and how DIL is used in LLDB
+
+### `frame variable`, `print`, and `expr` commands
+
+LLDB has three main commands that allow users to look at the values of program
+data, and to evaluate various types of expressions on them. These three commands
+are:
+
+- `frame variable`, aka `frame var` or `v`
+- `dwim-print`, aka `print` or `p`
+- `expression`, aka `expr`
+
+
+Historically `frame variable` was intended to handle path expressions (including
+re-formatted values), and `expression` was intended to handle any other
+expressions users wanted to evaluate. Also `expression` will run code (in the
+target) if it needs to, whereas `frame variable` will not. In general this meant
+that languages that don't support IR interpretations must always run code in the
+target, which will be slow.
+
+
+`dwim-print` was introduced in 2022.  Before that time, `p` was an alias
+for `expr`. The problem was that many LLDB users came to LLDB from GDB, where
+users could use one command either for accessing variable values (expression
+paths) or for evaluating more complex expressions. The single GDB command was
+`print`, usually abbreviated `p`. The result of this was that many LLDB users
+would just use `p` all the time, including times when it wasn't really necessary
+or even appropriate. `dwim-print` was introduced in an attempt to alleviate this
+problem. "dwim" stands for "do-what-I-mean". `dwim-print` looks at the
+expression and attempts to decide whether it `frame variable` would produce the
+same value as the full expression evaluator, and calls the appropriate mechanism
+accordingly. Since `p` was made an alias for `dwim-print`, this went a
+long way towards solving the problem of users running code in the target when
+that was not necessary.
+
+
+After being introduced in 2025, DIL became the default implementation for `frame
+variable` (aka `v`), so now that command is capable of handling many expressions
+that formerly needed to go through the full expression evaluator. This is now
+the recommended way for users to request to see the values of their variables,
+and to evaluate simple expressions on them.
+
+
+For full or complex expressions (e.g. things involving function calls or
+templates), users should still use the full expression evaluator (`expr`).
----------------
jimingham wrote:

might mention "overloaded operators" here since that's a point you've been making throughout.

https://github.com/llvm/llvm-project/pull/228275