[Lldb-commits] [lldb] [LLDB] Add user documentation for DIL. (PR #228275)

[via lldb-commits]

================
@@ -0,0 +1,278 @@
+# DIL: The Data Inspection Language
+
+This page explains the Data Inspection Language (DIL) in LLDB. Most of this
+document is intended for LLDB users, from both the command line and through
+IDE's (Integrated Development Environments) via lldb-dap. At the end there is
+some additional information for LLDB developers.
+
+## Background: What is DIL and why did we implement it?
+
+As the name implies, the Data Inspection Language is a language that was created
+specifically to improve the stability and performance of program data
+introspection in LLDB, particularly when that introspection requires evaluating
+any simple expressions (which is often the case when data formatters and
+synthetic children are involved).
+
+Prior to the introduction of DIL, nearly all expression evaluations in LLDB had
+to be done through the full expression evaluator. This is a heavy-weight
+mechanism that uses Clang to fully parse C++ expressions, building up Clang ASTs
+to represent the expressions and then evaluating these Clang ASTs within the
+current LLDB context. This approach has historicaly had several drawbacks,
+including being somewhat slow and also sometimes crashing (mostly due to Clang
+being desing to expect complete, correct programs rather than partial contexts
+and expressions).
+
+DIL was designed to addresses these issues by avoiding Clang altogether. Instead
+it defines its own simple grammar, AST representation, lexer, parser and
+interpreter. By bypassing Clang, and only handling comparatively simple
+expressions, DIL can be both faster and more stable than the full expression
+evaluator.
+
+
+## The DIL Language
+
+First and foremost, It is important to understand that DIL is **its own
+language**.
+
+While it is very similar to C-based languages, DIL is NOT C, nor C++, nor any
+other specific programming language. Deciding to make DIL its own language, for
+the specific purpose of helping introspect program data in LLDB, allows us to
+both simplify the language (we do not need to support all the complexities of
+C++, e.g.), and to easily support multiple programmling languages -- we are free
+to add features outside of standard C/C++, that might facilitate supporting Rust
+or Swift or Fortran, e.g.
+
+The actual formal definition of DIL can be found in an EBNF file in the LLDB
+source code repository,
+[here](https://github.com/llvm/llvm-project/blob/main/lldb/docs/dil-expr-lang.ebnf).
+
+
+Here is a  quick summary of the types of expressions DIL can support:
+
+- Identifier names, boolean values, numbers, register names, nullptr
+- Logical-and, logical-or, logical-not:  '&&', '||', '!'
+- Exclusize-and, exclusive-or: '&', '|'
+- Relational and equality epressions: '<', '>', '>=', '<=', '==', '!="
+- Shift expressions: '>>', '<<'
+- Basic arithmetic expressions: '+', '-', '*', '/', '%'
+- Array/Vector indexing expression: '[' <indexing-expression> ']'
+- Member-of, both via '.' and '->' (as appropriate)
+- Pointer dereferencing: '->', '*'
+- Address-of: '&'
+- Binary Not: '~'
+- Type casting, using C-Style syntax, for builtin types, class names, enum names and typedef names
+- Assignment and Composite Assignment: '=', '+=, '-=', '*=', '/=', '%=', '&=', '|=', '<<=', '>>='
+- Parenthesized expressions: '(', ')'
+
+NOTE: DIL only allows ONE assignment of any kind in any given expression. This
+is because, with multiple assignments it becomes impossible to guarantee a
+deterministic evaluation order, which could have unexpected consequeces (think
+about 'a=0; a++ + a++ + a++' -- what should that return?).
+
+## CAUTION: DIL vs. Expression Evaluator
+
+As we took pains to point out above, DIL is NOT identical to C/C++. Therefore
+you cannot count on it behaving identically to C/C++.
+
+DIL, and those places in LLDB where it is used to evaluate expressions, now can
+handle many expressions that only the full expression evaluator could handle in
+the past.
+
+Most of the time, this is not a problem, as the two different execution paths
+will usually return the same value for the same expression. HOWEVER, there are
+cases where they will both return apparently valid but DIFFERENT RESULTS. One
+example of this is in the case of operator overloading. If DIL is used in a
+situation where an operator has an overloaded definition, DIL will probably
+ignore the overloaded definition, and therefore return a result different from
+what you might expect. The full expression evaluator handles operator
+overloading properly.
+
+Because of this potential for DIL returning a different value that the full
+expression evaluator might, we have been cautious about using the full DIL
+capabilities in those places in LLDB where we now call DIL before falling back
+on the full expression evaluator. For more information on this, see the section
+'Programmer settings and changes for controlling or using DIL' below.
+
+## Where and how is DIL used in LLDB
+
+### Historical Background: 'frame variable', 'print', and 'expr' commands
+
+The LLDB interactive commands for examining program variables can be rather
+confusing. Currently there are three main commands that allow users to look at
+the values of program data, and to evaluate various types of expressions on
+them.  These three commands are:
+
+- 'frame variable', aka 'frame var' or 'v'
+- 'dwim-print', aka 'print' or 'p'
+- 'expr'
+
+As if that were not confusing enough, 'dwim-print' did not exist
+until 2022. Before that, 'print' and 'p' were abbreviations for 'expr'.
+
+So what do these various commands do, and how did we get into this confusing
+mess?
+
+Originally 'frame variable' was meant to allow only plain access to
+variables. It recognized and handled a few basic operators as part of this:
+Address-of, pointer dereferencing, finding fields/members, and vector/array
+indexing. Any more complicated expression evaluation was meant to go through the
+expression evaluator. The expectation was that users woukd use 'v' for simple
+accesses and 'p' for more complex evaulations.
+
+The problem was that nearly all the users to came to LLDB were coming from using
----------------
jimingham wrote:

That's not true, gdb actually had exactly the same setup, with its pseudo-expression evaluator and `info var` which was a reduced-functionality version of `frame var`.  gdb has also over time grown a re-presentation system like lldb's, but because it's expression evaluator is not language accurate, but rather a hand-grown thing, they could incorporate their re-presentations in the expression language.  

We decided that having a language accurate expression evaluator was valuable, and so we needed to introduce a way to act on the value representations using our own separate hand-grown thing - the DIL...

https://github.com/llvm/llvm-project/pull/228275