[clang] [docs] Rewrite selected Clang docs to Markdown (PR #209283)
Reid Kleckner via cfe-commits
cfe-commits at lists.llvm.org
Wed Jul 15 13:51:45 PDT 2026
https://github.com/rnk updated https://github.com/llvm/llvm-project/pull/209283
>From 4241fda8df078ddc81e4fd3c0ada9ac8f0115c04 Mon Sep 17 00:00:00 2001
From: Reid Kleckner <rkleckner at nvidia.com>
Date: Mon, 13 Jul 2026 19:05:57 +0000
Subject: [PATCH 1/2] [docs] Convert selected rst docs with rst2myst
---
clang/docs/AMDGPUSupport.md | 45 +-
clang/docs/APINotes.md | 77 +-
clang/docs/AllocToken.md | 260 ++--
clang/docs/BoundsSafety.md | 1160 +++++++-------
clang/docs/BoundsSafetyAdoptionGuide.md | 104 +-
clang/docs/BoundsSafetyImplPlans.md | 213 ++-
clang/docs/CIR/index.md | 30 +-
clang/docs/CXXTypeAwareAllocators.md | 135 +-
clang/docs/ClangPlugins.md | 270 ++--
clang/docs/CommandGuide/index.md | 20 +-
clang/docs/ControlFlowIntegrity.md | 324 ++--
clang/docs/DebuggingCoroutines.md | 1837 +++++++++++------------
clang/docs/ExternalClangExamples.md | 179 ++-
clang/docs/FAQ.md | 81 +-
clang/docs/IntroductionToTheClangAST.md | 139 +-
clang/docs/LibClang.md | 661 ++++----
clang/docs/LibFormat.md | 88 +-
clang/docs/LibTooling.md | 290 ++--
clang/docs/RISCVSupport.md | 41 +-
clang/docs/Tooling.md | 87 +-
20 files changed, 2922 insertions(+), 3119 deletions(-)
diff --git a/clang/docs/AMDGPUSupport.md b/clang/docs/AMDGPUSupport.md
index 8ca537fa5d729..21180587dc065 100644
--- a/clang/docs/AMDGPUSupport.md
+++ b/clang/docs/AMDGPUSupport.md
@@ -1,29 +1,34 @@
-.. raw:: html
-
- <style type="text/css">
- .none { background-color: #FFCCCC }
- .part { background-color: #FFFF99 }
- .good { background-color: #CCFF99 }
- </style>
+```{raw} html
+<style type="text/css">
+ .none { background-color: #FFCCCC }
+ .part { background-color: #FFFF99 }
+ .good { background-color: #CCFF99 }
+</style>
+```
+```{eval-rst}
.. role:: none
+```
+
+```{eval-rst}
.. role:: part
+```
+
+```{eval-rst}
.. role:: good
+```
-.. contents::
- :local:
+```{contents}
+:local: true
+```
-==============
-AMDGPU Support
-==============
+# AMDGPU Support
Clang supports OpenCL, HIP and OpenMP on AMD GPU targets.
+## Predefined Macros
-Predefined Macros
-=================
-
-
+```{eval-rst}
.. list-table::
:header-rows: 1
@@ -55,12 +60,12 @@ Predefined Macros
- Defined if LDEXPF instruction is available (deprecated).
* - ``__HAS_FP64__``
- Defined if FP64 instruction is available (deprecated).
+```
Please note that the specific architecture and feature names will vary depending on the GPU. Also, some macros are deprecated and may be removed in future releases.
-
-Target-Specific Builtins
-========================
+## Target-Specific Builtins
Clang exposes AMDGPU hardware intrinsics as target-specific builtins with the
-``__builtin_amdgcn_`` prefix. These are documented in :doc:`AMDGPUBuiltinReference`.
+`__builtin_amdgcn_` prefix. These are documented in {doc}`AMDGPUBuiltinReference`.
+
diff --git a/clang/docs/APINotes.md b/clang/docs/APINotes.md
index 8a1018011d023..68857cbdbd2fd 100644
--- a/clang/docs/APINotes.md
+++ b/clang/docs/APINotes.md
@@ -1,6 +1,4 @@
-================================================
-API Notes: Annotations Without Modifying Headers
-================================================
+# API Notes: Annotations Without Modifying Headers
**The Problem:** You have headers you want to use, but you also want to add
extra information to the API. You don't want to put that information in the
@@ -11,9 +9,9 @@ want to modify them at all.
**Incomplete solution:** Redeclare all the interesting parts of the API in your
own header and add the attributes you want. Unfortunately, this:
-* doesn't work with attributes that must be present on a definition
-* doesn't allow changing the definition in other ways
-* requires your header to be included in any client code to take effect
+- doesn't work with attributes that must be present on a definition
+- doesn't allow changing the definition in other ways
+- requires your header to be included in any client code to take effect
**Better solution:** Provide a "sidecar" file with the information you want to
add, and have that automatically get picked up by the module-building logic in
@@ -22,14 +20,11 @@ the compiler.
That's API notes.
API notes use a YAML-based file format. YAML is a format best explained by
-example, so here is a `small example
-<https://github.com/llvm/llvm-project/blob/main/clang/test/APINotes/Inputs/Frameworks/SomeKit.framework/Headers/SomeKit.apinotes>`_
+example, so here is a [small example](https://github.com/llvm/llvm-project/blob/main/clang/test/APINotes/Inputs/Frameworks/SomeKit.framework/Headers/SomeKit.apinotes)
from the compiler test suite of API
notes for a hypothetical "SomeKit" framework.
-
-Usage
-=====
+## Usage
API notes files are found relative to the module map that defines a module,
under the name "SomeKit.apinotes" for a module named "SomeKit". Additionally, a
@@ -39,46 +34,32 @@ directory as the corresponding module map; for framework modules, they should
be placed in the Headers and PrivateHeaders directories, respectively. The
module map for a private top-level framework module should be placed in the
PrivateHeaders directory as well, though it does not need an additional
-"_private" suffix on its name.
+"\_private" suffix on its name.
Clang will search for API notes files next to module maps only when passed the
-``-fapinotes-modules`` option.
-
+`-fapinotes-modules` option.
-Limitations
-===========
+## Limitations
- Since they're identified by module name, API notes cannot be used to modify
arbitrary textual headers.
-
-"Versioned" API Notes
-=====================
+## "Versioned" API Notes
Many API notes affect how a C API is imported into Swift. In order to change
that behavior while still remaining backwards-compatible, API notes can be
selectively applied based on the Swift compatibility version provided to the
-compiler (e.g. ``-fapi-notes-swift-version=5``). The rule is that an
+compiler (e.g. `-fapi-notes-swift-version=5`). The rule is that an
explicitly-versioned API note applies to that version *and all earlier
versions,* and any applicable explicitly-versioned API note takes precedence
over an unversioned API note.
-
-Reference
-=========
+## Reference
An API notes file contains a YAML dictionary with the following top-level
entries:
-:Name:
-
- The name of the module (the framework name, for frameworks). Note that this
- is always the name of a top-level module, even within a private API notes
- file.
-
- ::
-
- Name: MyFramework
+```{eval-rst}
:Classes, Protocols, Tags, Typedefs, Globals, Enumerators, Functions, Namespaces:
@@ -117,15 +98,12 @@ Each entry under 'Classes' and 'Protocols' can contain "Methods" and
Identified by 'Selector' and 'MethodKind'; the MethodKind is either
"Instance" or "Class".
+```
- ::
+Each entry under 'Classes' and 'Protocols' can contain "Methods" and
+"Properties" arrays, in addition to the attributes described below:
- Classes:
- - Name: UIViewController
- Methods:
- - Selector: "presentViewController:animated:"
- MethodKind: Instance
- …
+```{eval-rst}
:Properties:
@@ -153,9 +131,13 @@ arrays. Methods under Tags are C++ methods identified by 'Name' (rather than
Tags:
- Name: MyClass
- Methods:
- - Name: doSomething
- …
+```
+
+Each entry under "Tags" can contain "Methods", "Fields", and nested "Tags"
+arrays. Methods under Tags are C++ methods identified by 'Name' (rather than
+'Selector' and 'MethodKind' as used for Objective-C methods).
+
+```{eval-rst}
:Fields:
@@ -191,13 +173,12 @@ declaration kind), all of which are optional:
with all arguments. Use "_" to omit an argument label.
::
+```
- - Selector: "presentViewController:animated:"
- MethodKind: Instance
- SwiftName: "present(_:animated:)"
+Each declaration supports the following annotations (if relevant to that
+declaration kind), all of which are optional:
- - Class: NSBundle
- SwiftName: Bundle
+```{eval-rst}
:SwiftImportAs:
@@ -513,3 +494,5 @@ declaration kind), all of which are optional:
- Selector: "initWithFrame:"
MethodKind: Instance
DesignatedInit: true
+```
+
diff --git a/clang/docs/AllocToken.md b/clang/docs/AllocToken.md
index a7d8043734ee6..a26bce344b4dc 100644
--- a/clang/docs/AllocToken.md
+++ b/clang/docs/AllocToken.md
@@ -1,12 +1,10 @@
-=================
-Allocation Tokens
-=================
+# Allocation Tokens
-.. contents::
- :local:
+```{contents}
+:local: true
+```
-Introduction
-============
+## Introduction
Clang provides support for allocation tokens to enable allocator-level heap
organization strategies. Clang assigns mode-dependent token IDs to allocation
@@ -15,55 +13,49 @@ compatible memory allocator.
Possible allocator strategies include:
-* **Security Hardening**: Placing allocations into separate, isolated heap
+- **Security Hardening**: Placing allocations into separate, isolated heap
partitions. For example, separating pointer-containing types from raw data
can mitigate exploits that rely on overflowing a primitive buffer to corrupt
object metadata.
-
-* **Memory Layout Optimization**: Grouping related allocations to improve data
+- **Memory Layout Optimization**: Grouping related allocations to improve data
locality and cache utilization.
-
-* **Custom Allocation Policies**: Applying different management strategies to
+- **Custom Allocation Policies**: Applying different management strategies to
different partitions.
-Token Assignment Mode
-=====================
+## Token Assignment Mode
The default mode to calculate tokens is:
-* ``typehashpointersplit``: This mode assigns a token ID based on the hash of
+- `typehashpointersplit`: This mode assigns a token ID based on the hash of
the allocated type's name, where the top half ID-space is reserved for types
that contain pointers and the bottom half for types that do not contain
pointers.
Other token ID assignment modes are supported, but they may be subject to
-change or removal. These may (experimentally) be selected with ``-Xclang
--falloc-token-mode=<mode>``:
+change or removal. These may (experimentally) be selected with `-Xclang
+-falloc-token-mode=<mode>`:
-* ``typehash``: This mode assigns a token ID based on the hash of the allocated
+- `typehash`: This mode assigns a token ID based on the hash of the allocated
type's name.
-
-* ``random``: This mode assigns a statically-determined random token ID to each
+- `random`: This mode assigns a statically-determined random token ID to each
allocation site.
-
-* ``increment``: This mode assigns a simple, incrementally increasing token ID
+- `increment`: This mode assigns a simple, incrementally increasing token ID
to each allocation site.
The following command-line options affect generated token IDs:
-* ``-falloc-token-max=<N>``
- Configures the maximum number of token IDs. By default the number of tokens
- is bounded by ``SIZE_MAX``.
+- `-falloc-token-max=<N>`
+ : Configures the maximum number of token IDs. By default the number of tokens
+ is bounded by `SIZE_MAX`.
-Querying Token IDs with ``__builtin_infer_alloc_token``
-=======================================================
+## Querying Token IDs with `__builtin_infer_alloc_token`
For use cases where the token ID must be known at compile time, Clang provides
a builtin function:
-.. code-block:: c
-
- size_t __builtin_infer_alloc_token(<args>, ...);
+```c
+size_t __builtin_infer_alloc_token(<args>, ...);
+```
This builtin returns the token ID inferred from its argument expressions, which
mirror arguments normally passed to any allocation function. The argument
@@ -72,168 +64,158 @@ have side effects without any runtime impact.
For example, it can be used as follows:
-.. code-block:: c
+```c
+struct MyType { ... };
+void *__partition_alloc(size_t size, size_t partition);
+#define partition_alloc(...) __partition_alloc(__VA_ARGS__, __builtin_infer_alloc_token(__VA_ARGS__))
- struct MyType { ... };
- void *__partition_alloc(size_t size, size_t partition);
- #define partition_alloc(...) __partition_alloc(__VA_ARGS__, __builtin_infer_alloc_token(__VA_ARGS__))
+void foo(void) {
+ MyType *x = partition_alloc(sizeof(*x));
+}
+```
- void foo(void) {
- MyType *x = partition_alloc(sizeof(*x));
- }
-
-Allocation Token Instrumentation
-================================
+## Allocation Token Instrumentation
To enable instrumentation of allocation functions, code can be compiled with
-the ``-fsanitize=alloc-token`` flag:
-
-.. code-block:: console
+the `-fsanitize=alloc-token` flag:
- % clang++ -fsanitize=alloc-token example.cc
+```console
+% clang++ -fsanitize=alloc-token example.cc
+```
The instrumentation transforms allocation calls to include a token ID. For
example:
-.. code-block:: c
+```c
+// Original:
+ptr = malloc(size);
- // Original:
- ptr = malloc(size);
+// Instrumented:
+ptr = __alloc_token_malloc(size, <token id>);
+```
- // Instrumented:
- ptr = __alloc_token_malloc(size, <token id>);
-
-Runtime Interface
------------------
+### Runtime Interface
A compatible runtime must be provided that implements the token-enabled
allocation functions. The instrumentation generates calls to functions that
-take a final ``size_t token_id`` argument.
-
-.. code-block:: c
-
- // C standard library functions
- void *__alloc_token_malloc(size_t size, size_t token_id);
- void *__alloc_token_calloc(size_t count, size_t size, size_t token_id);
- void *__alloc_token_realloc(void *ptr, size_t size, size_t token_id);
- // ...
-
- // C++ operators (mangled names)
- // operator new(size_t, size_t)
- void *__alloc_token__Znwm(size_t size, size_t token_id);
- // operator new[](size_t, size_t)
- void *__alloc_token__Znam(size_t size, size_t token_id);
- // ... other variants like nothrow, etc., are also instrumented.
-
-Fast ABI
---------
-
-An alternative ABI can be enabled with ``-fsanitize-alloc-token-fast-abi``,
+take a final `size_t token_id` argument.
+
+```c
+// C standard library functions
+void *__alloc_token_malloc(size_t size, size_t token_id);
+void *__alloc_token_calloc(size_t count, size_t size, size_t token_id);
+void *__alloc_token_realloc(void *ptr, size_t size, size_t token_id);
+// ...
+
+// C++ operators (mangled names)
+// operator new(size_t, size_t)
+void *__alloc_token__Znwm(size_t size, size_t token_id);
+// operator new[](size_t, size_t)
+void *__alloc_token__Znam(size_t size, size_t token_id);
+// ... other variants like nothrow, etc., are also instrumented.
+```
+
+### Fast ABI
+
+An alternative ABI can be enabled with `-fsanitize-alloc-token-fast-abi`,
which encodes the token ID in the allocation function name.
-.. code-block:: c
-
- void *__alloc_token_0_malloc(size_t size);
- void *__alloc_token_1_malloc(size_t size);
- void *__alloc_token_2_malloc(size_t size);
- ...
- void *__alloc_token_0_Znwm(size_t size);
- void *__alloc_token_1_Znwm(size_t size);
- void *__alloc_token_2_Znwm(size_t size);
- ...
+```c
+void *__alloc_token_0_malloc(size_t size);
+void *__alloc_token_1_malloc(size_t size);
+void *__alloc_token_2_malloc(size_t size);
+...
+void *__alloc_token_0_Znwm(size_t size);
+void *__alloc_token_1_Znwm(size_t size);
+void *__alloc_token_2_Znwm(size_t size);
+...
+```
This ABI provides a more efficient alternative where
-``-falloc-token-max`` is small.
+`-falloc-token-max` is small.
-Instrumenting Non-Standard Allocation Functions
------------------------------------------------
+### Instrumenting Non-Standard Allocation Functions
By default, AllocToken only instruments standard library allocation functions.
This simplifies adoption, as a compatible allocator only needs to provide
token-enabled variants for a well-defined set of standard functions.
To extend instrumentation to custom allocation functions, enable broader
-coverage with ``-fsanitize-alloc-token-extended``. Such functions require being
-marked with the `malloc
-<https://clang.llvm.org/docs/AttributeReference.html#malloc>`_ or `alloc_size
-<https://clang.llvm.org/docs/AttributeReference.html#alloc-size>`_ attributes
+coverage with `-fsanitize-alloc-token-extended`. Such functions require being
+marked with the [malloc](https://clang.llvm.org/docs/AttributeReference.html#malloc) or [alloc_size](https://clang.llvm.org/docs/AttributeReference.html#alloc-size) attributes
(or a combination).
For example:
-.. code-block:: c
+```c
+void *custom_malloc(size_t size) __attribute__((malloc));
+void *my_malloc(size_t size) __attribute__((alloc_size(1)));
- void *custom_malloc(size_t size) __attribute__((malloc));
- void *my_malloc(size_t size) __attribute__((alloc_size(1)));
+// Original:
+ptr1 = custom_malloc(size);
+ptr2 = my_malloc(size);
- // Original:
- ptr1 = custom_malloc(size);
- ptr2 = my_malloc(size);
+// Instrumented:
+ptr1 = __alloc_token_custom_malloc(size, token_id);
+ptr2 = __alloc_token_my_malloc(size, token_id);
+```
- // Instrumented:
- ptr1 = __alloc_token_custom_malloc(size, token_id);
- ptr2 = __alloc_token_my_malloc(size, token_id);
-
-Note: Even in the default mode (without ``-fsanitize-alloc-token-extended``),
-an *inline* allocation wrapper marked with the `malloc
-<https://clang.llvm.org/docs/AttributeReference.html#malloc>`_ or `alloc_size
-<https://clang.llvm.org/docs/AttributeReference.html#alloc-size>`_ attribute is
+Note: Even in the default mode (without `-fsanitize-alloc-token-extended`),
+an *inline* allocation wrapper marked with the [malloc](https://clang.llvm.org/docs/AttributeReference.html#malloc) or [alloc_size](https://clang.llvm.org/docs/AttributeReference.html#alloc-size) attribute is
supported if it is inlined into its caller: the inferred token is propagated to
the allocation call the wrapper returns, which is then instrumented normally.
Wrappers that are not inlined still require
-``-fsanitize-alloc-token-extended``.
+`-fsanitize-alloc-token-extended`.
-Disabling Instrumentation
--------------------------
+### Disabling Instrumentation
To exclude specific functions from instrumentation, you can use the
-``no_sanitize("alloc-token")`` attribute:
-
-.. code-block:: c
+`no_sanitize("alloc-token")` attribute:
- __attribute__((no_sanitize("alloc-token")))
- void* custom_allocator(size_t size) {
- return malloc(size); // Uses original malloc
- }
+```c
+__attribute__((no_sanitize("alloc-token")))
+void* custom_allocator(size_t size) {
+ return malloc(size); // Uses original malloc
+}
+```
Note: Independent of any given allocator support, the instrumentation aims to
-remain performance neutral. As such, ``no_sanitize("alloc-token")``
+remain performance neutral. As such, `no_sanitize("alloc-token")`
functions may be inlined into instrumented functions and vice-versa. If
correctness is affected, such functions should explicitly be marked
-``noinline``.
+`noinline`.
-The ``__attribute__((disable_sanitizer_instrumentation))`` is also supported to
+The `__attribute__((disable_sanitizer_instrumentation))` is also supported to
disable this and other sanitizer instrumentations.
-Suppressions File (Ignorelist)
-------------------------------
+### Suppressions File (Ignorelist)
-AllocToken respects the ``src`` and ``fun`` entity types in the
-:doc:`SanitizerSpecialCaseList`, which can be used to omit specified source
+AllocToken respects the `src` and `fun` entity types in the
+{doc}`SanitizerSpecialCaseList`, which can be used to omit specified source
files or functions from instrumentation.
-.. code-block:: bash
-
- [alloc-token]
- # Exclude specific source files
- src:third_party/allocator.c
- # Exclude function name patterns
- fun:*custom_malloc*
- fun:LowLevel::*
-
-.. code-block:: console
+```bash
+[alloc-token]
+# Exclude specific source files
+src:third_party/allocator.c
+# Exclude function name patterns
+fun:*custom_malloc*
+fun:LowLevel::*
+```
- % clang++ -fsanitize=alloc-token -fsanitize-ignorelist=my_ignorelist.txt example.cc
+```console
+% clang++ -fsanitize=alloc-token -fsanitize-ignorelist=my_ignorelist.txt example.cc
+```
-Conditional Compilation with ``__SANITIZE_ALLOC_TOKEN__``
------------------------------------------------------------
+### Conditional Compilation with `__SANITIZE_ALLOC_TOKEN__`
In some cases, one may need to execute different code depending on whether
-AllocToken instrumentation is enabled. The ``__SANITIZE_ALLOC_TOKEN__`` macro
+AllocToken instrumentation is enabled. The `__SANITIZE_ALLOC_TOKEN__` macro
can be used for this purpose.
-.. code-block:: c
+```c
+#ifdef __SANITIZE_ALLOC_TOKEN__
+// Code specific to -fsanitize=alloc-token builds
+#endif
+```
- #ifdef __SANITIZE_ALLOC_TOKEN__
- // Code specific to -fsanitize=alloc-token builds
- #endif
diff --git a/clang/docs/BoundsSafety.md b/clang/docs/BoundsSafety.md
index b82e68eb7555a..ef1bb190668bf 100644
--- a/clang/docs/BoundsSafety.md
+++ b/clang/docs/BoundsSafety.md
@@ -1,29 +1,27 @@
-==================================================
-``-fbounds-safety``: Enforcing bounds safety for C
-==================================================
+# `-fbounds-safety`: Enforcing bounds safety for C
-.. contents::
- :local:
+```{contents}
+:local: true
+```
-Overview
-========
+## Overview
**NOTE:** This is a design document and the feature is not available for users yet.
-Please see :doc:`BoundsSafetyImplPlans` for more details.
+Please see {doc}`BoundsSafetyImplPlans` for more details.
-``-fbounds-safety`` is a C extension to enforce bounds safety to prevent
+`-fbounds-safety` is a C extension to enforce bounds safety to prevent
out-of-bounds (OOB) memory accesses, which remain a major source of security
-vulnerabilities in C. ``-fbounds-safety`` aims to eliminate this class of bugs
+vulnerabilities in C. `-fbounds-safety` aims to eliminate this class of bugs
by turning OOB accesses into deterministic traps.
-The ``-fbounds-safety`` extension offers bounds annotations that programmers can
+The `-fbounds-safety` extension offers bounds annotations that programmers can
use to attach bounds to pointers. For example, programmers can add the
-``__counted_by(N)`` annotation to parameter ``ptr``, indicating that the pointer
-has ``N`` valid elements:
+`__counted_by(N)` annotation to parameter `ptr`, indicating that the pointer
+has `N` valid elements:
-.. code-block:: c
-
- void foo(int *__counted_by(N) ptr, size_t N);
+```c
+void foo(int *__counted_by(N) ptr, size_t N);
+```
Using this bounds information, the compiler inserts bounds checks on every
pointer dereference, ensuring that the program does not access memory outside
@@ -31,14 +29,14 @@ the specified bounds. The compiler requires programmers to provide enough bounds
information so that the accesses can be checked at either run time or compile
time — and it rejects code if it cannot.
-The most important contribution of ``-fbounds-safety`` is how it reduces the
+The most important contribution of `-fbounds-safety` is how it reduces the
programmer's annotation burden by reconciling bounds annotations at ABI
boundaries with the use of implicit wide pointers (a.k.a. "fat" pointers) that
carry bounds information on local variables without the need for annotations. We
designed this model so that it preserves ABI compatibility with C while
minimizing adoption effort.
-The ``-fbounds-safety`` extension has been adopted on millions of lines of
+The `-fbounds-safety` extension has been adopted on millions of lines of
production C code and proven to work in a consumer operating system setting. The
extension was designed to enable incremental adoption — a key requirement in
real-world settings where modifying an entire project and its dependencies all
@@ -46,71 +44,69 @@ at once is often not possible. It also addresses multiple of other practical
challenges that have made existing approaches to safer C dialects difficult to
adopt, offering these properties that make it widely adoptable in practice:
-* It is designed to preserve the Application Binary Interface (ABI).
-* It interoperates well with plain C code.
-* It can be adopted partially and incrementally while still providing safety
+- It is designed to preserve the Application Binary Interface (ABI).
+- It interoperates well with plain C code.
+- It can be adopted partially and incrementally while still providing safety
benefits.
-* It is a conforming extension to C.
-* Consequently, source code that adopts the extension can continue to be
+- It is a conforming extension to C.
+- Consequently, source code that adopts the extension can continue to be
compiled by toolchains that do not support the extension (CAVEAT: this still
requires inclusion of a header file macro-defining bounds annotations to
empty).
-* It has a relatively low adoption cost.
+- It has a relatively low adoption cost.
-This document discusses the key designs of ``-fbounds-safety``. The document is
+This document discusses the key designs of `-fbounds-safety`. The document is
subject to active updates with a more detailed specification.
-Programming Model
-=================
+## Programming Model
-Overview
---------
+### Overview
-``-fbounds-safety`` ensures that pointers are not used to access memory beyond
+`-fbounds-safety` ensures that pointers are not used to access memory beyond
their bounds by performing bounds checking. If a bounds check fails, the program
will deterministically trap before out-of-bounds memory is accessed.
In our model, every pointer has an explicit or implicit bounds attribute that
determines its bounds and ensures guaranteed bounds checking. Consider the
-example below where the ``__counted_by(count)`` annotation indicates that
-parameter ``p`` points to a buffer of integers containing ``count`` elements. An
-off-by-one error is present in the loop condition, leading to ``p[i]`` being
+example below where the `__counted_by(count)` annotation indicates that
+parameter `p` points to a buffer of integers containing `count` elements. An
+off-by-one error is present in the loop condition, leading to `p[i]` being
out-of-bounds access during the loop's final iteration. The compiler inserts a
-bounds check before ``p`` is dereferenced to ensure that the access remains
+bounds check before `p` is dereferenced to ensure that the access remains
within the specified bounds.
-.. code-block:: c
-
- void fill_array_with_indices(int *__counted_by(count) p, unsigned count) {
- // off-by-one error (i < count)
- for (unsigned i = 0; i <= count; ++i) {
- // bounds check inserted:
- // if (i >= count) trap();
- p[i] = i;
- }
+```c
+void fill_array_with_indices(int *__counted_by(count) p, unsigned count) {
+ // off-by-one error (i < count)
+ for (unsigned i = 0; i <= count; ++i) {
+ // bounds check inserted:
+ // if (i >= count) trap();
+ p[i] = i;
}
+}
+```
A bounds annotation defines an invariant for the pointer type, and the model
-ensures that this invariant remains true. In the example below, pointer ``p``
-annotated with ``__counted_by(count)`` must always point to a memory buffer
-containing at least ``count`` elements of the pointee type. Changing the value
-of ``count``, like in the example below, may violate this invariant and permit
+ensures that this invariant remains true. In the example below, pointer `p`
+annotated with `__counted_by(count)` must always point to a memory buffer
+containing at least `count` elements of the pointee type. Changing the value
+of `count`, like in the example below, may violate this invariant and permit
out-of-bounds access to the pointer. To avoid this, the compiler employs
compile-time restrictions and emits run-time checks as necessary to ensure the
new count value doesn't exceed the actual length of the buffer. Section
-`Maintaining correctness of bounds annotations`_ provides more details about
+[Maintaining correctness of bounds annotations] provides more details about
this programming model.
-.. code-block:: c
+```c
+int g;
- int g;
-
- void foo(int *__counted_by(count) p, size_t count) {
- count++; // may violate the invariant of __counted_by
- count--; // may violate the invariant of __counted_by if count was 0.
- count = g; // may violate the invariant of __counted_by
- // depending on the value of `g`.
- }
+void foo(int *__counted_by(count) p, size_t count) {
+ count++; // may violate the invariant of __counted_by
+ count--; // may violate the invariant of __counted_by if count was 0.
+ count = g; // may violate the invariant of __counted_by
+ // depending on the value of `g`.
+}
+```
The requirement to annotate all pointers with explicit bounds information could
present a significant adoption burden. To tackle this issue, the model
@@ -123,17 +119,17 @@ which may result in incompatibilities with the application binary interface
(ABI). Breaking the ABI complicates interoperability with external code that has
not adopted the same programming model.
-``-fbounds-safety`` harmonizes the wide pointer and the bounds annotation
+`-fbounds-safety` harmonizes the wide pointer and the bounds annotation
approaches to reduce the adoption burden while maintaining the ABI. In this
model, local variables of pointer type are implicitly treated as wide pointers,
allowing them to carry bounds information without requiring explicit bounds
annotations. Please note that this approach doesn't apply to function parameters
which are considered ABI-visible. As local variables are typically hidden from
the ABI, this approach has a marginal impact on it. In addition,
-``-fbounds-safety`` employs compile-time restrictions to prevent implicit wide
-pointers from silently breaking the ABI (see `ABI implications of default bounds
-annotations`_). Pointers associated with any other variables, including function
-parameters, are treated as single object pointers (i.e., ``__single``), ensuring
+`-fbounds-safety` employs compile-time restrictions to prevent implicit wide
+pointers from silently breaking the ABI (see [ABI implications of default bounds
+annotations][abi implications of default bounds annotations]). Pointers associated with any other variables, including function
+parameters, are treated as single object pointers (i.e., `__single`), ensuring
that they always have the tightest bounds by default and offering a strong
bounds safety guarantee.
@@ -144,88 +140,84 @@ programming model, reducing the adoption burden.
The rest of the section will discuss individual bounds annotations and the
programming model in more detail.
-Bounds annotations
-------------------
+### Bounds annotations
-Annotation for pointers to a single object
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+#### Annotation for pointers to a single object
The C language allows pointer arithmetic on arbitrary pointers and this has been
a source of many bounds safety issues. In practice, many pointers are merely
pointing to a single object and incrementing or decrementing such a pointer
immediately makes the pointer go out-of-bounds. To prevent this unsafety,
-``-fbounds-safety`` provides the annotation ``__single`` that causes pointer
+`-fbounds-safety` provides the annotation `__single` that causes pointer
arithmetic on annotated pointers to be a compile time error.
-* ``__single`` : indicates that the pointer is either pointing to a single
- object or null. Hence, pointers with ``__single`` do not permit pointer
+- `__single` : indicates that the pointer is either pointing to a single
+ object or null. Hence, pointers with `__single` do not permit pointer
arithmetic nor being subscripted with a non-zero index. Dereferencing a
- ``__single`` pointer is allowed but it requires a null check. Upper and lower
- bounds checks are not required because the ``__single`` pointer should point
+ `__single` pointer is allowed but it requires a null check. Upper and lower
+ bounds checks are not required because the `__single` pointer should point
to a valid object unless it's null.
-``__single`` is the default annotation for ABI-visible pointers. This
+`__single` is the default annotation for ABI-visible pointers. This
gives strong security guarantees in that these pointers cannot be incremented or
decremented unless they have an explicit, overriding bounds annotation that can
be used to verify the safety of the operation. The compiler issues an error when
-a ``__single`` pointer is utilized for pointer arithmetic or array access, as
+a `__single` pointer is utilized for pointer arithmetic or array access, as
these operations would immediately cause the pointer to exceed its bounds.
Consequently, this prompts programmers to provide sufficient bounds information
to pointers. In the following example, the pointer on parameter p is
single-by-default, and is employed for array access. As a result, the compiler
-generates an error suggesting to add ``__counted_by`` to the pointer.
+generates an error suggesting to add `__counted_by` to the pointer.
-.. code-block:: c
-
- void fill_array_with_indices(int *p, unsigned count) {
- for (unsigned i = 0; i < count; ++i) {
- p[i] = i; // error
- }
+```c
+void fill_array_with_indices(int *p, unsigned count) {
+ for (unsigned i = 0; i < count; ++i) {
+ p[i] = i; // error
}
+}
+```
-
-External bounds annotations
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
+#### External bounds annotations
"External" bounds annotations provide a way to express a relationship between a
pointer variable and another variable (or expression) containing the bounds
-information of the pointer. In the following example, ``__counted_by(count)``
+information of the pointer. In the following example, `__counted_by(count)`
annotation expresses the bounds of parameter p using another parameter count.
This model works naturally with many C interfaces and structs because the bounds
of a pointer is often available adjacent to the pointer itself, e.g., at another
parameter of the same function prototype, or at another field of the same struct
declaration.
-.. code-block:: c
-
- void fill_array_with_indices(int *__counted_by(count) p, size_t count) {
- // off-by-one error
- for (size_t i = 0; i <= count; ++i)
- p[i] = i;
- }
+```c
+void fill_array_with_indices(int *__counted_by(count) p, size_t count) {
+ // off-by-one error
+ for (size_t i = 0; i <= count; ++i)
+ p[i] = i;
+}
+```
-External bounds annotations include ``__counted_by``, ``__sized_by``, and
-``__ended_by``. These annotations do not change the pointer representation,
+External bounds annotations include `__counted_by`, `__sized_by`, and
+`__ended_by`. These annotations do not change the pointer representation,
meaning they do not have ABI implications.
-* ``__counted_by(N)`` : The pointer points to memory that contains ``N``
- elements of pointee type. ``N`` is an expression of integer type which can be
+- `__counted_by(N)` : The pointer points to memory that contains `N`
+ elements of pointee type. `N` is an expression of integer type which can be
a simple reference to declaration, a constant including calls to constant
functions, or an arithmetic expression that does not have side effect. The
- ``__counted_by`` annotation cannot apply to pointers to incomplete types or
- types without size such as ``void *``. Instead, ``__sized_by`` can be used to
+ `__counted_by` annotation cannot apply to pointers to incomplete types or
+ types without size such as `void *`. Instead, `__sized_by` can be used to
describe the byte count.
-* ``__sized_by(N)`` : The pointer points to memory that contains ``N`` bytes.
- Just like the argument of ``__counted_by``, ``N`` is an expression of integer
+- `__sized_by(N)` : The pointer points to memory that contains `N` bytes.
+ Just like the argument of `__counted_by`, `N` is an expression of integer
type which can be a constant, a simple reference to a declaration, or an
arithmetic expression that does not have side effects. This is mainly used for
- pointers to incomplete types or types without size such as ``void *``.
-* ``__ended_by(P)`` : The pointer has the upper bound of value ``P``, which is
+ pointers to incomplete types or types without size such as `void *`.
+- `__ended_by(P)` : The pointer has the upper bound of value `P`, which is
one past the last element of the pointer. In other words, this annotation
describes a range that starts with the pointer that has this annotation and
- ends with ``P`` which is the argument of the annotation. ``P`` itself may be
- annotated with ``__ended_by(Q)``. In this case, the end of the range extends
- to the pointer ``Q``. This is used for "iterator" support in C where you're
+ ends with `P` which is the argument of the annotation. `P` itself may be
+ annotated with `__ended_by(Q)`. In this case, the end of the range extends
+ to the pointer `Q`. This is used for "iterator" support in C where you're
iterating from one pointer value to another until a final pointer value is
reached (and the final pointer value is not dereferenceable).
@@ -233,31 +225,30 @@ Accessing a pointer outside the specified bounds causes a run-time trap or a
compile-time error. Also, the model maintains correctness of bounds annotations
when the pointer and/or the related value containing the bounds information are
updated or passed as arguments. This is done by compile-time restrictions or
-run-time checks (see `Maintaining correctness of bounds annotations`_
-for more detail). For instance, initializing ``buf`` with ``null`` while
-assigning non-zero value to ``count``, as shown in the following example, would
-violate the ``__counted_by`` annotation because a null pointer does not point to
+run-time checks (see [Maintaining correctness of bounds annotations]
+for more detail). For instance, initializing `buf` with `null` while
+assigning non-zero value to `count`, as shown in the following example, would
+violate the `__counted_by` annotation because a null pointer does not point to
any valid memory location. To avoid this, the compiler produces either a
compile-time error or run-time trap.
-.. code-block:: c
-
- void null_with_count_10(int *__counted_by(count) buf, unsigned count) {
- buf = 0;
- // This is not allowed as it creates a null pointer with non-zero length
- count = 10;
- }
+```c
+void null_with_count_10(int *__counted_by(count) buf, unsigned count) {
+ buf = 0;
+ // This is not allowed as it creates a null pointer with non-zero length
+ count = 10;
+}
+```
However, there are use cases where a pointer is either a null pointer or is
pointing to memory of the specified size. To support this idiom,
-``-fbounds-safety`` provides ``*_or_null`` variants,
-``__counted_by_or_null(N)``, ``__sized_by_or_null(N)``, and
-``__ended_by_or_null(P)``. Accessing a pointer with any of these bounds
+`-fbounds-safety` provides `*_or_null` variants,
+`__counted_by_or_null(N)`, `__sized_by_or_null(N)`, and
+`__ended_by_or_null(P)`. Accessing a pointer with any of these bounds
annotations will require an extra null check to avoid a null pointer
dereference.
-Internal bounds annotations
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
+#### Internal bounds annotations
A wide pointer (sometimes known as a "fat" pointer) is a pointer that carries
additional bounds information internally (as part of its data). The bounds
@@ -266,562 +257,541 @@ pointers, hence the name "wide pointer". The memory layout of a wide pointer is
equivalent to a struct with the pointer, upper bound, and (optionally) lower
bound as its fields as shown below.
-.. code-block:: c
-
- struct wide_pointer_datalayout {
- void* pointer; // Address used for dereferences and pointer arithmetic
- void* upper_bound; // Points one past the highest address that can be
- // accessed
- void* lower_bound; // (Optional) Points to lowest address that can be
- // accessed
- };
+```c
+struct wide_pointer_datalayout {
+ void* pointer; // Address used for dereferences and pointer arithmetic
+ void* upper_bound; // Points one past the highest address that can be
+ // accessed
+ void* lower_bound; // (Optional) Points to lowest address that can be
+ // accessed
+};
+```
Even with this representational change, wide pointers act syntactically as
normal pointers to allow standard pointer operations, such as pointer
-dereference (``*p``), array subscript (``p[i]``), member access (``p->``), and
+dereference (`*p`), array subscript (`p[i]`), member access (`p->`), and
pointer arithmetic, with some restrictions on bounds-unsafe uses.
-``-fbounds-safety`` has a set of "internal" bounds annotations to turn pointers
-into wide pointers. These are ``__bidi_indexable`` and ``__indexable``. When a
+`-fbounds-safety` has a set of "internal" bounds annotations to turn pointers
+into wide pointers. These are `__bidi_indexable` and `__indexable`. When a
pointer has either of these annotations, the compiler changes the pointer to the
corresponding wide pointer. This means these annotations will break the ABI and
will not be compatible with plain C, and thus they should generally not be used
in ABI surfaces.
-* ``__bidi_indexable`` : A pointer with this annotation becomes a wide pointer
+- `__bidi_indexable` : A pointer with this annotation becomes a wide pointer
to carry the upper bound and the lower bound, the layout of which is
- equivalent to ``struct { T *ptr; T *upper_bound; T *lower_bound; };``. As the
+ equivalent to `struct { T *ptr; T *upper_bound; T *lower_bound; };`. As the
name indicates, pointers with this annotation are "bidirectionally indexable",
meaning that they can be indexed with either a negative or a positive offset
and the pointers can be incremented or decremented using pointer arithmetic. A
- ``__bidi_indexable`` pointer is allowed to hold an out-of-bounds pointer
+ `__bidi_indexable` pointer is allowed to hold an out-of-bounds pointer
value. While creating an OOB pointer is undefined behavior in C,
- ``-fbounds-safety`` makes it well-defined behavior. That is, pointer
- arithmetic overflow with ``__bidi_indexable`` is defined as equivalent of
+ `-fbounds-safety` makes it well-defined behavior. That is, pointer
+ arithmetic overflow with `__bidi_indexable` is defined as equivalent of
two's complement integer computation, and at the LLVM IR level this means
- ``getelementptr`` won't get ``inbounds`` keyword. Accessing memory using the
+ `getelementptr` won't get `inbounds` keyword. Accessing memory using the
OOB pointer is prevented via a run-time bounds check.
-
-* ``__indexable`` : A pointer with this annotation becomes a wide pointer
+- `__indexable` : A pointer with this annotation becomes a wide pointer
carrying the upper bound (but no explicit lower bound), the layout of which is
- equivalent to ``struct { T *ptr; T *upper_bound; };``. Since ``__indexable``
+ equivalent to `struct { T *ptr; T *upper_bound; };`. Since `__indexable`
pointers do not have a separate lower bound, the pointer value itself acts as
- the lower bound. An ``__indexable`` pointer can only be incremented or indexed
+ the lower bound. An `__indexable` pointer can only be incremented or indexed
in the positive direction. Indexing it in the negative direction will trigger
a compile-time error. Otherwise, the compiler inserts a run-time
check to ensure pointer arithmetic doesn't make the pointer smaller than the
- original ``__indexable`` pointer (Note that ``__indexable`` doesn't have a
+ original `__indexable` pointer (Note that `__indexable` doesn't have a
lower bound so the pointer value is effectively the lower bound). As pointer
arithmetic overflow will make the pointer smaller than the original pointer,
- it will cause a trap at runtime. Similar to ``__bidi_indexable``, an
- ``__indexable`` pointer is allowed to have a pointer value above the upper
+ it will cause a trap at runtime. Similar to `__bidi_indexable`, an
+ `__indexable` pointer is allowed to have a pointer value above the upper
bound and creating such a pointer is well-defined behavior. Dereferencing such
a pointer, however, will cause a run-time trap.
-
-* ``__bidi_indexable`` offers the best flexibility out of all the pointer
- annotations in this model, as ``__bidi_indexable`` pointers can be used for
+- `__bidi_indexable` offers the best flexibility out of all the pointer
+ annotations in this model, as `__bidi_indexable` pointers can be used for
any pointer operation. However, this comes with the largest code size and
memory cost out of the available pointer annotations in this model. In some
- cases, use of the ``__bidi_indexable`` annotation may be duplicating bounds
+ cases, use of the `__bidi_indexable` annotation may be duplicating bounds
information that exists elsewhere in the program. In such cases, using
external bounds annotations may be a better choice.
-``__bidi_indexable`` is the default annotation for non-ABI visible pointers,
+`__bidi_indexable` is the default annotation for non-ABI visible pointers,
such as local pointer variables — that is, if the programmer does not specify
another bounds annotation, a local pointer variable is implicitly
-``__bidi_indexable``. Since ``__bidi_indexable`` pointers automatically carry
+`__bidi_indexable`. Since `__bidi_indexable` pointers automatically carry
bounds information and have no restrictions on kinds of pointer operations that
can be used with these pointers, most code inside a function works as is without
-modification. In the example below, ``int *buf`` doesn't require manual
-annotation as it's implicitly ``int *__bidi_indexable buf``, carrying the bounds
+modification. In the example below, `int *buf` doesn't require manual
+annotation as it's implicitly `int *__bidi_indexable buf`, carrying the bounds
information passed from the return value of malloc, which is necessary to insert
-bounds checking for ``buf[i]``.
-
-.. code-block:: c
+bounds checking for `buf[i]`.
- void *__sized_by(size) malloc(size_t size);
+```c
+void *__sized_by(size) malloc(size_t size);
- int *__counted_by(n) get_array_with_0_to_n_1(size_t n) {
- int *buf = malloc(sizeof(int) * n);
- for (size_t i = 0; i < n; ++i)
- buf[i] = i;
- return buf;
- }
+int *__counted_by(n) get_array_with_0_to_n_1(size_t n) {
+ int *buf = malloc(sizeof(int) * n);
+ for (size_t i = 0; i < n; ++i)
+ buf[i] = i;
+ return buf;
+}
+```
-Annotations for sentinel-delimited arrays
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+#### Annotations for sentinel-delimited arrays
A C string is an array of characters. The null terminator — the first null
-character (``'\0'``) element in the array — marks the end of the string.
-``-fbounds-safety`` provides ``__null_terminated`` to annotate C strings and the
-generalized form ``__terminated_by(T)`` to annotate pointers and arrays with an
+character (`'\0'`) element in the array — marks the end of the string.
+`-fbounds-safety` provides `__null_terminated` to annotate C strings and the
+generalized form `__terminated_by(T)` to annotate pointers and arrays with an
end marked by a sentinel value. The model prevents dereferencing a
-``__terminated_by`` pointer beyond its end. Calculating the location of the end
+`__terminated_by` pointer beyond its end. Calculating the location of the end
(i.e., the address of the sentinel value), requires reading the entire array in
memory and would have some performance costs. To avoid an unintended performance
hit, the model puts some restrictions on how these pointers can be used.
-``__terminated_by`` pointers cannot be indexed and can only be incremented one
+`__terminated_by` pointers cannot be indexed and can only be incremented one
element at a time. To allow these operations, the pointers must be explicitly
-converted to ``__indexable`` pointers using the intrinsic function
-``__unsafe_terminated_by_to_indexable(P, T)`` (or
-``__unsafe_null_terminated_to_indexable(P)``) which converts the
-``__terminated_by`` pointer ``P`` to an ``__indexable`` pointer.
+converted to `__indexable` pointers using the intrinsic function
+`__unsafe_terminated_by_to_indexable(P, T)` (or
+`__unsafe_null_terminated_to_indexable(P)`) which converts the
+`__terminated_by` pointer `P` to an `__indexable` pointer.
-* ``__null_terminated`` : The pointer or array is terminated by ``NULL`` or
- ``0``. Modifying the terminator or incrementing the pointer beyond it is
+- `__null_terminated` : The pointer or array is terminated by `NULL` or
+ `0`. Modifying the terminator or incrementing the pointer beyond it is
prevented at run time.
-
-* ``__terminated_by(T)`` : The pointer or array is terminated by ``T`` which is
+- `__terminated_by(T)` : The pointer or array is terminated by `T` which is
a constant expression. Accessing or incrementing the pointer beyond the
- terminator is not allowed. This is a generalization of ``__null_terminated``
- which is defined as ``__terminated_by(0)``.
+ terminator is not allowed. This is a generalization of `__null_terminated`
+ which is defined as `__terminated_by(0)`.
-Annotation for interoperating with bounds-unsafe code
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+#### Annotation for interoperating with bounds-unsafe code
-A pointer with the ``__unsafe_indexable`` annotation behaves the same as a plain
+A pointer with the `__unsafe_indexable` annotation behaves the same as a plain
C pointer. That is, the pointer does not have any bounds information and pointer
operations are not checked.
-``__unsafe_indexable`` can be used to mark pointers from system headers or
+`__unsafe_indexable` can be used to mark pointers from system headers or
pointers from code that has not adopted -fbounds safety. This enables
-interoperation between code using ``-fbounds-safety`` and code that does not.
+interoperation between code using `-fbounds-safety` and code that does not.
-Default pointer types
----------------------
+### Default pointer types
-ABI visibility and default annotations
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+#### ABI visibility and default annotations
-Requiring ``-fbounds-safety`` adopters to add bounds annotations to all pointers
+Requiring `-fbounds-safety` adopters to add bounds annotations to all pointers
in the codebase would be a significant adoption burden. To avoid this and to
-secure all pointers by default, ``-fbounds-safety`` applies default bounds
+secure all pointers by default, `-fbounds-safety` applies default bounds
annotations to pointer types.
Default annotations apply to pointer types of declarations
-``-fbounds-safety`` applies default bounds annotations to pointer types used in
+`-fbounds-safety` applies default bounds annotations to pointer types used in
declarations. The default annotations are determined by the ABI visibility of
the pointer. A pointer type is ABI-visible if changing its size or
representation affects the ABI. For instance, changing the size of a type used
in a function parameter will affect the ABI and thus pointers used in function
parameters are ABI-visible pointers. On the other hand, changing the types of
-local variables won't have such ABI implications. Hence, ``-fbounds-safety``
+local variables won't have such ABI implications. Hence, `-fbounds-safety`
considers the outermost pointer types of local variables as non-ABI visible. The
rest of the pointers such as nested pointer types, pointer types of global
variables, struct fields, and function prototypes are considered ABI-visible.
-All ABI-visible pointers are treated as ``__single`` by default unless annotated
+All ABI-visible pointers are treated as `__single` by default unless annotated
otherwise. This default both preserves ABI and makes these pointers safe by
default. This behavior can be controlled with macros, i.e.,
-``__ptrcheck_abi_assume_*ATTR*()``, to set the default annotation for
-ABI-visible pointers to be either ``__single``, ``__bidi_indexable``,
-``__indexable``, or ``__unsafe_indexable``. For instance,
-``__ptrcheck_abi_assume_unsafe_indexable()`` will make all ABI-visible pointers
-be ``__unsafe_indexable``. Non-ABI visible pointers — the outermost pointer
-types of local variables — are ``__bidi_indexable`` by default, so that these
+`__ptrcheck_abi_assume_*ATTR*()`, to set the default annotation for
+ABI-visible pointers to be either `__single`, `__bidi_indexable`,
+`__indexable`, or `__unsafe_indexable`. For instance,
+`__ptrcheck_abi_assume_unsafe_indexable()` will make all ABI-visible pointers
+be `__unsafe_indexable`. Non-ABI visible pointers — the outermost pointer
+types of local variables — are `__bidi_indexable` by default, so that these
pointers have the bounds information necessary to perform bounds checks without
-the need for a manual annotation. All ``const char`` pointers or any typedefs
-equivalent to ``const char`` pointers are ``__null_terminated`` by default. This
-means that ``char8_t`` is ``unsigned char`` so ``const char8_t *`` won't be
-``__null_terminated`` by default. Similarly, ``const wchar_t *`` won't be
-``__null_terminated`` by default unless the platform defines it as ``typedef
-char wchar_t``. Please note, however, that the programmers can still explicitly
-use ``__null_terminated`` in any other pointers, e.g., ``char8_t
-*__null_terminated``, ``wchar_t *__null_terminated``, ``int
-*__null_terminated``, etc. if they should be treated as ``__null_terminated``.
+the need for a manual annotation. All `const char` pointers or any typedefs
+equivalent to `const char` pointers are `__null_terminated` by default. This
+means that `char8_t` is `unsigned char` so `const char8_t *` won't be
+`__null_terminated` by default. Similarly, `const wchar_t *` won't be
+`__null_terminated` by default unless the platform defines it as `typedef
+char wchar_t`. Please note, however, that the programmers can still explicitly
+use `__null_terminated` in any other pointers, e.g., `char8_t
+*__null_terminated`, `wchar_t *__null_terminated`, `int
+*__null_terminated`, etc. if they should be treated as `__null_terminated`.
The same applies to other annotations.
In system headers, the default pointer attribute for ABI-visible pointers is set
-to ``__unsafe_indexable`` by default.
-
-The ``__ptrcheck_abi_assume_*ATTR*()`` macros are defined as pragmas in the
-toolchain header (See `Portability with toolchains that do not support the
-extension`_ for more details about the toolchain header):
+to `__unsafe_indexable` by default.
-.. code-block:: C
+The `__ptrcheck_abi_assume_*ATTR*()` macros are defined as pragmas in the
+toolchain header (See [Portability with toolchains that do not support the
+extension][portability with toolchains that do not support the extension] for more details about the toolchain header):
- #define __ptrcheck_abi_assume_single() \
- _Pragma("clang abi_ptr_attr set(single)")
+```C
+#define __ptrcheck_abi_assume_single() \
+ _Pragma("clang abi_ptr_attr set(single)")
- #define __ptrcheck_abi_assume_indexable() \
- _Pragma("clang abi_ptr_attr set(indexable)")
+#define __ptrcheck_abi_assume_indexable() \
+ _Pragma("clang abi_ptr_attr set(indexable)")
- #define __ptrcheck_abi_assume_bidi_indexable() \
- _Pragma("clang abi_ptr_attr set(bidi_indexable)")
+#define __ptrcheck_abi_assume_bidi_indexable() \
+ _Pragma("clang abi_ptr_attr set(bidi_indexable)")
- #define __ptrcheck_abi_assume_unsafe_indexable() \
- _Pragma("clang abi_ptr_attr set(unsafe_indexable)")
+#define __ptrcheck_abi_assume_unsafe_indexable() \
+ _Pragma("clang abi_ptr_attr set(unsafe_indexable)")
+```
-
-ABI implications of default bounds annotations
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+#### ABI implications of default bounds annotations
Although simply modifying types of a local variable doesn't normally impact the
ABI, taking the address of such a modified type could create a pointer type that
-has an ABI mismatch. Looking at the following example, ``int *local`` is
-implicitly ``int *__bidi_indexable`` and thus the type of ``&local`` is a
-pointer to ``int *__bidi_indexable``. On the other hand, in ``void foo(int
-**)``, the parameter type is a pointer to ``int *__single`` (i.e., ``void
-foo(int *__single *__single)``) (or a pointer to ``int *__unsafe_indexable`` if
+has an ABI mismatch. Looking at the following example, `int *local` is
+implicitly `int *__bidi_indexable` and thus the type of `&local` is a
+pointer to `int *__bidi_indexable`. On the other hand, in `void foo(int
+**)`, the parameter type is a pointer to `int *__single` (i.e., `void
+foo(int *__single *__single)`) (or a pointer to `int *__unsafe_indexable` if
it's from a system header). The compiler reports an error for casts between
pointers whose elements have incompatible pointer attributes. This way,
-``-fbounds-safety`` prevents pointers that are implicitly ``__bidi_indexable``
+`-fbounds-safety` prevents pointers that are implicitly `__bidi_indexable`
from silently escaping thereby breaking the ABI.
-.. code-block:: c
+```c
+void foo(int **);
- void foo(int **);
+void bar(void) {
+ int *local = 0;
+ // error: passing 'int *__bidi_indexable*__bidi_indexable' to parameter of
+ // incompatible nested pointer type 'int *__single*__single'
+ foo(&local);
+}
+```
- void bar(void) {
- int *local = 0;
- // error: passing 'int *__bidi_indexable*__bidi_indexable' to parameter of
- // incompatible nested pointer type 'int *__single*__single'
- foo(&local);
- }
-
-A local variable may still be exposed to the ABI if ``typeof()`` takes the type
+A local variable may still be exposed to the ABI if `typeof()` takes the type
of local variable to define an interface as shown in the following example.
-.. code-block:: C
-
- // bar.c
- void bar(int *) { ... }
+```C
+// bar.c
+void bar(int *) { ... }
- // foo.c
- void foo(void) {
- int *p; // implicitly `int *__bidi_indexable p`
- extern void bar(typeof(p)); // creates an interface of type
- // `void bar(int *__bidi_indexable)`
- }
+// foo.c
+void foo(void) {
+ int *p; // implicitly `int *__bidi_indexable p`
+ extern void bar(typeof(p)); // creates an interface of type
+ // `void bar(int *__bidi_indexable)`
+}
+```
-Doing this may break the ABI if the parameter is not ``__bidi_indexable`` at the
-definition of function ``bar()`` which is likely the case because parameters are
-``__single`` by default without an explicit annotation.
+Doing this may break the ABI if the parameter is not `__bidi_indexable` at the
+definition of function `bar()` which is likely the case because parameters are
+`__single` by default without an explicit annotation.
In order to avoid an implicitly wide pointer from silently breaking the ABI, the
-compiler reports a warning when ``typeof()`` is used on an implicit wide pointer
+compiler reports a warning when `typeof()` is used on an implicit wide pointer
at any ABI visible context (e.g., function prototype, struct definition, etc.).
-.. _Default pointer types in typeof:
+(default-pointer-types-in-typeof)=
-Default pointer types in ``typeof()``
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+#### Default pointer types in `typeof()`
-When ``typeof()`` takes an expression, it respects the bounds annotation on
+When `typeof()` takes an expression, it respects the bounds annotation on
the expression type, including the bounds annotation is implicit. For example,
-the global variable ``g`` in the following code is implicitly ``__single`` so
-``typeof(g)`` gets ``char *__single``. The similar is true for the parameter
-``p``, so ``typeof(p)`` returns ``void *__single``. The local variable ``l`` is
-implicitly ``__bidi_indexable``, so ``typeof(l)`` becomes
-``int *__bidi_indexable``.
+the global variable `g` in the following code is implicitly `__single` so
+`typeof(g)` gets `char *__single`. The similar is true for the parameter
+`p`, so `typeof(p)` returns `void *__single`. The local variable `l` is
+implicitly `__bidi_indexable`, so `typeof(l)` becomes
+`int *__bidi_indexable`.
-.. code-block:: C
+```C
+char *g; // typeof(g) == char *__single
- char *g; // typeof(g) == char *__single
+void foo(void *p) {
+ // typeof(p) == void *__single
- void foo(void *p) {
- // typeof(p) == void *__single
-
- int *l; // typeof(l) == int *__bidi_indexable
- }
+ int *l; // typeof(l) == int *__bidi_indexable
+}
+```
When the type of expression has an "external" bounds annotation, e.g.,
-``__sized_by``, ``__counted_by``, etc., the compiler may report an error on
-``typeof`` if the annotation creates a dependency with another declaration or
-variable. For example, the compiler reports an error on ``typeof(p1)`` shown in
+`__sized_by`, `__counted_by`, etc., the compiler may report an error on
+`typeof` if the annotation creates a dependency with another declaration or
+variable. For example, the compiler reports an error on `typeof(p1)` shown in
the following code because allowing it can potentially create another type
-dependent on the parameter ``size`` in a different context (Please note that an
+dependent on the parameter `size` in a different context (Please note that an
external bounds annotation on a parameter may only refer to another parameter of
-the same function). On the other hand, ``typeof(p2)`` works resulting in ``int
-*__counted_by(10)``, since it doesn't depend on any other declaration.
+the same function). On the other hand, `typeof(p2)` works resulting in `int
+*__counted_by(10)`, since it doesn't depend on any other declaration.
-.. TODO: add a section describing constraints on external bounds annotations
+% TODO: add a section describing constraints on external bounds annotations
-.. code-block:: C
+```C
+void foo(int *__counted_by(size) p1, size_t size) {
+ // typeof(p1) == int *__counted_by(size)
+ // -> a compiler error as it tries to create another type
+ // dependent on `size`.
- void foo(int *__counted_by(size) p1, size_t size) {
- // typeof(p1) == int *__counted_by(size)
- // -> a compiler error as it tries to create another type
- // dependent on `size`.
+ int *__counted_by(10) p2; // typeof(p2) == int *__counted_by(10)
+ // -> no error
- int *__counted_by(10) p2; // typeof(p2) == int *__counted_by(10)
- // -> no error
-
- }
+}
+```
-When ``typeof()`` takes a type name, the compiler doesn't apply an implicit
-bounds annotation on the named pointer types. For example, ``typeof(int*)``
-returns ``int *`` without any bounds annotation. A bounds annotation may be
+When `typeof()` takes a type name, the compiler doesn't apply an implicit
+bounds annotation on the named pointer types. For example, `typeof(int*)`
+returns `int *` without any bounds annotation. A bounds annotation may be
added after the fact depending on the context. In the following example,
-``typeof(int *)`` returns ``int *`` so it's equivalent as the local variable is
-declared as ``int *l``, so it eventually becomes implicitly
-``__bidi_indexable``.
+`typeof(int *)` returns `int *` so it's equivalent as the local variable is
+declared as `int *l`, so it eventually becomes implicitly
+`__bidi_indexable`.
-.. code-block:: c
-
- void foo(void) {
- typeof(int *) l; // `int *__bidi_indexable` (same as `int *l`)
- }
+```c
+void foo(void) {
+ typeof(int *) l; // `int *__bidi_indexable` (same as `int *l`)
+}
+```
The programmers can still explicitly add a bounds annotation on the types named
-inside ``typeof``, e.g., ``typeof(int *__bidi_indexable)``, which evaluates to
-``int *__bidi_indexable``.
-
+inside `typeof`, e.g., `typeof(int *__bidi_indexable)`, which evaluates to
+`int *__bidi_indexable`.
-Default pointer types in ``sizeof()``
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+#### Default pointer types in `sizeof()`
-When ``sizeof()`` takes a type name, the compiler doesn't apply an implicit
+When `sizeof()` takes a type name, the compiler doesn't apply an implicit
bounds annotation on the named pointer types. This means if a bounds annotation
is not specified, the evaluated pointer type is treated identically to a plain C
-pointer type. Therefore, ``sizeof(int*)`` remains the same with or without
-``-fbounds-safety``. That said, programmers can explicitly add attributes to the
-types, e.g., ``sizeof(int *__bidi_indexable)``, in which case the sizeof
-evaluates to the size of type ``int *__bidi_indexable`` (the value equivalent to
-``3 * sizeof(int*)``).
-
-When ``sizeof()`` takes an expression, i.e., ``sizeof(expr``, it behaves as
-``sizeof(typeof(expr))``, except that ``sizeof(expr)`` does not report an error
-with ``expr`` that has a type with an external bounds annotation dependent on
-another declaration, whereas ``typeof()`` on the same expression would be an
-error as described in :ref:`Default pointer types in typeof`.
+pointer type. Therefore, `sizeof(int*)` remains the same with or without
+`-fbounds-safety`. That said, programmers can explicitly add attributes to the
+types, e.g., `sizeof(int *__bidi_indexable)`, in which case the sizeof
+evaluates to the size of type `int *__bidi_indexable` (the value equivalent to
+`3 * sizeof(int*)`).
+
+When `sizeof()` takes an expression, i.e., `sizeof(expr`, it behaves as
+`sizeof(typeof(expr))`, except that `sizeof(expr)` does not report an error
+with `expr` that has a type with an external bounds annotation dependent on
+another declaration, whereas `typeof()` on the same expression would be an
+error as described in {ref}`Default pointer types in typeof`.
The following example describes this behavior.
-.. code-block:: c
+```c
+void foo(int *__counted_by(size) p, size_t size) {
+ // sizeof(p) == sizeof(int *__counted_by(size)) == sizeof(int *)
+ // typeof(p): error
+};
+```
- void foo(int *__counted_by(size) p, size_t size) {
- // sizeof(p) == sizeof(int *__counted_by(size)) == sizeof(int *)
- // typeof(p): error
- };
+#### Default pointer types in `alignof()`
-Default pointer types in ``alignof()``
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-``alignof()`` only takes a type name as the argument and it doesn't take an
-expression. Similar to ``sizeof()`` and ``typeof``, the compiler doesn't apply
-an implicit bounds annotation on the pointer types named inside ``alignof()``.
-Therefore, ``alignof(T *)`` remains the same with or without
-``-fbounds-safety``, evaluating into the alignment of the raw pointer ``T *``.
+`alignof()` only takes a type name as the argument and it doesn't take an
+expression. Similar to `sizeof()` and `typeof`, the compiler doesn't apply
+an implicit bounds annotation on the pointer types named inside `alignof()`.
+Therefore, `alignof(T *)` remains the same with or without
+`-fbounds-safety`, evaluating into the alignment of the raw pointer `T *`.
The programmers can explicitly add a bounds annotation to the types, e.g.,
-``alignof(int *__bidi_indexable)``, which returns the alignment of ``int
-*__bidi_indexable``. A bounds annotation including an internal bounds annotation
-(i.e., ``__indexable`` and ``__bidi_indexable``) doesn't affect the alignment of
-the original pointer. Therefore, ``alignof(int *__bidi_indexable)`` is equal to
-``alignof(int *)``.
-
-
-Default pointer types used in C-style casts
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-A pointer type used in a C-style cast (e.g., ``(int *)src``) inherits the same
-pointer attribute in the type of src. For instance, if the type of src is ``T
-*__single`` (with ``T`` being an arbitrary C type), ``(int *)src`` will be ``int
-*__single``. The reasoning behind this behavior is so that a C-style cast
+`alignof(int *__bidi_indexable)`, which returns the alignment of `int
+*__bidi_indexable`. A bounds annotation including an internal bounds annotation
+(i.e., `__indexable` and `__bidi_indexable`) doesn't affect the alignment of
+the original pointer. Therefore, `alignof(int *__bidi_indexable)` is equal to
+`alignof(int *)`.
+
+#### Default pointer types used in C-style casts
+
+A pointer type used in a C-style cast (e.g., `(int *)src`) inherits the same
+pointer attribute in the type of src. For instance, if the type of src is `T
+*__single` (with `T` being an arbitrary C type), `(int *)src` will be `int
+*__single`. The reasoning behind this behavior is so that a C-style cast
doesn't introduce any unexpected side effects caused by an implicit cast of
bounds attribute.
-Pointer casts can have explicit bounds annotations. For instance, ``(int
-*__bidi_indexable)src`` casts to ``int *__bidi_indexable`` as long as src has a
-bounds annotation that can implicitly convert to ``__bidi_indexable``. If
-``src`` has type ``int *__single``, it can implicitly convert to ``int
-*__bidi_indexable`` which then will have the upper bound pointing to one past
-the first element. However, if src has type ``int *__unsafe_indexable``, the
-explicit cast ``(int *__bidi_indexable)src`` will cause an error because
-``__unsafe_indexable`` cannot cast to ``__bidi_indexable`` as
-``__unsafe_indexable`` doesn't have bounds information. `Cast rules`_ describes
+Pointer casts can have explicit bounds annotations. For instance, `(int
+*__bidi_indexable)src` casts to `int *__bidi_indexable` as long as src has a
+bounds annotation that can implicitly convert to `__bidi_indexable`. If
+`src` has type `int *__single`, it can implicitly convert to `int
+*__bidi_indexable` which then will have the upper bound pointing to one past
+the first element. However, if src has type `int *__unsafe_indexable`, the
+explicit cast `(int *__bidi_indexable)src` will cause an error because
+`__unsafe_indexable` cannot cast to `__bidi_indexable` as
+`__unsafe_indexable` doesn't have bounds information. [Cast rules] describes
in more detail what kinds of casts are allowed between pointers with different
bounds annotations.
-Default pointer types in typedef
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+#### Default pointer types in typedef
-Pointer types in ``typedef``\s do not have implicit default bounds annotations.
-Instead, the bounds annotation is determined when the ``typedef`` is used. The
-following example shows that no pointer annotation is specified in the ``typedef
-pint_t`` while each instance of ``typedef``'ed pointer gets its bounds
+Pointer types in `typedef`s do not have implicit default bounds annotations.
+Instead, the bounds annotation is determined when the `typedef` is used. The
+following example shows that no pointer annotation is specified in the `typedef
+pint_t` while each instance of `typedef`'ed pointer gets its bounds
annotation based on the context in which the type is used.
-.. code-block:: c
+```c
+typedef int * pint_t; // int *
- typedef int * pint_t; // int *
+pint_t glob; // int *__single glob;
- pint_t glob; // int *__single glob;
+void foo(void) {
+ pint_t local; // int *__bidi_indexable local;
+}
+```
- void foo(void) {
- pint_t local; // int *__bidi_indexable local;
- }
-
-Pointer types in a ``typedef`` can still have explicit annotations, e.g.,
-``typedef int *__single``, in which case the bounds annotation ``__single`` will
-apply to every use of the ``typedef``.
+Pointer types in a `typedef` can still have explicit annotations, e.g.,
+`typedef int *__single`, in which case the bounds annotation `__single` will
+apply to every use of the `typedef`.
-Array to pointer promotion to secure arrays (including VLAs)
-------------------------------------------------------------
+### Array to pointer promotion to secure arrays (including VLAs)
-Arrays on function prototypes
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+#### Arrays on function prototypes
In C, arrays on function prototypes are promoted (or "decayed") to a pointer to
-its first element (e.g., ``&arr[0]``). In ``-fbounds-safety``, arrays are also
+its first element (e.g., `&arr[0]`). In `-fbounds-safety`, arrays are also
decayed to pointers, but with the addition of an implicit bounds annotation,
which includes variable-length arrays (VLAs). As shown in the following example,
-arrays on function prototypes are decayed to corresponding ``__counted_by``
+arrays on function prototypes are decayed to corresponding `__counted_by`
pointers.
-.. code-block:: c
-
- // Function prototype: void foo(int n, int *__counted_by(n) arr);
- void foo(int n, int arr[n]);
+```c
+// Function prototype: void foo(int n, int *__counted_by(n) arr);
+void foo(int n, int arr[n]);
- // Function prototype: void bar(int *__counted_by(10) arr);
- void bar(int arr[10]);
+// Function prototype: void bar(int *__counted_by(10) arr);
+void bar(int arr[10]);
+```
This means the array parameters are treated as `__counted_by` pointers within
the function and callers of the function also see them as the corresponding
`__counted_by` pointers.
Incomplete arrays on function prototypes will cause a compiler error unless it
-has ``__counted_by`` annotation in its bracket.
+has `__counted_by` annotation in its bracket.
-.. code-block:: c
+```c
+void f1(int n, int arr[]); // error
- void f1(int n, int arr[]); // error
+void f3(int n, int arr[__counted_by(n)]); // ok
- void f3(int n, int arr[__counted_by(n)]); // ok
+void f2(int n, int arr[n]); // ok, decays to int *__counted_by(n)
- void f2(int n, int arr[n]); // ok, decays to int *__counted_by(n)
+void f4(int n, int *__counted_by(n) arr); // ok
- void f4(int n, int *__counted_by(n) arr); // ok
+void f5(int n, int *arr); // ok, but decays to int *__single,
+ // and cannot be used for pointer arithmetic
+```
- void f5(int n, int *arr); // ok, but decays to int *__single,
- // and cannot be used for pointer arithmetic
-
-Array references
-^^^^^^^^^^^^^^^^
+#### Array references
In C, similar to arrays on the function prototypes, a reference to array is
automatically promoted (or "decayed") to a pointer to its first element (e.g.,
-``&arr[0]``).
+`&arr[0]`).
-In `-fbounds-safety`, array references are promoted to ``__bidi_indexable``
+In `-fbounds-safety`, array references are promoted to `__bidi_indexable`
pointers which contain the upper and lower bounds of the array, with the
-equivalent of ``&arr[0]`` serving as the lower bound and ``&arr[array_size]``
+equivalent of `&arr[0]` serving as the lower bound and `&arr[array_size]`
(or one past the last element) serving as the upper bound. This applies to all
types of arrays including constant-length arrays, variable-length arrays (VLAs),
and flexible array members annotated with `__counted_by`.
-In the following example, reference to ``vla`` promotes to ``int
-*__bidi_indexable``, with ``&vla[n]`` as the upper bound and ``&vla[0]`` as the
-lower bound. Then, it's copied to ``int *p``, which is implicitly ``int
-*__bidi_indexable p``. Please note that value of ``n`` used to create the upper
-bound is ``10``, not ``100``, in this case because ``10`` is the actual length
-of ``vla``, the value of ``n`` at the time when the array is being allocated.
-
-.. code-block:: c
-
- void foo(void) {
- int n = 10;
- int vla[n];
- n = 100;
- int *p = vla; // { .ptr: &vla[0], .upper: &vla[10], .lower: &vla[0] }
- // it's `&vla[10]` because the value of `n` was 10 at the
- // time when the array is actually allocated.
- // ...
- }
-
-By promoting array references to ``__bidi_indexable``, all array accesses are
-bounds checked in ``-fbounds-safety``, just as ``__bidi_indexable`` pointers
+In the following example, reference to `vla` promotes to `int
+*__bidi_indexable`, with `&vla[n]` as the upper bound and `&vla[0]` as the
+lower bound. Then, it's copied to `int *p`, which is implicitly `int
+*__bidi_indexable p`. Please note that value of `n` used to create the upper
+bound is `10`, not `100`, in this case because `10` is the actual length
+of `vla`, the value of `n` at the time when the array is being allocated.
+
+```c
+void foo(void) {
+ int n = 10;
+ int vla[n];
+ n = 100;
+ int *p = vla; // { .ptr: &vla[0], .upper: &vla[10], .lower: &vla[0] }
+ // it's `&vla[10]` because the value of `n` was 10 at the
+ // time when the array is actually allocated.
+ // ...
+}
+```
+
+By promoting array references to `__bidi_indexable`, all array accesses are
+bounds checked in `-fbounds-safety`, just as `__bidi_indexable` pointers
are.
-Maintaining correctness of bounds annotations
----------------------------------------------
+### Maintaining correctness of bounds annotations
-``-fbounds-safety`` maintains correctness of bounds annotations by performing
+`-fbounds-safety` maintains correctness of bounds annotations by performing
additional checks when a pointer object and/or its related value containing the
bounds information is updated.
-For example, ``__single`` expresses an invariant that the pointer must either
+For example, `__single` expresses an invariant that the pointer must either
point to a single valid object or be a null pointer. To maintain this invariant,
-the compiler inserts checks when initializing a ``__single`` pointer, as shown
+the compiler inserts checks when initializing a `__single` pointer, as shown
in the following example:
-.. code-block:: c
-
- void foo(void *__sized_by(size) vp, size_t size) {
- // Inserted check:
- // if ((int*)upper_bound(vp) - (int*)vp < sizeof(int) && !!vp) trap();
- int *__single ip = (int *)vp;
- }
-
-Additionally, an explicit bounds annotation such as ``int *__counted_by(count)
-buf`` defines a relationship between two variables, ``buf`` and ``count``:
-namely, that ``buf`` has ``count`` number of elements available. This
+```c
+void foo(void *__sized_by(size) vp, size_t size) {
+ // Inserted check:
+ // if ((int*)upper_bound(vp) - (int*)vp < sizeof(int) && !!vp) trap();
+ int *__single ip = (int *)vp;
+}
+```
+
+Additionally, an explicit bounds annotation such as `int *__counted_by(count)
+buf` defines a relationship between two variables, `buf` and `count`:
+namely, that `buf` has `count` number of elements available. This
relationship must hold even after any of these related variables are updated. To
-this end, the model requires that assignments to ``buf`` and ``count`` must be
-side by side, with no side effects between them. This prevents ``buf`` and
-``count`` from temporarily falling out of sync due to updates happening at a
+this end, the model requires that assignments to `buf` and `count` must be
+side by side, with no side effects between them. This prevents `buf` and
+`count` from temporarily falling out of sync due to updates happening at a
distance.
-The example below shows a function ``alloc_buf`` that initializes a struct that
-members that use the ``__counted_by`` annotation. The compiler allows these
-assignments because ``sbuf->buf`` and ``sbuf->count`` are updated side by side
+The example below shows a function `alloc_buf` that initializes a struct that
+members that use the `__counted_by` annotation. The compiler allows these
+assignments because `sbuf->buf` and `sbuf->count` are updated side by side
without any side effects in between the assignments.
Furthermore, the compiler inserts additional run-time checks to ensure the new
-``buf`` has at least as many elements as the new ``count`` indicates as shown in
-the transformed pseudo code of function ``alloc_buf()`` in the example below.
-
-.. code-block:: c
-
- typedef struct {
- int *__counted_by(count) buf;
- size_t count;
- } sized_buf_t;
-
- void alloc_buf(sized_buf_t *sbuf, size_t nelems) {
- sbuf->buf = (int *)malloc(sizeof(int) * nelems);
- sbuf->count = nelems;
- }
-
- // Transformed pseudo code:
- void alloc_buf(sized_buf_t *sbuf, size_t nelems) {
- // Materialize RHS values:
- int *tmp_ptr = (int *)malloc(sizeof(int) * nelems);
- int tmp_count = nelems;
- // Inserted check:
- // - checks to ensure that `lower <= tmp_ptr <= upper`
- // - if (upper(tmp_ptr) - tmp_ptr < tmp_count) trap();
- sbuf->buf = tmp_ptr;
- sbuf->count = tmp_count;
- }
+`buf` has at least as many elements as the new `count` indicates as shown in
+the transformed pseudo code of function `alloc_buf()` in the example below.
+
+```c
+typedef struct {
+ int *__counted_by(count) buf;
+ size_t count;
+} sized_buf_t;
+
+void alloc_buf(sized_buf_t *sbuf, size_t nelems) {
+ sbuf->buf = (int *)malloc(sizeof(int) * nelems);
+ sbuf->count = nelems;
+}
+
+// Transformed pseudo code:
+void alloc_buf(sized_buf_t *sbuf, size_t nelems) {
+ // Materialize RHS values:
+ int *tmp_ptr = (int *)malloc(sizeof(int) * nelems);
+ int tmp_count = nelems;
+ // Inserted check:
+ // - checks to ensure that `lower <= tmp_ptr <= upper`
+ // - if (upper(tmp_ptr) - tmp_ptr < tmp_count) trap();
+ sbuf->buf = tmp_ptr;
+ sbuf->count = tmp_count;
+}
+```
Whether the compiler can optimize such run-time checks depends on how the upper
-bound of the pointer is derived. If the source pointer has ``__sized_by``,
-``__counted_by``, or a variant of such, the compiler assumes that the upper
-bound calculation doesn't overflow, e.g., ``ptr + size`` (where the type of
-``ptr`` is ``void *__sized_by(size)``), because when the ``__sized_by`` pointer
-is initialized, ``-fbounds-safety`` inserts run-time checks to ensure that ``ptr
-+ size`` doesn't overflow and that ``size >= 0``.
+bound of the pointer is derived. If the source pointer has `__sized_by`,
+`__counted_by`, or a variant of such, the compiler assumes that the upper
+bound calculation doesn't overflow, e.g., `ptr + size` (where the type of
+`ptr` is `void *__sized_by(size)`), because when the `__sized_by` pointer
+is initialized, `-fbounds-safety` inserts run-time checks to ensure that `ptr
+\+ size` doesn't overflow and that `size >= 0`.
Assuming the upper bound calculation doesn't overflow, the compiler can simplify
-the trap condition ``upper(tmp_ptr) - tmp_ptr < tmp_count`` to ``size <
-tmp_count`` so if both ``size`` and ``tmp_count`` values are known at compile
-time such that ``0 <= tmp_count <= size``, the optimizer can remove the check.
+the trap condition `upper(tmp_ptr) - tmp_ptr < tmp_count` to `size <
+tmp_count` so if both `size` and `tmp_count` values are known at compile
+time such that `0 <= tmp_count <= size`, the optimizer can remove the check.
-``ptr + size`` may still overflow if the ``__sized_by`` pointer is created from
-code that doesn't enable ``-fbounds-safety``, which is undefined behavior.
+`ptr + size` may still overflow if the `__sized_by` pointer is created from
+code that doesn't enable `-fbounds-safety`, which is undefined behavior.
-In the previous code example with the transformed ``alloc_buf()``, the upper
-bound of ``tmp_ptr`` is derived from ``void *__sized_by_or_null(size)``, which
-is the return type of ``malloc()``. Hence, the pointer arithmetic doesn't
-overflow or ``tmp_ptr`` is null. Therefore, if ``nelems`` was given as a
+In the previous code example with the transformed `alloc_buf()`, the upper
+bound of `tmp_ptr` is derived from `void *__sized_by_or_null(size)`, which
+is the return type of `malloc()`. Hence, the pointer arithmetic doesn't
+overflow or `tmp_ptr` is null. Therefore, if `nelems` was given as a
compile-time constant, the compiler could remove the checks.
-Cast rules
-----------
+### Cast rules
-``-fbounds-safety`` does not enforce overall type safety and bounds invariants
+`-fbounds-safety` does not enforce overall type safety and bounds invariants
can still be violated by incorrect casts in some cases. That said,
-``-fbounds-safety`` prevents type conversions that change bounds attributes in a
+`-fbounds-safety` prevents type conversions that change bounds attributes in a
way to violate the bounds invariant of the destination's pointer annotation.
Type conversions that change bounds attributes may be allowed if it does not
violate the invariant of the destination or that can be verified at run time.
@@ -829,179 +799,175 @@ Here are some of the important cast rules.
Two pointers that have different bounds annotations on their nested pointer
types are incompatible and cannot implicitly cast to each other. For example,
-``T *__single *__single`` cannot be converted to ``T *__bidi_indexable
-*__single``. Such a conversion between incompatible nested bounds annotations
+`T *__single *__single` cannot be converted to `T *__bidi_indexable
+*__single`. Such a conversion between incompatible nested bounds annotations
can be allowed using an explicit cast (e.g., C-style cast). Hereafter, the rules
-only apply to the top pointer types. ``__unsafe_indexable`` cannot be converted
-to any other safe pointer types (``__single``, ``__bidi_indexable``,
-``__counted_by``, etc) using a cast. The extension provides builtins to force
-this conversion, ``__unsafe_forge_bidi_indexable(type, pointer, char_count)`` to
-convert pointer to a ``__bidi_indexable`` pointer of type with ``char_count``
-bytes available and ``__unsafe_forge_single(type, pointer)`` to convert pointer
+only apply to the top pointer types. `__unsafe_indexable` cannot be converted
+to any other safe pointer types (`__single`, `__bidi_indexable`,
+`__counted_by`, etc) using a cast. The extension provides builtins to force
+this conversion, `__unsafe_forge_bidi_indexable(type, pointer, char_count)` to
+convert pointer to a `__bidi_indexable` pointer of type with `char_count`
+bytes available and `__unsafe_forge_single(type, pointer)` to convert pointer
to a single pointer of type type. The following examples show the usage of these
-functions. Function ``example_forge_bidi()`` gets an external buffer from an
-unsafe library by calling ``get_buf()`` which returns ``void
-*__unsafe_indexable.`` Under the type rules, this cannot be directly assigned to
-``void *buf`` (implicitly ``void *__bidi_indexable``). Thus,
-``__unsafe_forge_bidi_indexable`` is used to manually create a
-``__bidi_indexable`` from the unsafe buffer.
-
-.. code-block:: c
-
- // unsafe_library.h
- void *__unsafe_indexable get_buf(void);
- size_t get_buf_size(void);
-
- // my_source1.c (enables -fbounds-safety)
- #include "unsafe_library.h"
- void example_forge_bidi(void) {
- void *buf =
- __unsafe_forge_bidi_indexable(void *, get_buf(), get_buf_size());
- // ...
- }
-
- // my_source2.c (enables -fbounds-safety)
- #include <stdio.h>
- void example_forge_single(void) {
- FILE *fp = __unsafe_forge_single(FILE *, fopen("mypath", "rb"));
- // ...
- }
-
-* Function ``example_forge_single`` takes a file handle by calling fopen defined
- in system header ``stdio.h``. Assuming ``stdio.h`` did not adopt
- ``-fbounds-safety``, the return type of ``fopen`` would implicitly be ``FILE
- *__unsafe_indexable`` and thus it cannot be directly assigned to ``FILE *fp``
- in the bounds-safe source. To allow this operation, ``__unsafe_forge_single``
- is used to create a ``__single`` from the return value of ``fopen``.
-
-* Similar to ``__unsafe_indexable``, any non-pointer type (including ``int``,
- ``intptr_t``, ``uintptr_t``, etc.) cannot be converted to any safe pointer
- type because these don't have bounds information. ``__unsafe_forge_single`` or
- ``__unsafe_forge_bidi_indexable`` must be used to force the conversion.
-
-* Any safe pointer types can cast to ``__unsafe_indexable`` because it doesn't
+functions. Function `example_forge_bidi()` gets an external buffer from an
+unsafe library by calling `get_buf()` which returns `void
+*__unsafe_indexable.` Under the type rules, this cannot be directly assigned to
+`void *buf` (implicitly `void *__bidi_indexable`). Thus,
+`__unsafe_forge_bidi_indexable` is used to manually create a
+`__bidi_indexable` from the unsafe buffer.
+
+```c
+// unsafe_library.h
+void *__unsafe_indexable get_buf(void);
+size_t get_buf_size(void);
+
+// my_source1.c (enables -fbounds-safety)
+#include "unsafe_library.h"
+void example_forge_bidi(void) {
+ void *buf =
+ __unsafe_forge_bidi_indexable(void *, get_buf(), get_buf_size());
+ // ...
+}
+
+// my_source2.c (enables -fbounds-safety)
+#include <stdio.h>
+void example_forge_single(void) {
+ FILE *fp = __unsafe_forge_single(FILE *, fopen("mypath", "rb"));
+ // ...
+}
+```
+
+- Function `example_forge_single` takes a file handle by calling fopen defined
+ in system header `stdio.h`. Assuming `stdio.h` did not adopt
+ `-fbounds-safety`, the return type of `fopen` would implicitly be `FILE
+ *__unsafe_indexable` and thus it cannot be directly assigned to `FILE *fp`
+ in the bounds-safe source. To allow this operation, `__unsafe_forge_single`
+ is used to create a `__single` from the return value of `fopen`.
+
+- Similar to `__unsafe_indexable`, any non-pointer type (including `int`,
+ `intptr_t`, `uintptr_t`, etc.) cannot be converted to any safe pointer
+ type because these don't have bounds information. `__unsafe_forge_single` or
+ `__unsafe_forge_bidi_indexable` must be used to force the conversion.
+
+- Any safe pointer types can cast to `__unsafe_indexable` because it doesn't
have any invariant to maintain.
-* ``__single`` casts to ``__bidi_indexable`` if the pointee type has a known
- size. After the conversion, the resulting ``__bidi_indexable`` has the size of
- a single object of the pointee type of ``__single``. ``__single`` cannot cast
- to ``__bidi_indexable`` if the pointee type is incomplete or sizeless. For
- example, ``void *__single`` cannot convert to ``void *__bidi_indexable``
+- `__single` casts to `__bidi_indexable` if the pointee type has a known
+ size. After the conversion, the resulting `__bidi_indexable` has the size of
+ a single object of the pointee type of `__single`. `__single` cannot cast
+ to `__bidi_indexable` if the pointee type is incomplete or sizeless. For
+ example, `void *__single` cannot convert to `void *__bidi_indexable`
because void is an incomplete type and thus the compiler cannot correctly
determine the upper bound of a single void pointer.
-* Similarly, ``__single`` can cast to ``__indexable`` if the pointee type has a
- known size. The resulting ``__indexable`` has the size of a single object of
+- Similarly, `__single` can cast to `__indexable` if the pointee type has a
+ known size. The resulting `__indexable` has the size of a single object of
the pointee type.
-* ``__single`` casts to ``__counted_by(E)`` only if ``E`` is 0 or 1.
+- `__single` casts to `__counted_by(E)` only if `E` is 0 or 1.
-* ``__single`` can cast to ``__single`` including when they have different
+- `__single` can cast to `__single` including when they have different
pointee types as long as it is allowed in the underlying C standard.
- ``-fbounds-safety`` doesn't guarantee type safety.
+ `-fbounds-safety` doesn't guarantee type safety.
-* ``__bidi_indexable`` and ``__indexable`` can cast to ``__single``. The
+- `__bidi_indexable` and `__indexable` can cast to `__single`. The
compiler may insert run-time checks to ensure the pointer has at least a
single element or is a null pointer.
-* ``__bidi_indexable`` casts to ``__indexable`` if the pointer does not have an
+- `__bidi_indexable` casts to `__indexable` if the pointer does not have an
underflow. The compiler may insert run-time checks to ensure the pointer is
not below the lower bound.
-* ``__indexable`` casts to ``__bidi_indexable``. The resulting
- ``__bidi_indexable`` gets the lower bound same as the pointer value.
+- `__indexable` casts to `__bidi_indexable`. The resulting
+ `__bidi_indexable` gets the lower bound same as the pointer value.
-* A type conversion may involve both a bitcast and a bounds annotation cast. For
- example, casting from ``int *__bidi_indexable`` to ``char *__single`` involves
- a bitcast (``int *`` to ``char *``) and a bounds annotation cast
- (``__bidi_indexable`` to ``__single``). In this case, the compiler performs
- the bitcast and then converts the bounds annotation. This means, ``int
- *__bidi_indexable`` will be converted to ``char *__bidi_indexable`` and then
- to ``char *__single``.
+- A type conversion may involve both a bitcast and a bounds annotation cast. For
+ example, casting from `int *__bidi_indexable` to `char *__single` involves
+ a bitcast (`int *` to `char *`) and a bounds annotation cast
+ (`__bidi_indexable` to `__single`). In this case, the compiler performs
+ the bitcast and then converts the bounds annotation. This means, `int
+ *__bidi_indexable` will be converted to `char *__bidi_indexable` and then
+ to `char *__single`.
-* ``__terminated_by(T)`` cannot cast to any safe pointer type without the same
- ``__terminated_by(T)`` attribute. To perform the cast, programmers can use an
- intrinsic function such as ``__unsafe_terminated_by_to_indexable(P)`` to force
+- `__terminated_by(T)` cannot cast to any safe pointer type without the same
+ `__terminated_by(T)` attribute. To perform the cast, programmers can use an
+ intrinsic function such as `__unsafe_terminated_by_to_indexable(P)` to force
the conversion.
-* ``__terminated_by(T)`` can cast to ``__unsafe_indexable``.
+- `__terminated_by(T)` can cast to `__unsafe_indexable`.
-* Any type without ``__terminated_by(T)`` cannot cast to ``__terminated_by(T)``
+- Any type without `__terminated_by(T)` cannot cast to `__terminated_by(T)`
without explicitly using an intrinsic function to allow it.
- + ``__unsafe_terminated_by_from_indexable(T, PTR [, PTR_TO_TERM])`` casts any
- safe pointer PTR to a ``__terminated_by(T)`` pointer. ``PTR_TO_TERM`` is an
+ - `__unsafe_terminated_by_from_indexable(T, PTR [, PTR_TO_TERM])` casts any
+ safe pointer PTR to a `__terminated_by(T)` pointer. `PTR_TO_TERM` is an
optional argument where the programmer can provide the exact location of the
terminator. With this argument, the function can skip reading the entire
array in order to locate the end of the pointer (or the upper bound).
- Providing an incorrect ``PTR_TO_TERM`` causes a run-time trap.
+ Providing an incorrect `PTR_TO_TERM` causes a run-time trap.
+ - `__unsafe_forge_terminated_by(T, P, E)` creates `T __terminated_by(E)`
+ pointer given any pointer `P`. Tmust be a pointer type.
- + ``__unsafe_forge_terminated_by(T, P, E)`` creates ``T __terminated_by(E)``
- pointer given any pointer ``P``. Tmust be a pointer type.
-
-Portability with toolchains that do not support the extension
--------------------------------------------------------------
+### Portability with toolchains that do not support the extension
The language model is designed so that it doesn't alter the semantics of the
original C program, other than introducing deterministic traps where otherwise
the behavior is undefined and/or unsafe. Clang provides a toolchain header
-(``ptrcheck.h``) that macro-defines the annotations as type attributes when
-``-fbounds-safety`` is enabled and defines them to empty when the extension is
-disabled. Thus, the code adopting ``-fbounds-safety`` can compile with
+(`ptrcheck.h`) that macro-defines the annotations as type attributes when
+`-fbounds-safety` is enabled and defines them to empty when the extension is
+disabled. Thus, the code adopting `-fbounds-safety` can compile with
toolchains that do not support this extension, by including the header or adding
macros to define the annotations to empty. For example, the toolchain not
-supporting this extension may not have a header defining ``__counted_by``, so
-the code using ``__counted_by`` must define it as nothing or include a header
+supporting this extension may not have a header defining `__counted_by`, so
+the code using `__counted_by` must define it as nothing or include a header
that has the define.
-.. code-block:: c
-
- #if defined(__has_feature) && __has_feature(bounds_safety)
- #define __counted_by(T) __attribute__((__counted_by__(T)))
- // ... other bounds annotations
- #else
- #define __counted_by(T) // defined as nothing
- // ... other bounds annotations
- #endif
+```c
+#if defined(__has_feature) && __has_feature(bounds_safety)
+#define __counted_by(T) __attribute__((__counted_by__(T)))
+// ... other bounds annotations
+#else
+#define __counted_by(T) // defined as nothing
+// ... other bounds annotations
+#endif
- // expands to `void foo(int * ptr, size_t count);`
- // when extension is not enabled or not available
- void foo(int *__counted_by(count) ptr, size_t count);
+// expands to `void foo(int * ptr, size_t count);`
+// when extension is not enabled or not available
+void foo(int *__counted_by(count) ptr, size_t count);
+```
-Other potential applications of bounds annotations
-==================================================
+## Other potential applications of bounds annotations
-The bounds annotations provided by the ``-fbounds-safety`` programming model
+The bounds annotations provided by the `-fbounds-safety` programming model
have potential use cases beyond the language extension itself. For example,
static and dynamic analysis tools could use the bounds information to improve
-diagnostics for out-of-bounds accesses, even if ``-fbounds-safety`` is not used.
+diagnostics for out-of-bounds accesses, even if `-fbounds-safety` is not used.
The bounds annotations could be used to improve C interoperability with
bounds-safe languages, providing a better mapping to bounds-safe types in the
safe language interface. The bounds annotations can also serve as documentation
specifying the relationship between declarations.
-Limitations
-===========
+## Limitations
-``-fbounds-safety`` aims to bring the bounds safety guarantee to the C language,
+`-fbounds-safety` aims to bring the bounds safety guarantee to the C language,
and it does not guarantee other types of memory safety properties. Consequently,
it may not prevent some of the secondary bounds safety violations caused by
other types of safety violations such as type confusion. For instance,
-``-fbounds-safety`` does not perform type-safety checks on conversions between
-``__single`` pointers of different pointee types (e.g., ``char *__single`` →
-``void *__single`` → ``int *__single``) beyond what the foundation languages
+`-fbounds-safety` does not perform type-safety checks on conversions between
+`__single` pointers of different pointee types (e.g., `char *__single` →
+`void *__single` → `int *__single`) beyond what the foundation languages
(C/C++) already offer.
-``-fbounds-safety`` heavily relies on run-time checks to keep the bounds safety
+`-fbounds-safety` heavily relies on run-time checks to keep the bounds safety
and the soundness of the type system. This may incur significant code size
overhead in unoptimized builds and leave some of the adoption mistakes to be
caught only at run time. This is not a fundamental limitation, however, because
incrementally adding necessary static analysis will allow us to catch issues
early on and remove unnecessary bounds checks in unoptimized builds.
-Try it out
-==========
+## Try it out
Your feedback on the programming model is valuable. You may want to follow the
-instruction in :doc:`BoundsSafetyAdoptionGuide` to play with ``-fbounds-safety``
-and please send your feedback to `Yeoul Na <mailto:yeoul_na at apple.com>`_.
+instruction in {doc}`BoundsSafetyAdoptionGuide` to play with `-fbounds-safety`
+and please send your feedback to [Yeoul Na](mailto:yeoul_na at apple.com).
+
diff --git a/clang/docs/BoundsSafetyAdoptionGuide.md b/clang/docs/BoundsSafetyAdoptionGuide.md
index 09deae37abbf7..f7a2d3532d4c6 100644
--- a/clang/docs/BoundsSafetyAdoptionGuide.md
+++ b/clang/docs/BoundsSafetyAdoptionGuide.md
@@ -1,90 +1,82 @@
-======================================
-Adoption Guide for ``-fbounds-safety``
-======================================
+# Adoption Guide for `-fbounds-safety`
-.. contents::
- :local:
+```{contents}
+:local: true
+```
-Where to get ``-fbounds-safety``
-================================
+## Where to get `-fbounds-safety`
-The open sourcing to llvm.org's ``llvm-project`` is still on going and the
+The open sourcing to llvm.org's `llvm-project` is still on going and the
feature is not available yet. In the mean time, the preview implementation is
available
-`here <https://github.com/swiftlang/llvm-project/tree/stable/20240723>`_ in a
-fork of ``llvm-project``. Please follow
-`Building LLVM with CMake <https://llvm.org/docs/CMake.html>`_ to build the
+[here](https://github.com/swiftlang/llvm-project/tree/stable/20240723) in a
+fork of `llvm-project`. Please follow
+[Building LLVM with CMake](https://llvm.org/docs/CMake.html) to build the
compiler.
-Feature flag
-============
+## Feature flag
-Pass ``-fbounds-safety`` as a Clang compilation flag for the C file that you
+Pass `-fbounds-safety` as a Clang compilation flag for the C file that you
want to adopt. We recommend adopting the model file by file, because adoption
requires some effort to add bounds annotations and fix compiler diagnostics.
-Include ``ptrcheck.h``
-======================
+## Include `ptrcheck.h`
-``ptrcheck.h`` is a Clang toolchain header to provide definition of the bounds
-annotations such as ``__counted_by``, ``__counted_by_or_null``, ``__sized_by``,
+`ptrcheck.h` is a Clang toolchain header to provide definition of the bounds
+annotations such as `__counted_by`, `__counted_by_or_null`, `__sized_by`,
etc. In the LLVM source tree, the header is located in
-``llvm-project/clang/lib/Headers/ptrcheck.h``.
+`llvm-project/clang/lib/Headers/ptrcheck.h`.
-
-Add bounds annotations on pointers as necessary
-===============================================
+## Add bounds annotations on pointers as necessary
Annotate pointers on struct fields and function parameters if they are pointing
to an array of object, with appropriate bounds annotations. Please see
-:doc:`BoundsSafety` to learn what kind of bounds annotations are available and
+{doc}`BoundsSafety` to learn what kind of bounds annotations are available and
their semantics. Note that local pointer variables typically don't need bounds
-annotations because they are implicitly a wide pointer (``__bidi_indexable``)
+annotations because they are implicitly a wide pointer (`__bidi_indexable`)
that automatically carries the bounds information.
-Address compiler diagnostics
-============================
+## Address compiler diagnostics
-Once you pass ``-fbounds-safety`` to compile a C file, you will see some new
-compiler warnings and errors, which guide adoption of ``-fbounds-safety``.
+Once you pass `-fbounds-safety` to compile a C file, you will see some new
+compiler warnings and errors, which guide adoption of `-fbounds-safety`.
Consider the following example:
-.. code-block:: c
-
- #include <ptrcheck.h>
-
- void init_buf(int *p, int n) {
- for (int i = 0; i < n; ++i)
- p[i] = 0; // error: array subscript on single pointer 'p' must use a constant index of 0 to be in bounds
- }
-
-The parameter ``int *p`` doesn't have a bounds annotation, so the compiler will
-complain about the code indexing into it (``p[i]``) as it assumes that ``p`` is
-pointing to a single ``int`` object or null. To address the diagnostics, you
-should add a bounds annotation on ``int *p`` so that the compiler can reason
-about the safety of the array subscript. In the following example, ``p`` is now
-``int *__counted_by(n)``, so the compiler will allow the array subscript with
+```c
+#include <ptrcheck.h>
+
+void init_buf(int *p, int n) {
+ for (int i = 0; i < n; ++i)
+ p[i] = 0; // error: array subscript on single pointer 'p' must use a constant index of 0 to be in bounds
+}
+```
+
+The parameter `int *p` doesn't have a bounds annotation, so the compiler will
+complain about the code indexing into it (`p[i]`) as it assumes that `p` is
+pointing to a single `int` object or null. To address the diagnostics, you
+should add a bounds annotation on `int *p` so that the compiler can reason
+about the safety of the array subscript. In the following example, `p` is now
+`int *__counted_by(n)`, so the compiler will allow the array subscript with
additional run-time checks as necessary.
-.. code-block:: c
-
- #include <ptrcheck.h>
+```c
+#include <ptrcheck.h>
- void init_buf(int *__counted_by(n) p, int n) {
- for (int i = 0; i < n; ++i)
- p[i] = 0; // ok; `p` is now has a type with bounds annotation.
- }
+void init_buf(int *__counted_by(n) p, int n) {
+ for (int i = 0; i < n; ++i)
+ p[i] = 0; // ok; `p` is now has a type with bounds annotation.
+}
+```
-Run test suites to fix new run-time traps
-=========================================
+## Run test suites to fix new run-time traps
-Adopting ``-fbounds-safety`` may cause your program to trap if it violates
+Adopting `-fbounds-safety` may cause your program to trap if it violates
bounds safety or it has incorrect adoption. Thus, it is necessary to perform
run-time testing of your program to gain confidence that it won't trap at
run time.
-Repeat the process for each remaining file
-==========================================
+## Repeat the process for each remaining file
Once you've done with adopting a single C file, please repeat the same process
-for each remaining C file that you want to adopt.
\ No newline at end of file
+for each remaining C file that you want to adopt.
+
diff --git a/clang/docs/BoundsSafetyImplPlans.md b/clang/docs/BoundsSafetyImplPlans.md
index b374b0a0c68a4..f8b4501d91272 100644
--- a/clang/docs/BoundsSafetyImplPlans.md
+++ b/clang/docs/BoundsSafetyImplPlans.md
@@ -1,60 +1,53 @@
-============================================
-Implementation plans for ``-fbounds-safety``
-============================================
+# Implementation plans for `-fbounds-safety`
-.. contents::
- :local:
+```{contents}
+:local: true
+```
-Gradual updates with experimental flag
-======================================
+## Gradual updates with experimental flag
The feature will be implemented as a series of smaller PRs and we will guard our
-implementation with an experimental flag ``-fexperimental-bounds-safety`` until
+implementation with an experimental flag `-fexperimental-bounds-safety` until
the usable model is fully available. Once the model is ready for use, we will
-expose the flag ``-fbounds-safety``.
+expose the flag `-fbounds-safety`.
-Possible patch sets
--------------------
+### Possible patch sets
-* External bounds annotations and the (late) parsing logic.
-* Internal bounds annotations (wide pointers) and their parsing logic.
-* Clang code generation for wide pointers with debug information.
-* Pointer cast semantics involving bounds annotations (this could be divided
+- External bounds annotations and the (late) parsing logic.
+- Internal bounds annotations (wide pointers) and their parsing logic.
+- Clang code generation for wide pointers with debug information.
+- Pointer cast semantics involving bounds annotations (this could be divided
into multiple sub-PRs).
-* CFG analysis for pairs of related pointer and count assignments and the likes.
-* Bounds check expressions in AST and the Clang code generation (this could also
+- CFG analysis for pairs of related pointer and count assignments and the likes.
+- Bounds check expressions in AST and the Clang code generation (this could also
be divided into multiple sub-PRs).
-Proposed implementation
-=======================
+## Proposed implementation
-External bounds annotations
----------------------------
+### External bounds annotations
The bounds annotations are C type attributes appertaining to pointer types. If
-an attribute is added to the position of a declaration attribute, e.g., ``int
-*ptr __counted_by(size)``, the attribute appertains to the outermost pointer
-type of the declaration (``int *``).
+an attribute is added to the position of a declaration attribute, e.g., `int
+*ptr __counted_by(size)`, the attribute appertains to the outermost pointer
+type of the declaration (`int *`).
-New sugar types
----------------
+### New sugar types
An external bounds annotation creates a type sugar of the underlying pointer
-types. We will introduce a new sugar type, ``DynamicBoundsPointerType`` to
-represent ``__counted_by`` or ``__sized_by``. Using ``AttributedType`` would not
+types. We will introduce a new sugar type, `DynamicBoundsPointerType` to
+represent `__counted_by` or `__sized_by`. Using `AttributedType` would not
be sufficient because the type needs to hold the count or size expression as
well as some metadata necessary for analysis, while this type may be implemented
-through inheritance from ``AttributedType``. Treating the annotations as type
+through inheritance from `AttributedType`. Treating the annotations as type
sugars means two types with incompatible external bounds annotations may be
considered canonically the same types. This is sometimes necessary, for example,
-to make the ``__counted_by`` and friends not participate in function
+to make the `__counted_by` and friends not participate in function
overloading. However, this design requires a separate logic to walk through the
entire type hierarchy to check type compatibility of bounds annotations.
-Late parsing for C
-------------------
+### Late parsing for C
-A bounds annotation such as ``__counted_by(count)`` can be added to type of a
+A bounds annotation such as `__counted_by(count)` can be added to type of a
struct field declaration where count is another field of the same struct
declared later. Similarly, the annotation may apply to type of a function
parameter declaration which precedes the parameter count in the same function.
@@ -65,14 +58,13 @@ while the C declaration attributes and C/C++ type attributes do not have the
same logic. This requires introducing late parsing logic for C/C++ type
attributes.
-Internal bounds annotations
----------------------------
+### Internal bounds annotations
-``__indexable`` and ``__bidi_indexable`` alter pointer representations to be
+`__indexable` and `__bidi_indexable` alter pointer representations to be
equivalent to a struct with the pointer and the corresponding bounds fields.
Despite this difference in their representations, they are still pointers in
terms of types of operations that are allowed and their semantics. For instance,
-a pointer dereference on a ``__bidi_indexable`` pointer will return the
+a pointer dereference on a `__bidi_indexable` pointer will return the
dereferenced value same as plain C pointers, modulo the extra bounds checks
being performed before dereferencing the wide pointer. This means mapping the
wide pointers to struct types with equivalent layout won’t be sufficient. To
@@ -83,16 +75,15 @@ while they are still treated as pointers.
In LLVM IR, wide pointers will be emitted as structs of equivalent
representations. Clang CodeGen will handle them as Aggregate in
-``TypeEvaluationKind (TEK)``. ``AggExprEmitter`` was extended to handle pointer
-operations returning wide pointers. Alternatively, a new ``TEK`` and an
+`TypeEvaluationKind (TEK)`. `AggExprEmitter` was extended to handle pointer
+operations returning wide pointers. Alternatively, a new `TEK` and an
expression emitter dedicated to wide pointers could be introduced.
-Default bounds annotations
---------------------------
+### Default bounds annotations
-The model may implicitly add ``__bidi_indexable`` or ``__single`` depending on
-the context of the declaration that has the pointer type. ``__bidi_indexable``
-implicitly adds to local variables, while ``__single`` implicitly adds to
+The model may implicitly add `__bidi_indexable` or `__single` depending on
+the context of the declaration that has the pointer type. `__bidi_indexable`
+implicitly adds to local variables, while `__single` implicitly adds to
pointer types specifying struct fields, function parameters, or global
variables. This means the parser may first create the pointer type without any
default pointer attribute and then recreate the type once the parser has the
@@ -101,154 +92,144 @@ declaration context and determined the default attribute accordingly.
This also requires the parser to reset the type of the declaration with the
newly created type with the right default attribute.
-Promotion expression
---------------------
+### Promotion expression
A new expression will be introduced to represent the conversion from a pointer
-with an external bounds annotation, such as ``__counted_by``, to
-``__bidi_indexable``. This type of conversion cannot be handled by normal
+with an external bounds annotation, such as `__counted_by`, to
+`__bidi_indexable`. This type of conversion cannot be handled by normal
CastExprs because it requires an extra subexpression(s) to provide the bounds
information necessary to create a wide pointer.
-Bounds check expression
------------------------
+### Bounds check expression
-Bounds checks are part of semantics defined in the ``-fbounds-safety`` language
+Bounds checks are part of semantics defined in the `-fbounds-safety` language
model. Hence, exposing the bounds checks and other semantic actions in the AST
is desirable. A new expression for bounds checks has been added to the AST. The
-bounds check expression has a ``BoundsCheckKind`` to indicate the kind of checks
+bounds check expression has a `BoundsCheckKind` to indicate the kind of checks
and has the additional sub-expressions that are necessary to perform the check
according to the kind.
-Paired assignment check
------------------------
+### Paired assignment check
-``-fbounds-safety`` enforces that variables or fields related with the same
-external bounds annotation (e.g., ``buf`` and ``count`` related with
-``__counted_by`` in the example below) must be updated side by side within the
+`-fbounds-safety` enforces that variables or fields related with the same
+external bounds annotation (e.g., `buf` and `count` related with
+`__counted_by` in the example below) must be updated side by side within the
same basic block and without side effect in between.
-.. code-block:: c
+```c
+typedef struct {
+ int *__counted_by(count) buf; size_t count;
+} sized_buf_t;
- typedef struct {
- int *__counted_by(count) buf; size_t count;
- } sized_buf_t;
-
- void alloc_buf(sized_buf_t *sbuf, size_t nelems) {
- sbuf->buf = (int *)malloc(sizeof(int) * nelems);
- sbuf->count = nelems;
- }
+void alloc_buf(sized_buf_t *sbuf, size_t nelems) {
+ sbuf->buf = (int *)malloc(sizeof(int) * nelems);
+ sbuf->count = nelems;
+}
+```
To implement this rule, the compiler requires a linear representation of
statements to understand the ordering and the adjacency between the two or more
assignments. The Clang CFG is used to implement this analysis as Clang CFG
-provides a linear view of statements within each ``CFGBlock`` (Clang
-``CFGBlock`` represents a single basic block in a source-level CFG).
+provides a linear view of statements within each `CFGBlock` (Clang
+`CFGBlock` represents a single basic block in a source-level CFG).
-Bounds check optimizations
---------------------------
+### Bounds check optimizations
-In ``-fbounds-safety``, the Clang frontend emits run-time checks for every
+In `-fbounds-safety`, the Clang frontend emits run-time checks for every
memory dereference if the type system or analyses in the frontend couldn’t
verify its bounds safety. The implementation relies on LLVM optimizations to
remove redundant run-time checks. Using this optimization strategy, if the
original source code already has bounds checks, the fewer additional checks
-``-fbounds-safety`` will introduce. The LLVM ``ConstraintElimination`` pass is
+`-fbounds-safety` will introduce. The LLVM `ConstraintElimination` pass is
designed to remove provable redundant checks (please check Florian Hahn’s
presentation in 2021 LLVM Dev Meeting and the implementation to learn more). In
-the following example, ``-fbounds-safety`` implicitly adds the redundant bounds
+the following example, `-fbounds-safety` implicitly adds the redundant bounds
checks that the optimizer can remove:
-.. code-block:: c
-
- void fill_array_with_indices(int *__counted_by(count) p, size_t count) {
- for (size_t i = 0; i < count; ++i) {
- // implicit bounds checks:
- // if (p + i < p || p + i + 1 > p + count) trap();
- p[i] = i;
- }
+```c
+void fill_array_with_indices(int *__counted_by(count) p, size_t count) {
+ for (size_t i = 0; i < count; ++i) {
+ // implicit bounds checks:
+ // if (p + i < p || p + i + 1 > p + count) trap();
+ p[i] = i;
}
+}
+```
-``ConstraintElimination`` collects the following facts and determines if the
+`ConstraintElimination` collects the following facts and determines if the
bounds checks can be safely removed:
-* Inside the for-loop, ``0 <= i < count``, hence ``1 <= i + 1 <= count``.
-* Pointer arithmetic ``p + count`` in the if-condition doesn’t wrap.
-* ``-fbounds-safety`` treats pointer arithmetic overflow as deterministically
+- Inside the for-loop, `0 <= i < count`, hence `1 <= i + 1 <= count`.
+- Pointer arithmetic `p + count` in the if-condition doesn’t wrap.
+- `-fbounds-safety` treats pointer arithmetic overflow as deterministically
two’s complement computation, not an undefined behavior. Therefore,
getelementptr does not typically have inbounds keyword. However, the compiler
- does emit inbounds for ``p + count`` in this case because
- ``__counted_by(count)`` has the invariant that p has at least as many as
- elements as count. Using this information, ``ConstraintElimination`` is able
- to determine ``p + count`` doesn’t wrap.
-* Accordingly, ``p + i`` and ``p + i + 1`` also don’t wrap.
-* Therefore, ``p <= p + i`` and ``p + i + 1 <= p + count``.
-* The if-condition simplifies to false and becomes dead code that the subsequent
+ does emit inbounds for `p + count` in this case because
+ `__counted_by(count)` has the invariant that p has at least as many as
+ elements as count. Using this information, `ConstraintElimination` is able
+ to determine `p + count` doesn’t wrap.
+- Accordingly, `p + i` and `p + i + 1` also don’t wrap.
+- Therefore, `p <= p + i` and `p + i + 1 <= p + count`.
+- The if-condition simplifies to false and becomes dead code that the subsequent
optimization passes can remove.
-``OptRemarks`` can be utilized to provide insights into performance tuning. It
+`OptRemarks` can be utilized to provide insights into performance tuning. It
has the capability to report on checks that it cannot eliminate, possibly with
reasons, allowing programmers to adjust their code to unlock further
optimizations.
-Debugging
-=========
+## Debugging
-Internal bounds annotations
----------------------------
+### Internal bounds annotations
Internal bounds annotations change a pointer into a wide pointer. The debugger
needs to understand that wide pointers are essentially pointers with a struct
layout. To handle this, a wide pointer is described as a record type in the
debug info. The type name has a special name prefix (e.g.,
-``__bounds_safety$bidi_indexable``) which can be recognized by a debug info
+`__bounds_safety$bidi_indexable`) which can be recognized by a debug info
consumer to provide support that goes beyond showing the internal structure of
the wide pointer. There are no DWARF extensions needed to support wide pointers.
In our implementation, LLDB recognizes wide pointer types by name and
reconstructs them as wide pointer Clang AST types for use in the expression
evaluator.
-External bounds annotations
----------------------------
+### External bounds annotations
Similar to internal bounds annotations, external bound annotations are described
as a typedef to their underlying pointer type in the debug info, and the bounds
are encoded as strings in the typedef’s name (e.g.,
-``__bounds_safety$counted_by:N``).
+`__bounds_safety$counted_by:N`).
-Recognizing ``-fbounds-safety`` traps
--------------------------------------
+### Recognizing `-fbounds-safety` traps
-Clang emits debug info for ``-fbounds-safety`` traps as inlined functions, where
+Clang emits debug info for `-fbounds-safety` traps as inlined functions, where
the function name encodes the error message. LLDB implements a frame recognizer
to surface a human-readable error cause to the end user. A debug info consumer
that is unaware of this sees an inlined function whose name encodes an error
-message (e.g., : ``__bounds_safety$Bounds check failed``).
+message (e.g., : `__bounds_safety$Bounds check failed`).
-Expression Parsing
-------------------
+### Expression Parsing
In our implementation, LLDB’s expression evaluator does not enable the
-``-fbounds-safety`` language option because it’s currently unable to fully
+`-fbounds-safety` language option because it’s currently unable to fully
reconstruct the pointers with external bounds annotations, and also because the
evaluator operates in C++ mode, utilizing C++ reference types, while
-``-fbounds-safety`` does not currently support C++. This means LLDB’s expression
-evaluator can only evaluate a subset of the ``-fbounds-safety`` language model.
+`-fbounds-safety` does not currently support C++. This means LLDB’s expression
+evaluator can only evaluate a subset of the `-fbounds-safety` language model.
Specifically, it’s capable of evaluating the wide pointers that already exist in
the source code. All other expressions are evaluated according to C/C++
semantics.
-C++ support
-===========
+## C++ support
C++ has multiple options to write code in a bounds-safe manner, such as
following the bounds-safety core guidelines and/or using hardened libc++ along
-with the `C++ Safe Buffer model
-<https://discourse.llvm.org/t/rfc-c-buffer-hardening/65734>`_. However, these
+with the [C++ Safe Buffer model](https://discourse.llvm.org/t/rfc-c-buffer-hardening/65734). However, these
techniques may require ABI changes and may not be applicable to code
interoperating with C. When the ABI of an existing program needs to be preserved
-and for headers shared between C and C++, ``-fbounds-safety`` offers a potential
+and for headers shared between C and C++, `-fbounds-safety` offers a potential
solution.
-``-fbounds-safety`` is not currently supported in C++, but we believe the
+`-fbounds-safety` is not currently supported in C++, but we believe the
general approach would be applicable for future efforts.
+
diff --git a/clang/docs/CIR/index.md b/clang/docs/CIR/index.md
index e4b2acba15502..a594ac50dbf67 100644
--- a/clang/docs/CIR/index.md
+++ b/clang/docs/CIR/index.md
@@ -1,23 +1,23 @@
-=======
-ClangIR
-=======
+# ClangIR
-.. warning::
- The project of upstreaming ClangIR support from the incubator repository is
- still in progress, and ClangIR is not included in a default clang build. The
- documentation may be incomplete and out-of-date.
+:::{warning}
+The project of upstreaming ClangIR support from the incubator repository is
+still in progress, and ClangIR is not included in a default clang build. The
+documentation may be incomplete and out-of-date.
+:::
ClangIR is a high-level representation in Clang that reflects aspects of the
C/C++ languages and their extensions. It is implemented using MLIR and occupies
a position between Clang's AST and LLVM IR.
-ClangIR Design Documents
-========================
+## ClangIR Design Documents
-.. toctree::
- :numbered:
- :maxdepth: 1
+```{toctree}
+:maxdepth: 1
+:numbered: true
+
+ABILowering
+CleanupAndEHDesign
+CodeDuplication
+```
- ABILowering
- CleanupAndEHDesign
- CodeDuplication
diff --git a/clang/docs/CXXTypeAwareAllocators.md b/clang/docs/CXXTypeAwareAllocators.md
index c65d20a5c7d47..fabaca7b534ef 100644
--- a/clang/docs/CXXTypeAwareAllocators.md
+++ b/clang/docs/CXXTypeAwareAllocators.md
@@ -1,12 +1,10 @@
-=========================
-C++ Type Aware Allocators
-=========================
+# C++ Type Aware Allocators
-.. contents::
- :local:
+```{contents}
+:local: true
+```
-Introduction
-============
+## Introduction
Clang includes an implementation of P2719 "Type-aware allocation and deallocation
functions".
@@ -19,27 +17,27 @@ allocators.
P2719 introduces a type-identity tag as valid parameter type for all allocation
operators. This tag is a default initialized value of type `std::type_identity<T>`
-where T is the type being allocated or deallocated. Unlike the other placement
+where T is the type being allocated or deallocated. Unlike the other placement
arguments this tag is passed as the first parameter to the operator.
The most basic use case is as follows
-.. code-block:: c++
+```c++
+#include <new>
+#include <type_traits>
- #include <new>
- #include <type_traits>
+struct S {
+ // ...
+};
- struct S {
- // ...
- };
+void *operator new(std::type_identity<S>, size_t, std::align_val_t);
+void operator delete(std::type_identity<S>, void *, size_t, std::align_val_t);
- void *operator new(std::type_identity<S>, size_t, std::align_val_t);
- void operator delete(std::type_identity<S>, void *, size_t, std::align_val_t);
-
- void f() {
- S *s = new S; // calls ::operator new(std::type_identity<S>(), sizeof(S), alignof(S))
- delete s; // calls ::operator delete(std::type_identity<S>(), s, sizeof(S), alignof(S))
- }
+void f() {
+ S *s = new S; // calls ::operator new(std::type_identity<S>(), sizeof(S), alignof(S))
+ delete s; // calls ::operator delete(std::type_identity<S>(), s, sizeof(S), alignof(S))
+}
+```
While this functionality alone is powerful and useful, the true power comes
by using templates. In addition to adding the type-identity tag, P2719 allows
@@ -53,38 +51,38 @@ This allows arbitrarily constrained definitions of the operators that resolve
as would be expected for any other template function resolution, e.g (only
showing `operator new` for brevity)
-.. code-block:: c++
-
- template <typename T, unsigned Size> struct Array {
- T buffer[Size];
- };
+```c++
+template <typename T, unsigned Size> struct Array {
+ T buffer[Size];
+};
- // Starting with a concrete type
- void *operator new(std::type_identity<Array<int, 5>>, size_t, std::align_val_t);
+// Starting with a concrete type
+void *operator new(std::type_identity<Array<int, 5>>, size_t, std::align_val_t);
- // Only care about five element arrays
- template <typename T>
- void *operator new(std::type_identity<Array<T, 5>>, size_t, std::align_val_t);
+// Only care about five element arrays
+template <typename T>
+void *operator new(std::type_identity<Array<T, 5>>, size_t, std::align_val_t);
- // An array of N floats
- template <unsigned N>
- void *operator new(std::type_identity<Array<float, N>>, size_t, std::align_val_t);
+// An array of N floats
+template <unsigned N>
+void *operator new(std::type_identity<Array<float, N>>, size_t, std::align_val_t);
- // Any array
- template <typename T, unsigned N>
- void *operator new(std::type_identity<Array<T, N>>, size_t, std::align_val_t);
+// Any array
+template <typename T, unsigned N>
+void *operator new(std::type_identity<Array<T, N>>, size_t, std::align_val_t);
- // A handy concept
- template <typename T> concept Polymorphic = std::is_polymorphic_v<T>;
+// A handy concept
+template <typename T> concept Polymorphic = std::is_polymorphic_v<T>;
- // Only applies is T is Polymorphic
- template <Polymorphic T, unsigned N>
- void *operator new(std::type_identity<Array<T, N>>, size_t, std::align_val_t);
+// Only applies is T is Polymorphic
+template <Polymorphic T, unsigned N>
+void *operator new(std::type_identity<Array<T, N>>, size_t, std::align_val_t);
- // Any even length array
- template <typename T, unsigned N>
- void *operator new(std::type_identity<Array<T, N>>, size_t, std::align_val_t)
- requires(N%2 == 0);
+// Any even length array
+template <typename T, unsigned N>
+void *operator new(std::type_identity<Array<T, N>>, size_t, std::align_val_t)
+ requires(N%2 == 0);
+```
Operator selection then proceeds according to the usual rules for choosing
the best/most constrained match.
@@ -92,19 +90,16 @@ the best/most constrained match.
Any declaration of a type aware operator new or operator delete must include a
matching complementary operator defined in the same scope.
-Notes
-=====
+## Notes
-Unconstrained Global Operators
-------------------------------
+### Unconstrained Global Operators
Declaring an unconstrained type aware global operator `new` or `delete` (or
`[]` variants) creates numerous hazards, similar to, but different from, those
created by attempting to replace the non-type aware global operators. For that
reason unconstrained operators are strongly discouraged.
-Mismatching Constraints
------------------------
+### Mismatching Constraints
When declaring global type aware operators you should ensure the constraints
applied to new and delete match exactly, and declare them together. This
@@ -112,8 +107,7 @@ limits the risk of having mismatching operators selected due to differing
constraints resulting in changes to prioritization when determining the most
viable candidate.
-Declarations Across Libraries
------------------------------
+### Declarations Across Libraries
Declaring a typed allocator for a type in a separate TU or library creates
similar hazards as different libraries and TUs may see (or select) different
@@ -121,35 +115,34 @@ definitions.
Under this model something like this would be risky
-.. code-block:: c++
-
- template<typename T>
- void *operator new(std::type_identity<std::vector<T>>, size_t, std::align_val_t);
+```c++
+template<typename T>
+void *operator new(std::type_identity<std::vector<T>>, size_t, std::align_val_t);
+```
However this hazard is not present simply due to the use of the a type from
another library:
-.. code-block:: c++
-
- template<typename T>
- struct MyType {
- T thing;
- };
- template<typename T>
- void *operator new(std::type_identity<MyType<std::vector<T>>>, size_t, std::align_val_t);
+```c++
+template<typename T>
+struct MyType {
+ T thing;
+};
+template<typename T>
+void *operator new(std::type_identity<MyType<std::vector<T>>>, size_t, std::align_val_t);
+```
Here we see `std::vector` being used, but that is not the actual type being
allocated.
-Implicit and Placement Parameters
----------------------------------
+### Implicit and Placement Parameters
Type aware allocators are always passed both the implicit alignment and size
parameters in all cases. Explicit placement parameters are supported after the
mandatory implicit parameters.
-Publication
-===========
+## Publication
-`Type-aware allocation and deallocation functions <https://wg21.link/P2719>`_.
+[Type-aware allocation and deallocation functions](https://wg21.link/P2719).
Louis Dionne, Oliver Hunt.
+
diff --git a/clang/docs/ClangPlugins.md b/clang/docs/ClangPlugins.md
index 3bd9e963d48ab..6bb6778204f50 100644
--- a/clang/docs/ClangPlugins.md
+++ b/clang/docs/ClangPlugins.md
@@ -1,139 +1,125 @@
-=============
-Clang Plugins
-=============
+# Clang Plugins
Clang Plugins make it possible to run extra user defined actions during a
compilation. This document will provide a basic walkthrough of how to write and
run a Clang Plugin.
-Introduction
-============
+## Introduction
-Clang Plugins run FrontendActions over code. See the :doc:`FrontendAction
-tutorial <RAVFrontendAction>` on how to write a ``FrontendAction`` using the
-``RecursiveASTVisitor``. In this tutorial, we'll demonstrate how to write a
+Clang Plugins run FrontendActions over code. See the {doc}`FrontendAction
+tutorial <RAVFrontendAction>` on how to write a `FrontendAction` using the
+`RecursiveASTVisitor`. In this tutorial, we'll demonstrate how to write a
simple clang plugin.
-Writing a ``PluginASTAction``
-=============================
+## Writing a `PluginASTAction`
-The main difference from writing normal ``FrontendActions`` is that you can
-handle plugin command line options. The ``PluginASTAction`` base class declares
-a ``ParseArgs`` method which you have to implement in your plugin.
+The main difference from writing normal `FrontendActions` is that you can
+handle plugin command line options. The `PluginASTAction` base class declares
+a `ParseArgs` method which you have to implement in your plugin.
-.. code-block:: c++
-
- bool ParseArgs(const CompilerInstance &CI,
- const std::vector<std::string>& args) {
- for (unsigned i = 0, e = args.size(); i != e; ++i) {
- if (args[i] == "-some-arg") {
- // Handle the command line argument.
- }
+```c++
+bool ParseArgs(const CompilerInstance &CI,
+ const std::vector<std::string>& args) {
+ for (unsigned i = 0, e = args.size(); i != e; ++i) {
+ if (args[i] == "-some-arg") {
+ // Handle the command line argument.
}
- return true;
}
+ return true;
+}
+```
-Registering a plugin
-====================
+## Registering a plugin
A plugin is loaded from a dynamic library at runtime by the compiler. To
-register a plugin in a library, use ``FrontendPluginRegistry::Add<>``:
-
-.. code-block:: c++
-
- static FrontendPluginRegistry::Add<MyPlugin> X("my-plugin-name", "my plugin description");
+register a plugin in a library, use `FrontendPluginRegistry::Add<>`:
-Defining pragmas
-================
+```c++
+static FrontendPluginRegistry::Add<MyPlugin> X("my-plugin-name", "my plugin description");
+```
-Plugins can also define pragmas by declaring a ``PragmaHandler`` and
-registering it using ``PragmaHandlerRegistry::Add<>``:
+## Defining pragmas
-.. code-block:: c++
-
- // Define a pragma handler for #pragma example_pragma
- class ExamplePragmaHandler : public PragmaHandler {
- public:
- ExamplePragmaHandler() : PragmaHandler("example_pragma") { }
- void HandlePragma(Preprocessor &PP, PragmaIntroducer Introducer,
- Token &PragmaTok) {
- // Handle the pragma
- }
- };
+Plugins can also define pragmas by declaring a `PragmaHandler` and
+registering it using `PragmaHandlerRegistry::Add<>`:
- static PragmaHandlerRegistry::Add<ExamplePragmaHandler> Y("example_pragma","example pragma description");
+```c++
+// Define a pragma handler for #pragma example_pragma
+class ExamplePragmaHandler : public PragmaHandler {
+public:
+ ExamplePragmaHandler() : PragmaHandler("example_pragma") { }
+ void HandlePragma(Preprocessor &PP, PragmaIntroducer Introducer,
+ Token &PragmaTok) {
+ // Handle the pragma
+ }
+};
-Defining attributes
-===================
+static PragmaHandlerRegistry::Add<ExamplePragmaHandler> Y("example_pragma","example pragma description");
+```
-Plugins can define attributes by declaring a ``ParsedAttrInfo`` and registering
-it using ``ParsedAttrInfoRegister::Add<>``:
+## Defining attributes
-.. code-block:: c++
+Plugins can define attributes by declaring a `ParsedAttrInfo` and registering
+it using `ParsedAttrInfoRegister::Add<>`:
- class ExampleAttrInfo : public ParsedAttrInfo {
- public:
- ExampleAttrInfo() {
- Spellings.push_back({ParsedAttr::AS_GNU,"example"});
- }
- AttrHandling handleDeclAttribute(Sema &S, Decl *D,
- const ParsedAttr &Attr) const override {
- // Handle the attribute
- return AttributeApplied;
- }
- };
+```c++
+class ExampleAttrInfo : public ParsedAttrInfo {
+public:
+ ExampleAttrInfo() {
+ Spellings.push_back({ParsedAttr::AS_GNU,"example"});
+ }
+ AttrHandling handleDeclAttribute(Sema &S, Decl *D,
+ const ParsedAttr &Attr) const override {
+ // Handle the attribute
+ return AttributeApplied;
+ }
+};
- static ParsedAttrInfoRegistry::Add<ExampleAttrInfo> Z("example_attr","example attribute description");
+static ParsedAttrInfoRegistry::Add<ExampleAttrInfo> Z("example_attr","example attribute description");
+```
-The members of ``ParsedAttrInfo`` that a plugin attribute must define are:
+The members of `ParsedAttrInfo` that a plugin attribute must define are:
- * ``Spellings``, which must be populated with every `Spelling
- </doxygen/structclang_1_1ParsedAttrInfo_1_1Spelling.html>`_ of the
- attribute, each of which consists of an attribute syntax and how the
- attribute name is spelled for that syntax. If the syntax allows a scope then
- the spelling must be "scope::attr" if a scope is present or "::attr" if not.
+> - `Spellings`, which must be populated with every [Spelling](/doxygen/structclang_1_1ParsedAttrInfo_1_1Spelling.html) of the
+> attribute, each of which consists of an attribute syntax and how the
+> attribute name is spelled for that syntax. If the syntax allows a scope then
+> the spelling must be "scope::attr" if a scope is present or "::attr" if not.
-The members of ``ParsedAttrInfo`` that may need to be defined, depending on the
+The members of `ParsedAttrInfo` that may need to be defined, depending on the
attribute, are:
- * ``NumArgs`` and ``OptArgs``, which set the number of required and optional
- arguments to the attribute.
- * ``diagAppertainsToDecl``, which checks if the attribute has been used on the
- right kind of declaration and issues a diagnostic if not.
- * ``handleDeclAttribute``, which is the function that applies the attribute to
- a declaration. It is responsible for checking that the attribute's arguments
- are valid, and typically applies the attribute by adding an ``Attr`` to the
- ``Decl``. It returns either ``AttributeApplied``, to indicate that the
- attribute was successfully applied, or ``AttributeNotApplied`` if it wasn't.
- * ``diagAppertainsToStmt``, which checks if the attribute has been used on the
- right kind of statement and issues a diagnostic if not.
- * ``handleStmtAttribute``, which is the function that applies the attribute to
- a statement. It is responsible for checking that the attribute's arguments
- are valid, and typically applies the attribute by adding an ``Attr`` to the
- ``Stmt``. It returns either ``AttributeApplied``, to indicate that the
- attribute was successfully applied, or ``AttributeNotApplied`` if it wasn't.
- * ``diagLangOpts``, which checks if the attribute is permitted for the current
- language mode and issues a diagnostic if not.
- * ``existsInTarget``, which checks if the attribute is permitted for the given
- target.
-
-To see a working example of an attribute plugin, see `the Attribute.cpp example
-<https://github.com/llvm/llvm-project/blob/main/clang/examples/Attribute/Attribute.cpp>`_.
-
-Putting it all together
-=======================
-
-Let's look at an example plugin that prints top-level function names. This
+> - `NumArgs` and `OptArgs`, which set the number of required and optional
+> arguments to the attribute.
+> - `diagAppertainsToDecl`, which checks if the attribute has been used on the
+> right kind of declaration and issues a diagnostic if not.
+> - `handleDeclAttribute`, which is the function that applies the attribute to
+> a declaration. It is responsible for checking that the attribute's arguments
+> are valid, and typically applies the attribute by adding an `Attr` to the
+> `Decl`. It returns either `AttributeApplied`, to indicate that the
+> attribute was successfully applied, or `AttributeNotApplied` if it wasn't.
+> - `diagAppertainsToStmt`, which checks if the attribute has been used on the
+> right kind of statement and issues a diagnostic if not.
+> - `handleStmtAttribute`, which is the function that applies the attribute to
+> a statement. It is responsible for checking that the attribute's arguments
+> are valid, and typically applies the attribute by adding an `Attr` to the
+> `Stmt`. It returns either `AttributeApplied`, to indicate that the
+> attribute was successfully applied, or `AttributeNotApplied` if it wasn't.
+> - `diagLangOpts`, which checks if the attribute is permitted for the current
+> language mode and issues a diagnostic if not.
+> - `existsInTarget`, which checks if the attribute is permitted for the given
+> target.
+
+To see a working example of an attribute plugin, see [the Attribute.cpp example](https://github.com/llvm/llvm-project/blob/main/clang/examples/Attribute/Attribute.cpp).
+
+## Putting it all together
+
+Let's look at an example plugin that prints top-level function names. This
example is checked into the clang repository; please take a look at
-the `latest version of PrintFunctionNames.cpp
-<https://github.com/llvm/llvm-project/blob/main/clang/examples/PrintFunctionNames/PrintFunctionNames.cpp>`_.
-
-Running the plugin
-==================
+the [latest version of PrintFunctionNames.cpp](https://github.com/llvm/llvm-project/blob/main/clang/examples/PrintFunctionNames/PrintFunctionNames.cpp).
+## Running the plugin
-Using the compiler driver
---------------------------
+### Using the compiler driver
The Clang driver accepts the `-fplugin` option to load a plugin.
Clang plugins can receive arguments from the compiler driver command
@@ -141,21 +127,18 @@ line via the `fplugin-arg-<plugin name>-<argument>` option. Using this
method, the plugin name cannot contain dashes itself, but the argument
passed to the plugin can.
-
-.. code-block:: console
-
- $ export BD=/path/to/build/directory
- $ make -C $BD CallSuperAttr
- $ clang++ -fplugin=$BD/lib/CallSuperAttr.so \
- -fplugin-arg-call_super_plugin-help \
- test.cpp
+```console
+$ export BD=/path/to/build/directory
+$ make -C $BD CallSuperAttr
+$ clang++ -fplugin=$BD/lib/CallSuperAttr.so \
+ -fplugin-arg-call_super_plugin-help \
+ test.cpp
+```
If your plugin name contains dashes, either rename the plugin or use the
cc1 command line options listed below.
-
-Using the cc1 command line
---------------------------
+### Using the cc1 command line
To run a plugin, the dynamic library containing the plugin registry must be
loaded via the `-load` command line option. This will load all plugins
@@ -166,58 +149,55 @@ that are registered, and you can select the plugins to run by specifying the
Note that those options must reach clang's cc1 process. There are two
ways to do so:
-* Directly call the parsing process by using the `-cc1` option; this
+- Directly call the parsing process by using the `-cc1` option; this
has the downside of not configuring the default header search paths, so
you'll need to specify the full system path configuration on the command
line.
-* Use clang as usual, but prefix all arguments to the cc1 process with
+- Use clang as usual, but prefix all arguments to the cc1 process with
`-Xclang`.
-For example, to run the ``print-function-names`` plugin over a source file in
+For example, to run the `print-function-names` plugin over a source file in
clang, first build the plugin, and then call clang with the plugin from the
source tree:
-.. code-block:: console
-
- $ export BD=/path/to/build/directory
- $ (cd $BD && make PrintFunctionNames )
- $ clang++ -D_GNU_SOURCE -D_DEBUG -D__STDC_CONSTANT_MACROS \
- -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D_GNU_SOURCE \
- -I$BD/tools/clang/include -Itools/clang/include -I$BD/include -Iinclude \
- tools/clang/tools/clang-check/ClangCheck.cpp -fsyntax-only \
- -Xclang -load -Xclang $BD/lib/PrintFunctionNames.so -Xclang \
- -plugin -Xclang print-fns
+```console
+$ export BD=/path/to/build/directory
+$ (cd $BD && make PrintFunctionNames )
+$ clang++ -D_GNU_SOURCE -D_DEBUG -D__STDC_CONSTANT_MACROS \
+ -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D_GNU_SOURCE \
+ -I$BD/tools/clang/include -Itools/clang/include -I$BD/include -Iinclude \
+ tools/clang/tools/clang-check/ClangCheck.cpp -fsyntax-only \
+ -Xclang -load -Xclang $BD/lib/PrintFunctionNames.so -Xclang \
+ -plugin -Xclang print-fns
+```
Also see the print-function-name plugin example's
-`README <https://github.com/llvm/llvm-project/blob/main/clang/examples/PrintFunctionNames/README.txt>`_
-
+[README](https://github.com/llvm/llvm-project/blob/main/clang/examples/PrintFunctionNames/README.txt)
-Using the clang command line
-----------------------------
+### Using the clang command line
Using `-fplugin=plugin` on the clang command line passes the plugin
through as an argument to `-load` on the cc1 command line. If the plugin
-class implements the ``getActionType`` method then the plugin is run
+class implements the `getActionType` method then the plugin is run
automatically. For example, to run the plugin automatically after the main AST
action (i.e. the same as using `-add-plugin`):
-.. code-block:: c++
-
- // Automatically run the plugin after the main AST action
- PluginASTAction::ActionType getActionType() override {
- return AddAfterMainAction;
- }
+```c++
+// Automatically run the plugin after the main AST action
+PluginASTAction::ActionType getActionType() override {
+ return AddAfterMainAction;
+}
+```
-Interaction with ``-clear-ast-before-backend``
-----------------------------------------------
+### Interaction with `-clear-ast-before-backend`
To reduce peak memory usage of the compiler, plugins are recommended to run
*before* the main action, which is usually code generation. This is because
having any plugins that run after the codegen action automatically turns off
-``-clear-ast-before-backend``. ``-clear-ast-before-backend`` reduces peak
+`-clear-ast-before-backend`. `-clear-ast-before-backend` reduces peak
memory by clearing the Clang AST after generating IR and before running IR
-optimizations. Use ``CmdlineBeforeMainAction`` or ``AddBeforeMainAction`` as
-``getActionType`` to run plugins while still benefitting from
-``-clear-ast-before-backend``. Plugins must make sure not to modify the AST,
+optimizations. Use `CmdlineBeforeMainAction` or `AddBeforeMainAction` as
+`getActionType` to run plugins while still benefitting from
+`-clear-ast-before-backend`. Plugins must make sure not to modify the AST,
otherwise they should run after the main action.
diff --git a/clang/docs/CommandGuide/index.md b/clang/docs/CommandGuide/index.md
index 83a91182e9ca6..af7678f905190 100644
--- a/clang/docs/CommandGuide/index.md
+++ b/clang/docs/CommandGuide/index.md
@@ -1,18 +1,18 @@
-Clang "man" pages
------------------
+# Clang "man" pages
The following documents are command descriptions for all of the Clang tools.
These pages describe how to use the Clang commands and what their options are.
Note that these pages do not describe all of the options available for all
-tools. To get a complete listing, pass the ``--help`` (general options) or
-``--help-hidden`` (general and debugging options) arguments to the tool you are
+tools. To get a complete listing, pass the `--help` (general options) or
+`--help-hidden` (general and debugging options) arguments to the tool you are
interested in.
-Basic Commands
-~~~~~~~~~~~~~~
+## Basic Commands
-.. toctree::
- :maxdepth: 1
+```{toctree}
+:maxdepth: 1
+
+clang
+diagtool
+```
- clang
- diagtool
diff --git a/clang/docs/ControlFlowIntegrity.md b/clang/docs/ControlFlowIntegrity.md
index cfe5bd836cacf..d45526c9e9760 100644
--- a/clang/docs/ControlFlowIntegrity.md
+++ b/clang/docs/ControlFlowIntegrity.md
@@ -1,17 +1,16 @@
-======================
-Control Flow Integrity
-======================
+# Control Flow Integrity
-.. toctree::
- :hidden:
+```{toctree}
+:hidden: true
- ControlFlowIntegrityDesign
+ControlFlowIntegrityDesign
+```
-.. contents::
- :local:
+```{contents}
+:local: true
+```
-Introduction
-============
+## Introduction
Clang includes an implementation of a number of control flow integrity (CFI)
schemes, which are designed to abort the program upon detecting certain forms
@@ -20,11 +19,11 @@ program's control flow. These schemes have been optimized for performance,
allowing developers to enable them in release builds.
To enable Clang's available CFI schemes, use the flag
-``-fsanitize=cfi``. You can also enable a subset of available
-:ref:`schemes <cfi-schemes>`. As currently implemented, all schemes
-except for ``kcfi`` rely on link-time optimization (LTO); so it is
-required to specify ``-flto`` or ``-flto=thin``, and the linker used
-must support LTO, for example via the `gold plugin`_.
+`-fsanitize=cfi`. You can also enable a subset of available
+{ref}`schemes <cfi-schemes>`. As currently implemented, all schemes
+except for `kcfi` rely on link-time optimization (LTO); so it is
+required to specify `-flto` or `-flto=thin`, and the linker used
+must support LTO, for example via the [gold plugin].
To allow the checks to be implemented efficiently, the program must
be structured such that certain object files are compiled with CFI
@@ -34,92 +33,85 @@ the use of shared libraries in some cases.
The compiler will only produce CFI checks for a class if it can infer hidden
LTO visibility for that class. LTO visibility is a property of a class that
is inferred from flags and attributes. For more details, see the documentation
-for :doc:`LTO visibility <LTOVisibility>`.
+for {doc}`LTO visibility <LTOVisibility>`.
-The ``-fsanitize=cfi-{vcall,nvcall,derived-cast,unrelated-cast}`` flags
-require that a ``-fvisibility=`` flag also be specified. This is because the
-default visibility setting is ``-fvisibility=default``, which would disable
+The `-fsanitize=cfi-{vcall,nvcall,derived-cast,unrelated-cast}` flags
+require that a `-fvisibility=` flag also be specified. This is because the
+default visibility setting is `-fvisibility=default`, which would disable
CFI checks for classes without visibility attributes. Most users will want
-to specify ``-fvisibility=hidden``, which enables CFI checks for such classes.
+to specify `-fvisibility=hidden`, which enables CFI checks for such classes.
-When using ``-fsanitize=cfi*`` with ``-flto=thin``, it is recommended
-to reduce link times by passing `-funique-source-file-names
-<UsersManual.html#cmdoption-f-no-unique-source-file-names>`_, provided
+When using `-fsanitize=cfi*` with `-flto=thin`, it is recommended
+to reduce link times by passing [-funique-source-file-names](UsersManual.html#cmdoption-f-no-unique-source-file-names), provided
that your program is compatible with it.
-Experimental support for :ref:`cross-DSO control flow integrity
+Experimental support for {ref}`cross-DSO control flow integrity
<cfi-cross-dso>` exists that does not require classes to have hidden LTO
visibility. This cross-DSO support has unstable ABI at this time.
-.. _gold plugin: https://llvm.org/docs/GoldPlugin.html
+(cfi-schemes)=
-.. _cfi-schemes:
-
-Available schemes
-=================
+## Available schemes
Available schemes are:
- - ``-fsanitize=cfi-cast-strict``: Enables :ref:`strict cast checks
- <cfi-strictness>`.
- - ``-fsanitize=cfi-derived-cast``: Base-to-derived cast to the wrong
- dynamic type.
- - ``-fsanitize=cfi-unrelated-cast``: Cast from ``void*`` or another
- unrelated type to the wrong dynamic type.
- - ``-fsanitize=cfi-nvcall``: Non-virtual call via an object whose vptr is of
- the wrong dynamic type.
- - ``-fsanitize=cfi-vcall``: Virtual call via an object whose vptr is of the
- wrong dynamic type.
- - ``-fsanitize=cfi-icall``: Indirect call of a function with wrong dynamic
- type.
- - ``-fsanitize=cfi-mfcall``: Indirect call via a member function pointer with
- wrong dynamic type.
-
-You can use ``-fsanitize=cfi`` to enable all the schemes and use
-``-fno-sanitize`` flag to narrow down the set of schemes as desired.
+> - `-fsanitize=cfi-cast-strict`: Enables {ref}`strict cast checks
+> <cfi-strictness>`.
+> - `-fsanitize=cfi-derived-cast`: Base-to-derived cast to the wrong
+> dynamic type.
+> - `-fsanitize=cfi-unrelated-cast`: Cast from `void*` or another
+> unrelated type to the wrong dynamic type.
+> - `-fsanitize=cfi-nvcall`: Non-virtual call via an object whose vptr is of
+> the wrong dynamic type.
+> - `-fsanitize=cfi-vcall`: Virtual call via an object whose vptr is of the
+> wrong dynamic type.
+> - `-fsanitize=cfi-icall`: Indirect call of a function with wrong dynamic
+> type.
+> - `-fsanitize=cfi-mfcall`: Indirect call via a member function pointer with
+> wrong dynamic type.
+
+You can use `-fsanitize=cfi` to enable all the schemes and use
+`-fno-sanitize` flag to narrow down the set of schemes as desired.
For example, you can build your program with
-``-fsanitize=cfi -fno-sanitize=cfi-nvcall,cfi-icall``
+`-fsanitize=cfi -fno-sanitize=cfi-nvcall,cfi-icall`
to use all schemes except for non-virtual member function call and indirect call
checking.
-Remember that you have to provide ``-flto`` or ``-flto=thin`` if at
+Remember that you have to provide `-flto` or `-flto=thin` if at
least one CFI scheme is enabled.
-Trapping and Diagnostics
-========================
+## Trapping and Diagnostics
By default, CFI will abort the program immediately upon detecting a control
-flow integrity violation. You can use the :ref:`-fno-sanitize-trap=
+flow integrity violation. You can use the {ref}`-fno-sanitize-trap=
<controlling-code-generation>` flag to cause CFI to print a diagnostic
similar to the one below before the program aborts.
-.. code-block:: console
-
- bad-cast.cpp:109:7: runtime error: control flow integrity check for type 'B' failed during base-to-derived cast (vtable address 0x000000425a50)
- 0x000000425a50: note: vtable is of type 'A'
- 00 00 00 00 f0 f1 41 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20 5a 42 00
- ^
+```console
+bad-cast.cpp:109:7: runtime error: control flow integrity check for type 'B' failed during base-to-derived cast (vtable address 0x000000425a50)
+0x000000425a50: note: vtable is of type 'A'
+ 00 00 00 00 f0 f1 41 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20 5a 42 00
+ ^
+```
If diagnostics are enabled, you can also configure CFI to continue program
-execution instead of aborting by using the :ref:`-fsanitize-recover=
+execution instead of aborting by using the {ref}`-fsanitize-recover=
<controlling-code-generation>` flag.
-Forward-Edge CFI for Virtual Calls
-==================================
+## Forward-Edge CFI for Virtual Calls
This scheme checks that virtual calls take place using a vptr of the correct
dynamic type; that is, the dynamic type of the called object must be a
derived class of the static type of the object used to make the call.
-This CFI scheme can be enabled on its own using ``-fsanitize=cfi-vcall``.
+This CFI scheme can be enabled on its own using `-fsanitize=cfi-vcall`.
For this scheme to work, all translation units containing the definition
of a virtual member function (whether inline or not), other than members
-of :ref:`ignored <cfi-ignorelist>` types or types with public :doc:`LTO
-visibility <LTOVisibility>`, must be compiled with ``-flto`` or ``-flto=thin``
+of {ref}`ignored <cfi-ignorelist>` types or types with public {doc}`LTO
+visibility <LTOVisibility>`, must be compiled with `-flto` or `-flto=thin`
enabled and be statically linked into the program.
-Performance
------------
+### Performance
A performance overhead of less than 1% has been measured by running the
Dromaeo benchmark suite against an instrumented version of the Chromium
@@ -129,8 +121,7 @@ virtual-call-heavy SPEC 2006 xalancbmk.
Note that this scheme has not yet been optimized for binary size; an increase
of up to 15% has been observed for Chromium.
-Bad Cast Checking
-=================
+## Bad Cast Checking
This scheme checks that pointer casts are made to an object of the correct
dynamic type; that is, the dynamic type of the object must be a derived class
@@ -143,48 +134,46 @@ of the same mechanisms.
There are two types of bad cast that may be forbidden: bad casts
from a base class to a derived class (which can be checked with
-``-fsanitize=cfi-derived-cast``), and bad casts from a pointer of
-type ``void*`` or another unrelated type (which can be checked with
-``-fsanitize=cfi-unrelated-cast``).
+`-fsanitize=cfi-derived-cast`), and bad casts from a pointer of
+type `void*` or another unrelated type (which can be checked with
+`-fsanitize=cfi-unrelated-cast`).
The difference between these two types of casts is that the first is defined
by the C++ standard to produce an undefined value, while the second is not
in itself undefined behavior (it is well defined to cast the pointer back
to its original type) unless the object is uninitialized and the cast is a
-``static_cast`` (see C++14 [basic.life]p5).
+`static_cast` (see C++14 [basic.life]p5).
If a program as a matter of policy forbids the second type of cast, that
restriction can normally be enforced. However it may in some cases be necessary
for a function to perform a forbidden cast to conform with an external API
-(e.g. the ``allocate`` member function of a standard library allocator). Such
-functions may be :ref:`ignored <cfi-ignorelist>`.
+(e.g. the `allocate` member function of a standard library allocator). Such
+functions may be {ref}`ignored <cfi-ignorelist>`.
For this scheme to work, all translation units containing the definition
of a virtual member function (whether inline or not), other than members
-of :ref:`ignored <cfi-ignorelist>` types or types with public :doc:`LTO
-visibility <LTOVisibility>`, must be compiled with ``-flto`` or ``-flto=thin``
+of {ref}`ignored <cfi-ignorelist>` types or types with public {doc}`LTO
+visibility <LTOVisibility>`, must be compiled with `-flto` or `-flto=thin`
enabled and be statically linked into the program.
-Non-Virtual Member Function Call Checking
-=========================================
+## Non-Virtual Member Function Call Checking
This scheme checks that non-virtual calls take place using an object of
the correct dynamic type; that is, the dynamic type of the called object
must be a derived class of the static type of the object used to make the
call. The checks are currently only introduced where the object is of a
-polymorphic class type. This CFI scheme can be enabled on its own using
-``-fsanitize=cfi-nvcall``.
+polymorphic class type. This CFI scheme can be enabled on its own using
+`-fsanitize=cfi-nvcall`.
For this scheme to work, all translation units containing the definition
of a virtual member function (whether inline or not), other than members
-of :ref:`ignored <cfi-ignorelist>` types or types with public :doc:`LTO
-visibility <LTOVisibility>`, must be compiled with ``-flto`` or ``-flto=thin``
+of {ref}`ignored <cfi-ignorelist>` types or types with public {doc}`LTO
+visibility <LTOVisibility>`, must be compiled with `-flto` or `-flto=thin`
enabled and be statically linked into the program.
-.. _cfi-strictness:
+(cfi-strictness)=
-Strictness
-----------
+### Strictness
If a class has a single non-virtual base and does not introduce or override
virtual member functions or fields other than an implicitly defined virtual
@@ -196,56 +185,53 @@ Casting an instance of a base class to such a derived class is technically
undefined behavior, but it is a relatively common hack for introducing
member functions on class instances with specific properties that works under
most compilers and should not have security implications, so we allow it by
-default. It can be disabled with ``-fsanitize=cfi-cast-strict``.
+default. It can be disabled with `-fsanitize=cfi-cast-strict`.
-Indirect Function Call Checking
-===============================
+## Indirect Function Call Checking
This scheme checks that function calls take place using a function of the
correct dynamic type; that is, the dynamic type of the function must match
the static type used at the call. This CFI scheme can be enabled on its own
-using ``-fsanitize=cfi-icall``.
+using `-fsanitize=cfi-icall`.
For this scheme to work, each indirect function call in the program, other
-than calls in :ref:`ignored <cfi-ignorelist>` functions, must call a
-function which was either compiled with ``-fsanitize=cfi-icall`` enabled,
+than calls in {ref}`ignored <cfi-ignorelist>` functions, must call a
+function which was either compiled with `-fsanitize=cfi-icall` enabled,
or whose address was taken by a function in a translation unit compiled with
-``-fsanitize=cfi-icall``.
+`-fsanitize=cfi-icall`.
-If a function in a translation unit compiled with ``-fsanitize=cfi-icall``
-takes the address of a function not compiled with ``-fsanitize=cfi-icall``,
+If a function in a translation unit compiled with `-fsanitize=cfi-icall`
+takes the address of a function not compiled with `-fsanitize=cfi-icall`,
that address may differ from the address taken by a function in a translation
-unit not compiled with ``-fsanitize=cfi-icall``. This is technically a
+unit not compiled with `-fsanitize=cfi-icall`. This is technically a
violation of the C and C++ standards, but it should not affect most programs.
-Each translation unit compiled with ``-fsanitize=cfi-icall`` must be
+Each translation unit compiled with `-fsanitize=cfi-icall` must be
statically linked into the program or shared library, and calls across
shared library boundaries are handled as if the callee was not compiled with
-``-fsanitize=cfi-icall``.
+`-fsanitize=cfi-icall`.
This scheme is currently supported on a limited set of targets: x86,
x86_64, arm, arch64 and wasm.
-``-fsanitize-cfi-icall-generalize-pointers``
---------------------------------------------
+### `-fsanitize-cfi-icall-generalize-pointers`
Mismatched pointer types are a common cause of cfi-icall check failures.
-Translation units compiled with the ``-fsanitize-cfi-icall-generalize-pointers``
+Translation units compiled with the `-fsanitize-cfi-icall-generalize-pointers`
flag relax pointer type checking for call sites in that translation unit,
-applied across all functions compiled with ``-fsanitize=cfi-icall``.
+applied across all functions compiled with `-fsanitize=cfi-icall`.
Specifically, pointers in return and argument types are treated as equivalent as
-long as the qualifiers for the type they point to match. For example, ``char*``,
-``char**``, and ``int*`` are considered equivalent types. However, ``char*`` and
-``const char*`` are considered separate types.
+long as the qualifiers for the type they point to match. For example, `char*`,
+`char**`, and `int*` are considered equivalent types. However, `char*` and
+`const char*` are considered separate types.
-``-fsanitize-cfi-icall-generalize-pointers`` is not compatible with
-``-fsanitize-cfi-cross-dso``.
+`-fsanitize-cfi-icall-generalize-pointers` is not compatible with
+`-fsanitize-cfi-cross-dso`.
-.. _cfi-icall-experimental-normalize-integers:
+(cfi-icall-experimental-normalize-integers)=
-``-fsanitize-cfi-icall-experimental-normalize-integers``
---------------------------------------------------------
+### `-fsanitize-cfi-icall-experimental-normalize-integers`
This option enables normalizing integer types as vendor extended types for
cross-language LLVM CFI/KCFI support with other languages that can't represent
@@ -256,21 +242,20 @@ Specifically, integer types are encoded as their defined representations (e.g.,
compatibility with languages that define explicitly-sized integer types (e.g.,
i8, i16, i32, ..., in Rust).
-``-fsanitize-cfi-icall-experimental-normalize-integers`` is compatible with
-``-fsanitize-cfi-icall-generalize-pointers``.
+`-fsanitize-cfi-icall-experimental-normalize-integers` is compatible with
+`-fsanitize-cfi-icall-generalize-pointers`.
This option is currently experimental.
-.. _cfi-canonical-jump-tables:
+(cfi-canonical-jump-tables)=
-``-fsanitize-cfi-canonical-jump-tables``
-----------------------------------------
+### `-fsanitize-cfi-canonical-jump-tables`
The default behavior of Clang's indirect function call checker will replace
the address of each CFI-checked function in the output file's symbol table
with the address of a jump table entry which will pass CFI checks. We refer
to this as making the jump table `canonical`. This property allows code that
-was not compiled with ``-fsanitize=cfi-icall`` to take a CFI-valid address
+was not compiled with `-fsanitize=cfi-icall` to take a CFI-valid address
of a function, but it comes with a couple of caveats that are especially
relevant for users of cross-DSO CFI:
@@ -279,7 +264,6 @@ relevant for users of cross-DSO CFI:
jump table entry, which must be emitted even in the common case where the
function is never address-taken anywhere in the program, and must be used
even for direct calls between DSOs, in addition to the PLT overhead.
-
- There is no good way to take a CFI-valid address of a function written in
assembly or a language not supported by Clang. The reason is that the code
generator would need to insert a jump table in order to form a CFI-valid
@@ -292,7 +276,7 @@ relevant for users of cross-DSO CFI:
addition to adding runtime overhead.
For these reasons, we provide the option of making the jump table non-canonical
-with the flag ``-fno-sanitize-cfi-canonical-jump-tables``. When the jump
+with the flag `-fno-sanitize-cfi-canonical-jump-tables`. When the jump
table is made non-canonical, symbol table entries point directly to the
function body. Any instances of a function's address being taken in C will
be replaced with a jump table address.
@@ -303,47 +287,44 @@ especially in cross-DSO mode which normally preserves function address
equality entirely.
Furthermore, it is occasionally necessary for code not compiled with
-``-fsanitize=cfi-icall`` to take a function address that is valid
+`-fsanitize=cfi-icall` to take a function address that is valid
for CFI. For example, this is necessary when a function's address
is taken by assembly code and then called by CFI-checking C code. The
-``__attribute__((cfi_canonical_jump_table))`` attribute may be used to make
+`__attribute__((cfi_canonical_jump_table))` attribute may be used to make
the jump table entry of a specific function canonical so that the external
code will end up taking an address for the function that will pass CFI checks.
-``-fsanitize=cfi-icall`` and ``-fsanitize=function``
-----------------------------------------------------
+### `-fsanitize=cfi-icall` and `-fsanitize=function`
-This tool is similar to ``-fsanitize=function`` in that both tools check
+This tool is similar to `-fsanitize=function` in that both tools check
the types of function calls. However, the two tools occupy different points
-on the design space; ``-fsanitize=function`` is a developer tool designed
-to find bugs in local development builds, whereas ``-fsanitize=cfi-icall``
+on the design space; `-fsanitize=function` is a developer tool designed
+to find bugs in local development builds, whereas `-fsanitize=cfi-icall`
is a security hardening mechanism designed to be deployed in release builds.
-``-fsanitize=function`` has a higher space and time overhead due to a more
+`-fsanitize=function` has a higher space and time overhead due to a more
complex type check at indirect call sites, which may make it unsuitable for
deployment.
-On the other hand, ``-fsanitize=function`` conforms more closely with the C++
+On the other hand, `-fsanitize=function` conforms more closely with the C++
standard and user expectations around interaction with shared libraries;
the identity of function pointers is maintained, and calls across shared
library boundaries are no different from calls within a single program or
shared library.
-.. _kcfi:
+(kcfi)=
-``-fsanitize=kcfi``
--------------------
+### `-fsanitize=kcfi`
This is an alternative indirect call control-flow integrity scheme designed
for low-level system software, such as operating system kernels. Unlike
-``-fsanitize=cfi-icall``, it doesn't require ``-flto``, won't result in
+`-fsanitize=cfi-icall`, it doesn't require `-flto`, won't result in
function pointers being replaced with jump table references, and never breaks
cross-DSO function address equality. These properties make KCFI easier to
adopt in low-level software. KCFI is limited to checking only function
pointers, and isn't compatible with executable-only memory.
-``-fsanitize-kcfi-arity``
------------------------------
+### `-fsanitize-kcfi-arity`
For supported targets, this feature extends kCFI by telling the compiler to
record information about each indirect-callable function's arity (i.e., the
@@ -351,8 +332,7 @@ number of arguments passed in registers) into the binary. Some kernel CFI
techniques, such as FineIBT, may be able to use this information to provide
enhanced security.
-Member Function Pointer Call Checking
-=====================================
+## Member Function Pointer Call Checking
This scheme checks that indirect calls via a member function pointer
take place using an object of the correct dynamic type. Specifically, we
@@ -360,78 +340,76 @@ check that the dynamic type of the member function referenced by the member
function pointer matches the "function pointer" part of the member function
pointer, and that the member function's class type is related to the base
type of the member function. This CFI scheme can be enabled on its own using
-``-fsanitize=cfi-mfcall``.
+`-fsanitize=cfi-mfcall`.
The compiler will only emit a full CFI check if the member function pointer's
base type is complete. This is because the complete definition of the base
type contains information that is necessary to correctly compile the CFI
check. To ensure that the compiler always emits a full CFI check, it is
-recommended to also pass the flag ``-fcomplete-member-pointers``, which
+recommended to also pass the flag `-fcomplete-member-pointers`, which
enables a non-conforming language extension that requires member pointer
base types to be complete if they may be used for a call.
For this scheme to work, all translation units containing the definition
of a virtual member function (whether inline or not), other than members
-of :ref:`ignored <cfi-ignorelist>` types or types with public :doc:`LTO
-visibility <LTOVisibility>`, must be compiled with ``-flto`` or ``-flto=thin``
+of {ref}`ignored <cfi-ignorelist>` types or types with public {doc}`LTO
+visibility <LTOVisibility>`, must be compiled with `-flto` or `-flto=thin`
enabled and be statically linked into the program.
This scheme is currently not compatible with cross-DSO CFI or the
Microsoft ABI.
-.. _cfi-ignorelist:
+(cfi-ignorelist)=
-Ignorelist
-==========
+## Ignorelist
-A :doc:`SanitizerSpecialCaseList` can be used to relax CFI checks for certain
-source files, functions and types using the ``src``, ``fun`` and ``type``
-entity types. Specific CFI modes can be be specified using ``[section]``
+A {doc}`SanitizerSpecialCaseList` can be used to relax CFI checks for certain
+source files, functions and types using the `src`, `fun` and `type`
+entity types. Specific CFI modes can be be specified using `[section]`
headers.
-.. code-block:: bash
-
- # Suppress all CFI checking for code in a file.
- src:bad_file.cpp
- src:bad_header.h
- # Ignore all functions with names containing MyFooBar.
- fun:*MyFooBar*
- # Ignore all types in the standard library.
- type:std::*
- # Disable only unrelated cast checks for this function
- [cfi-unrelated-cast]
- fun:*UnrelatedCast*
- # Disable CFI call checks for this function without affecting cast checks
- [cfi-vcall|cfi-nvcall|cfi-icall]
- fun:*BadCall*
-
-
-.. _cfi-cross-dso:
-
-Shared library support
-======================
+```bash
+# Suppress all CFI checking for code in a file.
+src:bad_file.cpp
+src:bad_header.h
+# Ignore all functions with names containing MyFooBar.
+fun:*MyFooBar*
+# Ignore all types in the standard library.
+type:std::*
+# Disable only unrelated cast checks for this function
+[cfi-unrelated-cast]
+fun:*UnrelatedCast*
+# Disable CFI call checks for this function without affecting cast checks
+[cfi-vcall|cfi-nvcall|cfi-icall]
+fun:*BadCall*
+```
+
+(cfi-cross-dso)=
+
+## Shared library support
Use **-f[no-]sanitize-cfi-cross-dso** to enable the cross-DSO control
flow integrity mode, which allows all CFI schemes listed above to
apply across DSO boundaries. As in the regular CFI, each DSO must be
-built with ``-flto`` or ``-flto=thin``.
+built with `-flto` or `-flto=thin`.
Normally, CFI checks will only be performed for classes that have hidden LTO
visibility. With this flag enabled, the compiler will emit cross-DSO CFI
checks for all classes, except for those which appear in the CFI ignorelist
-or which use a ``no_sanitize`` attribute.
+or which use a `no_sanitize` attribute.
-Design
-======
+## Design
-Please refer to the :doc:`design document<ControlFlowIntegrityDesign>`.
+Please refer to the {doc}`design document<ControlFlowIntegrityDesign>`.
-Publications
-============
+## Publications
-`Control-Flow Integrity: Principles, Implementations, and Applications <https://research.microsoft.com/pubs/64250/ccs05.pdf>`_.
+[Control-Flow Integrity: Principles, Implementations, and Applications](https://research.microsoft.com/pubs/64250/ccs05.pdf).
Martin Abadi, Mihai Budiu, Úlfar Erlingsson, Jay Ligatti.
-`Enforcing Forward-Edge Control-Flow Integrity in GCC & LLVM <https://www.usenix.org/system/files/conference/usenixsecurity14/sec14-paper-tice.pdf>`_.
+[Enforcing Forward-Edge Control-Flow Integrity in GCC & LLVM](https://www.usenix.org/system/files/conference/usenixsecurity14/sec14-paper-tice.pdf).
Caroline Tice, Tom Roeder, Peter Collingbourne, Stephen Checkoway,
Úlfar Erlingsson, Luis Lozano, Geoff Pike.
+
+[gold plugin]: https://llvm.org/docs/GoldPlugin.html
+
diff --git a/clang/docs/DebuggingCoroutines.md b/clang/docs/DebuggingCoroutines.md
index c62e2ea0e32ab..a9a2ba3a8ea32 100644
--- a/clang/docs/DebuggingCoroutines.md
+++ b/clang/docs/DebuggingCoroutines.md
@@ -1,12 +1,10 @@
-========================
-Debugging C++ Coroutines
-========================
+# Debugging C++ Coroutines
-.. contents::
- :local:
+```{contents}
+:local: true
+```
-Introduction
-============
+## Introduction
Coroutines in C++ were introduced in C++20, and the user experience for
debugging them can still be challenging. This document guides you on how to most
@@ -24,7 +22,7 @@ still improving their support for coroutines. As such, we recommend using the
latest available version of your toolchain.
This document focuses on clang and lldb. The screenshots show
-`lldb-dap <https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.lldb-dap>`_
+[lldb-dap](https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.lldb-dap)
in combination with VS Code. The same techniques can also be used in other
IDEs.
@@ -39,177 +37,171 @@ this document, the more low-level, technical the content will become. If
you are on an up-to-date toolchain, you will hopefully be able to stop reading
earlier.
-Debugging generators
-====================
+## Debugging generators
One of the two major use cases for coroutines in C++ is generators, i.e.,
-functions which can produce values via ``co_yield``. Values are produced
+functions which can produce values via `co_yield`. Values are produced
lazily, on-demand. For this purpose, every time a new value is requested, the
-coroutine gets resumed. As soon as it reaches a ``co_yield`` and thereby
+coroutine gets resumed. As soon as it reaches a `co_yield` and thereby
returns the requested value, the coroutine is suspended again.
-This logic is encapsulated in a ``generator`` type similar to this one:
+This logic is encapsulated in a `generator` type similar to this one:
-.. code-block:: c++
+```c++
+// generator.hpp
+#include <coroutine>
- // generator.hpp
- #include <coroutine>
+// `generator` is a stripped down, minimal generator type.
+template<typename T>
+struct generator {
+ struct promise_type {
+ T current_value{};
- // `generator` is a stripped down, minimal generator type.
- template<typename T>
- struct generator {
- struct promise_type {
- T current_value{};
-
- auto get_return_object() {
- return std::coroutine_handle<promise_type>::from_promise(*this);
- }
- auto initial_suspend() { return std::suspend_always(); }
- auto final_suspend() noexcept { return std::suspend_always(); }
- auto return_void() { return std::suspend_always(); }
- void unhandled_exception() { __builtin_unreachable(); }
- auto yield_value(T v) {
- current_value = v;
- return std::suspend_always();
- }
- };
-
- generator(std::coroutine_handle<promise_type> h) : hdl(h) { hdl.resume(); }
- ~generator() { hdl.destroy(); }
-
- generator<T>& operator++() { hdl.resume(); return *this; } // resume the coroutine
- T operator*() const { return hdl.promise().current_value; }
-
- private:
- std::coroutine_handle<promise_type> hdl;
+ auto get_return_object() {
+ return std::coroutine_handle<promise_type>::from_promise(*this);
+ }
+ auto initial_suspend() { return std::suspend_always(); }
+ auto final_suspend() noexcept { return std::suspend_always(); }
+ auto return_void() { return std::suspend_always(); }
+ void unhandled_exception() { __builtin_unreachable(); }
+ auto yield_value(T v) {
+ current_value = v;
+ return std::suspend_always();
+ }
};
-We can then use this ``generator`` class to print the Fibonacci sequence:
-
-.. code-block:: c++
+ generator(std::coroutine_handle<promise_type> h) : hdl(h) { hdl.resume(); }
+ ~generator() { hdl.destroy(); }
- #include "generator.hpp"
- #include <iostream>
+ generator<T>& operator++() { hdl.resume(); return *this; } // resume the coroutine
+ T operator*() const { return hdl.promise().current_value; }
- generator<int> fibonacci() {
- co_yield 0;
- int prev = 0;
- co_yield 1;
- int current = 1;
- while (true) {
- int next = current + prev;
- co_yield next;
- prev = current;
- current = next;
- }
+ private:
+ std::coroutine_handle<promise_type> hdl;
+};
+```
+
+We can then use this `generator` class to print the Fibonacci sequence:
+
+```c++
+#include "generator.hpp"
+#include <iostream>
+
+generator<int> fibonacci() {
+ co_yield 0;
+ int prev = 0;
+ co_yield 1;
+ int current = 1;
+ while (true) {
+ int next = current + prev;
+ co_yield next;
+ prev = current;
+ current = next;
}
+}
- template<typename T>
- void print10Elements(generator<T>& gen) {
- for (unsigned i = 0; i < 10; ++i) {
- std::cerr << *gen << "\n";
- ++gen;
- }
+template<typename T>
+void print10Elements(generator<T>& gen) {
+ for (unsigned i = 0; i < 10; ++i) {
+ std::cerr << *gen << "\n";
+ ++gen;
}
+}
- int main() {
- std::cerr << "Fibonacci sequence - here we go\n";
- generator<int> fib = fibonacci();
- for (unsigned i = 0; i < 5; ++i) {
- ++fib;
- }
- print10Elements(fib);
+int main() {
+ std::cerr << "Fibonacci sequence - here we go\n";
+ generator<int> fib = fibonacci();
+ for (unsigned i = 0; i < 5; ++i) {
+ ++fib;
}
+ print10Elements(fib);
+}
+```
-To compile this code, use ``clang++ --std=c++23 generator-example.cpp -g``.
+To compile this code, use `clang++ --std=c++23 generator-example.cpp -g`.
-Breakpoints inside the generators
----------------------------------
+### Breakpoints inside the generators
We can set breakpoints inside coroutines just as we set them in regular
functions. For VS Code, that means clicking next the line number in the editor.
-In the ``lldb`` CLI or in ``gdb``, you can use ``b`` to set a breakpoint.
+In the `lldb` CLI or in `gdb`, you can use `b` to set a breakpoint.
-Inspecting variables in a coroutine
------------------------------------
+### Inspecting variables in a coroutine
-If you hit a breakpoint inside the ``fibonacci`` function, you should be able
-to inspect all local variables (``prev``, ``current``, ``next``) just like in
+If you hit a breakpoint inside the `fibonacci` function, you should be able
+to inspect all local variables (`prev`, `current`, `next`) just like in
a regular function.
-.. image:: ./coro-generator-variables.png
+```{image} ./coro-generator-variables.png
+```
-Note the two additional variables ``__promise`` and ``__coro_frame``. Those
+Note the two additional variables `__promise` and `__coro_frame`. Those
show the internal state of the coroutine. They are not relevant for our
generator example but will be relevant for asynchronous programming described
in the next section.
-Stepping out of a coroutine
----------------------------
+### Stepping out of a coroutine
When single-stepping, you will notice that the debugger will leave the
-``fibonacci`` function as soon as you hit a ``co_yield`` statement. You might
+`fibonacci` function as soon as you hit a `co_yield` statement. You might
find yourself inside some standard library code. After stepping out of the
-library code, you will be back in the ``main`` function.
+library code, you will be back in the `main` function.
-Stepping into a coroutine
--------------------------
+### Stepping into a coroutine
-If you stop at ``++fib`` and try to step into the generator, you will first
-find yourself inside ``operator++``. Stepping into the ``handle.resume()`` will
+If you stop at `++fib` and try to step into the generator, you will first
+find yourself inside `operator++`. Stepping into the `handle.resume()` will
not work by default.
This is because lldb does not step into functions from the standard library by
-default. To make this work, you first need to run ``settings set
-target.process.thread.step-avoid-regexp ""``. You can do so from the "Debug
+default. To make this work, you first need to run `settings set
+target.process.thread.step-avoid-regexp ""`. You can do so from the "Debug
Console" towards the bottom of the screen. With that setting change, you can
-step through ``coroutine_handle::resume`` and into your generator.
+step through `coroutine_handle::resume` and into your generator.
You might find yourself at the top of the coroutine at first, instead of at
your previous suspension point. In that case, single-step and you will arrive
-at the previously suspended ``co_yield`` statement.
-
+at the previously suspended `co_yield` statement.
-Inspecting a suspended coroutine
---------------------------------
+### Inspecting a suspended coroutine
-The ``print10Elements`` function receives an opaque ``generator`` type. Let's
-assume we are suspended at the ``++gen;`` line and want to inspect the
+The `print10Elements` function receives an opaque `generator` type. Let's
+assume we are suspended at the `++gen;` line and want to inspect the
generator and its internal state.
-To do so, we can simply look into the ``gen.hdl`` variable. LLDB comes with a
-pretty printer for ``std::coroutine_handle`` which will show us the internal
+To do so, we can simply look into the `gen.hdl` variable. LLDB comes with a
+pretty printer for `std::coroutine_handle` which will show us the internal
state of the coroutine. For GDB, the pretty printer is provided by a script,
-see :ref:`gdb-script` for setup instructions.
+see {ref}`gdb-script` for setup instructions.
-.. image:: ./coro-generator-suspended.png
+```{image} ./coro-generator-suspended.png
+```
-We can see two function pointers ``resume`` and ``destroy``. These pointers
+We can see two function pointers `resume` and `destroy`. These pointers
point to the resume / destroy functions. By inspecting those function pointers,
-we can see that our ``generator`` is actually backed by our ``fibonacci``
+we can see that our `generator` is actually backed by our `fibonacci`
coroutine. When using VS Code + lldb-dap, you can Cmd+Click on the function
-address (``0x555...`` in the screenshot) to jump directly to the function
+address (`0x555...` in the screenshot) to jump directly to the function
definition backing your coroutine handle.
-Next, we see the ``promise``. In our case, this reveals the current value of
+Next, we see the `promise`. In our case, this reveals the current value of
our generator.
-The ``coro_frame`` member represents the internal state of the coroutine. It
-contains our internal coroutine state ``prev``, ``current``, ``next``.
+The `coro_frame` member represents the internal state of the coroutine. It
+contains our internal coroutine state `prev`, `current`, `next`.
Furthermore, it contains many internal, compiler-specific members, which are
named based on their type. These represent temporary values which the compiler
decided to spill across suspension points, but which were not declared in our
original source code and hence have no proper user-provided name.
-Tracking the exact suspension point
------------------------------------
+### Tracking the exact suspension point
-Among the compiler-generated members, the ``__coro_index`` is particularly
+Among the compiler-generated members, the `__coro_index` is particularly
important. This member identifies the suspension point at which the coroutine
is currently suspended. However, it is non-trivial to map this number back to
a source code location.
-For GDB, the provided :ref:`gdb-script` already takes care of this and provides
+For GDB, the provided {ref}`gdb-script` already takes care of this and provides
the exact line number of the suspension point as part of the coroutine handle's
summary string. Unfortunately, LLDB's pretty-printer does not support this, yet.
Furthermore, those labels are only emitted starting with clang 21.0.
@@ -221,55 +213,54 @@ For simple cases, you might still be able to guess the suspension point correctl
Alternatively, you might also want to modify your coroutine library to store
the line number of the current suspension point in the promise:
-.. code-block:: c++
-
- // For all promise_types we need a new `_coro_return_address` variable:
- class promise_type {
- ...
- void* _coro_return_address = nullptr;
- };
-
- // For all the awaiter types we need:
- class awaiter {
- ...
- template <typename Promise>
- __attribute__((noinline)) auto await_suspend(std::coroutine_handle<Promise> handle) {
- ...
- handle.promise()._coro_return_address = __builtin_return_address(0);
- }
- };
+```c++
+// For all promise_types we need a new `_coro_return_address` variable:
+class promise_type {
+ ...
+ void* _coro_return_address = nullptr;
+};
+
+// For all the awaiter types we need:
+class awaiter {
+ ...
+ template <typename Promise>
+ __attribute__((noinline)) auto await_suspend(std::coroutine_handle<Promise> handle) {
+ ...
+ handle.promise()._coro_return_address = __builtin_return_address(0);
+ }
+};
+```
-This stores the return address of ``await_suspend`` within the promise.
+This stores the return address of `await_suspend` within the promise.
Thereby, we can read it back from the promise of a suspended coroutine and map
-it to an exact source code location. For a complete example, see the ``task``
+it to an exact source code location. For a complete example, see the `task`
type used below for asynchronous programming.
Alternatively, we can modify the C++ code to store the line number in the
-promise type. We can use ``std::source_location`` to get the line number of
-the await and store it inside the ``promise_type``. In the debugger, we can
+promise type. We can use `std::source_location` to get the line number of
+the await and store it inside the `promise_type`. In the debugger, we can
then read the line number from the promise of the suspended coroutine.
-.. code-block:: c++
-
- // For all the awaiter types we need:
- class awaiter {
- ...
- template <typename Promise>
- void await_suspend(std::coroutine_handle<Promise> handle,
- std::source_location sl = std::source_location::current()) {
- ...
- handle.promise().line_number = sl.line();
- }
- };
+```c++
+// For all the awaiter types we need:
+class awaiter {
+ ...
+ template <typename Promise>
+ void await_suspend(std::coroutine_handle<Promise> handle,
+ std::source_location sl = std::source_location::current()) {
+ ...
+ handle.promise().line_number = sl.line();
+ }
+};
+```
The downside of both approaches is that they come at the price of additional
runtime cost. In particular, the second approach increases binary size, since it
-requires additional ``std::source_location`` objects, and those source
+requires additional `std::source_location` objects, and those source
locations are not stripped by split-dwarf. Whether the first approach is worth
the additional runtime cost is a trade-off you need to make yourself.
-Async stack traces
-==================
+## Async stack traces
Besides generators, the second common use case for coroutines in C++ is
asynchronous programming, usually involving libraries such as stdexec, folly,
@@ -278,174 +269,172 @@ provide custom debugging support, so in addition to this guide, you might want
to check out their documentation.
When using coroutines for asynchronous programming, your library usually
-provides you with some ``task`` type. This type usually looks similar to this:
-
-.. code-block:: c++
-
- // async-task-library.hpp
- #include <coroutine>
- #include <utility>
-
- struct task {
- struct promise_type {
- task get_return_object() { return std::coroutine_handle<promise_type>::from_promise(*this); }
- auto initial_suspend() { return std::suspend_always{}; }
-
- void unhandled_exception() noexcept {}
-
- auto final_suspend() noexcept {
- struct FinalSuspend {
- std::coroutine_handle<> continuation;
- auto await_ready() noexcept { return false; }
- auto await_suspend(std::coroutine_handle<> handle) noexcept {
- return continuation;
- }
- void await_resume() noexcept {}
- };
- return FinalSuspend{continuation};
- }
-
- void return_value(int res) { result = res; }
+provides you with some `task` type. This type usually looks similar to this:
+
+```c++
+// async-task-library.hpp
+#include <coroutine>
+#include <utility>
+
+struct task {
+ struct promise_type {
+ task get_return_object() { return std::coroutine_handle<promise_type>::from_promise(*this); }
+ auto initial_suspend() { return std::suspend_always{}; }
+
+ void unhandled_exception() noexcept {}
+
+ auto final_suspend() noexcept {
+ struct FinalSuspend {
+ std::coroutine_handle<> continuation;
+ auto await_ready() noexcept { return false; }
+ auto await_suspend(std::coroutine_handle<> handle) noexcept {
+ return continuation;
+ }
+ void await_resume() noexcept {}
+ };
+ return FinalSuspend{continuation};
+ }
- std::coroutine_handle<> continuation = std::noop_coroutine();
- int result = 0;
- #ifndef NDEBUG
- void* _coro_suspension_point_addr = nullptr;
- #endif
- };
+ void return_value(int res) { result = res; }
- task(std::coroutine_handle<promise_type> handle) : handle(handle) {}
- ~task() {
- if (handle)
- handle.destroy();
- }
+ std::coroutine_handle<> continuation = std::noop_coroutine();
+ int result = 0;
+ #ifndef NDEBUG
+ void* _coro_suspension_point_addr = nullptr;
+ #endif
+ };
- struct Awaiter {
- std::coroutine_handle<promise_type> handle;
- auto await_ready() { return false; }
+ task(std::coroutine_handle<promise_type> handle) : handle(handle) {}
+ ~task() {
+ if (handle)
+ handle.destroy();
+ }
- template <typename P>
+ struct Awaiter {
+ std::coroutine_handle<promise_type> handle;
+ auto await_ready() { return false; }
+
+ template <typename P>
+ #ifndef NDEBUG
+ __attribute__((noinline))
+ #endif
+ auto await_suspend(std::coroutine_handle<P> continuation) {
+ handle.promise().continuation = continuation;
#ifndef NDEBUG
- __attribute__((noinline))
+ continuation.promise()._coro_suspension_point_addr = __builtin_return_address(0);
#endif
- auto await_suspend(std::coroutine_handle<P> continuation) {
- handle.promise().continuation = continuation;
- #ifndef NDEBUG
- continuation.promise()._coro_suspension_point_addr = __builtin_return_address(0);
- #endif
- return handle;
- }
- int await_resume() {
- return handle.promise().result;
- }
- };
-
- auto operator co_await() {
- return Awaiter{handle};
+ return handle;
}
-
- int syncStart() {
- handle.resume();
+ int await_resume() {
return handle.promise().result;
}
-
- private:
- std::coroutine_handle<promise_type> handle;
};
-Note how the ``task::promise_type`` has a member variable
-``std::coroutine_handle<> continuation``. This is the handle of the coroutine
+ auto operator co_await() {
+ return Awaiter{handle};
+ }
+
+ int syncStart() {
+ handle.resume();
+ return handle.promise().result;
+ }
+
+private:
+ std::coroutine_handle<promise_type> handle;
+};
+```
+
+Note how the `task::promise_type` has a member variable
+`std::coroutine_handle<> continuation`. This is the handle of the coroutine
that will be resumed when the current coroutine is finished executing (see
-``final_suspend``). In a sense, this is the "return address" of the coroutine.
-It is set inside ``operator co_await`` when another coroutine calls our
+`final_suspend`). In a sense, this is the "return address" of the coroutine.
+It is set inside `operator co_await` when another coroutine calls our
generator and awaits for the next value to be produced.
-The result value is returned via the ``int result`` member. It is written in
-``return_value`` and read by ``Awaiter::await_resume``. Usually, the result
+The result value is returned via the `int result` member. It is written in
+`return_value` and read by `Awaiter::await_resume`. Usually, the result
type of a task is a template argument. For simplicity's sake, we hard-coded the
-``int`` type in this example.
+`int` type in this example.
-Stack traces of in-flight coroutines
-------------------------------------
+### Stack traces of in-flight coroutines
Let's assume you have the following program and set a breakpoint inside the
-``write_output`` function. There are multiple call paths through which this
+`write_output` function. There are multiple call paths through which this
function could have been reached. How can we find out said call path?
-.. code-block:: c++
-
- #include <iostream>
- #include <string_view>
- #include "async-task-library.hpp"
-
- static task write_output(std::string_view contents) {
- std::cout << contents << "\n";
- co_return contents.size();
- }
-
- static task greet() {
- int bytes_written = 0;
- bytes_written += co_await write_output("Hello");
- bytes_written += co_await write_output("World");
- co_return bytes_written;
- }
-
- int main() {
- int bytes_written = greet().syncStart();
- std::cout << "Bytes written: " << bytes_written << "\n";
- return 0;
- }
-
-To do so, let's break inside ``write_output``. We can understand our call-stack
-by looking into the special ``__promise`` variable. This artificial variable is
-generated by the compiler and points to the ``promise_type`` instance
+```c++
+#include <iostream>
+#include <string_view>
+#include "async-task-library.hpp"
+
+static task write_output(std::string_view contents) {
+ std::cout << contents << "\n";
+ co_return contents.size();
+}
+
+static task greet() {
+ int bytes_written = 0;
+ bytes_written += co_await write_output("Hello");
+ bytes_written += co_await write_output("World");
+ co_return bytes_written;
+}
+
+int main() {
+ int bytes_written = greet().syncStart();
+ std::cout << "Bytes written: " << bytes_written << "\n";
+ return 0;
+}
+```
+
+To do so, let's break inside `write_output`. We can understand our call-stack
+by looking into the special `__promise` variable. This artificial variable is
+generated by the compiler and points to the `promise_type` instance
corresponding to the currently in-flight coroutine. In this case, the
-``__promise`` variable contains the ``continuation`` which points to our
-caller. That caller again contains a ``promise`` with a ``continuation`` which
+`__promise` variable contains the `continuation` which points to our
+caller. That caller again contains a `promise` with a `continuation` which
points to our caller's caller.
-.. image:: ./coro-async-task-continuations.png
+```{image} ./coro-async-task-continuations.png
+```
We can figure out the involved coroutine functions and their current suspension
points as discussed above in the "Inspecting a suspended coroutine" section.
-When using LLDB's CLI, the command ``p --ptr-depth 4 __promise`` might also be
+When using LLDB's CLI, the command `p --ptr-depth 4 __promise` might also be
useful to automatically dereference all the pointers up to the given depth.
To get a flat representation of that call stack, we can use a debugger script,
-such as the one shown in the :ref:`lldb-script` section. With that
-script, we can run ``coro bt`` to get the following stack trace:
-
-.. code-block::
-
- (lldb) coro bt
- frame #0: write_output(std::basic_string_view<char, std::char_traits<char>>) at /home/avogelsgesang/Documents/corotest/async-task-example.cpp:6:16
- [async] frame #1: greet() at /home/avogelsgesang/Documents/corotest/async-task-example.cpp:12:20
- [async] frame #2: std::__n4861::coroutine_handle<std::__n4861::noop_coroutine_promise>::__frame::__dummy_resume_destroy() at /usr/include/c++/14/coroutine:298, suspension point unknown
- frame #3: std::__n4861::coroutine_handle<task::promise_type>::resume() const at /usr/include/c++/14/coroutine:242:29
- frame #4: task::syncStart() at /home/avogelsgesang/Documents/corotest/async-task-library.hpp:78:14
- frame #5: main at /home/avogelsgesang/Documents/corotest/async-task-example.cpp:18:11
- frame #6: __libc_start_call_main at sysdeps/nptl/libc_start_call_main.h:58:16
- frame #7: __libc_start_main_impl at csu/libc-start.c:360:3
- frame #8: _start at :4294967295
+such as the one shown in the {ref}`lldb-script` section. With that
+script, we can run `coro bt` to get the following stack trace:
+
+```
+(lldb) coro bt
+frame #0: write_output(std::basic_string_view<char, std::char_traits<char>>) at /home/avogelsgesang/Documents/corotest/async-task-example.cpp:6:16
+[async] frame #1: greet() at /home/avogelsgesang/Documents/corotest/async-task-example.cpp:12:20
+[async] frame #2: std::__n4861::coroutine_handle<std::__n4861::noop_coroutine_promise>::__frame::__dummy_resume_destroy() at /usr/include/c++/14/coroutine:298, suspension point unknown
+frame #3: std::__n4861::coroutine_handle<task::promise_type>::resume() const at /usr/include/c++/14/coroutine:242:29
+frame #4: task::syncStart() at /home/avogelsgesang/Documents/corotest/async-task-library.hpp:78:14
+frame #5: main at /home/avogelsgesang/Documents/corotest/async-task-example.cpp:18:11
+frame #6: __libc_start_call_main at sysdeps/nptl/libc_start_call_main.h:58:16
+frame #7: __libc_start_main_impl at csu/libc-start.c:360:3
+frame #8: _start at :4294967295
+```
Note how the frames #1 and #2 are async frames.
-The ``coro bt`` command already includes logic to identify the exact suspension
-point of each frame based on the ``_coro_suspension_point_addr`` stored inside
+The `coro bt` command already includes logic to identify the exact suspension
+point of each frame based on the `_coro_suspension_point_addr` stored inside
the promise.
-Stack traces of suspended coroutines
-------------------------------------
+### Stack traces of suspended coroutines
Usually, while a coroutine is waiting for, e.g., an in-flight network request,
-the suspended ``coroutine_handle`` is stored within the work queues inside the
+the suspended `coroutine_handle` is stored within the work queues inside the
IO scheduler. As soon as we get hold of the coroutine handle, we can backtrace
-it by using ``coro bt <coro_handle>`` where ``<coro_handle>`` is an expression
+it by using `coro bt <coro_handle>` where `<coro_handle>` is an expression
evaluating to the coroutine handle of the suspended coroutine.
-Keeping track of all existing coroutines
-----------------------------------------
+### Keeping track of all existing coroutines
Usually, we should be able to get hold of all currently suspended coroutines by
inspecting the worker queues of the IO scheduler. In cases where this is not
@@ -454,77 +443,74 @@ suspended coroutines.
One such solution is to store the list of in-flight coroutines in a collection:
-.. code-block:: c++
-
- inline std::unordered_set<std::coroutine_handle<void>> inflight_coroutines;
- inline std::mutex inflight_coroutines_mutex;
-
- class promise_type {
- public:
- promise_type() {
- std::unique_lock<std::mutex> lock(inflight_coroutines_mutex);
- inflight_coroutines.insert(std::coroutine_handle<promise_type>::from_promise(*this));
- }
- ~promise_type() {
- std::unique_lock<std::mutex> lock(inflight_coroutines_mutex);
- inflight_coroutines.erase(std::coroutine_handle<promise_type>::from_promise(*this));
- }
- };
+```c++
+inline std::unordered_set<std::coroutine_handle<void>> inflight_coroutines;
+inline std::mutex inflight_coroutines_mutex;
-With this in place, it is possible to inspect ``inflight_coroutines`` from the
-debugger and rely on LLDB's ``std::coroutine_handle`` pretty-printer to
+class promise_type {
+public:
+ promise_type() {
+ std::unique_lock<std::mutex> lock(inflight_coroutines_mutex);
+ inflight_coroutines.insert(std::coroutine_handle<promise_type>::from_promise(*this));
+ }
+ ~promise_type() {
+ std::unique_lock<std::mutex> lock(inflight_coroutines_mutex);
+ inflight_coroutines.erase(std::coroutine_handle<promise_type>::from_promise(*this));
+ }
+};
+```
+
+With this in place, it is possible to inspect `inflight_coroutines` from the
+debugger and rely on LLDB's `std::coroutine_handle` pretty-printer to
inspect the coroutines.
This technique will track *all* coroutines, also the ones which are currently
awaiting another coroutine, though. To identify just the "roots" of our
-in-flight coroutines, we can use the ``coro in-flight inflight_coroutines``
-command provided by the :ref:`lldb-script`.
+in-flight coroutines, we can use the `coro in-flight inflight_coroutines`
+command provided by the {ref}`lldb-script`.
Please note that the above is expensive from a runtime performance perspective,
and requires locking to prevent data races. As such, it is not recommended to
use this approach in production code.
-Known issues & workarounds for older LLDB versions
-==================================================
+## Known issues & workarounds for older LLDB versions
-LLDB before 21.0 did not yet show the ``__coro_frame`` inside
-``coroutine_handle``. To inspect the coroutine frame, you had to use the
-approach described in the :ref:`devirtualization` section.
+LLDB before 21.0 did not yet show the `__coro_frame` inside
+`coroutine_handle`. To inspect the coroutine frame, you had to use the
+approach described in the {ref}`devirtualization` section.
-LLDB before 18.0 hid the ``__promise`` and ``__coro_frame``
+LLDB before 18.0 hid the `__promise` and `__coro_frame`
variables by default. The variables are still present, but they need to be
explicitly added to the "watch" pane in VS Code or requested via
-``print __promise`` and ``print __coro_frame`` from the debugger console.
+`print __promise` and `print __coro_frame` from the debugger console.
LLDB before 16.0 did not yet provide a pretty-printer for
-``std::coroutine_handle``. To inspect the coroutine handle, you had to manually
-use the approach described in the :ref:`devirtualization`
+`std::coroutine_handle`. To inspect the coroutine handle, you had to manually
+use the approach described in the {ref}`devirtualization`
section.
-Toolchain Implementation Details
-================================
+## Toolchain Implementation Details
This section covers the ABI as well as additional compiler-specific behavior.
The ABI is followed by all compilers, on all major systems, including Windows,
Linux, and macOS. Different compilers emit different debug information, though.
-Ramp, resume and destroy functions
-----------------------------------
+### Ramp, resume and destroy functions
Every coroutine is split into three parts:
-* The ramp function allocates the coroutine frame and initializes it, usually
+- The ramp function allocates the coroutine frame and initializes it, usually
copying over all variables into the coroutine frame
-* The resume function continues the coroutine from its previous suspension point
-* The destroy function destroys and deallocates the coroutine frame
-* The cleanup function destroys the coroutine frame but does not deallocate it.
+- The resume function continues the coroutine from its previous suspension point
+- The destroy function destroys and deallocates the coroutine frame
+- The cleanup function destroys the coroutine frame but does not deallocate it.
It is used when the coroutine's allocation was elided thanks to
- `Heap Allocation Elision (HALO) <https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p0981r0.html>`_
+ [Heap Allocation Elision (HALO)](https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p0981r0.html)
The ramp function is called by the coroutine's caller, and available under the
original function name used in the C++ source code. The resume function is
-called via ``std::coroutine_handle::resume``. The destroy function is called
-via ``std::coroutine_handle::destroy``.
+called via `std::coroutine_handle::resume`. The destroy function is called
+via `std::coroutine_handle::destroy`.
Information between the three functions is passed via the coroutine frame, a
compiler-synthesized struct that contains all necessary internal state. The
@@ -538,52 +524,50 @@ coroutine frame. When a coroutine frame was elided thanks to HALO, only the
destructors need to be called, but the coroutine frame must not be deallocated.
In those cases, the cleanup function is used instead of the destroy function.
-For coroutines allocated with ``[[clang::coro_await_elidable]]``, clang also
-generates a ``.noalloc`` variant of the ramp function, which does not allocate
+For coroutines allocated with `[[clang::coro_await_elidable]]`, clang also
+generates a `.noalloc` variant of the ramp function, which does not allocate
the coroutine frame by itself, but instead expects the caller to allocate the
coroutine frame and pass it to the ramp function.
When trying to intercept all creations of new coroutines in the debugger, you
-hence might have to set breakpoints in the ramp function and its ``.noalloc``
+hence might have to set breakpoints in the ramp function and its `.noalloc`
variant.
-Artificial ``__promise`` and ``__coro_frame`` variables
--------------------------------------------------------
+### Artificial `__promise` and `__coro_frame` variables
-Inside all coroutine functions, clang / LLVM synthesize a ``__promise`` and
-``__coro_frame`` variable. These variables are used to store the coroutine's
+Inside all coroutine functions, clang / LLVM synthesize a `__promise` and
+`__coro_frame` variable. These variables are used to store the coroutine's
state. When inside the coroutine function, those can be used to directly
inspect the promise and the coroutine frame of the own function.
-The ABI of a coroutine
-----------------------
+### The ABI of a coroutine
-A ``std::coroutine_handle`` essentially only holds a pointer to a coroutine
+A `std::coroutine_handle` essentially only holds a pointer to a coroutine
frame. It resembles the following struct:
-.. code-block:: c++
-
- template<typename promise_type>
- struct coroutine_handle {
- void* __coroutine_frame = nullptr;
- };
+```c++
+template<typename promise_type>
+struct coroutine_handle {
+ void* __coroutine_frame = nullptr;
+};
+```
The structure of coroutine frames is defined as
-.. code-block:: c++
-
- struct my_coroutine_frame {
- void (*__resume)(coroutine_frame*); // function pointer to the `resume` function
- void (*__destroy)(coroutine_frame*); // function pointer to the `destroy` function
- promise_type promise; // the corresponding `promise_type`
- ... // Internal coroutine state
- }
+```c++
+struct my_coroutine_frame {
+ void (*__resume)(coroutine_frame*); // function pointer to the `resume` function
+ void (*__destroy)(coroutine_frame*); // function pointer to the `destroy` function
+ promise_type promise; // the corresponding `promise_type`
+ ... // Internal coroutine state
+}
+```
For each coroutine, the compiler synthesizes a different coroutine type,
storing all necessary internal state. The actual coroutine type is type-erased
-behind the ``std::coroutine_handle``.
+behind the `std::coroutine_handle`.
-However, all coroutine frames always contain the ``resume`` and ``destroy``
+However, all coroutine frames always contain the `resume` and `destroy`
functions as their first two members. As such, we can read the function
pointers from the coroutine frame and then obtain the function's name from its
address.
@@ -592,19 +576,18 @@ The promise is guaranteed to be at a 16-byte offset from the coroutine frame.
If we have a coroutine handle at address 0x416eb0, we can hence reinterpret-cast
the promise as follows:
-.. code-block:: text
+```text
+print (task::promise_type)*(0x416eb0+16)
+```
- print (task::promise_type)*(0x416eb0+16)
-
-Implementation in clang / LLVM
-------------------------------
+### Implementation in clang / LLVM
The C++ Coroutines feature in the Clang compiler is implemented in two parts of
the compiler. Semantic analysis is performed in Clang, and coroutine
construction and optimization take place in the LLVM middle-end.
For each coroutine function, the frontend generates a single corresponding
-LLVM-IR function. This function uses special ``llvm.coro.suspend`` intrinsics
+LLVM-IR function. This function uses special `llvm.coro.suspend` intrinsics
to mark the suspension points of the coroutine. The middle end first optimizes
this function and applies, e.g., constant propagation across the whole,
non-split coroutine.
@@ -620,9 +603,9 @@ gets emitted, but none of them are really interesting regarding debugging
information.
For more details on the IR representation of coroutines and the relevant
-optimization passes, see `Coroutines in LLVM <https://llvm.org/docs/Coroutines.html>`_.
+optimization passes, see [Coroutines in LLVM](https://llvm.org/docs/Coroutines.html).
-Emitting debug information inside ``CoroSplit`` forces us to generate
+Emitting debug information inside `CoroSplit` forces us to generate
insufficient debugging information. Usually, the compiler generates debug
information in the frontend, as debug information is highly language specific.
However, this is not possible for coroutine frames because the frames are
@@ -632,42 +615,40 @@ To mitigate this problem, the LLVM middle end attempts to generate some debug
information, which is unfortunately incomplete, since much of the
language-specific information is missing in the middle end.
-.. _devirtualization:
+(devirtualization)=
-Devirtualization of coroutine handles
--------------------------------------
+### Devirtualization of coroutine handles
Figuring out the promise type and the coroutine frame type of a coroutine
-handle requires inspecting the ``resume`` and ``destroy`` function pointers.
+handle requires inspecting the `resume` and `destroy` function pointers.
There are two possible approaches to do so:
-1. clang always names the type by appending ``.coro_frame_ty`` to the
+1. clang always names the type by appending `.coro_frame_ty` to the
linkage name of the ramp function.
-2. Both clang and GCC add the function-local ``__promise`` and
- ``__coro_frame`` variables to the resume and destroy functions.
+2. Both clang and GCC add the function-local `__promise` and
+ `__coro_frame` variables to the resume and destroy functions.
We can lookup their types and thereby get the types of promise
and coroutine frame.
In general, the second approach is preferred, as it is more portable.
To do so, we look up the types in the destroy function and not the resume function
-because the resume function pointer will be set to a ``nullptr`` as soon as a
+because the resume function pointer will be set to a `nullptr` as soon as a
coroutine reaches its final suspension point. If we used the resume function,
devirtualization would hence fail for all coroutines that have reached their final
suspension point.
LLDB comes with devirtualization support out of the box, as part of the
-pretty-printer for ``std::coroutine_handle``. For GDB, a similar pretty-printer
-is provided by the :ref:`gdb-script`.
+pretty-printer for `std::coroutine_handle`. For GDB, a similar pretty-printer
+is provided by the {ref}`gdb-script`.
-Interpreting the coroutine frame in optimized builds
-----------------------------------------------------
+### Interpreting the coroutine frame in optimized builds
-The ``__coro_frame`` variable usually refers to the coroutine frame of an
+The `__coro_frame` variable usually refers to the coroutine frame of an
*in-flight* coroutine. This means the coroutine is currently executing.
However, the compiler only guarantees the coroutine frame to be in a consistent
state while the coroutine is suspended. As such, the variables inside the
-``__coro_frame`` variable might be outdated, particularly when optimizations
+`__coro_frame` variable might be outdated, particularly when optimizations
are enabled.
Furthermore, when optimizations are enabled, the compiler will layout the
@@ -675,621 +656,615 @@ coroutine frame more aggressively. Unused values are optimized out, and the
state will usually contain only the minimal information required to reconstruct
the coroutine's state.
-clang / LLVM usually use variables like ``__int_32_0`` to represent this
+clang / LLVM usually use variables like `__int_32_0` to represent this
optimized storage. Those values usually do not directly correspond to variables
in the source code.
When compiling the program
-.. code-block:: c++
-
- static task coro_task(int v) {
- int a = v;
- co_await some_other_task();
- a++; // __int_32_0 is 43 here
- std::cout << a << "\n";
- a++; // __int_32_0 is still 43 here
- std::cout << a << "\n";
- a++; // __int_32_0 is still 43 here!
- std::cout << a << "\n";
- co_await some_other_task();
- a++; // __int_32_0 is still 43 here!!
- std::cout << a << "\n";
- a++; // Why is __int_32_0 still 43 here?
- std::cout << a << "\n";
- }
-
-clang creates a single entry ``__int_32_0`` in the coroutine state.
-
-Intuitively, one might assume that ``__int_32_0`` represents the value of the
-local variable ``a``. However, inspecting ``__int_32_0`` in the debugger while
-single-stepping will reveal that the value of ``__int_32_0`` stays constant,
-despite ``a`` being frequently incremented.
+```c++
+static task coro_task(int v) {
+ int a = v;
+ co_await some_other_task();
+ a++; // __int_32_0 is 43 here
+ std::cout << a << "\n";
+ a++; // __int_32_0 is still 43 here
+ std::cout << a << "\n";
+ a++; // __int_32_0 is still 43 here!
+ std::cout << a << "\n";
+ co_await some_other_task();
+ a++; // __int_32_0 is still 43 here!!
+ std::cout << a << "\n";
+ a++; // Why is __int_32_0 still 43 here?
+ std::cout << a << "\n";
+}
+```
+
+clang creates a single entry `__int_32_0` in the coroutine state.
+
+Intuitively, one might assume that `__int_32_0` represents the value of the
+local variable `a`. However, inspecting `__int_32_0` in the debugger while
+single-stepping will reveal that the value of `__int_32_0` stays constant,
+despite `a` being frequently incremented.
While this might be surprising, this is a result of the optimizer recognizing
that it can eliminate most of the load/store operations.
The above code is optimized to the equivalent of:
-.. code-block:: c++
-
- static task coro_task(int v) {
- store v into __int_32_0 in the frame
- co_await await_counter{};
- a = load __int_32_0
- std::cout << a+1 << "\n";
- std::cout << a+2 << "\n";
- std::cout << a+3 << "\n";
- co_await await_counter{};
- a = load __int_32_0
- std::cout << a+4 << "\n";
- std::cout << a+5 << "\n";
- }
-
-It should now be obvious why the value of ``__int_32_0`` remains unchanged
-throughout the function. It is important to recognize that ``__int_32_0`` does
-not directly correspond to ``a``, but is instead a variable generated to assist
+```c++
+static task coro_task(int v) {
+ store v into __int_32_0 in the frame
+ co_await await_counter{};
+ a = load __int_32_0
+ std::cout << a+1 << "\n";
+ std::cout << a+2 << "\n";
+ std::cout << a+3 << "\n";
+ co_await await_counter{};
+ a = load __int_32_0
+ std::cout << a+4 << "\n";
+ std::cout << a+5 << "\n";
+}
+```
+
+It should now be obvious why the value of `__int_32_0` remains unchanged
+throughout the function. It is important to recognize that `__int_32_0` does
+not directly correspond to `a`, but is instead a variable generated to assist
the compiler in code generation. The variables in an optimized coroutine frame
should not be thought of as directly representing the variables in the C++
source.
+### Mapping suspension point indices to source code locations
-Mapping suspension point indices to source code locations
----------------------------------------------------------
-
-To aid in mapping a ``__coro_index`` back to a source code location, clang 21.0
+To aid in mapping a `__coro_index` back to a source code location, clang 21.0
and newer emit special, compiler-generated labels for the suspension points.
-In gdb, we can use the ``info line`` command to get the source code location of
+In gdb, we can use the `info line` command to get the source code location of
the suspension point.
-::
-
- (gdb) info line -function coro_task -label __coro_resume_2
- Line 45 of "llvm-example.cpp" starts at address 0x1b1b <_ZL9coro_taski.resume+555> and ends at 0x1b46 <_ZL9coro_taski.resume+598>.
- Line 45 of "llvm-example.cpp" starts at address 0x201b <_ZL9coro_taski.destroy+555> and ends at 0x2046 <_ZL9coro_taski.destroy+598>.
- Line 45 of "llvm-example.cpp" starts at address 0x253b <_ZL9coro_taski.cleanup+555> and ends at 0x2566 <_ZL9coro_taski.cleanup+598>.
+```
+(gdb) info line -function coro_task -label __coro_resume_2
+Line 45 of "llvm-example.cpp" starts at address 0x1b1b <_ZL9coro_taski.resume+555> and ends at 0x1b46 <_ZL9coro_taski.resume+598>.
+Line 45 of "llvm-example.cpp" starts at address 0x201b <_ZL9coro_taski.destroy+555> and ends at 0x2046 <_ZL9coro_taski.destroy+598>.
+Line 45 of "llvm-example.cpp" starts at address 0x253b <_ZL9coro_taski.cleanup+555> and ends at 0x2566 <_ZL9coro_taski.cleanup+598>.
+```
LLDB does not support looking up labels, yet. For this reason, LLDB's pretty-printer
does not show the exact line number of the suspension point.
+## Resources
-Resources
-=========
+(lldb-script)=
-.. _lldb-script:
+### LLDB Debugger Script
-LLDB Debugger Script
---------------------
-
-The following script provides the ``coro bt`` and ``coro in-flight`` commands
-discussed above. It can be loaded into LLDB using ``command script import
-lldb_coro_debugging.py``. To load this by default, add this command to your
-``~/.lldbinit`` file.
+The following script provides the `coro bt` and `coro in-flight` commands
+discussed above. It can be loaded into LLDB using `command script import
+lldb_coro_debugging.py`. To load this by default, add this command to your
+`~/.lldbinit` file.
Note that this script requires LLDB 21.0 or newer.
-.. code-block:: python
-
- # lldb_coro_debugging.py
- import lldb
- from lldb.plugins.parsed_cmd import ParsedCommand
-
- def _get_first_var_path(v, paths):
- """
- Tries multiple variable paths via `GetValueForExpressionPath`
- and returns the first one that succeeds, or None if none succeed.
- """
- for path in paths:
- var = v.GetValueForExpressionPath(path)
- if var.error.Success():
- return var
- return None
-
-
- def _print_async_bt(coro_hdl, result, *, curr_idx, start, limit, continuation_paths, prefix=""):
- """
- Prints a backtrace for an async coroutine stack starting from `coro_hdl`,
- using the given `continuation_paths` to get the next coroutine from the promise.
- """
- target = coro_hdl.GetTarget()
- while curr_idx < limit and coro_hdl is not None and coro_hdl.error.Success():
- # Print the stack frame, if in range
- if curr_idx >= start:
- # Figure out the function name
- destroy_func_var = coro_hdl.GetValueForExpressionPath(".destroy")
- destroy_addr = target.ResolveLoadAddress(destroy_func_var.GetValueAsAddress())
- func_name = destroy_addr.function.name
- # Figure out the line entry to show
- suspension_addr_var = coro_hdl.GetValueForExpressionPath(".promise._coro_suspension_point_addr")
- if suspension_addr_var.error.Success():
- line_entry = target.ResolveLoadAddress(suspension_addr_var.GetValueAsAddress()).line_entry
- print(f"{prefix} frame #{curr_idx}: {func_name} at {line_entry}", file=result)
- else:
- # We don't know the exact line, print the suspension point ID, so we at least show
- # the id of the current suspension point
- suspension_point_var = coro_hdl.GetValueForExpressionPath(".coro_frame.__coro_index")
- if suspension_point_var.error.Success():
- suspension_point = suspension_point_var.GetValueAsUnsigned()
- else:
- suspension_point = "unknown"
- line_entry = destroy_addr.line_entry
- print(f"{prefix} frame #{curr_idx}: {func_name} at {line_entry}, suspension point {suspension_point}", file=result)
- # Move to the next stack frame
- curr_idx += 1
- promise_var = coro_hdl.GetChildMemberWithName("promise")
- coro_hdl = _get_first_var_path(promise_var, continuation_paths)
- return curr_idx
-
- def _print_combined_bt(frame, result, *, unfiltered, curr_idx, start, limit, continuation_paths):
- """
- Prints a backtrace starting from `frame`, interleaving async coroutine frames
- with regular frames.
- """
- while curr_idx < limit and frame.IsValid():
- if curr_idx >= start and (unfiltered or not frame.IsHidden()):
- print(f"frame #{curr_idx}: {frame.name} at {frame.line_entry}", file=result)
- curr_idx += 1
- coro_var = _get_first_var_path(frame.GetValueForVariablePath("__promise"), continuation_paths)
- if coro_var:
- curr_idx = _print_async_bt(coro_var, result,
- curr_idx=curr_idx, start=start, limit=limit,
- continuation_paths=continuation_paths, prefix="[async]")
- frame = frame.parent
-
-
- class CoroBacktraceCommand(ParsedCommand):
- def get_short_help(self):
- return "Create a backtrace for C++-20 coroutines"
-
- def get_flags(self):
- return lldb.eCommandRequiresFrame | lldb.eCommandProcessMustBePaused
-
- def setup_command_definition(self):
- ov_parser = self.get_parser()
- ov_parser.add_option(
- "e",
- "continuation-expr",
- help = (
- "Semi-colon-separated list of expressions evaluated against the promise object"
- "to get the next coroutine (e.g. `.continuation;.coro_parent`)"
- ),
- value_type = lldb.eArgTypeNone,
- dest = "continuation_expr_arg",
- default = ".continuation",
- )
- ov_parser.add_option(
- "c",
- "count",
- help = "How many frames to display (0 for all)",
- value_type = lldb.eArgTypeCount,
- dest = "count_arg",
- default = 20,
- )
- ov_parser.add_option(
- "s",
- "start",
- help = "Frame in which to start the backtrace",
- value_type = lldb.eArgTypeIndex,
- dest = "frame_index_arg",
- default = 0,
- )
- ov_parser.add_option(
- "u",
- "unfiltered",
- help = "Do not filter out frames according to installed frame recognizers",
- value_type = lldb.eArgTypeBoolean,
- dest = "unfiltered_arg",
- default = False,
- )
- ov_parser.add_argument_set([
- ov_parser.make_argument_element(
- lldb.eArgTypeExpression,
- repeat="optional"
- )
- ])
-
- def __call__(self, debugger, args_array, exe_ctx, result):
- ov_parser = self.get_parser()
- continuation_paths = ov_parser.continuation_expr_arg.split(";")
- count = ov_parser.count_arg
- if count == 0:
- count = 99999
- frame_index = ov_parser.frame_index_arg
- unfiltered = ov_parser.unfiltered_arg
-
- frame = exe_ctx.GetFrame()
- if not frame.IsValid():
- result.SetError("invalid frame")
- return
-
- if len(args_array) > 1:
- result.SetError("At most one expression expected")
- return
- elif len(args_array) == 1:
- expr = args_array.GetItemAtIndex(0).GetStringValue(9999)
- coro_hdl = frame.EvaluateExpression(expr)
- if not coro_hdl.error.Success():
- result.AppendMessage(
- f'error: expression failed {expr} => {coro_hdl.error}'
- )
- result.SetError(f"Expression `{expr}` failed to evaluate")
- return
- _print_async_bt(coro_hdl, result,
- curr_idx = 0, start = frame_index, limit = frame_index + count,
- continuation_paths = continuation_paths)
- else:
- _print_combined_bt(frame, result, unfiltered=unfiltered,
- curr_idx = 0, start = frame_index, limit = frame_index + count,
- continuation_paths = continuation_paths)
-
-
- class CoroInflightCommand(ParsedCommand):
- def get_short_help(self):
- return "Identify all in-flight coroutines"
-
- def get_flags(self):
- return lldb.eCommandRequiresTarget | lldb.eCommandProcessMustBePaused
-
- def setup_command_definition(self):
- ov_parser = self.get_parser()
- ov_parser.add_option(
- "e",
- "continuation-expr",
- help = (
- "Semi-colon-separated list of expressions evaluated against the promise object"
- "to get the next coroutine (e.g. `.continuation;.coro_parent`)"
- ),
- value_type = lldb.eArgTypeNone,
- dest = "continuation_expr_arg",
- default = ".continuation",
- )
- ov_parser.add_option(
- "c",
- "count",
- help = "How many frames to display (0 for all)",
- value_type = lldb.eArgTypeCount,
- dest = "count_arg",
- default = 5,
- )
- ov_parser.add_argument_set([
- ov_parser.make_argument_element(
- lldb.eArgTypeExpression,
- repeat="plus"
- )
- ])
-
- def __call__(self, debugger, args_array, exe_ctx, result):
- ov_parser = self.get_parser()
- continuation_paths = ov_parser.continuation_expr_arg.split(";")
- count = ov_parser.count_arg
-
- # Collect all coroutine_handles from the provided containers
- all_coros = []
- for entry in args_array:
- expr = entry.GetStringValue(9999)
- if exe_ctx.frame.IsValid():
- coro_container = exe_ctx.frame.EvaluateExpression(expr)
- else:
- coro_container = exe_ctx.target.EvaluateExpression(expr)
- if not coro_container.error.Success():
- result.AppendMessage(
- f'error: expression failed {expr} => {coro_container.error}'
- )
- result.SetError(f"Expression `{expr}` failed to evaluate")
- return
- for entry in coro_container.children:
- if "coroutine_handle" not in entry.GetType().name:
- result.SetError(f"Found entry of type {entry.GetType().name} in {expr},"
- " expected a coroutine handle")
- return
- all_coros.append(entry)
-
- # Remove all coroutines that are currently waiting for other coroutines to finish
- coro_roots = {c.GetChildMemberWithName("coro_frame").GetValueAsAddress(): c for c in all_coros}
- for coro_hdl in all_coros:
- parent_coro = _get_first_var_path(coro_hdl.GetChildMemberWithName("promise"), continuation_paths)
- parent_addr = parent_coro.GetChildMemberWithName("coro_frame").GetValueAsAddress()
- if parent_addr in coro_roots:
- del coro_roots[parent_addr]
-
- # Print all remaining coroutines
- for addr, root_hdl in coro_roots.items():
- print(f"coroutine root 0x{addr:x}", file=result)
- _print_async_bt(root_hdl, result,
- curr_idx=0, start=0, limit=count,
- continuation_paths=continuation_paths, prefix=" ")
-
-
- def __lldb_init_module(debugger, internal_dict):
- debugger.HandleCommand("command container add -h 'Debugging utilities for C++20 coroutines' coro")
- debugger.HandleCommand(f"command script add -o -p -c {__name__}.CoroBacktraceCommand coro bt")
- debugger.HandleCommand(f"command script add -o -p -c {__name__}.CoroInflightCommand coro in-flight")
- print("Coro debugging utilities installed. Use `help coro` to see available commands.")
-
- if __name__ == '__main__':
- print("This script should be loaded from LLDB using `command script import <filename>`")
-
-.. _gdb-script:
-
-GDB Debugger Script
--------------------
+```python
+# lldb_coro_debugging.py
+import lldb
+from lldb.plugins.parsed_cmd import ParsedCommand
+
+def _get_first_var_path(v, paths):
+ """
+ Tries multiple variable paths via `GetValueForExpressionPath`
+ and returns the first one that succeeds, or None if none succeed.
+ """
+ for path in paths:
+ var = v.GetValueForExpressionPath(path)
+ if var.error.Success():
+ return var
+ return None
+
+
+def _print_async_bt(coro_hdl, result, *, curr_idx, start, limit, continuation_paths, prefix=""):
+ """
+ Prints a backtrace for an async coroutine stack starting from `coro_hdl`,
+ using the given `continuation_paths` to get the next coroutine from the promise.
+ """
+ target = coro_hdl.GetTarget()
+ while curr_idx < limit and coro_hdl is not None and coro_hdl.error.Success():
+ # Print the stack frame, if in range
+ if curr_idx >= start:
+ # Figure out the function name
+ destroy_func_var = coro_hdl.GetValueForExpressionPath(".destroy")
+ destroy_addr = target.ResolveLoadAddress(destroy_func_var.GetValueAsAddress())
+ func_name = destroy_addr.function.name
+ # Figure out the line entry to show
+ suspension_addr_var = coro_hdl.GetValueForExpressionPath(".promise._coro_suspension_point_addr")
+ if suspension_addr_var.error.Success():
+ line_entry = target.ResolveLoadAddress(suspension_addr_var.GetValueAsAddress()).line_entry
+ print(f"{prefix} frame #{curr_idx}: {func_name} at {line_entry}", file=result)
+ else:
+ # We don't know the exact line, print the suspension point ID, so we at least show
+ # the id of the current suspension point
+ suspension_point_var = coro_hdl.GetValueForExpressionPath(".coro_frame.__coro_index")
+ if suspension_point_var.error.Success():
+ suspension_point = suspension_point_var.GetValueAsUnsigned()
+ else:
+ suspension_point = "unknown"
+ line_entry = destroy_addr.line_entry
+ print(f"{prefix} frame #{curr_idx}: {func_name} at {line_entry}, suspension point {suspension_point}", file=result)
+ # Move to the next stack frame
+ curr_idx += 1
+ promise_var = coro_hdl.GetChildMemberWithName("promise")
+ coro_hdl = _get_first_var_path(promise_var, continuation_paths)
+ return curr_idx
+
+def _print_combined_bt(frame, result, *, unfiltered, curr_idx, start, limit, continuation_paths):
+ """
+ Prints a backtrace starting from `frame`, interleaving async coroutine frames
+ with regular frames.
+ """
+ while curr_idx < limit and frame.IsValid():
+ if curr_idx >= start and (unfiltered or not frame.IsHidden()):
+ print(f"frame #{curr_idx}: {frame.name} at {frame.line_entry}", file=result)
+ curr_idx += 1
+ coro_var = _get_first_var_path(frame.GetValueForVariablePath("__promise"), continuation_paths)
+ if coro_var:
+ curr_idx = _print_async_bt(coro_var, result,
+ curr_idx=curr_idx, start=start, limit=limit,
+ continuation_paths=continuation_paths, prefix="[async]")
+ frame = frame.parent
+
+
+class CoroBacktraceCommand(ParsedCommand):
+ def get_short_help(self):
+ return "Create a backtrace for C++-20 coroutines"
+
+ def get_flags(self):
+ return lldb.eCommandRequiresFrame | lldb.eCommandProcessMustBePaused
+
+ def setup_command_definition(self):
+ ov_parser = self.get_parser()
+ ov_parser.add_option(
+ "e",
+ "continuation-expr",
+ help = (
+ "Semi-colon-separated list of expressions evaluated against the promise object"
+ "to get the next coroutine (e.g. `.continuation;.coro_parent`)"
+ ),
+ value_type = lldb.eArgTypeNone,
+ dest = "continuation_expr_arg",
+ default = ".continuation",
+ )
+ ov_parser.add_option(
+ "c",
+ "count",
+ help = "How many frames to display (0 for all)",
+ value_type = lldb.eArgTypeCount,
+ dest = "count_arg",
+ default = 20,
+ )
+ ov_parser.add_option(
+ "s",
+ "start",
+ help = "Frame in which to start the backtrace",
+ value_type = lldb.eArgTypeIndex,
+ dest = "frame_index_arg",
+ default = 0,
+ )
+ ov_parser.add_option(
+ "u",
+ "unfiltered",
+ help = "Do not filter out frames according to installed frame recognizers",
+ value_type = lldb.eArgTypeBoolean,
+ dest = "unfiltered_arg",
+ default = False,
+ )
+ ov_parser.add_argument_set([
+ ov_parser.make_argument_element(
+ lldb.eArgTypeExpression,
+ repeat="optional"
+ )
+ ])
+
+ def __call__(self, debugger, args_array, exe_ctx, result):
+ ov_parser = self.get_parser()
+ continuation_paths = ov_parser.continuation_expr_arg.split(";")
+ count = ov_parser.count_arg
+ if count == 0:
+ count = 99999
+ frame_index = ov_parser.frame_index_arg
+ unfiltered = ov_parser.unfiltered_arg
+
+ frame = exe_ctx.GetFrame()
+ if not frame.IsValid():
+ result.SetError("invalid frame")
+ return
+
+ if len(args_array) > 1:
+ result.SetError("At most one expression expected")
+ return
+ elif len(args_array) == 1:
+ expr = args_array.GetItemAtIndex(0).GetStringValue(9999)
+ coro_hdl = frame.EvaluateExpression(expr)
+ if not coro_hdl.error.Success():
+ result.AppendMessage(
+ f'error: expression failed {expr} => {coro_hdl.error}'
+ )
+ result.SetError(f"Expression `{expr}` failed to evaluate")
+ return
+ _print_async_bt(coro_hdl, result,
+ curr_idx = 0, start = frame_index, limit = frame_index + count,
+ continuation_paths = continuation_paths)
+ else:
+ _print_combined_bt(frame, result, unfiltered=unfiltered,
+ curr_idx = 0, start = frame_index, limit = frame_index + count,
+ continuation_paths = continuation_paths)
+
+
+class CoroInflightCommand(ParsedCommand):
+ def get_short_help(self):
+ return "Identify all in-flight coroutines"
+
+ def get_flags(self):
+ return lldb.eCommandRequiresTarget | lldb.eCommandProcessMustBePaused
+
+ def setup_command_definition(self):
+ ov_parser = self.get_parser()
+ ov_parser.add_option(
+ "e",
+ "continuation-expr",
+ help = (
+ "Semi-colon-separated list of expressions evaluated against the promise object"
+ "to get the next coroutine (e.g. `.continuation;.coro_parent`)"
+ ),
+ value_type = lldb.eArgTypeNone,
+ dest = "continuation_expr_arg",
+ default = ".continuation",
+ )
+ ov_parser.add_option(
+ "c",
+ "count",
+ help = "How many frames to display (0 for all)",
+ value_type = lldb.eArgTypeCount,
+ dest = "count_arg",
+ default = 5,
+ )
+ ov_parser.add_argument_set([
+ ov_parser.make_argument_element(
+ lldb.eArgTypeExpression,
+ repeat="plus"
+ )
+ ])
+
+ def __call__(self, debugger, args_array, exe_ctx, result):
+ ov_parser = self.get_parser()
+ continuation_paths = ov_parser.continuation_expr_arg.split(";")
+ count = ov_parser.count_arg
+
+ # Collect all coroutine_handles from the provided containers
+ all_coros = []
+ for entry in args_array:
+ expr = entry.GetStringValue(9999)
+ if exe_ctx.frame.IsValid():
+ coro_container = exe_ctx.frame.EvaluateExpression(expr)
+ else:
+ coro_container = exe_ctx.target.EvaluateExpression(expr)
+ if not coro_container.error.Success():
+ result.AppendMessage(
+ f'error: expression failed {expr} => {coro_container.error}'
+ )
+ result.SetError(f"Expression `{expr}` failed to evaluate")
+ return
+ for entry in coro_container.children:
+ if "coroutine_handle" not in entry.GetType().name:
+ result.SetError(f"Found entry of type {entry.GetType().name} in {expr},"
+ " expected a coroutine handle")
+ return
+ all_coros.append(entry)
+
+ # Remove all coroutines that are currently waiting for other coroutines to finish
+ coro_roots = {c.GetChildMemberWithName("coro_frame").GetValueAsAddress(): c for c in all_coros}
+ for coro_hdl in all_coros:
+ parent_coro = _get_first_var_path(coro_hdl.GetChildMemberWithName("promise"), continuation_paths)
+ parent_addr = parent_coro.GetChildMemberWithName("coro_frame").GetValueAsAddress()
+ if parent_addr in coro_roots:
+ del coro_roots[parent_addr]
+
+ # Print all remaining coroutines
+ for addr, root_hdl in coro_roots.items():
+ print(f"coroutine root 0x{addr:x}", file=result)
+ _print_async_bt(root_hdl, result,
+ curr_idx=0, start=0, limit=count,
+ continuation_paths=continuation_paths, prefix=" ")
+
+
+def __lldb_init_module(debugger, internal_dict):
+ debugger.HandleCommand("command container add -h 'Debugging utilities for C++20 coroutines' coro")
+ debugger.HandleCommand(f"command script add -o -p -c {__name__}.CoroBacktraceCommand coro bt")
+ debugger.HandleCommand(f"command script add -o -p -c {__name__}.CoroInflightCommand coro in-flight")
+ print("Coro debugging utilities installed. Use `help coro` to see available commands.")
+
+if __name__ == '__main__':
+ print("This script should be loaded from LLDB using `command script import <filename>`")
+```
+
+(gdb-script)=
+
+### GDB Debugger Script
The following script provides:
-* a pretty-printer for coroutine handles
-* a frame filter to add coroutine frames to the built-in ``bt`` command
-* the ``get_coro_frame`` and ``get_coro_promise`` functions to be used in
- expressions, e.g. ``p get_coro_promise(fib.coro_hdl)->current_state``
-
-It can be loaded into GDB using ``source gdb_coro_debugging.py``.
-To load this by default, add this command to your ``~/.gdbinit`` file.
-
-.. code-block:: python
-
- # gdb_coro_debugging.py
- import gdb
- from gdb.FrameDecorator import FrameDecorator
-
- import typing
- import re
-
- def _load_pointer_at(addr: int):
- return gdb.Value(addr).reinterpret_cast(gdb.lookup_type('void').pointer().pointer()).dereference()
-
- """
- Devirtualized coroutine frame.
-
- Devirtualizes the promise and frame pointer types by inspecting
- the destroy function.
-
- Implements `to_string` and `children` to be used by `gdb.printing.PrettyPrinter`.
- Base class for `CoroutineHandlePrinter`.
- """
- class DevirtualizedCoroFrame:
- def __init__(self, frame_ptr_raw: int, val: gdb.Value | None = None):
- self.val = val
- self.frame_ptr_raw = frame_ptr_raw
-
- # Get the resume and destroy pointers.
- if frame_ptr_raw == 0:
- self.resume_ptr = None
- self.destroy_ptr = None
- self.promise_ptr = None
- self.frame_ptr = gdb.Value(frame_ptr_raw).reinterpret_cast(gdb.lookup_type("void").pointer())
- return
-
- # Get the resume and destroy pointers.
- self.resume_ptr = _load_pointer_at(frame_ptr_raw)
- self.destroy_ptr = _load_pointer_at(frame_ptr_raw + 8)
-
- # Devirtualize the promise and frame pointer types.
- frame_type = gdb.lookup_type("void")
- promise_type = gdb.lookup_type("void")
- self.destroy_func = gdb.block_for_pc(int(self.destroy_ptr))
- if self.destroy_func is not None:
- frame_var = gdb.lookup_symbol("__coro_frame", self.destroy_func, gdb.SYMBOL_VAR_DOMAIN)[0]
- if frame_var is not None:
- frame_type = frame_var.type
- promise_var = gdb.lookup_symbol("__promise", self.destroy_func, gdb.SYMBOL_VAR_DOMAIN)[0]
- if promise_var is not None:
- promise_type = promise_var.type.strip_typedefs()
-
- # If the type has a template argument, prefer it over the devirtualized type.
- if self.val is not None:
- promise_type_template_arg = self.val.type.template_argument(0)
- if promise_type_template_arg is not None and promise_type_template_arg.code != gdb.TYPE_CODE_VOID:
- promise_type = promise_type_template_arg
-
- self.promise_ptr = gdb.Value(frame_ptr_raw + 16).reinterpret_cast(promise_type.pointer())
- self.frame_ptr = gdb.Value(frame_ptr_raw).reinterpret_cast(frame_type.pointer())
-
- # Try to get the suspension point index and look up the exact line entry.
- self.suspension_point_index = int(self.frame_ptr.dereference()["__coro_index"]) if frame_type.code == gdb.TYPE_CODE_STRUCT else None
- self.resume_func = gdb.block_for_pc(int(self.resume_ptr))
- self.resume_label = None
- if self.resume_func is not None and self.suspension_point_index is not None:
- label_name = f"__coro_resume_{self.suspension_point_index}"
- self.resume_label = gdb.lookup_symbol(label_name, self.resume_func, gdb.SYMBOL_LABEL_DOMAIN)[0]
-
- def get_function_name(self):
- if self.destroy_func is None:
- return None
- name = self.destroy_func.function.name
- # Strip the "clone" suffix if it exists.
- if "() [clone " in name:
- name = name[:name.index("() [clone ")]
- return name
-
- def to_string(self):
- result = "coro(" + str(self.frame_ptr_raw) + ")"
- if self.destroy_func is not None:
- result += ": " + self.get_function_name()
- if self.resume_label is not None:
- result += ", line " + str(self.resume_label.line)
- if self.suspension_point_index is not None:
- result += ", suspension point " + str(self.suspension_point_index)
- return result
-
- def children(self):
- if self.resume_ptr is None:
- return [
- ("coro_frame", self.frame_ptr),
- ]
- else:
- return [
- ("resume", self.resume_ptr),
- ("destroy", self.destroy_ptr),
- ("promise", self.promise_ptr),
- ("coro_frame", self.frame_ptr)
- ]
-
-
- # Works for both libc++ and libstdc++.
- libcxx_corohdl_regex = re.compile('^std::__[A-Za-z0-9]+::coroutine_handle<.+>$|^std::coroutine_handle<.+>(( )?&)?$')
-
- def _extract_coro_frame_ptr_from_handle(val: gdb.Value):
- if libcxx_corohdl_regex.match(val.type.strip_typedefs().name) is None:
- raise ValueError("Expected a std::coroutine_handle, got %s" % val.type.strip_typedefs().name)
-
- # We expect the coroutine handle to have a single field, which is the frame pointer.
- # This heuristic works for both libc++ and libstdc++.
- fields = val.type.fields()
- if len(fields) != 1:
- raise ValueError("Expected 1 field, got %d" % len(fields))
- return int(val[fields[0]])
-
-
- """
- Pretty printer for `std::coroutine_handle<T>`
-
- Works for both libc++ and libstdc++.
-
- It prints the coroutine handle as a struct with the following fields:
- - resume: the resume function pointer
- - destroy: the destroy function pointer
- - promise: the promise pointer
- - coro_frame: the coroutine frame pointer
-
- Most of the functionality is implemented in `DevirtualizedCoroFrame`.
- """
- class CoroutineHandlePrinter(DevirtualizedCoroFrame):
- def __init__(self, val : gdb.Value):
- frame_ptr_raw = _extract_coro_frame_ptr_from_handle(val)
- super(CoroutineHandlePrinter, self).__init__(frame_ptr_raw, val)
-
-
- def build_pretty_printer():
- pp = gdb.printing.RegexpCollectionPrettyPrinter("coroutine")
- pp.add_printer('std::coroutine_handle', libcxx_corohdl_regex, CoroutineHandlePrinter)
- return pp
-
- gdb.printing.register_pretty_printer(
- gdb.current_objfile(),
- build_pretty_printer())
-
-
- """
- Get the coroutine frame pointer from a coroutine handle.
-
- Usage:
- ```
- p *get_coro_frame(coroutine_hdl)
- ```
- """
- class GetCoroFrame(gdb.Function):
- def __init__(self):
- super(GetCoroFrame, self).__init__("get_coro_frame")
-
- def invoke(self, coroutine_hdl_raw):
- return CoroutineHandlePrinter(coroutine_hdl_raw).frame_ptr
-
- GetCoroFrame()
-
-
- """
- Get the coroutine frame pointer from a coroutine handle.
-
- Usage:
- ```
- p *get_coro_promise(coroutine_hdl)
- ```
- """
- class GetCoroFrame(gdb.Function):
- def __init__(self):
- super(GetCoroFrame, self).__init__("get_coro_promise")
-
- def invoke(self, coroutine_hdl_raw):
- return CoroutineHandlePrinter(coroutine_hdl_raw).promise_ptr
-
- GetCoroFrame()
-
-
- """
- Decorator for coroutine frames.
-
- Used by `CoroutineFrameFilter` to add the coroutine frames to the built-in `bt` command.
- """
- class CoroutineFrameDecorator(FrameDecorator):
- def __init__(self, coro_frame: DevirtualizedCoroFrame, inferior_frame: gdb.Frame):
- super(CoroutineFrameDecorator, self).__init__(inferior_frame)
- self.coro_frame = coro_frame
-
- def function(self):
- func_name = self.coro_frame.get_function_name()
- if func_name is not None:
- return "[async] " + func_name
- return "[async] coroutine (coro_frame=" + str(self.coro_frame.frame_ptr_raw) + ")"
+- a pretty-printer for coroutine handles
+- a frame filter to add coroutine frames to the built-in `bt` command
+- the `get_coro_frame` and `get_coro_promise` functions to be used in
+ expressions, e.g. `p get_coro_promise(fib.coro_hdl)->current_state`
+
+It can be loaded into GDB using `source gdb_coro_debugging.py`.
+To load this by default, add this command to your `~/.gdbinit` file.
+
+````python
+# gdb_coro_debugging.py
+import gdb
+from gdb.FrameDecorator import FrameDecorator
+
+import typing
+import re
+
+def _load_pointer_at(addr: int):
+ return gdb.Value(addr).reinterpret_cast(gdb.lookup_type('void').pointer().pointer()).dereference()
+
+"""
+Devirtualized coroutine frame.
+
+Devirtualizes the promise and frame pointer types by inspecting
+the destroy function.
+
+Implements `to_string` and `children` to be used by `gdb.printing.PrettyPrinter`.
+Base class for `CoroutineHandlePrinter`.
+"""
+class DevirtualizedCoroFrame:
+ def __init__(self, frame_ptr_raw: int, val: gdb.Value | None = None):
+ self.val = val
+ self.frame_ptr_raw = frame_ptr_raw
+
+ # Get the resume and destroy pointers.
+ if frame_ptr_raw == 0:
+ self.resume_ptr = None
+ self.destroy_ptr = None
+ self.promise_ptr = None
+ self.frame_ptr = gdb.Value(frame_ptr_raw).reinterpret_cast(gdb.lookup_type("void").pointer())
+ return
+
+ # Get the resume and destroy pointers.
+ self.resume_ptr = _load_pointer_at(frame_ptr_raw)
+ self.destroy_ptr = _load_pointer_at(frame_ptr_raw + 8)
+
+ # Devirtualize the promise and frame pointer types.
+ frame_type = gdb.lookup_type("void")
+ promise_type = gdb.lookup_type("void")
+ self.destroy_func = gdb.block_for_pc(int(self.destroy_ptr))
+ if self.destroy_func is not None:
+ frame_var = gdb.lookup_symbol("__coro_frame", self.destroy_func, gdb.SYMBOL_VAR_DOMAIN)[0]
+ if frame_var is not None:
+ frame_type = frame_var.type
+ promise_var = gdb.lookup_symbol("__promise", self.destroy_func, gdb.SYMBOL_VAR_DOMAIN)[0]
+ if promise_var is not None:
+ promise_type = promise_var.type.strip_typedefs()
+
+ # If the type has a template argument, prefer it over the devirtualized type.
+ if self.val is not None:
+ promise_type_template_arg = self.val.type.template_argument(0)
+ if promise_type_template_arg is not None and promise_type_template_arg.code != gdb.TYPE_CODE_VOID:
+ promise_type = promise_type_template_arg
+
+ self.promise_ptr = gdb.Value(frame_ptr_raw + 16).reinterpret_cast(promise_type.pointer())
+ self.frame_ptr = gdb.Value(frame_ptr_raw).reinterpret_cast(frame_type.pointer())
+
+ # Try to get the suspension point index and look up the exact line entry.
+ self.suspension_point_index = int(self.frame_ptr.dereference()["__coro_index"]) if frame_type.code == gdb.TYPE_CODE_STRUCT else None
+ self.resume_func = gdb.block_for_pc(int(self.resume_ptr))
+ self.resume_label = None
+ if self.resume_func is not None and self.suspension_point_index is not None:
+ label_name = f"__coro_resume_{self.suspension_point_index}"
+ self.resume_label = gdb.lookup_symbol(label_name, self.resume_func, gdb.SYMBOL_LABEL_DOMAIN)[0]
+
+ def get_function_name(self):
+ if self.destroy_func is None:
+ return None
+ name = self.destroy_func.function.name
+ # Strip the "clone" suffix if it exists.
+ if "() [clone " in name:
+ name = name[:name.index("() [clone ")]
+ return name
+
+ def to_string(self):
+ result = "coro(" + str(self.frame_ptr_raw) + ")"
+ if self.destroy_func is not None:
+ result += ": " + self.get_function_name()
+ if self.resume_label is not None:
+ result += ", line " + str(self.resume_label.line)
+ if self.suspension_point_index is not None:
+ result += ", suspension point " + str(self.suspension_point_index)
+ return result
+
+ def children(self):
+ if self.resume_ptr is None:
+ return [
+ ("coro_frame", self.frame_ptr),
+ ]
+ else:
+ return [
+ ("resume", self.resume_ptr),
+ ("destroy", self.destroy_ptr),
+ ("promise", self.promise_ptr),
+ ("coro_frame", self.frame_ptr)
+ ]
+
+
+# Works for both libc++ and libstdc++.
+libcxx_corohdl_regex = re.compile('^std::__[A-Za-z0-9]+::coroutine_handle<.+>$|^std::coroutine_handle<.+>(( )?&)?$')
+
+def _extract_coro_frame_ptr_from_handle(val: gdb.Value):
+ if libcxx_corohdl_regex.match(val.type.strip_typedefs().name) is None:
+ raise ValueError("Expected a std::coroutine_handle, got %s" % val.type.strip_typedefs().name)
+
+ # We expect the coroutine handle to have a single field, which is the frame pointer.
+ # This heuristic works for both libc++ and libstdc++.
+ fields = val.type.fields()
+ if len(fields) != 1:
+ raise ValueError("Expected 1 field, got %d" % len(fields))
+ return int(val[fields[0]])
+
+
+"""
+Pretty printer for `std::coroutine_handle<T>`
+
+Works for both libc++ and libstdc++.
+
+It prints the coroutine handle as a struct with the following fields:
+- resume: the resume function pointer
+- destroy: the destroy function pointer
+- promise: the promise pointer
+- coro_frame: the coroutine frame pointer
+
+Most of the functionality is implemented in `DevirtualizedCoroFrame`.
+"""
+class CoroutineHandlePrinter(DevirtualizedCoroFrame):
+ def __init__(self, val : gdb.Value):
+ frame_ptr_raw = _extract_coro_frame_ptr_from_handle(val)
+ super(CoroutineHandlePrinter, self).__init__(frame_ptr_raw, val)
+
+
+def build_pretty_printer():
+ pp = gdb.printing.RegexpCollectionPrettyPrinter("coroutine")
+ pp.add_printer('std::coroutine_handle', libcxx_corohdl_regex, CoroutineHandlePrinter)
+ return pp
+
+gdb.printing.register_pretty_printer(
+ gdb.current_objfile(),
+ build_pretty_printer())
+
+
+"""
+Get the coroutine frame pointer from a coroutine handle.
+
+Usage:
+```
+p *get_coro_frame(coroutine_hdl)
+```
+"""
+class GetCoroFrame(gdb.Function):
+ def __init__(self):
+ super(GetCoroFrame, self).__init__("get_coro_frame")
+
+ def invoke(self, coroutine_hdl_raw):
+ return CoroutineHandlePrinter(coroutine_hdl_raw).frame_ptr
+
+GetCoroFrame()
+
+
+"""
+Get the coroutine frame pointer from a coroutine handle.
+
+Usage:
+```
+p *get_coro_promise(coroutine_hdl)
+```
+"""
+class GetCoroFrame(gdb.Function):
+ def __init__(self):
+ super(GetCoroFrame, self).__init__("get_coro_promise")
+
+ def invoke(self, coroutine_hdl_raw):
+ return CoroutineHandlePrinter(coroutine_hdl_raw).promise_ptr
+
+GetCoroFrame()
+
+
+"""
+Decorator for coroutine frames.
+
+Used by `CoroutineFrameFilter` to add the coroutine frames to the built-in `bt` command.
+"""
+class CoroutineFrameDecorator(FrameDecorator):
+ def __init__(self, coro_frame: DevirtualizedCoroFrame, inferior_frame: gdb.Frame):
+ super(CoroutineFrameDecorator, self).__init__(inferior_frame)
+ self.coro_frame = coro_frame
+
+ def function(self):
+ func_name = self.coro_frame.get_function_name()
+ if func_name is not None:
+ return "[async] " + func_name
+ return "[async] coroutine (coro_frame=" + str(self.coro_frame.frame_ptr_raw) + ")"
- def address(self):
- return None
+ def address(self):
+ return None
- def filename(self):
- if self.coro_frame.destroy_func is not None:
- return self.coro_frame.destroy_func.function.symtab.filename
- return None
+ def filename(self):
+ if self.coro_frame.destroy_func is not None:
+ return self.coro_frame.destroy_func.function.symtab.filename
+ return None
- def line(self):
- if self.coro_frame.resume_label is not None:
- return self.coro_frame.resume_label.line
- return None
+ def line(self):
+ if self.coro_frame.resume_label is not None:
+ return self.coro_frame.resume_label.line
+ return None
- def frame_args(self):
- return []
+ def frame_args(self):
+ return []
- def frame_locals(self):
- return []
+ def frame_locals(self):
+ return []
- def _get_continuation(promise: gdb.Value) -> DevirtualizedCoroFrame | None:
- try:
- # TODO: adjust this according for your coroutine framework
- return DevirtualizedCoroFrame(_extract_coro_frame_ptr_from_handle(promise["continuation"]))
- except Exception as e:
- return None
-
+def _get_continuation(promise: gdb.Value) -> DevirtualizedCoroFrame | None:
+ try:
+ # TODO: adjust this according for your coroutine framework
+ return DevirtualizedCoroFrame(_extract_coro_frame_ptr_from_handle(promise["continuation"]))
+ except Exception as e:
+ return None
+
- def _create_coroutine_frames(coro_frame: DevirtualizedCoroFrame, inferior_frame: gdb.Frame):
- while coro_frame is not None:
- yield CoroutineFrameDecorator(coro_frame, inferior_frame)
- coro_frame = _get_continuation(coro_frame.promise_ptr)
+def _create_coroutine_frames(coro_frame: DevirtualizedCoroFrame, inferior_frame: gdb.Frame):
+ while coro_frame is not None:
+ yield CoroutineFrameDecorator(coro_frame, inferior_frame)
+ coro_frame = _get_continuation(coro_frame.promise_ptr)
- """
- Frame filter to add coroutine frames to the built-in `bt` command.
- """
- class CppCoroutineFrameFilter():
- def __init__(self):
- self.name = "CppCoroutineFrameFilter"
- self.priority = 50
- self.enabled = True
- # Register this frame filter with the global frame_filters dictionary.
- gdb.frame_filters[self.name] = self
+"""
+Frame filter to add coroutine frames to the built-in `bt` command.
+"""
+class CppCoroutineFrameFilter():
+ def __init__(self):
+ self.name = "CppCoroutineFrameFilter"
+ self.priority = 50
+ self.enabled = True
+ # Register this frame filter with the global frame_filters dictionary.
+ gdb.frame_filters[self.name] = self
- def filter(self, frame_iter: typing.Iterable[gdb.FrameDecorator]):
- for frame in frame_iter:
- yield frame
- inferior_frame = frame.inferior_frame()
- try:
- promise_ptr = inferior_frame.read_var("__promise")
- except Exception:
- continue
- parent_coro = _get_continuation(promise_ptr)
- if parent_coro is not None:
- yield from _create_coroutine_frames(parent_coro, inferior_frame)
+ def filter(self, frame_iter: typing.Iterable[gdb.FrameDecorator]):
+ for frame in frame_iter:
+ yield frame
+ inferior_frame = frame.inferior_frame()
+ try:
+ promise_ptr = inferior_frame.read_var("__promise")
+ except Exception:
+ continue
+ parent_coro = _get_continuation(promise_ptr)
+ if parent_coro is not None:
+ yield from _create_coroutine_frames(parent_coro, inferior_frame)
- CppCoroutineFrameFilter()
+CppCoroutineFrameFilter()
+````
-Further Reading
----------------
+### Further Reading
The authors of the Folly libraries wrote a blog post series on how they debug coroutines:
-* `Async stack traces in folly: Introduction <https://developers.facebook.com/blog/post/2021/09/16/async-stack-traces-folly-Introduction/>`_
-* `Async stack traces in folly: Synchronous and asynchronous stack traces <https://developers.facebook.com/blog/post/2021/09/23/async-stack-traces-folly-synchronous-asynchronous-stack-traces/>`_
-* `Async stack traces in folly: Forming an async stack from individual frames <https://developers.facebook.com/blog/post/2021/09/30/async-stack-traces-folly-forming-async-stack-individual-frames/>`_
-* `Async Stack Traces for C++ Coroutines in Folly: Walking the async stack <https://developers.facebook.com/blog/post/2021/10/14/async-stack-traces-c-plus-plus-coroutines-folly-walking-async-stack/>`_
-* `Async stack traces in folly: Improving debugging in the developer lifecycle <https://developers.facebook.com/blog/post/2021/10/21/async-stack-traces-folly-improving-debugging-developer-lifecycle/>`_
+- [Async stack traces in folly: Introduction](https://developers.facebook.com/blog/post/2021/09/16/async-stack-traces-folly-Introduction/)
+- [Async stack traces in folly: Synchronous and asynchronous stack traces](https://developers.facebook.com/blog/post/2021/09/23/async-stack-traces-folly-synchronous-asynchronous-stack-traces/)
+- [Async stack traces in folly: Forming an async stack from individual frames](https://developers.facebook.com/blog/post/2021/09/30/async-stack-traces-folly-forming-async-stack-individual-frames/)
+- [Async Stack Traces for C++ Coroutines in Folly: Walking the async stack](https://developers.facebook.com/blog/post/2021/10/14/async-stack-traces-c-plus-plus-coroutines-folly-walking-async-stack/)
+- [Async stack traces in folly: Improving debugging in the developer lifecycle](https://developers.facebook.com/blog/post/2021/10/21/async-stack-traces-folly-improving-debugging-developer-lifecycle/)
Besides some topics also covered here (stack traces from the debugger), Folly's blog post series also covers
additional topics, such as capturing async stack traces in performance profiles via eBPF filters
and printing async stack traces on crashes.
+
diff --git a/clang/docs/ExternalClangExamples.md b/clang/docs/ExternalClangExamples.md
index ec95106b4697d..80e7f51cd3f89 100644
--- a/clang/docs/ExternalClangExamples.md
+++ b/clang/docs/ExternalClangExamples.md
@@ -1,9 +1,6 @@
-=======================
-External Clang Examples
-=======================
+# External Clang Examples
-Introduction
-============
+## Introduction
This page provides some examples of the kinds of things that people have
done with Clang that might serve as useful guides (or starting points) from
@@ -19,85 +16,101 @@ where Clang is used are:
- Documentation/cross-reference generation.
If you know of (or wrote!) a tool or project using Clang, please post on
-`the Discourse forums (Clang Frontend category)
-<https://discourse.llvm.org/c/clang/6>`_ to have it added.
+[the Discourse forums (Clang Frontend category)](https://discourse.llvm.org/c/clang/6) to have it added.
(or if you are already a Clang contributor, feel free to directly commit
additions). Since the primary purpose of this page is to provide examples
that can help developers, generally they must have code available.
-List of projects and tools
-==========================
-
-`<https://github.com/Andersbakken/rtags/>`_
- "RTags is a client/server application that indexes c/c++ code and keeps
- a persistent in-memory database of references, symbolnames, completions
- etc."
-
-`<https://rprichard.github.io/CxxCodeBrowser/>`_
- "A C/C++ source code indexer and navigator."
-
-`<https://github.com/etaoins/qconnectlint>`_
- "qconnectlint is a Clang tool for statically verifying the consistency
- of signal and slot connections made with Qt's ``QObject::connect``."
-
-`<https://github.com/woboq/woboq_codebrowser>`_
- "The Woboq Code Browser is a web-based code browser for C/C++ projects.
- Check out `<https://code.woboq.org/>`_ for an example!"
-
-`<https://github.com/mozilla/dxr>`_
- "DXR is a source code cross-reference tool that uses static analysis
- data collected by instrumented compilers."
-
-`<https://github.com/eschulte/clang-mutate>`_
- "This tool performs a number of operations on C-language source files."
-
-`<https://github.com/gmarpons/Crisp>`_
- "A coding rule validation add-on for LLVM/clang. Crisp rules are written
- in Prolog. A high-level declarative DSL to easily write new rules is under
- development. It will be called CRISP, an acronym for *Coding Rules in
- Sugared Prolog*."
-
-`<https://github.com/drothlis/clang-ctags>`_
- "Generate tag file for C++ source code."
-
-`<https://github.com/exclipy/clang_indexer>`_
- "This is an indexer for C and C++ based on the libclang library."
-
-`<https://github.com/holtgrewe/linty>`_
- "Linty - C/C++ Style Checking with Python & libclang."
-
-`<https://github.com/axw/cmonster>`_
- "cmonster is a Python wrapper for the Clang C++ parser."
-
-`<https://github.com/rizsotto/Constantine>`_
- "Constantine is a toy project to learn how to write clang plugin.
- Implements pseudo const analysis. Generates warnings about variables,
- which were declared without const qualifier."
-
-`<https://github.com/jessevdk/cldoc>`_
- "cldoc is a Clang based documentation generator for C and C++.
- cldoc tries to solve the issue of writing C/C++ software documentation
- with a modern, non-intrusive and robust approach."
-
-`<https://github.com/AlexDenisov/ToyClangPlugin>`_
- "The simplest Clang plugin implementing a semantic check for Objective-C.
- This example shows how to use the ``DiagnosticsEngine`` (emit warnings,
- errors, fixit hints). See also `<http://l.rw.rw/clang_plugin>`_ for
- step-by-step instructions."
-
-`<https://phabricator.kde.org/source/clazy>`_
- "clazy is a compiler plugin which allows clang to understand Qt semantics.
- You get more than 50 Qt related compiler warnings, ranging from unneeded
- memory allocations to misusage of API, including fix-its for automatic
- refactoring."
-
-`<https://gerrit.libreoffice.org/gitweb?p=core.git;a=blob_plain;f=compilerplugins/README;hb=HEAD>`_
- "LibreOffice uses a Clang plugin infrastructure to check during the build
- various things, some more, some less specific to the LibreOffice source code.
- There are currently around 50 such checkers, from flagging C-style casts and
- uses of reserved identifiers to ensuring that code adheres to lifecycle
- protocols for certain LibreOffice-specific classes. They may serve as
- examples for writing RecursiveASTVisitor-based plugins."
-
-`<https://github.com/banach-space/clang-tutor>`_
- "A collection of out-of-tree Clang plugins for teaching and learning."
+## List of projects and tools
+
+[https://github.com/Andersbakken/rtags/](https://github.com/Andersbakken/rtags/)
+
+: "RTags is a client/server application that indexes c/c++ code and keeps
+ a persistent in-memory database of references, symbolnames, completions
+ etc."
+
+[https://rprichard.github.io/CxxCodeBrowser/](https://rprichard.github.io/CxxCodeBrowser/)
+
+: "A C/C++ source code indexer and navigator."
+
+[https://github.com/etaoins/qconnectlint](https://github.com/etaoins/qconnectlint)
+
+: "qconnectlint is a Clang tool for statically verifying the consistency
+ of signal and slot connections made with Qt's `QObject::connect`."
+
+[https://github.com/woboq/woboq_codebrowser](https://github.com/woboq/woboq_codebrowser)
+
+: "The Woboq Code Browser is a web-based code browser for C/C++ projects.
+ Check out [https://code.woboq.org/](https://code.woboq.org/) for an example!"
+
+[https://github.com/mozilla/dxr](https://github.com/mozilla/dxr)
+
+: "DXR is a source code cross-reference tool that uses static analysis
+ data collected by instrumented compilers."
+
+[https://github.com/eschulte/clang-mutate](https://github.com/eschulte/clang-mutate)
+
+: "This tool performs a number of operations on C-language source files."
+
+[https://github.com/gmarpons/Crisp](https://github.com/gmarpons/Crisp)
+
+: "A coding rule validation add-on for LLVM/clang. Crisp rules are written
+ in Prolog. A high-level declarative DSL to easily write new rules is under
+ development. It will be called CRISP, an acronym for *Coding Rules in
+ Sugared Prolog*."
+
+[https://github.com/drothlis/clang-ctags](https://github.com/drothlis/clang-ctags)
+
+: "Generate tag file for C++ source code."
+
+[https://github.com/exclipy/clang_indexer](https://github.com/exclipy/clang_indexer)
+
+: "This is an indexer for C and C++ based on the libclang library."
+
+[https://github.com/holtgrewe/linty](https://github.com/holtgrewe/linty)
+
+: "Linty - C/C++ Style Checking with Python & libclang."
+
+[https://github.com/axw/cmonster](https://github.com/axw/cmonster)
+
+: "cmonster is a Python wrapper for the Clang C++ parser."
+
+[https://github.com/rizsotto/Constantine](https://github.com/rizsotto/Constantine)
+
+: "Constantine is a toy project to learn how to write clang plugin.
+ Implements pseudo const analysis. Generates warnings about variables,
+ which were declared without const qualifier."
+
+[https://github.com/jessevdk/cldoc](https://github.com/jessevdk/cldoc)
+
+: "cldoc is a Clang based documentation generator for C and C++.
+ cldoc tries to solve the issue of writing C/C++ software documentation
+ with a modern, non-intrusive and robust approach."
+
+[https://github.com/AlexDenisov/ToyClangPlugin](https://github.com/AlexDenisov/ToyClangPlugin)
+
+: "The simplest Clang plugin implementing a semantic check for Objective-C.
+ This example shows how to use the `DiagnosticsEngine` (emit warnings,
+ errors, fixit hints). See also [http://l.rw.rw/clang_plugin](http://l.rw.rw/clang_plugin) for
+ step-by-step instructions."
+
+[https://phabricator.kde.org/source/clazy](https://phabricator.kde.org/source/clazy)
+
+: "clazy is a compiler plugin which allows clang to understand Qt semantics.
+ You get more than 50 Qt related compiler warnings, ranging from unneeded
+ memory allocations to misusage of API, including fix-its for automatic
+ refactoring."
+
+[https://gerrit.libreoffice.org/gitweb?p=core.git;a=blob_plain;f=compilerplugins/README;hb=HEAD](https://gerrit.libreoffice.org/gitweb?p=core.git;a=blob_plain;f=compilerplugins/README;hb=HEAD)
+
+: "LibreOffice uses a Clang plugin infrastructure to check during the build
+ various things, some more, some less specific to the LibreOffice source code.
+ There are currently around 50 such checkers, from flagging C-style casts and
+ uses of reserved identifiers to ensuring that code adheres to lifecycle
+ protocols for certain LibreOffice-specific classes. They may serve as
+ examples for writing RecursiveASTVisitor-based plugins."
+
+[https://github.com/banach-space/clang-tutor](https://github.com/banach-space/clang-tutor)
+
+: "A collection of out-of-tree Clang plugins for teaching and learning."
+
diff --git a/clang/docs/FAQ.md b/clang/docs/FAQ.md
index 4c4f8a87e3bc7..47be96fa36e43 100644
--- a/clang/docs/FAQ.md
+++ b/clang/docs/FAQ.md
@@ -1,64 +1,59 @@
-================================
-Frequently Asked Questions (FAQ)
-================================
+# Frequently Asked Questions (FAQ)
-.. contents::
- :local:
+```{contents}
+:local: true
+```
-Driver
-======
+## Driver
-I run ``clang -cc1 ...`` and get weird errors about missing headers
--------------------------------------------------------------------
+### I run `clang -cc1 ...` and get weird errors about missing headers
Given this source file:
-.. code-block:: c
-
- #include <stdio.h>
-
- int main() {
- printf("Hello world\n");
- }
+```c
+#include <stdio.h>
+int main() {
+ printf("Hello world\n");
+}
+```
If you run:
-.. code-block:: console
-
- $ clang -cc1 hello.c
- hello.c:1:10: fatal error: 'stdio.h' file not found
- #include <stdio.h>
- ^
- 1 error generated.
-
-``clang -cc1`` is the frontend, ``clang`` is the :doc:`driver
-<DriverInternals>`. The driver invokes the frontend with options appropriate
-for your system. To see these options, run:
+```console
+$ clang -cc1 hello.c
+hello.c:1:10: fatal error: 'stdio.h' file not found
+#include <stdio.h>
+ ^
+1 error generated.
+```
-.. code-block:: console
+`clang -cc1` is the frontend, `clang` is the {doc}`driver
+<DriverInternals>`. The driver invokes the frontend with options appropriate
+for your system. To see these options, run:
- $ clang -### -c hello.c
+```console
+$ clang -### -c hello.c
+```
Some clang command line options are driver-only options, some are frontend-only
-options. Frontend-only options are intended to be used only by clang developers.
-Users should not run ``clang -cc1`` directly, because ``-cc1`` options are not
+options. Frontend-only options are intended to be used only by clang developers.
+Users should not run `clang -cc1` directly, because `-cc1` options are not
guaranteed to be stable.
-If you want to use a frontend-only option ("a ``-cc1`` option"), for example
-``-ast-dump``, then you need to take the ``clang -cc1`` line generated by the
-driver and add the option you need. Alternatively, you can run
-``clang -Xclang <option> ...`` to force the driver pass ``<option>`` to
-``clang -cc1``.
+If you want to use a frontend-only option ("a `-cc1` option"), for example
+`-ast-dump`, then you need to take the `clang -cc1` line generated by the
+driver and add the option you need. Alternatively, you can run
+`clang -Xclang <option> ...` to force the driver pass `<option>` to
+`clang -cc1`.
-I get errors about some headers being missing (``stddef.h``, ``stdarg.h``)
---------------------------------------------------------------------------
+### I get errors about some headers being missing (`stddef.h`, `stdarg.h`)
-Some header files (``stddef.h``, ``stdarg.h``, and others) are shipped with
-Clang --- these are called builtin includes. Clang searches for them in a
-directory relative to the location of the ``clang`` binary. If you moved the
-``clang`` binary, you need to move the builtin headers, too.
+Some header files (`stddef.h`, `stdarg.h`, and others) are shipped with
+Clang --- these are called builtin includes. Clang searches for them in a
+directory relative to the location of the `clang` binary. If you moved the
+`clang` binary, you need to move the builtin headers, too.
-More information can be found in the :ref:`libtooling_builtin_includes`
+More information can be found in the {ref}`libtooling_builtin_includes`
section.
diff --git a/clang/docs/IntroductionToTheClangAST.md b/clang/docs/IntroductionToTheClangAST.md
index 6fbb8f1d6440e..56431b9ee5fec 100644
--- a/clang/docs/IntroductionToTheClangAST.md
+++ b/clang/docs/IntroductionToTheClangAST.md
@@ -1,20 +1,17 @@
-=============================
-Introduction to the Clang AST
-=============================
+# Introduction to the Clang AST
This document gives a gentle introduction to the mysteries of the Clang
AST. It is targeted at developers who either want to contribute to
Clang, or use tools that work based on Clang's AST, like the AST
matchers.
-.. raw:: html
+```{raw} html
+<center><iframe width="560" height="315" src="https://www.youtube.com/embed/VqCkCDFLSsc?vq=hd720" frameborder="0" allowfullscreen></iframe></center>
+```
- <center><iframe width="560" height="315" src="https://www.youtube.com/embed/VqCkCDFLSsc?vq=hd720" frameborder="0" allowfullscreen></iframe></center>
+[Slides](https://llvm.org/devmtg/2013-04/klimek-slides.pdf)
-`Slides <https://llvm.org/devmtg/2013-04/klimek-slides.pdf>`_
-
-Introduction
-============
+## Introduction
Clang's AST is different from ASTs produced by some other compilers in
that it closely resembles both the written C++ code and the C++
@@ -23,104 +20,102 @@ constants are available in an unreduced form in the AST. This makes
Clang's AST a good fit for refactoring tools.
Documentation for all Clang AST nodes is available via the generated
-`Doxygen <https://clang.llvm.org/doxygen>`_. The doxygen online
+[Doxygen](https://clang.llvm.org/doxygen). The doxygen online
documentation is also indexed by your favorite search engine, which will
make a search for clang and the AST node's class name usually turn up
the doxygen of the class you're looking for (for example, search for:
clang ParenExpr).
-Examining the AST
-=================
+## Examining the AST
A good way to familiarize yourself with the Clang AST is to actually look
at it on some simple example code. Clang has a builtin AST-dump mode,
-which can be enabled with the flag ``-ast-dump``.
+which can be enabled with the flag `-ast-dump`.
Let's look at a simple example AST:
-::
-
- $ cat test.cc
- int f(int x) {
- int result = (x / 42);
- return result;
- }
-
- # Clang by default is a frontend for many tools; -Xclang is used to pass
- # options directly to the C++ frontend.
- $ clang -Xclang -ast-dump -fsyntax-only test.cc
- TranslationUnitDecl 0x5aea0d0 <<invalid sloc>>
- ... cutting out internal declarations of clang ...
- `-FunctionDecl 0x5aeab50 <test.cc:1:1, line:4:1> f 'int (int)'
- |-ParmVarDecl 0x5aeaa90 <line:1:7, col:11> x 'int'
- `-CompoundStmt 0x5aead88 <col:14, line:4:1>
- |-DeclStmt 0x5aead10 <line:2:3, col:24>
- | `-VarDecl 0x5aeac10 <col:3, col:23> result 'int'
- | `-ParenExpr 0x5aeacf0 <col:16, col:23> 'int'
- | `-BinaryOperator 0x5aeacc8 <col:17, col:21> 'int' '/'
- | |-ImplicitCastExpr 0x5aeacb0 <col:17> 'int' <LValueToRValue>
- | | `-DeclRefExpr 0x5aeac68 <col:17> 'int' lvalue ParmVar 0x5aeaa90 'x' 'int'
- | `-IntegerLiteral 0x5aeac90 <col:21> 'int' 42
- `-ReturnStmt 0x5aead68 <line:3:3, col:10>
- `-ImplicitCastExpr 0x5aead50 <col:10> 'int' <LValueToRValue>
- `-DeclRefExpr 0x5aead28 <col:10> 'int' lvalue Var 0x5aeac10 'result' 'int'
+```
+$ cat test.cc
+int f(int x) {
+ int result = (x / 42);
+ return result;
+}
+
+# Clang by default is a frontend for many tools; -Xclang is used to pass
+# options directly to the C++ frontend.
+$ clang -Xclang -ast-dump -fsyntax-only test.cc
+TranslationUnitDecl 0x5aea0d0 <<invalid sloc>>
+... cutting out internal declarations of clang ...
+`-FunctionDecl 0x5aeab50 <test.cc:1:1, line:4:1> f 'int (int)'
+ |-ParmVarDecl 0x5aeaa90 <line:1:7, col:11> x 'int'
+ `-CompoundStmt 0x5aead88 <col:14, line:4:1>
+ |-DeclStmt 0x5aead10 <line:2:3, col:24>
+ | `-VarDecl 0x5aeac10 <col:3, col:23> result 'int'
+ | `-ParenExpr 0x5aeacf0 <col:16, col:23> 'int'
+ | `-BinaryOperator 0x5aeacc8 <col:17, col:21> 'int' '/'
+ | |-ImplicitCastExpr 0x5aeacb0 <col:17> 'int' <LValueToRValue>
+ | | `-DeclRefExpr 0x5aeac68 <col:17> 'int' lvalue ParmVar 0x5aeaa90 'x' 'int'
+ | `-IntegerLiteral 0x5aeac90 <col:21> 'int' 42
+ `-ReturnStmt 0x5aead68 <line:3:3, col:10>
+ `-ImplicitCastExpr 0x5aead50 <col:10> 'int' <LValueToRValue>
+ `-DeclRefExpr 0x5aead28 <col:10> 'int' lvalue Var 0x5aeac10 'result' 'int'
+```
The toplevel declaration in
-a translation unit is always the `translation unit
-declaration <https://clang.llvm.org/doxygen/classclang_1_1TranslationUnitDecl.html>`_.
-In this example, our first user written declaration is the `function
-declaration <https://clang.llvm.org/doxygen/classclang_1_1FunctionDecl.html>`_
-of "``f``". The body of "``f``" is a `compound
-statement <https://clang.llvm.org/doxygen/classclang_1_1CompoundStmt.html>`_,
-whose child nodes are a `declaration
-statement <https://clang.llvm.org/doxygen/classclang_1_1DeclStmt.html>`_
-that declares our result variable, and the `return
-statement <https://clang.llvm.org/doxygen/classclang_1_1ReturnStmt.html>`_.
-
-AST Context
-===========
+a translation unit is always the [translation unit
+declaration](https://clang.llvm.org/doxygen/classclang_1_1TranslationUnitDecl.html).
+In this example, our first user written declaration is the [function
+declaration](https://clang.llvm.org/doxygen/classclang_1_1FunctionDecl.html)
+of "`f`". The body of "`f`" is a [compound
+statement](https://clang.llvm.org/doxygen/classclang_1_1CompoundStmt.html),
+whose child nodes are a [declaration
+statement](https://clang.llvm.org/doxygen/classclang_1_1DeclStmt.html)
+that declares our result variable, and the [return
+statement](https://clang.llvm.org/doxygen/classclang_1_1ReturnStmt.html).
+
+## AST Context
All information about the AST for a translation unit is bundled up in
the class
-`ASTContext <https://clang.llvm.org/doxygen/classclang_1_1ASTContext.html>`_.
+[ASTContext](https://clang.llvm.org/doxygen/classclang_1_1ASTContext.html).
It allows traversal of the whole translation unit starting from
-`getTranslationUnitDecl <https://clang.llvm.org/doxygen/classclang_1_1ASTContext.html#abd909fb01ef10cfd0244832a67b1dd64>`_,
-or to access Clang's `table of
-identifiers <https://clang.llvm.org/doxygen/classclang_1_1ASTContext.html#a4f95adb9958e22fbe55212ae6482feb4>`_
+[getTranslationUnitDecl](https://clang.llvm.org/doxygen/classclang_1_1ASTContext.html#abd909fb01ef10cfd0244832a67b1dd64),
+or to access Clang's [table of
+identifiers](https://clang.llvm.org/doxygen/classclang_1_1ASTContext.html#a4f95adb9958e22fbe55212ae6482feb4)
for the parsed translation unit.
-AST Nodes
-=========
+## AST Nodes
Clang's AST nodes are modeled on a class hierarchy that does not have a
common ancestor. Instead, there are multiple larger hierarchies for
basic node types like
-`Decl <https://clang.llvm.org/doxygen/classclang_1_1Decl.html>`_ and
-`Stmt <https://clang.llvm.org/doxygen/classclang_1_1Stmt.html>`_. Many
+[Decl](https://clang.llvm.org/doxygen/classclang_1_1Decl.html) and
+[Stmt](https://clang.llvm.org/doxygen/classclang_1_1Stmt.html). Many
important AST nodes derive from
-`Type <https://clang.llvm.org/doxygen/classclang_1_1Type.html>`_,
-`Decl <https://clang.llvm.org/doxygen/classclang_1_1Decl.html>`_,
-`DeclContext <https://clang.llvm.org/doxygen/classclang_1_1DeclContext.html>`_
-or `Stmt <https://clang.llvm.org/doxygen/classclang_1_1Stmt.html>`_, with
+[Type](https://clang.llvm.org/doxygen/classclang_1_1Type.html),
+[Decl](https://clang.llvm.org/doxygen/classclang_1_1Decl.html),
+[DeclContext](https://clang.llvm.org/doxygen/classclang_1_1DeclContext.html)
+or [Stmt](https://clang.llvm.org/doxygen/classclang_1_1Stmt.html), with
some classes deriving from both Decl and DeclContext.
There are also a multitude of nodes in the AST that are not part of a
larger hierarchy, and are only reachable from specific other nodes, like
-`CXXBaseSpecifier <https://clang.llvm.org/doxygen/classclang_1_1CXXBaseSpecifier.html>`_.
+[CXXBaseSpecifier](https://clang.llvm.org/doxygen/classclang_1_1CXXBaseSpecifier.html).
Thus, to traverse the full AST, one starts from the
-`TranslationUnitDecl <https://clang.llvm.org/doxygen/classclang_1_1TranslationUnitDecl.html>`_
+[TranslationUnitDecl](https://clang.llvm.org/doxygen/classclang_1_1TranslationUnitDecl.html)
and then recursively traverses everything that can be reached from that
node - this information has to be encoded for each specific node type.
This algorithm is encoded in the
-`RecursiveASTVisitor <https://clang.llvm.org/doxygen/classclang_1_1RecursiveASTVisitor.html>`_.
-See the `RecursiveASTVisitor
-tutorial <https://clang.llvm.org/docs/RAVFrontendAction.html>`_.
+[RecursiveASTVisitor](https://clang.llvm.org/doxygen/classclang_1_1RecursiveASTVisitor.html).
+See the [RecursiveASTVisitor
+tutorial](https://clang.llvm.org/docs/RAVFrontendAction.html).
The two most basic nodes in the Clang AST are statements
-(`Stmt <https://clang.llvm.org/doxygen/classclang_1_1Stmt.html>`_) and
+([Stmt](https://clang.llvm.org/doxygen/classclang_1_1Stmt.html)) and
declarations
-(`Decl <https://clang.llvm.org/doxygen/classclang_1_1Decl.html>`_). Note
+([Decl](https://clang.llvm.org/doxygen/classclang_1_1Decl.html)). Note
that expressions
-(`Expr <https://clang.llvm.org/doxygen/classclang_1_1Expr.html>`_) are
+([Expr](https://clang.llvm.org/doxygen/classclang_1_1Expr.html)) are
also statements in Clang's AST.
+
diff --git a/clang/docs/LibClang.md b/clang/docs/LibClang.md
index 6c62bcb5f8c29..f8278a678a768 100644
--- a/clang/docs/LibClang.md
+++ b/clang/docs/LibClang.md
@@ -1,401 +1,388 @@
+```{eval-rst}
.. role:: raw-html(raw)
:format: html
+```
-Libclang tutorial
-=================
-The C Interface to Clang provides a relatively small API that exposes facilities for parsing source code into an abstract syntax tree (AST), loading already-parsed ASTs, traversing the AST, associating physical source locations with elements within the AST, and other facilities that support Clang-based development tools.
-This C interface to Clang will never provide all of the information representation stored in Clang's C++ AST, nor should it: the intent is to maintain an API that is :ref:`relatively stable <Stability>` from one release to the next, providing only the basic functionality needed to support development tools.
-The entire C interface of libclang is available in the file `Index.h`_
-
-Essential types overview
--------------------------
-
-All types of libclang are prefixed with ``CX``
+# Libclang tutorial
-CXIndex
-~~~~~~~
-An Index that consists of a set of translation units that would typically be linked together into an executable or library.
-
-CXTranslationUnit
-~~~~~~~~~~~~~~~~~
-A single translation unit, which resides in an index.
-
-CXCursor
-~~~~~~~~
-A cursor representing a pointer to some element in the abstract syntax tree of a translation unit.
+The C Interface to Clang provides a relatively small API that exposes facilities for parsing source code into an abstract syntax tree (AST), loading already-parsed ASTs, traversing the AST, associating physical source locations with elements within the AST, and other facilities that support Clang-based development tools.
+This C interface to Clang will never provide all of the information representation stored in Clang's C++ AST, nor should it: the intent is to maintain an API that is {ref}`relatively stable <Stability>` from one release to the next, providing only the basic functionality needed to support development tools.
+The entire C interface of libclang is available in the file [Index.h]
+## Essential types overview
-Code example
-""""""""""""
+All types of libclang are prefixed with `CX`
-.. code-block:: cpp
+### CXIndex
- // file.cpp
- struct foo{
- int bar;
- int* bar_pointer;
- };
+An Index that consists of a set of translation units that would typically be linked together into an executable or library.
-.. code-block:: cpp
+### CXTranslationUnit
- // main.cpp
- #include <clang-c/Index.h>
- #include <iostream>
+A single translation unit, which resides in an index.
- int main(){
- CXIndex index = clang_createIndex(0, 0); //Create index
- CXTranslationUnit unit = clang_parseTranslationUnit(
- index,
- "file.cpp", nullptr, 0,
- nullptr, 0,
- CXTranslationUnit_None); //Parse "file.cpp"
+### CXCursor
+A cursor representing a pointer to some element in the abstract syntax tree of a translation unit.
- if (unit == nullptr){
- std::cerr << "Unable to parse translation unit. Quitting.\n";
- return 0;
- }
- CXCursor cursor = clang_getTranslationUnitCursor(unit); //Obtain a cursor at the root of the translation unit
+#### Code example
+
+```cpp
+// file.cpp
+struct foo{
+ int bar;
+ int* bar_pointer;
+};
+```
+
+```cpp
+// main.cpp
+#include <clang-c/Index.h>
+#include <iostream>
+
+int main(){
+ CXIndex index = clang_createIndex(0, 0); //Create index
+ CXTranslationUnit unit = clang_parseTranslationUnit(
+ index,
+ "file.cpp", nullptr, 0,
+ nullptr, 0,
+ CXTranslationUnit_None); //Parse "file.cpp"
+
+
+ if (unit == nullptr){
+ std::cerr << "Unable to parse translation unit. Quitting.\n";
+ return 0;
}
+ CXCursor cursor = clang_getTranslationUnitCursor(unit); //Obtain a cursor at the root of the translation unit
+}
+```
-.. code-block:: cmake
-
- # CMakeLists.txt
- cmake_minimum_required(VERSION 3.20)
- project(my_clang_tool VERSION 0.1.0)
-
- # This will find the default system installation of Clang; if you want to
- # use a different build of clang, pass -DClang_DIR=/foobar/lib/cmake/clang
- # to the CMake configure command, where /foobar is the build directory where
- # you built Clang.
- find_package(Clang CONFIG REQUIRED)
-
- add_executable(my_clang_tool main.cpp)
- target_include_directories(my_clang_tool PRIVATE ${CLANG_INCLUDE_DIRS})
- target_link_libraries(my_clang_tool PRIVATE libclang)
+```cmake
+# CMakeLists.txt
+cmake_minimum_required(VERSION 3.20)
+project(my_clang_tool VERSION 0.1.0)
-Visiting elements of an AST
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The elements of an AST can be recursively visited with pre-order traversal with ``clang_visitChildren``.
+# This will find the default system installation of Clang; if you want to
+# use a different build of clang, pass -DClang_DIR=/foobar/lib/cmake/clang
+# to the CMake configure command, where /foobar is the build directory where
+# you built Clang.
+find_package(Clang CONFIG REQUIRED)
-.. code-block:: cpp
+add_executable(my_clang_tool main.cpp)
+target_include_directories(my_clang_tool PRIVATE ${CLANG_INCLUDE_DIRS})
+target_link_libraries(my_clang_tool PRIVATE libclang)
+```
- clang_visitChildren(
- cursor, //Root cursor
- [](CXCursor current_cursor, CXCursor parent, CXClientData client_data){
+### Visiting elements of an AST
- CXString current_display_name = clang_getCursorDisplayName(current_cursor);
- //Allocate a CXString representing the name of the current cursor
+The elements of an AST can be recursively visited with pre-order traversal with `clang_visitChildren`.
- std::cout << "Visiting element " << clang_getCString(current_display_name) << "\n";
- //Print the char* value of current_display_name
+```cpp
+clang_visitChildren(
+ cursor, //Root cursor
+ [](CXCursor current_cursor, CXCursor parent, CXClientData client_data){
- clang_disposeString(current_display_name);
- //Since clang_getCursorDisplayName allocates a new CXString, it must be freed. This applies
- //to all functions returning a CXString
+ CXString current_display_name = clang_getCursorDisplayName(current_cursor);
+ //Allocate a CXString representing the name of the current cursor
- return CXChildVisit_Recurse;
+ std::cout << "Visiting element " << clang_getCString(current_display_name) << "\n";
+ //Print the char* value of current_display_name
+ clang_disposeString(current_display_name);
+ //Since clang_getCursorDisplayName allocates a new CXString, it must be freed. This applies
+ //to all functions returning a CXString
- }, //CXCursorVisitor: a function pointer
- nullptr //client_data
- );
+ return CXChildVisit_Recurse;
-The return value of ``CXCursorVisitor``, the callable argument of ``clang_visitChildren``, can return one of the three:
-#. ``CXChildVisit_Break``: Terminates the cursor traversal
+ }, //CXCursorVisitor: a function pointer
+ nullptr //client_data
+ );
+```
-#. ``CXChildVisit_Continue``: Continues the cursor traversal with the next sibling of the cursor just visited, without visiting its children.
+The return value of `CXCursorVisitor`, the callable argument of `clang_visitChildren`, can return one of the three:
-#. ``CXChildVisit_Recurse``: Recursively traverse the children of this cursor, using the same visitor and client data
+1. `CXChildVisit_Break`: Terminates the cursor traversal
+2. `CXChildVisit_Continue`: Continues the cursor traversal with the next sibling of the cursor just visited, without visiting its children.
+3. `CXChildVisit_Recurse`: Recursively traverse the children of this cursor, using the same visitor and client data
The expected output of that program is
-.. code-block::
-
- Visiting element foo
- Visiting element bar
- Visiting element bar_pointer
-
-
-Extracting information from a Cursor
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-.. The following functions take a ``CXCursor`` as an argument and return associated information.
+```
+Visiting element foo
+Visiting element bar
+Visiting element bar_pointer
+```
+### Extracting information from a Cursor
+% The following functions take a ``CXCursor`` as an argument and return associated information.
-Extracting the Cursor kind
-""""""""""""""""""""""""""
+#### Extracting the Cursor kind
-``CXCursorKind clang_getCursorKind(CXCursor)`` Describes the kind of entity that a cursor refers to. Example values:
+`CXCursorKind clang_getCursorKind(CXCursor)` Describes the kind of entity that a cursor refers to. Example values:
-- ``CXCursor_StructDecl``: A C or C++ struct.
-- ``CXCursor_FieldDecl``: A field in a struct, union, or C++ class.
-- ``CXCursor_CallExpr``: An expression that calls a function.
+- `CXCursor_StructDecl`: A C or C++ struct.
+- `CXCursor_FieldDecl`: A field in a struct, union, or C++ class.
+- `CXCursor_CallExpr`: An expression that calls a function.
+#### Extracting the Cursor type
-Extracting the Cursor type
-""""""""""""""""""""""""""
-``CXType clang_getCursorType(CXCursor)``: Retrieve the type of a CXCursor (if any).
+`CXType clang_getCursorType(CXCursor)`: Retrieve the type of a CXCursor (if any).
-A ``CXType`` represents a complete C++ type, including qualifiers and pointers. It has a member field ``CXTypeKind kind`` and additional opaque data.
+A `CXType` represents a complete C++ type, including qualifiers and pointers. It has a member field `CXTypeKind kind` and additional opaque data.
-Example values for ``CXTypeKind kind``
+Example values for `CXTypeKind kind`
-- ``CXType_Invalid``: Represents an invalid type (e.g., where no type is available)
-- ``CXType_Pointer``: A pointer to another type
-- ``CXType_Int``: Regular ``int``
-- ``CXType_Elaborated``: Represents a type that was referred to using an elaborated type keyword e.g. struct S, or via a qualified name, e.g., N::M::type, or both.
+- `CXType_Invalid`: Represents an invalid type (e.g., where no type is available)
+- `CXType_Pointer`: A pointer to another type
+- `CXType_Int`: Regular `int`
+- `CXType_Elaborated`: Represents a type that was referred to using an elaborated type keyword e.g. struct S, or via a qualified name, e.g., N::M::type, or both.
-Any ``CXTypeKind`` can be converted to a ``CXString`` using ``clang_getTypeKindSpelling(CXTypeKind)``.
+Any `CXTypeKind` can be converted to a `CXString` using `clang_getTypeKindSpelling(CXTypeKind)`.
-A ``CXType`` holds additional necessary opaque type info, such as:
+A `CXType` holds additional necessary opaque type info, such as:
- Which struct was referred to?
- What type is the pointer pointing to?
-- Qualifiers (e.g. ``const``, ``volatile``)?
-
-Qualifiers of a ``CXType`` can be queried with:
-
-- ``clang_isConstQualifiedType(CXType)`` to check for ``const``
-- ``clang_isRestrictQualifiedType(CXType)`` to check for ``restrict``
-- ``clang_isVolatileQualifiedType(CXType)`` to check for ``volatile``
-
-Code example
-""""""""""""
-.. code-block:: cpp
-
- //structs.cpp
- struct A{
- int value;
- };
- struct B{
- int value;
- A struct_value;
- };
-
-.. code-block:: cpp
-
- #include <clang-c/Index.h>
- #include <iostream>
-
- int main(){
- CXIndex index = clang_createIndex(0, 0); //Create index
- CXTranslationUnit unit = clang_parseTranslationUnit(
- index,
- "structs.cpp", nullptr, 0,
- nullptr, 0,
- CXTranslationUnit_None); //Parse "structs.cpp"
-
- if (unit == nullptr){
- std::cerr << "Unable to parse translation unit. Quitting.\n";
- return 0;
+- Qualifiers (e.g. `const`, `volatile`)?
+
+Qualifiers of a `CXType` can be queried with:
+
+- `clang_isConstQualifiedType(CXType)` to check for `const`
+- `clang_isRestrictQualifiedType(CXType)` to check for `restrict`
+- `clang_isVolatileQualifiedType(CXType)` to check for `volatile`
+
+#### Code example
+
+```cpp
+//structs.cpp
+struct A{
+ int value;
+};
+struct B{
+ int value;
+ A struct_value;
+};
+```
+
+```cpp
+#include <clang-c/Index.h>
+#include <iostream>
+
+int main(){
+ CXIndex index = clang_createIndex(0, 0); //Create index
+ CXTranslationUnit unit = clang_parseTranslationUnit(
+ index,
+ "structs.cpp", nullptr, 0,
+ nullptr, 0,
+ CXTranslationUnit_None); //Parse "structs.cpp"
+
+ if (unit == nullptr){
+ std::cerr << "Unable to parse translation unit. Quitting.\n";
+ return 0;
+ }
+ CXCursor cursor = clang_getTranslationUnitCursor(unit); //Obtain a cursor at the root of the translation unit
+
+ clang_visitChildren(
+ cursor,
+ [](CXCursor current_cursor, CXCursor parent, CXClientData client_data){
+ CXType cursor_type = clang_getCursorType(current_cursor);
+
+ CXString type_kind_spelling = clang_getTypeKindSpelling(cursor_type.kind);
+ std::cout << "Type Kind: " << clang_getCString(type_kind_spelling);
+ clang_disposeString(type_kind_spelling);
+
+ if(cursor_type.kind == CXType_Pointer || // If cursor_type is a pointer
+ cursor_type.kind == CXType_LValueReference || // or an LValue Reference (&)
+ cursor_type.kind == CXType_RValueReference){ // or an RValue Reference (&&),
+ CXType pointed_to_type = clang_getPointeeType(cursor_type);// retrieve the pointed-to type
+
+ CXString pointed_to_type_spelling = clang_getTypeSpelling(pointed_to_type); // Spell out the entire
+ std::cout << "pointing to type: " << clang_getCString(pointed_to_type_spelling);// pointed-to type
+ clang_disposeString(pointed_to_type_spelling);
}
- CXCursor cursor = clang_getTranslationUnitCursor(unit); //Obtain a cursor at the root of the translation unit
-
- clang_visitChildren(
- cursor,
- [](CXCursor current_cursor, CXCursor parent, CXClientData client_data){
- CXType cursor_type = clang_getCursorType(current_cursor);
-
- CXString type_kind_spelling = clang_getTypeKindSpelling(cursor_type.kind);
- std::cout << "Type Kind: " << clang_getCString(type_kind_spelling);
- clang_disposeString(type_kind_spelling);
-
- if(cursor_type.kind == CXType_Pointer || // If cursor_type is a pointer
- cursor_type.kind == CXType_LValueReference || // or an LValue Reference (&)
- cursor_type.kind == CXType_RValueReference){ // or an RValue Reference (&&),
- CXType pointed_to_type = clang_getPointeeType(cursor_type);// retrieve the pointed-to type
-
- CXString pointed_to_type_spelling = clang_getTypeSpelling(pointed_to_type); // Spell out the entire
- std::cout << "pointing to type: " << clang_getCString(pointed_to_type_spelling);// pointed-to type
- clang_disposeString(pointed_to_type_spelling);
- }
- else if(cursor_type.kind == CXType_Record){
- CXString type_spelling = clang_getTypeSpelling(cursor_type);
- std::cout << ", namely " << clang_getCString(type_spelling);
- clang_disposeString(type_spelling);
- }
- std::cout << "\n";
- return CXChildVisit_Recurse;
- },
- nullptr
- );
+ else if(cursor_type.kind == CXType_Record){
+ CXString type_spelling = clang_getTypeSpelling(cursor_type);
+ std::cout << ", namely " << clang_getCString(type_spelling);
+ clang_disposeString(type_spelling);
+ }
+ std::cout << "\n";
+ return CXChildVisit_Recurse;
+ },
+ nullptr
+ );
+```
The expected output of program is:
-.. code-block::
+```
+Type Kind: Record, namely A
+Type Kind: Int
+Type Kind: Record, namely B
+Type Kind: Int
+Type Kind: Record, namely A
+Type Kind: Record, namely A
+```
- Type Kind: Record, namely A
- Type Kind: Int
- Type Kind: Record, namely B
- Type Kind: Int
- Type Kind: Record, namely A
- Type Kind: Record, namely A
+Reiterating the difference between `CXType` and `CXTypeKind`: For an example
+```cpp
+const char* __restrict__ variable;
+```
-Reiterating the difference between ``CXType`` and ``CXTypeKind``: For an example
+- Type Kind will be: `CXType_Pointer` spelled `"Pointer"`
+- Type will be a complex `CXType` structure, spelled `"const char* __restrict__`
-.. code-block:: cpp
+#### Retrieving source locations
- const char* __restrict__ variable;
+`CXSourceRange clang_getCursorExtent(CXCursor)` returns a `CXSourceRange`, representing a half-open range in the source code.
-- Type Kind will be: ``CXType_Pointer`` spelled ``"Pointer"``
-- Type will be a complex ``CXType`` structure, spelled ``"const char* __restrict__``
+Use `clang_getRangeStart(CXSourceRange)` and `clang_getRangeEnd(CXSourceRange)` to retrieve the starting and end `CXSourceLocation` from a source range, respectively.
-Retrieving source locations
-"""""""""""""""""""""""""""
+Given a `CXSourceLocation`, use `clang_getExpansionLocation` to retrieve file, line and column of a source location.
-``CXSourceRange clang_getCursorExtent(CXCursor)`` returns a ``CXSourceRange``, representing a half-open range in the source code.
+#### Code example
-Use ``clang_getRangeStart(CXSourceRange)`` and ``clang_getRangeEnd(CXSourceRange)`` to retrieve the starting and end ``CXSourceLocation`` from a source range, respectively.
+```cpp
+// Again, file.cpp
+struct foo{
+ int bar;
+ int* bar_pointer;
+};
+```
-Given a ``CXSourceLocation``, use ``clang_getExpansionLocation`` to retrieve file, line and column of a source location.
-
-Code example
-""""""""""""
-.. code-block:: cpp
-
- // Again, file.cpp
- struct foo{
- int bar;
- int* bar_pointer;
- };
-.. code-block:: cpp
-
- clang_visitChildren(
- cursor,
- [](CXCursor current_cursor, CXCursor parent, CXClientData client_data){
-
- CXType cursor_type = clang_getCursorType(current_cursor);
- CXString cursor_spelling = clang_getCursorSpelling(current_cursor);
- CXSourceRange cursor_range = clang_getCursorExtent(current_cursor);
- std::cout << "Cursor " << clang_getCString(cursor_spelling);
-
- CXFile file;
- unsigned start_line, start_column, start_offset;
- unsigned end_line, end_column, end_offset;
-
- clang_getExpansionLocation(clang_getRangeStart(cursor_range), &file, &start_line, &start_column, &start_offset);
- clang_getExpansionLocation(clang_getRangeEnd (cursor_range), &file, &end_line , &end_column , &end_offset);
- std::cout << " spanning lines " << start_line << " to " << end_line;
- clang_disposeString(cursor_spelling);
-
- std::cout << "\n";
- return CXChildVisit_Recurse;
- },
- nullptr
- );
+```cpp
+clang_visitChildren(
+ cursor,
+ [](CXCursor current_cursor, CXCursor parent, CXClientData client_data){
-The expected output of this program is:
+ CXType cursor_type = clang_getCursorType(current_cursor);
+ CXString cursor_spelling = clang_getCursorSpelling(current_cursor);
+ CXSourceRange cursor_range = clang_getCursorExtent(current_cursor);
+ std::cout << "Cursor " << clang_getCString(cursor_spelling);
-.. code-block::
+ CXFile file;
+ unsigned start_line, start_column, start_offset;
+ unsigned end_line, end_column, end_offset;
- Cursor foo spanning lines 2 to 5
- Cursor bar spanning lines 3 to 3
- Cursor bar_pointer spanning lines 4 to 4
+ clang_getExpansionLocation(clang_getRangeStart(cursor_range), &file, &start_line, &start_column, &start_offset);
+ clang_getExpansionLocation(clang_getRangeEnd (cursor_range), &file, &end_line , &end_column , &end_offset);
+ std::cout << " spanning lines " << start_line << " to " << end_line;
+ clang_disposeString(cursor_spelling);
-Complete example code
-~~~~~~~~~~~~~~~~~~~~~
+ std::cout << "\n";
+ return CXChildVisit_Recurse;
+ },
+ nullptr
+);
+```
-.. code-block:: cpp
+The expected output of this program is:
- // main.cpp
- #include <clang-c/Index.h>
- #include <iostream>
+```
+Cursor foo spanning lines 2 to 5
+Cursor bar spanning lines 3 to 3
+Cursor bar_pointer spanning lines 4 to 4
+```
+
+### Complete example code
+
+```cpp
+// main.cpp
+#include <clang-c/Index.h>
+#include <iostream>
+
+int main(){
+ CXIndex index = clang_createIndex(0, 0); //Create index
+ CXTranslationUnit unit = clang_parseTranslationUnit(
+ index,
+ "file.cpp", nullptr, 0,
+ nullptr, 0,
+ CXTranslationUnit_None); //Parse "file.cpp"
+
+ if (unit == nullptr){
+ std::cerr << "Unable to parse translation unit. Quitting.\n";
+ return 0;
+ }
+ CXCursor cursor = clang_getTranslationUnitCursor(unit); //Obtain a cursor at the root of the translation unit
- int main(){
- CXIndex index = clang_createIndex(0, 0); //Create index
- CXTranslationUnit unit = clang_parseTranslationUnit(
- index,
- "file.cpp", nullptr, 0,
- nullptr, 0,
- CXTranslationUnit_None); //Parse "file.cpp"
- if (unit == nullptr){
- std::cerr << "Unable to parse translation unit. Quitting.\n";
- return 0;
+ clang_visitChildren(
+ cursor,
+ [](CXCursor current_cursor, CXCursor parent, CXClientData client_data){
+ CXType cursor_type = clang_getCursorType(current_cursor);
+
+ CXString type_kind_spelling = clang_getTypeKindSpelling(cursor_type.kind);
+ std::cout << "TypeKind: " << clang_getCString(type_kind_spelling);
+ clang_disposeString(type_kind_spelling);
+
+ if(cursor_type.kind == CXType_Pointer || // If cursor_type is a pointer
+ cursor_type.kind == CXType_LValueReference || // or an LValue Reference (&)
+ cursor_type.kind == CXType_RValueReference){ // or an RValue Reference (&&),
+ CXType pointed_to_type = clang_getPointeeType(cursor_type);// retrieve the pointed-to type
+
+ CXString pointed_to_type_spelling = clang_getTypeSpelling(pointed_to_type); // Spell out the entire
+ std::cout << "pointing to type: " << clang_getCString(pointed_to_type_spelling);// pointed-to type
+ clang_disposeString(pointed_to_type_spelling);
}
- CXCursor cursor = clang_getTranslationUnitCursor(unit); //Obtain a cursor at the root of the translation unit
-
-
- clang_visitChildren(
- cursor,
- [](CXCursor current_cursor, CXCursor parent, CXClientData client_data){
- CXType cursor_type = clang_getCursorType(current_cursor);
-
- CXString type_kind_spelling = clang_getTypeKindSpelling(cursor_type.kind);
- std::cout << "TypeKind: " << clang_getCString(type_kind_spelling);
- clang_disposeString(type_kind_spelling);
-
- if(cursor_type.kind == CXType_Pointer || // If cursor_type is a pointer
- cursor_type.kind == CXType_LValueReference || // or an LValue Reference (&)
- cursor_type.kind == CXType_RValueReference){ // or an RValue Reference (&&),
- CXType pointed_to_type = clang_getPointeeType(cursor_type);// retrieve the pointed-to type
-
- CXString pointed_to_type_spelling = clang_getTypeSpelling(pointed_to_type); // Spell out the entire
- std::cout << "pointing to type: " << clang_getCString(pointed_to_type_spelling);// pointed-to type
- clang_disposeString(pointed_to_type_spelling);
- }
- else if(cursor_type.kind == CXType_Record){
- CXString type_spelling = clang_getTypeSpelling(cursor_type);
- std::cout << ", namely " << clang_getCString(type_spelling);
- clang_disposeString(type_spelling);
- }
- std::cout << "\n";
- return CXChildVisit_Recurse;
- },
- nullptr
- );
-
-
- clang_visitChildren(
- cursor,
- [](CXCursor current_cursor, CXCursor parent, CXClientData client_data){
-
- CXType cursor_type = clang_getCursorType(current_cursor);
- CXString cursor_spelling = clang_getCursorSpelling(current_cursor);
- CXSourceRange cursor_range = clang_getCursorExtent(current_cursor);
- std::cout << "Cursor " << clang_getCString(cursor_spelling);
-
- CXFile file;
- unsigned start_line, start_column, start_offset;
- unsigned end_line, end_column, end_offset;
-
- clang_getExpansionLocation(clang_getRangeStart(cursor_range), &file, &start_line, &start_column, &start_offset);
- clang_getExpansionLocation(clang_getRangeEnd (cursor_range), &file, &end_line , &end_column , &end_offset);
- std::cout << " spanning lines " << start_line << " to " << end_line;
- clang_disposeString(cursor_spelling);
-
- std::cout << "\n";
- return CXChildVisit_Recurse;
- },
- nullptr
- );
- }
+ else if(cursor_type.kind == CXType_Record){
+ CXString type_spelling = clang_getTypeSpelling(cursor_type);
+ std::cout << ", namely " << clang_getCString(type_spelling);
+ clang_disposeString(type_spelling);
+ }
+ std::cout << "\n";
+ return CXChildVisit_Recurse;
+ },
+ nullptr
+ );
-.. code-block:: cmake
- # CMakeLists.txt
- cmake_minimum_required(VERSION 3.20)
- project(my_clang_tool VERSION 0.1.0)
+ clang_visitChildren(
+ cursor,
+ [](CXCursor current_cursor, CXCursor parent, CXClientData client_data){
+
+ CXType cursor_type = clang_getCursorType(current_cursor);
+ CXString cursor_spelling = clang_getCursorSpelling(current_cursor);
+ CXSourceRange cursor_range = clang_getCursorExtent(current_cursor);
+ std::cout << "Cursor " << clang_getCString(cursor_spelling);
+
+ CXFile file;
+ unsigned start_line, start_column, start_offset;
+ unsigned end_line, end_column, end_offset;
+
+ clang_getExpansionLocation(clang_getRangeStart(cursor_range), &file, &start_line, &start_column, &start_offset);
+ clang_getExpansionLocation(clang_getRangeEnd (cursor_range), &file, &end_line , &end_column , &end_offset);
+ std::cout << " spanning lines " << start_line << " to " << end_line;
+ clang_disposeString(cursor_spelling);
+
+ std::cout << "\n";
+ return CXChildVisit_Recurse;
+ },
+ nullptr
+ );
+}
+```
- # This will find the default system installation of Clang; if you want to
- # use a different build of clang, pass -DClang_DIR=/foobar/lib/cmake/clang
- # to the CMake configure command, where /foobar is the build directory where
- # you built Clang.
- find_package(Clang CONFIG REQUIRED)
+```cmake
+# CMakeLists.txt
+cmake_minimum_required(VERSION 3.20)
+project(my_clang_tool VERSION 0.1.0)
- add_executable(my_clang_tool main.cpp)
- target_include_directories(my_clang_tool PRIVATE ${CLANG_INCLUDE_DIRS})
- target_link_libraries(my_clang_tool PRIVATE libclang)
+# This will find the default system installation of Clang; if you want to
+# use a different build of clang, pass -DClang_DIR=/foobar/lib/cmake/clang
+# to the CMake configure command, where /foobar is the build directory where
+# you built Clang.
+find_package(Clang CONFIG REQUIRED)
-.. _Index.h: https://github.com/llvm/llvm-project/blob/main/clang/include/clang-c/Index.h
+add_executable(my_clang_tool main.cpp)
+target_include_directories(my_clang_tool PRIVATE ${CLANG_INCLUDE_DIRS})
+target_link_libraries(my_clang_tool PRIVATE libclang)
+```
-.. _Stability:
+(stability)=
-ABI and API Stability
----------------------
+## ABI and API Stability
The C interfaces in libclang are intended to be relatively stable. This allows
a programmer to use libclang without having to worry as much about Clang
@@ -408,38 +395,40 @@ fixed, features get implemented, etc.
The library should be ABI and API stable over time, but ABI- and API-breaking
changes can happen in the following (non-exhaustive) situations:
-* Adding new enumerator to an enumeration (can be ABI-breaking in C++).
-* Removing an explicitly deprecated API after a suitably long deprecation
+- Adding new enumerator to an enumeration (can be ABI-breaking in C++).
+- Removing an explicitly deprecated API after a suitably long deprecation
period.
-* Using implementation details, such as names or comments that say something
+- Using implementation details, such as names or comments that say something
is "private", "reserved", "internal", etc.
-* Bug fixes and changes to Clang's internal implementation happen routinely and
+- Bug fixes and changes to Clang's internal implementation happen routinely and
will change the behavior of callers.
-* Rarely, bug fixes to libclang itself.
+- Rarely, bug fixes to libclang itself.
-The library has version macros (``CINDEX_VERSION_MAJOR``,
-``CINDEX_VERSION_MINOR``, and ``CINDEX_VERSION``) which can be used to test for
-specific library versions at compile time. The ``CINDEX_VERSION_MAJOR`` macro
+The library has version macros (`CINDEX_VERSION_MAJOR`,
+`CINDEX_VERSION_MINOR`, and `CINDEX_VERSION`) which can be used to test for
+specific library versions at compile time. The `CINDEX_VERSION_MAJOR` macro
is only incremented if there are major source- or ABI-breaking changes. Except
for removing an explicitly deprecated API, the changes listed above are not
considered major source- or ABI-breaking changes. Historically, the value this
macro expands to has not changed, but may be incremented in the future should
-the need arise. The ``CINDEX_VERSION_MINOR`` macro is incremented as new APIs
-are added. The ``CINDEX_VERSION`` macro expands to a value based on the major
+the need arise. The `CINDEX_VERSION_MINOR` macro is incremented as new APIs
+are added. The `CINDEX_VERSION` macro expands to a value based on the major
and minor version macros.
In an effort to allow the library to be modified as new needs arise, the
following situations are explicitly unsupported:
-* Loading different library versions into the same executable and passing
+- Loading different library versions into the same executable and passing
objects between the libraries; despite general ABI stability, different
versions of the library may use different implementation details that are not
compatible across library versions.
-* For the same reason as above, serializing objects from one version of the
+- For the same reason as above, serializing objects from one version of the
library and deserializing with a different version is also not supported.
Note: because libclang is a wrapper around the compiler frontend, it is not a
-`security-sensitive component`_ of the LLVM Project. Consider using a sandbox
+[security-sensitive component] of the LLVM Project. Consider using a sandbox
or some other mitigation approach if processing untrusted input.
-.. _security-sensitive component: https://llvm.org/docs/Security.html#what-is-considered-a-security-issue
+[index.h]: https://github.com/llvm/llvm-project/blob/main/clang/include/clang-c/Index.h
+[security-sensitive component]: https://llvm.org/docs/Security.html#what-is-considered-a-security-issue
+
diff --git a/clang/docs/LibFormat.md b/clang/docs/LibFormat.md
index 9450073b4841c..f1a468ecf8814 100644
--- a/clang/docs/LibFormat.md
+++ b/clang/docs/LibFormat.md
@@ -1,78 +1,72 @@
-=========
-LibFormat
-=========
+# LibFormat
LibFormat is a library that implements automatic source code formatting based
on Clang. This document describes the LibFormat interface and design as well
as some basic style discussions.
If you just want to use `clang-format` as a tool or integrated into an editor,
-checkout :doc:`ClangFormat`.
+checkout {doc}`ClangFormat`.
-Design
-------
+## Design
FIXME: Write up design.
+## Interface
-Interface
----------
+The core routine of LibFormat is `reformat()`:
-The core routine of LibFormat is ``reformat()``:
+```c++
+tooling::Replacements reformat(const FormatStyle &Style, Lexer &Lex,
+ SourceManager &SourceMgr,
+ std::vector<CharSourceRange> Ranges);
+```
-.. code-block:: c++
+This reads a token stream out of the lexer `Lex` and reformats all the code
+ranges in `Ranges`. The `FormatStyle` controls basic decisions made during
+formatting. A list of options can be found under {ref}`style-options`.
- tooling::Replacements reformat(const FormatStyle &Style, Lexer &Lex,
- SourceManager &SourceMgr,
- std::vector<CharSourceRange> Ranges);
+The style options are described in {doc}`ClangFormatStyleOptions`.
-This reads a token stream out of the lexer ``Lex`` and reformats all the code
-ranges in ``Ranges``. The ``FormatStyle`` controls basic decisions made during
-formatting. A list of options can be found under :ref:`style-options`.
+(style-options)=
-The style options are described in :doc:`ClangFormatStyleOptions`.
-
-
-.. _style-options:
-
-Style Options
--------------
+## Style Options
The style options describe specific formatting options that can be used in
order to make `ClangFormat` comply with different style guides. Currently,
several style guides are hard-coded:
-.. code-block:: c++
+```c++
+/// Returns a format style complying with the LLVM coding standards:
+/// https://llvm.org/docs/CodingStandards.html.
+FormatStyle getLLVMStyle();
- /// Returns a format style complying with the LLVM coding standards:
- /// https://llvm.org/docs/CodingStandards.html.
- FormatStyle getLLVMStyle();
+/// Returns a format style complying with Google's C++ style guide:
+/// http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml.
+FormatStyle getGoogleStyle();
- /// Returns a format style complying with Google's C++ style guide:
- /// http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml.
- FormatStyle getGoogleStyle();
+/// Returns a format style complying with Chromium's style guide:
+/// https://chromium.googlesource.com/chromium/src/+/refs/heads/main/styleguide/styleguide.md
+FormatStyle getChromiumStyle();
- /// Returns a format style complying with Chromium's style guide:
- /// https://chromium.googlesource.com/chromium/src/+/refs/heads/main/styleguide/styleguide.md
- FormatStyle getChromiumStyle();
+/// Returns a format style complying with the GNU coding standards:
+/// https://www.gnu.org/prep/standards/standards.html
+FormatStyle getGNUStyle();
- /// Returns a format style complying with the GNU coding standards:
- /// https://www.gnu.org/prep/standards/standards.html
- FormatStyle getGNUStyle();
+/// Returns a format style complying with Mozilla's style guide
+/// https://firefox-source-docs.mozilla.org/code-quality/coding-style/index.html
+FormatStyle getMozillaStyle();
- /// Returns a format style complying with Mozilla's style guide
- /// https://firefox-source-docs.mozilla.org/code-quality/coding-style/index.html
- FormatStyle getMozillaStyle();
+/// Returns a format style complying with Webkit's style guide:
+/// https://webkit.org/code-style-guidelines/
+FormatStyle getWebkitStyle();
- /// Returns a format style complying with Webkit's style guide:
- /// https://webkit.org/code-style-guidelines/
- FormatStyle getWebkitStyle();
+/// Returns a format style complying with Microsoft's style guide:
+/// https://docs.microsoft.com/en-us/visualstudio/ide/editorconfig-code-style-settings-reference
+FormatStyle getMicrosoftStyle();
+```
- /// Returns a format style complying with Microsoft's style guide:
- /// https://docs.microsoft.com/en-us/visualstudio/ide/editorconfig-code-style-settings-reference
- FormatStyle getMicrosoftStyle();
-
-These options are also exposed in the :doc:`standalone tools <ClangFormat>`
+These options are also exposed in the {doc}`standalone tools <ClangFormat>`
through the `-style` option.
In the future, we plan on making this configurable.
+
diff --git a/clang/docs/LibTooling.md b/clang/docs/LibTooling.md
index c6687fb9642f9..0c9a6ffeba7d0 100644
--- a/clang/docs/LibTooling.md
+++ b/clang/docs/LibTooling.md
@@ -1,213 +1,201 @@
-==========
-LibTooling
-==========
+# LibTooling
LibTooling is a library to support writing standalone tools based on Clang.
This document will provide a basic walkthrough of how to write a tool using
LibTooling.
For the information on how to setup Clang Tooling for LLVM see
-:doc:`HowToSetupToolingForLLVM`
+{doc}`HowToSetupToolingForLLVM`
-Introduction
-------------
+## Introduction
-Tools built with LibTooling, like Clang Plugins, run ``FrontendActions`` over
+Tools built with LibTooling, like Clang Plugins, run `FrontendActions` over
code.
-.. See FIXME for a tutorial on how to write FrontendActions.
+% See FIXME for a tutorial on how to write FrontendActions.
In this tutorial, we'll demonstrate the different ways of running Clang's
-``SyntaxOnlyAction``, which runs a quick syntax check, over a bunch of code.
+`SyntaxOnlyAction`, which runs a quick syntax check, over a bunch of code.
-Parsing a code snippet in memory
---------------------------------
+## Parsing a code snippet in memory
-If you ever wanted to run a ``FrontendAction`` over some sample code, for
-example to unit test parts of the Clang AST, ``runToolOnCode`` is what you
-looked for. Let me give you an example:
+If you ever wanted to run a `FrontendAction` over some sample code, for
+example to unit test parts of the Clang AST, `runToolOnCode` is what you
+looked for. Let me give you an example:
-.. code-block:: c++
+```c++
+#include "clang/Tooling/Tooling.h"
- #include "clang/Tooling/Tooling.h"
+TEST(runToolOnCode, CanSyntaxCheckCode) {
+ // runToolOnCode returns whether the action was correctly run over the
+ // given code.
+ EXPECT_TRUE(runToolOnCode(std::make_unique<clang::SyntaxOnlyAction>(), "class X {};"));
+}
+```
- TEST(runToolOnCode, CanSyntaxCheckCode) {
- // runToolOnCode returns whether the action was correctly run over the
- // given code.
- EXPECT_TRUE(runToolOnCode(std::make_unique<clang::SyntaxOnlyAction>(), "class X {};"));
- }
-
-Writing a standalone tool
--------------------------
+## Writing a standalone tool
-Once you unit tested your ``FrontendAction`` to the point where it cannot
-possibly break, it's time to create a standalone tool. For a standalone tool
+Once you unit tested your `FrontendAction` to the point where it cannot
+possibly break, it's time to create a standalone tool. For a standalone tool
to run clang, it first needs to figure out what command line arguments to use
-for a specified file. To that end we create a ``CompilationDatabase``. There
+for a specified file. To that end we create a `CompilationDatabase`. There
are different ways to create a compilation database, and we need to support all
-of them depending on command-line options. There's the ``CommonOptionsParser``
+of them depending on command-line options. There's the `CommonOptionsParser`
class that takes the responsibility to parse command-line parameters related to
compilation databases and inputs, so that all tools share the implementation.
-Parsing common tools options
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+### Parsing common tools options
-``CompilationDatabase`` can be read from a build directory or the command line.
-Using ``CommonOptionsParser`` allows for explicit specification of a compile
-command line, specification of build path using the ``-p`` command-line option,
+`CompilationDatabase` can be read from a build directory or the command line.
+Using `CommonOptionsParser` allows for explicit specification of a compile
+command line, specification of build path using the `-p` command-line option,
and automatic location of the compilation database using source files paths.
-.. code-block:: c++
-
- #include "clang/Tooling/CommonOptionsParser.h"
- #include "llvm/Support/CommandLine.h"
-
- using namespace clang::tooling;
- using namespace llvm;
-
- // Apply a custom category to all command-line options so that they are the
- // only ones displayed.
- static cl::OptionCategory MyToolCategory("my-tool options");
-
- int main(int argc, const char **argv) {
- // CommonOptionsParser::create will parse arguments and create a
- // CompilationDatabase.
- auto ExpectedParser = CommonOptionsParser::create(argc, argv, MyToolCategory);
- if (!ExpectedParser) {
- // Fail gracefully for unsupported options.
- llvm::errs() << toString(ExpectedParser.takeError());
- return 1;
- }
- CommonOptionsParser& OptionsParser = ExpectedParser.get();
-
- // Use OptionsParser.getCompilations() and OptionsParser.getSourcePathList()
- // to retrieve CompilationDatabase and the list of input file paths.
+```c++
+#include "clang/Tooling/CommonOptionsParser.h"
+#include "llvm/Support/CommandLine.h"
+
+using namespace clang::tooling;
+using namespace llvm;
+
+// Apply a custom category to all command-line options so that they are the
+// only ones displayed.
+static cl::OptionCategory MyToolCategory("my-tool options");
+
+int main(int argc, const char **argv) {
+ // CommonOptionsParser::create will parse arguments and create a
+ // CompilationDatabase.
+ auto ExpectedParser = CommonOptionsParser::create(argc, argv, MyToolCategory);
+ if (!ExpectedParser) {
+ // Fail gracefully for unsupported options.
+ llvm::errs() << toString(ExpectedParser.takeError());
+ return 1;
}
+ CommonOptionsParser& OptionsParser = ExpectedParser.get();
-Creating and running a ClangTool
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+ // Use OptionsParser.getCompilations() and OptionsParser.getSourcePathList()
+ // to retrieve CompilationDatabase and the list of input file paths.
+}
+```
-Once we have a ``CompilationDatabase``, we can create a ``ClangTool`` and run
-our ``FrontendAction`` over some code. For example, to run the
-``SyntaxOnlyAction`` over the files "a.cc" and "b.cc" one would write:
+### Creating and running a ClangTool
-.. code-block:: c++
+Once we have a `CompilationDatabase`, we can create a `ClangTool` and run
+our `FrontendAction` over some code. For example, to run the
+`SyntaxOnlyAction` over the files "a.cc" and "b.cc" one would write:
- // A clang tool can run over a number of sources in the same process...
- std::vector<std::string> Sources;
- Sources.push_back("a.cc");
- Sources.push_back("b.cc");
+```c++
+// A clang tool can run over a number of sources in the same process...
+std::vector<std::string> Sources;
+Sources.push_back("a.cc");
+Sources.push_back("b.cc");
- // We hand the CompilationDatabase we created and the sources to run over into
- // the tool constructor.
- ClangTool Tool(OptionsParser.getCompilations(), Sources);
+// We hand the CompilationDatabase we created and the sources to run over into
+// the tool constructor.
+ClangTool Tool(OptionsParser.getCompilations(), Sources);
- // The ClangTool needs a new FrontendAction for each translation unit we run
- // on. Thus, it takes a FrontendActionFactory as parameter. To create a
- // FrontendActionFactory from a given FrontendAction type, we call
- // newFrontendActionFactory<clang::SyntaxOnlyAction>().
- int result = Tool.run(newFrontendActionFactory<clang::SyntaxOnlyAction>().get());
+// The ClangTool needs a new FrontendAction for each translation unit we run
+// on. Thus, it takes a FrontendActionFactory as parameter. To create a
+// FrontendActionFactory from a given FrontendAction type, we call
+// newFrontendActionFactory<clang::SyntaxOnlyAction>().
+int result = Tool.run(newFrontendActionFactory<clang::SyntaxOnlyAction>().get());
+```
-Putting it together --- the first tool
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+### Putting it together --- the first tool
-Now we combine the two previous steps into our first real tool. A more advanced
+Now we combine the two previous steps into our first real tool. A more advanced
version of this example tool is also checked into the clang tree at
-``tools/clang-check/ClangCheck.cpp``.
-
-.. code-block:: c++
-
- // Declares clang::SyntaxOnlyAction.
- #include "clang/Frontend/FrontendActions.h"
- #include "clang/Tooling/CommonOptionsParser.h"
- #include "clang/Tooling/Tooling.h"
- // Declares llvm::cl::extrahelp.
- #include "llvm/Support/CommandLine.h"
-
- using namespace clang::tooling;
- using namespace llvm;
-
- // Apply a custom category to all command-line options so that they are the
- // only ones displayed.
- static cl::OptionCategory MyToolCategory("my-tool options");
-
- // CommonOptionsParser declares HelpMessage with a description of the common
- // command-line options related to the compilation database and input files.
- // It's nice to have this help message in all tools.
- static cl::extrahelp CommonHelp(CommonOptionsParser::HelpMessage);
-
- // A help message for this specific tool can be added afterwards.
- static cl::extrahelp MoreHelp("\nMore help text...\n");
-
- int main(int argc, const char **argv) {
- auto ExpectedParser = CommonOptionsParser::create(argc, argv, MyToolCategory);
- if (!ExpectedParser) {
- llvm::errs() << toString(ExpectedParser.takeError());
- return 1;
- }
- CommonOptionsParser& OptionsParser = ExpectedParser.get();
- ClangTool Tool(OptionsParser.getCompilations(),
- OptionsParser.getSourcePathList());
- return Tool.run(newFrontendActionFactory<clang::SyntaxOnlyAction>().get());
+`tools/clang-check/ClangCheck.cpp`.
+
+```c++
+// Declares clang::SyntaxOnlyAction.
+#include "clang/Frontend/FrontendActions.h"
+#include "clang/Tooling/CommonOptionsParser.h"
+#include "clang/Tooling/Tooling.h"
+// Declares llvm::cl::extrahelp.
+#include "llvm/Support/CommandLine.h"
+
+using namespace clang::tooling;
+using namespace llvm;
+
+// Apply a custom category to all command-line options so that they are the
+// only ones displayed.
+static cl::OptionCategory MyToolCategory("my-tool options");
+
+// CommonOptionsParser declares HelpMessage with a description of the common
+// command-line options related to the compilation database and input files.
+// It's nice to have this help message in all tools.
+static cl::extrahelp CommonHelp(CommonOptionsParser::HelpMessage);
+
+// A help message for this specific tool can be added afterwards.
+static cl::extrahelp MoreHelp("\nMore help text...\n");
+
+int main(int argc, const char **argv) {
+ auto ExpectedParser = CommonOptionsParser::create(argc, argv, MyToolCategory);
+ if (!ExpectedParser) {
+ llvm::errs() << toString(ExpectedParser.takeError());
+ return 1;
}
+ CommonOptionsParser& OptionsParser = ExpectedParser.get();
+ ClangTool Tool(OptionsParser.getCompilations(),
+ OptionsParser.getSourcePathList());
+ return Tool.run(newFrontendActionFactory<clang::SyntaxOnlyAction>().get());
+}
+```
-Running the tool on some code
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+### Running the tool on some code
When you check out and build clang, clang-check is already built and available
to you in bin/clang-check inside your build directory.
You can run clang-check on a file in the llvm repository by specifying all the
-needed parameters after a "``--``" separator:
-
-.. code-block:: bash
+needed parameters after a "`--`" separator:
- $ cd /path/to/source/llvm
- $ export BD=/path/to/build/llvm
- $ $BD/bin/clang-check tools/clang/tools/clang-check/ClangCheck.cpp -- \
- clang++ -D__STDC_CONSTANT_MACROS -D__STDC_LIMIT_MACROS \
- -Itools/clang/include -I$BD/include -Iinclude \
- -Itools/clang/lib/Headers -c
+```bash
+$ cd /path/to/source/llvm
+$ export BD=/path/to/build/llvm
+$ $BD/bin/clang-check tools/clang/tools/clang-check/ClangCheck.cpp -- \
+ clang++ -D__STDC_CONSTANT_MACROS -D__STDC_LIMIT_MACROS \
+ -Itools/clang/include -I$BD/include -Iinclude \
+ -Itools/clang/lib/Headers -c
+```
As an alternative, you can also configure cmake to output a compile command
database into its build directory:
-.. code-block:: bash
-
- # Alternatively to calling cmake, use ccmake, toggle to advanced mode and
- # set the parameter CMAKE_EXPORT_COMPILE_COMMANDS from the UI.
- $ cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON .
+```bash
+# Alternatively to calling cmake, use ccmake, toggle to advanced mode and
+# set the parameter CMAKE_EXPORT_COMPILE_COMMANDS from the UI.
+$ cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON .
+```
-This creates a file called ``compile_commands.json`` in the build directory.
-Now you can run :program:`clang-check` over files in the project by specifying
+This creates a file called `compile_commands.json` in the build directory.
+Now you can run {program}`clang-check` over files in the project by specifying
the build path as first argument and some source files as further positional
arguments:
-.. code-block:: bash
+```bash
+$ cd /path/to/source/llvm
+$ export BD=/path/to/build/llvm
+$ $BD/bin/clang-check -p $BD tools/clang/tools/clang-check/ClangCheck.cpp
+```
- $ cd /path/to/source/llvm
- $ export BD=/path/to/build/llvm
- $ $BD/bin/clang-check -p $BD tools/clang/tools/clang-check/ClangCheck.cpp
+(libtooling-builtin-includes)=
-
-.. _libtooling_builtin_includes:
-
-Builtin includes
-^^^^^^^^^^^^^^^^
+### Builtin includes
Clang tools need their builtin headers and search for them the same way Clang
-does. Thus, the default location to look for builtin headers is in a path
-``$(dirname /path/to/tool)/../lib/clang/3.3/include`` relative to the tool
-binary. This works out-of-the-box for tools running from llvm's toplevel
+does. Thus, the default location to look for builtin headers is in a path
+`$(dirname /path/to/tool)/../lib/clang/3.3/include` relative to the tool
+binary. This works out-of-the-box for tools running from llvm's toplevel
binary directory after building clang-resource-headers, or if the tool is
running from the binary directory of a clang install next to the clang binary.
-Tips: if your tool fails to find ``stddef.h`` or similar headers, call the tool
-with ``-v`` and look at the search paths it looks through.
+Tips: if your tool fails to find `stddef.h` or similar headers, call the tool
+with `-v` and look at the search paths it looks through.
-Linking
-^^^^^^^
+### Linking
For a list of libraries to link, look at one of the tools' CMake files (for
-example `clang-check/CMakeList.txt
-<https://github.com/llvm/llvm-project/blob/main/clang/tools/clang-check/CMakeLists.txt>`_).
+example [clang-check/CMakeList.txt](https://github.com/llvm/llvm-project/blob/main/clang/tools/clang-check/CMakeLists.txt)).
+
diff --git a/clang/docs/RISCVSupport.md b/clang/docs/RISCVSupport.md
index 7d49c445756b6..084f5629c2cff 100644
--- a/clang/docs/RISCVSupport.md
+++ b/clang/docs/RISCVSupport.md
@@ -1,34 +1,33 @@
-==============
-RISC-V Support
-==============
+# RISC-V Support
-.. contents::
- :local:
+```{contents}
+:local: true
+```
-Intrinsic Detection Macros
-===========================
+## Intrinsic Detection Macros
Clang provides macros to detect which RISC-V intrinsics are supported by the
toolchain. This is only available if intrinsics are ratified, in other word,
experimental intrinsics do not have macro defined.
Note: This is independent from assembler support.
-Intrinsic Detection
----------------------------
+### Intrinsic Detection
-Macros of the form ``__riscv_intrinsic_<extension>`` indicate that the toolchain
+Macros of the form `__riscv_intrinsic_<extension>` indicate that the toolchain
supports intrinsics for a given extension:
-.. code-block:: c
-
- #if defined(__riscv_intrinsic_zbb)
- // Toolchain supports Zbb scalar intrinsics like __riscv_orc_b_*
- // These can be used with target attributes even if -march doesn't include Zbb
- __attribute__((target("arch=+zbb")))
- unsigned long process_with_zbb(unsigned long x) {
- return __riscv_orc_b_64(x);
- }
- #endif
+```c
+#if defined(__riscv_intrinsic_zbb)
+ // Toolchain supports Zbb scalar intrinsics like __riscv_orc_b_*
+ // These can be used with target attributes even if -march doesn't include Zbb
+ __attribute__((target("arch=+zbb")))
+ unsigned long process_with_zbb(unsigned long x) {
+ return __riscv_orc_b_64(x);
+ }
+#endif
+```
Composite extensions are also defined when all their sub-extensions are available, e.g.
- ``__riscv_intrinsic_zkn`` - zbkb + zbkc + zbkx + zkne + zknd + zknh
+
+: `__riscv_intrinsic_zkn` - zbkb + zbkc + zbkx + zkne + zknd + zknh
+
diff --git a/clang/docs/Tooling.md b/clang/docs/Tooling.md
index 141d4d8d8959b..de4781fe99fde 100644
--- a/clang/docs/Tooling.md
+++ b/clang/docs/Tooling.md
@@ -1,97 +1,92 @@
-=================================================
-Choosing the Right Interface for Your Application
-=================================================
+# Choosing the Right Interface for Your Application
Clang provides infrastructure to write tools that need syntactic and semantic
-information about a program. This document will give a short introduction of
+information about a program. This document will give a short introduction of
the different ways to write clang tools, and their pros and cons.
-LibClang
---------
+## LibClang
-`LibClang <https://clang.llvm.org/doxygen/group__CINDEX.html>`_ is a stable high
-level C interface to clang. When in doubt LibClang is probably the interface
-you want to use. Consider the other interfaces only when you have a good
+[LibClang](https://clang.llvm.org/doxygen/group__CINDEX.html) is a stable high
+level C interface to clang. When in doubt LibClang is probably the interface
+you want to use. Consider the other interfaces only when you have a good
reason not to use LibClang.
Canonical examples of when to use LibClang:
-* Xcode
-* Clang Python Bindings
+- Xcode
+- Clang Python Bindings
Use LibClang when you...:
-* want to interface with clang from other languages than C++
-* need a stable interface that takes care to be backwards compatible
-* want powerful high-level abstractions, like iterating through an AST with a
+- want to interface with clang from other languages than C++
+- need a stable interface that takes care to be backwards compatible
+- want powerful high-level abstractions, like iterating through an AST with a
cursor, and don't want to learn all the nitty gritty details of Clang's AST.
Do not use LibClang when you...:
-* want full control over the Clang AST
+- want full control over the Clang AST
-Clang Plugins
--------------
+## Clang Plugins
-:doc:`Clang Plugins <ClangPlugins>` allow you to run additional actions on the
-AST as part of a compilation. Plugins are dynamic libraries that are loaded at
+{doc}`Clang Plugins <ClangPlugins>` allow you to run additional actions on the
+AST as part of a compilation. Plugins are dynamic libraries that are loaded at
runtime by the compiler, and they're easy to integrate into your build
environment.
Canonical examples of when to use Clang Plugins:
-* special lint-style warnings or errors for your project
-* creating additional build artifacts from a single compile step
+- special lint-style warnings or errors for your project
+- creating additional build artifacts from a single compile step
Use Clang Plugins when you...:
-* need your tool to rerun if any of the dependencies change
-* want your tool to make or break a build
-* need full control over the Clang AST
+- need your tool to rerun if any of the dependencies change
+- want your tool to make or break a build
+- need full control over the Clang AST
Do not use Clang Plugins when you...:
-* want to run tools outside of your build environment
-* want full control on how Clang is set up, including mapping of in-memory
+- want to run tools outside of your build environment
+- want full control on how Clang is set up, including mapping of in-memory
virtual files
-* need to run over a specific subset of files in your project which is not
+- need to run over a specific subset of files in your project which is not
necessarily related to any changes which would trigger rebuilds
-LibTooling
-----------
+## LibTooling
-:doc:`LibTooling <LibTooling>` is a C++ interface aimed at writing standalone
-tools, as well as integrating into services that run clang tools. Canonical
+{doc}`LibTooling <LibTooling>` is a C++ interface aimed at writing standalone
+tools, as well as integrating into services that run clang tools. Canonical
examples of when to use LibTooling:
-* a simple syntax checker
-* refactoring tools
+- a simple syntax checker
+- refactoring tools
Use LibTooling when you...:
-* want to run tools over a single file, or a specific subset of files,
+- want to run tools over a single file, or a specific subset of files,
independently of the build system
-* want full control over the Clang AST
-* want to share code with Clang Plugins
+- want full control over the Clang AST
+- want to share code with Clang Plugins
Do not use LibTooling when you...:
-* want to run as part of the build triggered by dependency changes
-* want a stable interface so you don't need to change your code when the AST API
+- want to run as part of the build triggered by dependency changes
+- want a stable interface so you don't need to change your code when the AST API
changes
-* want high level abstractions like cursors and code completion out of the box
-* do not want to write your tools in C++
+- want high level abstractions like cursors and code completion out of the box
+- do not want to write your tools in C++
-:doc:`Clang tools <ClangTools>` are a collection of specific developer tools
+{doc}`Clang tools <ClangTools>` are a collection of specific developer tools
built on top of the LibTooling infrastructure as part of the Clang project.
They are targeted at automating and improving core development activities of
C/C++ developers.
Examples of tools we are building or planning as part of the Clang project:
-* Syntax checking (:program:`clang-check`)
-* Automatic fixing of compile errors (:program:`clang-fixit`)
-* Automatic code formatting (:program:`clang-format`)
-* Migration tools for new features in new language standards
-* Core refactoring tools
+- Syntax checking ({program}`clang-check`)
+- Automatic fixing of compile errors ({program}`clang-fixit`)
+- Automatic code formatting ({program}`clang-format`)
+- Migration tools for new features in new language standards
+- Core refactoring tools
>From 6a4a95ca8a60abb8c55b889e30f329cec2f87a6c Mon Sep 17 00:00:00 2001
From: Reid Kleckner <rkleckner at nvidia.com>
Date: Mon, 13 Jul 2026 19:06:34 +0000
Subject: [PATCH 2/2] [docs] Finish MyST migration for selected docs
---
clang/docs/AMDGPUSupport.md | 69 +++++++-----------------------
clang/docs/APINotes.md | 43 +++++++++++--------
clang/docs/AllocToken.md | 10 +++--
clang/docs/BoundsSafety.md | 23 +++++-----
clang/docs/ClangPlugins.md | 49 +++++++++++----------
clang/docs/ControlFlowIntegrity.md | 33 +++++++-------
clang/docs/LibClang.md | 6 ---
clang/docs/LibTooling.md | 3 +-
8 files changed, 100 insertions(+), 136 deletions(-)
diff --git a/clang/docs/AMDGPUSupport.md b/clang/docs/AMDGPUSupport.md
index 21180587dc065..8731af371b23f 100644
--- a/clang/docs/AMDGPUSupport.md
+++ b/clang/docs/AMDGPUSupport.md
@@ -1,23 +1,3 @@
-```{raw} html
-<style type="text/css">
- .none { background-color: #FFCCCC }
- .part { background-color: #FFFF99 }
- .good { background-color: #CCFF99 }
-</style>
-```
-
-```{eval-rst}
-.. role:: none
-```
-
-```{eval-rst}
-.. role:: part
-```
-
-```{eval-rst}
-.. role:: good
-```
-
```{contents}
:local: true
```
@@ -28,39 +8,21 @@ Clang supports OpenCL, HIP and OpenMP on AMD GPU targets.
## Predefined Macros
-```{eval-rst}
-.. list-table::
- :header-rows: 1
-
- * - Macro
- - Description
- * - ``__AMDGPU__``
- - Indicates that the code is being compiled for an AMD GPU.
- * - ``__AMDGCN__``
- - Defined if the GPU target is AMDGCN.
- * - ``__R600__``
- - Defined if the GPU target is R600.
- * - ``__<ArchName>__``
- - Defined with the name of the architecture (e.g., ``__gfx906__`` for the gfx906 architecture).
- * - ``__<GFXN>__``
- - Defines the GFX family (e.g., for gfx906, this macro would be ``__GFX9__``).
- * - ``__amdgcn_processor__``
- - Defined with the processor name as a string (e.g., ``"gfx906"``).
- * - ``__amdgcn_target_id__``
- - Defined with the target ID as a string.
- * - ``__amdgcn_feature_<feature-name>__``
- - Defined for each supported target feature. The value is 1 if the feature is enabled and 0 if it is disabled. Allowed feature names are sramecc and xnack.
- * - ``__AMDGCN_CUMODE__``
- - Defined as 1 if the CU mode is enabled and 0 if the WGP mode is enabled.
- * - ``__AMDGCN_UNSAFE_FP_ATOMICS__``
- - Defined if unsafe floating-point atomics are allowed.
- * - ``__HAS_FMAF__``
- - Defined if FMAF instruction is available (deprecated).
- * - ``__HAS_LDEXPF__``
- - Defined if LDEXPF instruction is available (deprecated).
- * - ``__HAS_FP64__``
- - Defined if FP64 instruction is available (deprecated).
-```
+| Macro | Description |
+| --- | --- |
+| `__AMDGPU__` | Indicates that the code is being compiled for an AMD GPU. |
+| `__AMDGCN__` | Defined if the GPU target is AMDGCN. |
+| `__R600__` | Defined if the GPU target is R600. |
+| `__<ArchName>__` | Defined with the name of the architecture (e.g., `__gfx906__` for the gfx906 architecture). |
+| `__<GFXN>__` | Defines the GFX family (e.g., for gfx906, this macro would be `__GFX9__`). |
+| `__amdgcn_processor__` | Defined with the processor name as a string (e.g., `"gfx906"`). |
+| `__amdgcn_target_id__` | Defined with the target ID as a string. |
+| `__amdgcn_feature_<feature-name>__` | Defined for each supported target feature. The value is 1 if the feature is enabled and 0 if it is disabled. Allowed feature names are sramecc and xnack. |
+| `__AMDGCN_CUMODE__` | Defined as 1 if the CU mode is enabled and 0 if the WGP mode is enabled. |
+| `__AMDGCN_UNSAFE_FP_ATOMICS__` | Defined if unsafe floating-point atomics are allowed. |
+| `__HAS_FMAF__` | Defined if FMAF instruction is available (deprecated). |
+| `__HAS_LDEXPF__` | Defined if LDEXPF instruction is available (deprecated). |
+| `__HAS_FP64__` | Defined if FP64 instruction is available (deprecated). |
Please note that the specific architecture and feature names will vary depending on the GPU. Also, some macros are deprecated and may be removed in future releases.
@@ -68,4 +30,3 @@ Please note that the specific architecture and feature names will vary depending
Clang exposes AMDGPU hardware intrinsics as target-specific builtins with the
`__builtin_amdgcn_` prefix. These are documented in {doc}`AMDGPUBuiltinReference`.
-
diff --git a/clang/docs/APINotes.md b/clang/docs/APINotes.md
index 68857cbdbd2fd..4d8646ec36c27 100644
--- a/clang/docs/APINotes.md
+++ b/clang/docs/APINotes.md
@@ -34,7 +34,7 @@ directory as the corresponding module map; for framework modules, they should
be placed in the Headers and PrivateHeaders directories, respectively. The
module map for a private top-level framework module should be placed in the
PrivateHeaders directory as well, though it does not need an additional
-"\_private" suffix on its name.
+"_private" suffix on its name.
Clang will search for API notes files next to module maps only when passed the
`-fapinotes-modules` option.
@@ -61,6 +61,16 @@ entries:
```{eval-rst}
+:Name:
+
+ The name of the module (the framework name, for frameworks). Note that this
+ is always the name of a top-level module, even within a private API notes
+ file.
+
+ ::
+
+ Name: MyFramework
+
:Classes, Protocols, Tags, Typedefs, Globals, Enumerators, Functions, Namespaces:
Arrays of top-level declarations. Each entry in the array must have a
@@ -98,12 +108,15 @@ Each entry under 'Classes' and 'Protocols' can contain "Methods" and
Identified by 'Selector' and 'MethodKind'; the MethodKind is either
"Instance" or "Class".
-```
-Each entry under 'Classes' and 'Protocols' can contain "Methods" and
-"Properties" arrays, in addition to the attributes described below:
+ ::
-```{eval-rst}
+ Classes:
+ - Name: UIViewController
+ Methods:
+ - Selector: "presentViewController:animated:"
+ MethodKind: Instance
+ …
:Properties:
@@ -131,13 +144,9 @@ arrays. Methods under Tags are C++ methods identified by 'Name' (rather than
Tags:
- Name: MyClass
-```
-
-Each entry under "Tags" can contain "Methods", "Fields", and nested "Tags"
-arrays. Methods under Tags are C++ methods identified by 'Name' (rather than
-'Selector' and 'MethodKind' as used for Objective-C methods).
-
-```{eval-rst}
+ Methods:
+ - Name: doSomething
+ …
:Fields:
@@ -173,12 +182,13 @@ declaration kind), all of which are optional:
with all arguments. Use "_" to omit an argument label.
::
-```
-Each declaration supports the following annotations (if relevant to that
-declaration kind), all of which are optional:
+ - Selector: "presentViewController:animated:"
+ MethodKind: Instance
+ SwiftName: "present(_:animated:)"
-```{eval-rst}
+ - Class: NSBundle
+ SwiftName: Bundle
:SwiftImportAs:
@@ -495,4 +505,3 @@ declaration kind), all of which are optional:
MethodKind: Instance
DesignatedInit: true
```
-
diff --git a/clang/docs/AllocToken.md b/clang/docs/AllocToken.md
index a26bce344b4dc..7083d7d655641 100644
--- a/clang/docs/AllocToken.md
+++ b/clang/docs/AllocToken.md
@@ -142,7 +142,8 @@ token-enabled variants for a well-defined set of standard functions.
To extend instrumentation to custom allocation functions, enable broader
coverage with `-fsanitize-alloc-token-extended`. Such functions require being
-marked with the [malloc](https://clang.llvm.org/docs/AttributeReference.html#malloc) or [alloc_size](https://clang.llvm.org/docs/AttributeReference.html#alloc-size) attributes
+marked with the [malloc][malloc-attribute] or
+[alloc_size][alloc-size-attribute] attributes
(or a combination).
For example:
@@ -161,12 +162,16 @@ ptr2 = __alloc_token_my_malloc(size, token_id);
```
Note: Even in the default mode (without `-fsanitize-alloc-token-extended`),
-an *inline* allocation wrapper marked with the [malloc](https://clang.llvm.org/docs/AttributeReference.html#malloc) or [alloc_size](https://clang.llvm.org/docs/AttributeReference.html#alloc-size) attribute is
+an *inline* allocation wrapper marked with the [malloc][malloc-attribute] or
+[alloc_size][alloc-size-attribute] attribute is
supported if it is inlined into its caller: the inferred token is propagated to
the allocation call the wrapper returns, which is then instrumented normally.
Wrappers that are not inlined still require
`-fsanitize-alloc-token-extended`.
+[malloc-attribute]: https://clang.llvm.org/docs/AttributeReference.html#malloc
+[alloc-size-attribute]: https://clang.llvm.org/docs/AttributeReference.html#alloc-size
+
### Disabling Instrumentation
To exclude specific functions from instrumentation, you can use the
@@ -218,4 +223,3 @@ can be used for this purpose.
// Code specific to -fsanitize=alloc-token builds
#endif
```
-
diff --git a/clang/docs/BoundsSafety.md b/clang/docs/BoundsSafety.md
index ef1bb190668bf..e9ef248a4f7c1 100644
--- a/clang/docs/BoundsSafety.md
+++ b/clang/docs/BoundsSafety.md
@@ -1,7 +1,7 @@
# `-fbounds-safety`: Enforcing bounds safety for C
```{contents}
-:local: true
+:local:
```
## Overview
@@ -94,8 +94,8 @@ of `count`, like in the example below, may violate this invariant and permit
out-of-bounds access to the pointer. To avoid this, the compiler employs
compile-time restrictions and emits run-time checks as necessary to ensure the
new count value doesn't exceed the actual length of the buffer. Section
-[Maintaining correctness of bounds annotations] provides more details about
-this programming model.
+[Maintaining correctness of bounds annotations](#maintaining-correctness-of-bounds-annotations)
+provides more details about this programming model.
```c
int g;
@@ -128,7 +128,7 @@ which are considered ABI-visible. As local variables are typically hidden from
the ABI, this approach has a marginal impact on it. In addition,
`-fbounds-safety` employs compile-time restrictions to prevent implicit wide
pointers from silently breaking the ABI (see [ABI implications of default bounds
-annotations][abi implications of default bounds annotations]). Pointers associated with any other variables, including function
+annotations](#abi-implications-of-default-bounds-annotations)). Pointers associated with any other variables, including function
parameters, are treated as single object pointers (i.e., `__single`), ensuring
that they always have the tightest bounds by default and offering a strong
bounds safety guarantee.
@@ -225,7 +225,7 @@ Accessing a pointer outside the specified bounds causes a run-time trap or a
compile-time error. Also, the model maintains correctness of bounds annotations
when the pointer and/or the related value containing the bounds information are
updated or passed as arguments. This is done by compile-time restrictions or
-run-time checks (see [Maintaining correctness of bounds annotations]
+run-time checks (see [Maintaining correctness of bounds annotations](#maintaining-correctness-of-bounds-annotations)
for more detail). For instance, initializing `buf` with `null` while
assigning non-zero value to `count`, as shown in the following example, would
violate the `__counted_by` annotation because a null pointer does not point to
@@ -419,7 +419,7 @@ to `__unsafe_indexable` by default.
The `__ptrcheck_abi_assume_*ATTR*()` macros are defined as pragmas in the
toolchain header (See [Portability with toolchains that do not support the
-extension][portability with toolchains that do not support the extension] for more details about the toolchain header):
+extension](#portability-with-toolchains-that-do-not-support-the-extension) for more details about the toolchain header):
```C
#define __ptrcheck_abi_assume_single() \
@@ -562,7 +562,7 @@ When `sizeof()` takes an expression, i.e., `sizeof(expr`, it behaves as
`sizeof(typeof(expr))`, except that `sizeof(expr)` does not report an error
with `expr` that has a type with an external bounds annotation dependent on
another declaration, whereas `typeof()` on the same expression would be an
-error as described in {ref}`Default pointer types in typeof`.
+error as described in [Default pointer types in typeof()](#default-pointer-types-in-typeof).
The following example describes this behavior.
```c
@@ -603,9 +603,9 @@ bounds annotation that can implicitly convert to `__bidi_indexable`. If
the first element. However, if src has type `int *__unsafe_indexable`, the
explicit cast `(int *__bidi_indexable)src` will cause an error because
`__unsafe_indexable` cannot cast to `__bidi_indexable` as
-`__unsafe_indexable` doesn't have bounds information. [Cast rules] describes
-in more detail what kinds of casts are allowed between pointers with different
-bounds annotations.
+`__unsafe_indexable` doesn't have bounds information. [Cast rules](#cast-rules)
+describes in more detail what kinds of casts are allowed between pointers with
+different bounds annotations.
#### Default pointer types in typedef
@@ -771,7 +771,7 @@ bound of the pointer is derived. If the source pointer has `__sized_by`,
bound calculation doesn't overflow, e.g., `ptr + size` (where the type of
`ptr` is `void *__sized_by(size)`), because when the `__sized_by` pointer
is initialized, `-fbounds-safety` inserts run-time checks to ensure that `ptr
-\+ size` doesn't overflow and that `size >= 0`.
++ size` doesn't overflow and that `size >= 0`.
Assuming the upper bound calculation doesn't overflow, the compiler can simplify
the trap condition `upper(tmp_ptr) - tmp_ptr < tmp_count` to `size <
@@ -970,4 +970,3 @@ early on and remove unnecessary bounds checks in unoptimized builds.
Your feedback on the programming model is valuable. You may want to follow the
instruction in {doc}`BoundsSafetyAdoptionGuide` to play with `-fbounds-safety`
and please send your feedback to [Yeoul Na](mailto:yeoul_na at apple.com).
-
diff --git a/clang/docs/ClangPlugins.md b/clang/docs/ClangPlugins.md
index 6bb6778204f50..10820833f4e4b 100644
--- a/clang/docs/ClangPlugins.md
+++ b/clang/docs/ClangPlugins.md
@@ -80,34 +80,34 @@ static ParsedAttrInfoRegistry::Add<ExampleAttrInfo> Z("example_attr","example at
The members of `ParsedAttrInfo` that a plugin attribute must define are:
-> - `Spellings`, which must be populated with every [Spelling](/doxygen/structclang_1_1ParsedAttrInfo_1_1Spelling.html) of the
-> attribute, each of which consists of an attribute syntax and how the
-> attribute name is spelled for that syntax. If the syntax allows a scope then
-> the spelling must be "scope::attr" if a scope is present or "::attr" if not.
+- `Spellings`, which must be populated with every [Spelling](https://clang.llvm.org/doxygen/structclang_1_1ParsedAttrInfo_1_1Spelling.html) of the
+ attribute, each of which consists of an attribute syntax and how the
+ attribute name is spelled for that syntax. If the syntax allows a scope then
+ the spelling must be "scope::attr" if a scope is present or "::attr" if not.
The members of `ParsedAttrInfo` that may need to be defined, depending on the
attribute, are:
-> - `NumArgs` and `OptArgs`, which set the number of required and optional
-> arguments to the attribute.
-> - `diagAppertainsToDecl`, which checks if the attribute has been used on the
-> right kind of declaration and issues a diagnostic if not.
-> - `handleDeclAttribute`, which is the function that applies the attribute to
-> a declaration. It is responsible for checking that the attribute's arguments
-> are valid, and typically applies the attribute by adding an `Attr` to the
-> `Decl`. It returns either `AttributeApplied`, to indicate that the
-> attribute was successfully applied, or `AttributeNotApplied` if it wasn't.
-> - `diagAppertainsToStmt`, which checks if the attribute has been used on the
-> right kind of statement and issues a diagnostic if not.
-> - `handleStmtAttribute`, which is the function that applies the attribute to
-> a statement. It is responsible for checking that the attribute's arguments
-> are valid, and typically applies the attribute by adding an `Attr` to the
-> `Stmt`. It returns either `AttributeApplied`, to indicate that the
-> attribute was successfully applied, or `AttributeNotApplied` if it wasn't.
-> - `diagLangOpts`, which checks if the attribute is permitted for the current
-> language mode and issues a diagnostic if not.
-> - `existsInTarget`, which checks if the attribute is permitted for the given
-> target.
+- `NumArgs` and `OptArgs`, which set the number of required and optional
+ arguments to the attribute.
+- `diagAppertainsToDecl`, which checks if the attribute has been used on the
+ right kind of declaration and issues a diagnostic if not.
+- `handleDeclAttribute`, which is the function that applies the attribute to
+ a declaration. It is responsible for checking that the attribute's arguments
+ are valid, and typically applies the attribute by adding an `Attr` to the
+ `Decl`. It returns either `AttributeApplied`, to indicate that the
+ attribute was successfully applied, or `AttributeNotApplied` if it wasn't.
+- `diagAppertainsToStmt`, which checks if the attribute has been used on the
+ right kind of statement and issues a diagnostic if not.
+- `handleStmtAttribute`, which is the function that applies the attribute to
+ a statement. It is responsible for checking that the attribute's arguments
+ are valid, and typically applies the attribute by adding an `Attr` to the
+ `Stmt`. It returns either `AttributeApplied`, to indicate that the
+ attribute was successfully applied, or `AttributeNotApplied` if it wasn't.
+- `diagLangOpts`, which checks if the attribute is permitted for the current
+ language mode and issues a diagnostic if not.
+- `existsInTarget`, which checks if the attribute is permitted for the given
+ target.
To see a working example of an attribute plugin, see [the Attribute.cpp example](https://github.com/llvm/llvm-project/blob/main/clang/examples/Attribute/Attribute.cpp).
@@ -200,4 +200,3 @@ optimizations. Use `CmdlineBeforeMainAction` or `AddBeforeMainAction` as
`getActionType` to run plugins while still benefitting from
`-clear-ast-before-backend`. Plugins must make sure not to modify the AST,
otherwise they should run after the main action.
-
diff --git a/clang/docs/ControlFlowIntegrity.md b/clang/docs/ControlFlowIntegrity.md
index d45526c9e9760..d8627aa791b7b 100644
--- a/clang/docs/ControlFlowIntegrity.md
+++ b/clang/docs/ControlFlowIntegrity.md
@@ -42,7 +42,7 @@ CFI checks for classes without visibility attributes. Most users will want
to specify `-fvisibility=hidden`, which enables CFI checks for such classes.
When using `-fsanitize=cfi*` with `-flto=thin`, it is recommended
-to reduce link times by passing [-funique-source-file-names](UsersManual.html#cmdoption-f-no-unique-source-file-names), provided
+to reduce link times by passing [-funique-source-file-names](https://clang.llvm.org/docs/UsersManual.html#cmdoption-f-no-unique-source-file-names), provided
that your program is compatible with it.
Experimental support for {ref}`cross-DSO control flow integrity
@@ -55,20 +55,20 @@ visibility. This cross-DSO support has unstable ABI at this time.
Available schemes are:
-> - `-fsanitize=cfi-cast-strict`: Enables {ref}`strict cast checks
-> <cfi-strictness>`.
-> - `-fsanitize=cfi-derived-cast`: Base-to-derived cast to the wrong
-> dynamic type.
-> - `-fsanitize=cfi-unrelated-cast`: Cast from `void*` or another
-> unrelated type to the wrong dynamic type.
-> - `-fsanitize=cfi-nvcall`: Non-virtual call via an object whose vptr is of
-> the wrong dynamic type.
-> - `-fsanitize=cfi-vcall`: Virtual call via an object whose vptr is of the
-> wrong dynamic type.
-> - `-fsanitize=cfi-icall`: Indirect call of a function with wrong dynamic
-> type.
-> - `-fsanitize=cfi-mfcall`: Indirect call via a member function pointer with
-> wrong dynamic type.
+- `-fsanitize=cfi-cast-strict`: Enables {ref}`strict cast checks
+ <cfi-strictness>`.
+- `-fsanitize=cfi-derived-cast`: Base-to-derived cast to the wrong
+ dynamic type.
+- `-fsanitize=cfi-unrelated-cast`: Cast from `void*` or another
+ unrelated type to the wrong dynamic type.
+- `-fsanitize=cfi-nvcall`: Non-virtual call via an object whose vptr is of
+ the wrong dynamic type.
+- `-fsanitize=cfi-vcall`: Virtual call via an object whose vptr is of the
+ wrong dynamic type.
+- `-fsanitize=cfi-icall`: Indirect call of a function with wrong dynamic
+ type.
+- `-fsanitize=cfi-mfcall`: Indirect call via a member function pointer with
+ wrong dynamic type.
You can use `-fsanitize=cfi` to enable all the schemes and use
`-fno-sanitize` flag to narrow down the set of schemes as desired.
@@ -254,7 +254,7 @@ This option is currently experimental.
The default behavior of Clang's indirect function call checker will replace
the address of each CFI-checked function in the output file's symbol table
with the address of a jump table entry which will pass CFI checks. We refer
-to this as making the jump table `canonical`. This property allows code that
+to this as making the jump table *canonical*. This property allows code that
was not compiled with `-fsanitize=cfi-icall` to take a CFI-valid address
of a function, but it comes with a couple of caveats that are especially
relevant for users of cross-DSO CFI:
@@ -412,4 +412,3 @@ Caroline Tice, Tom Roeder, Peter Collingbourne, Stephen Checkoway,
Úlfar Erlingsson, Luis Lozano, Geoff Pike.
[gold plugin]: https://llvm.org/docs/GoldPlugin.html
-
diff --git a/clang/docs/LibClang.md b/clang/docs/LibClang.md
index f8278a678a768..d2c7d1d29c50e 100644
--- a/clang/docs/LibClang.md
+++ b/clang/docs/LibClang.md
@@ -1,8 +1,3 @@
-```{eval-rst}
-.. role:: raw-html(raw)
- :format: html
-```
-
# Libclang tutorial
The C Interface to Clang provides a relatively small API that exposes facilities for parsing source code into an abstract syntax tree (AST), loading already-parsed ASTs, traversing the AST, associating physical source locations with elements within the AST, and other facilities that support Clang-based development tools.
@@ -431,4 +426,3 @@ or some other mitigation approach if processing untrusted input.
[index.h]: https://github.com/llvm/llvm-project/blob/main/clang/include/clang-c/Index.h
[security-sensitive component]: https://llvm.org/docs/Security.html#what-is-considered-a-security-issue
-
diff --git a/clang/docs/LibTooling.md b/clang/docs/LibTooling.md
index 0c9a6ffeba7d0..c8e11b60be2a3 100644
--- a/clang/docs/LibTooling.md
+++ b/clang/docs/LibTooling.md
@@ -180,7 +180,7 @@ $ export BD=/path/to/build/llvm
$ $BD/bin/clang-check -p $BD tools/clang/tools/clang-check/ClangCheck.cpp
```
-(libtooling-builtin-includes)=
+(libtooling_builtin_includes)=
### Builtin includes
@@ -198,4 +198,3 @@ with `-v` and look at the search paths it looks through.
For a list of libraries to link, look at one of the tools' CMake files (for
example [clang-check/CMakeList.txt](https://github.com/llvm/llvm-project/blob/main/clang/tools/clang-check/CMakeLists.txt)).
-
More information about the cfe-commits
mailing list