[libc-commits] [libc] [libc][docs] Add developer guide for source-level code coverage (PR #214692)

Fri Aug 7 03:13:07 PDT 2026

https://github.com/tapiwagonga created https://github.com/llvm/llvm-project/pull/214692

This patch introduces official developer documentation for configuring, extracting, and analyzing source-level code coverage for **LLVM-libc**.

### Motivation
Because LLVM-libc is built as a freestanding (`-nostdlib`) library, standard `compiler-rt` coverage workflows fail out of the box due to missing hosted standard C library file I/O (`fopen`, `fwrite`). This guide provides developers with the exact build configuration, test execution commands, and profile-merging steps required to generate accurate coverage reports across the library.

---

### Summary of Changes

1. **Added [`libc/docs/dev/code_coverage.md`](https://github.com/llvm/llvm-project/blob/main/libc/docs/dev/code_coverage.md):**
   * **Scope & Limitations:** Clarifies Linux platform support, unit-test execution boundaries (`-DLIBC_TEST_UNIT_TEST_ONLY=ON`), and report filtering (`-ignore-filename-regex=".*(test|utils).*"`).
   * **CMake Configuration:** Documents required flags (`-DLLVM_LIBC_ENABLE_COVERAGE=ON -DLIBC_TEST_UNIT_TEST_ONLY=ON`) to instrument freestanding binaries.
   * **Execution Options:** Details whole-codebase execution (`ninja -C build-cov libc-unit-tests`) alongside isolated single/targeted test execution (`libc.test.src.<subsystem>.<function>_test.__unit__`).
   * **Profile Merging & Reports:** Step-by-step instructions for `llvm-profdata merge -sparse`, multi-binary resolution, and dual reporting formats:
     * **Format A (Terminal):** High-level summary table via `llvm-cov report`.
     * **Format B (HTML):** Line-by-line interactive website via `llvm-cov show -format=html -show-branches=count`.

2. **Updated [`libc/docs/dev/index.md`](https://github.com/llvm/llvm-project/blob/main/libc/docs/dev/index.md):**
   * Registered `code_coverage` in the Sphinx `{toctree}` to make it discoverable in the navigation sidebar of `libc.llvm.org/dev/`.




>From 9a9aab8072341dcc4146a0770d9923c5ca8cd6fd Mon Sep 17 00:00:00 2001
From: Tapiwa Gonga <tapiwagonga at google.com>
Date: Thu, 6 Aug 2026 14:22:06 +0000
Subject: [PATCH] [libc][docs] Add developer guide for source-level code
 coverage

---
 libc/docs/dev/code_coverage.md | 110 +++++++++++++++++++++++++++++++++
 libc/docs/dev/index.md         |   1 +
 2 files changed, 111 insertions(+)
 create mode 100644 libc/docs/dev/code_coverage.md

diff --git a/libc/docs/dev/code_coverage.md b/libc/docs/dev/code_coverage.md
new file mode 100644
index 0000000000000..9d96d5634fc53
--- /dev/null
+++ b/libc/docs/dev/code_coverage.md
@@ -0,0 +1,110 @@
+(code_coverage)=
+
+# Source-Level Code Coverage
+
+LLVM-libc supports source-level code coverage for its unit tests.
+
+Because `llvm-libc` unit tests are built as freestanding binaries (`-nostdlib`), standard `compiler-rt` coverage workflows fail in this environment as they inherently rely on the host's standard C library (`fopen`, `fwrite`), which is intentionally omitted to prevent host contamination.
+
+To bypass this constraint, the `LibcTestMain.cpp` test harness implements a custom profiling dumper:
+
+1. It silences the default `compiler-rt` dumper by overriding the global symbol: `extern "C" char __llvm_profile_filename[] = "/dev/null";`.
+2. It hooks into `atexit()` to dump coverage before the process ends. *(Note: Death tests that terminate via `_exit()`, `abort()`, or unhandled signals inherently bypass this hook. The parent test runner still correctly dumps its overall profile).*
+3. It determines the required buffer size via `__llvm_profile_get_size_for_buffer()` and allocates memory using `LIBC_NAMESPACE::linux_syscalls::mmap`.
+4. It extracts the raw profiling data from the compiler into the memory segment using `__llvm_profile_write_buffer()`.
+5. It writes the segment to a `.profraw` file using internal Linux syscall wrappers (`open`, `write`, `close`, `munmap`).
+
+## Limitations
+
+- **OS Support:** Because this relies on Linux system call wrappers, coverage extraction is strictly gated behind `#if defined(__linux__)`. On macOS or Windows builds, the coverage dumping step is gracefully bypassed, allowing the tests to compile normally.
+
+## 1. Setup and Configuration
+
+Before running any tests, you must clear old profile data and configure your CMake build directory to generate coverage instrumentation.
+
+```bash
+# 1. Clear previous profile artifacts
+find . -name "libc_cov_*.profraw" -delete
+rm -f libc_full.profdata profraw_list.txt
+
+# 2. Configure the build directory
+cmake -G Ninja -S runtimes -B build-cov \
+  -DLLVM_ENABLE_RUNTIMES="libc" \
+  -DLLVM_LIBC_FULL_BUILD=ON \
+  -DLLVM_LIBC_ENABLE_COVERAGE=ON \
+  -DCMAKE_CXX_COMPILER=clang++-19 \
+  -DCMAKE_C_COMPILER=clang-19
+```
+
+## 2. Build and Run Tests
+
+You have three options depending on how much of the library you want to test. (Targeted testing is significantly faster for local debugging).
+
+Test targets in `llvm-libc` follow a strict naming convention based on their directory structure: `libc.test.<directory_path>.<test_name>.__unit__`.
+
+### Option A: Whole Codebase
+To run the entire unit test suite (approx. 1,776 tests) and generate a massive, whole-codebase report:
+
+```bash
+ninja -C build-cov check-libc
+
+# If using Option A, do not define FILES_TO_REPORT in Step 3.
+```
+
+### Option B: Single Target
+To instantly run coverage on a single test, specify its exact Ninja target. For example, to test `isalpha`:
+
+```bash
+# Run only the specific target
+ninja -C build-cov libc.test.src.ctype.isalpha_test.__unit__
+
+# Define the source file to filter the report in Step 3
+FILES_TO_REPORT="libc/src/ctype/isalpha.cpp"
+```
+
+### Option C: Multiple Targets
+To run multiple isolated tests simultaneously, pass them as a space-separated list. For example, testing `isalpha` and `isdigit`:
+
+```bash
+# Define your targets 
+TARGETS="libc.test.src.ctype.isalpha_test.__unit__ libc.test.src.ctype.isdigit_test.__unit__"
+
+# Run the targets
+ninja -C build-cov $TARGETS
+
+# Define the source files to filter the report in Step 3
+FILES_TO_REPORT="libc/src/ctype/isalpha.cpp libc/src/ctype/isdigit.cpp"
+```
+
+## 3. Generate the Report
+
+Once your tests have finished running, merge the raw profile data and extract the executables to map the coverage back to the source code.
+
+```bash
+# 1. Merge raw profiles
+find . -name "libc_cov_*.profraw" > profraw_list.txt
+llvm-profdata-19 merge -sparse --input-files=profraw_list.txt -o libc_full.profdata
+
+# 2. Extract executables
+EXECUTABLES=($(find build-cov -type f -executable -name "*__build__"))
+OBJECTS=("${EXECUTABLES[@]:1}")
+OBJECTS=("${OBJECTS[@]/#/-object=}")
+```
+
+### Choose Your Output Type
+
+You can specify the format of your final coverage report by changing the output command.
+
+**Option A: Terminal Summary Table (Text)**
+This provides a quick text-based summary of your coverage percentages directly in the terminal:
+```bash
+llvm-cov-19 report -instr-profile=libc_full.profdata "${EXECUTABLES[0]}" "${OBJECTS[@]}" $FILES_TO_REPORT
+```
+
+**Option B: Line-by-Line Interactive Webpage (HTML)**
+This generates an interactive HTML website so you can visually inspect exactly which lines of code are missing coverage. You can change the `OUTPUT_DIR` variable to save it wherever you prefer:
+```bash
+OUTPUT_DIR="coverage_html"
+llvm-cov-19 show -instr-profile=libc_full.profdata -format=html -output-dir=$OUTPUT_DIR "${EXECUTABLES[0]}" "${OBJECTS[@]}" $FILES_TO_REPORT
+```
+*(After running this, open `coverage_html/index.html` in your web browser).*
diff --git a/libc/docs/dev/index.md b/libc/docs/dev/index.md
index 4ce78540c8fad..be0d0018df249 100644
--- a/libc/docs/dev/index.md
+++ b/libc/docs/dev/index.md
@@ -8,6 +8,7 @@ Navigate to the links below for information on the respective topics:
 :maxdepth: 1
 
 building_docs
+code_coverage
 code_style
 source_tree_layout
 entrypoints

