[libc-commits] [libc] [libc][docs] Add developer guide for source-level code coverage (PR #214692)

[Tapiwa Gonga via libc-commits]

================
@@ -0,0 +1,311 @@
+(code_coverage)=
+
+# Code Coverage
+
+LLVM-libc supports generating statement, branch, and Modified Condition / Decision Coverage (MC/DC) reports locally. Because `llvm-libc` unit tests run in freestanding test harnesses, coverage counters and boolean decision bitmasks are captured directly and written to disk upon test completion using internal Linux kernel system calls.
+
+---
+
+## Prerequisites & Toolchain Setup
+
+Generating coverage reports requires Clang, LLVM profile tools, CMake, and Ninja:
+
+* **Compiler:** Clang 18 or later (Clang 21 or later is required for MC/DC instrumentation).
+* **LLVM Utilities:** Matching major versions of `llvm-profdata` and `llvm-cov`.
+* **Build System:** CMake 3.28+ and Ninja.
+
+### Toolchain Discovery
+
+If your Linux distribution packages version-suffixed binaries (e.g. `clang-21`, `llvm-profdata-21`), you can resolve them automatically:
+
+```bash
+CLANG_MAJOR=$(clang --version | sed -n 's/.*version \([0-9]*\).*/\1/p')
+LLVM_PROFDATA=$(which llvm-profdata-$CLANG_MAJOR 2>/dev/null || which llvm-profdata)
+LLVM_COV=$(which llvm-cov-$CLANG_MAJOR 2>/dev/null || which llvm-cov)
+```
+
+---
+
+## Cleaning Profile Counters
+
+Removes previously generated raw profile counter files (`.profraw`) and merged profile databases (`.profdata`) so that new coverage runs record clean, non-aggregated execution data:
+
+```bash
+find . -name "libc_cov_*.profraw" -delete 2>/dev/null || true
+rm -f libc_full.profdata libc_mcdc.profdata libc_single.profdata profraw_list.txt
+```
+
+---
+
+## Standard Statement & Branch Coverage
+
+Standard coverage measures physical line execution and conditional branch outcomes across all LLVM-libc entrypoints and internal support utilities.
+
+### 1. CMake Configuration
+
+Configures CMake to build LLVM-libc in overlay mode, setting `-DLIBC_ENABLE_COVERAGE=ON` to pass Clang's `-fprofile-instr-generate` and `-fcoverage-mapping` flags to the compiler:
+
+```bash
+cmake -G Ninja -S runtimes -B build-cov \
+  -DCMAKE_C_COMPILER=clang \
+  -DCMAKE_CXX_COMPILER=clang++ \
+  -DCMAKE_BUILD_TYPE=Debug \
+  -DLLVM_ENABLE_RUNTIMES="libc" \
+  -DLLVM_LIBC_FULL_BUILD=OFF \
+  -DLIBC_ENABLE_COVERAGE=ON
+```
+
+### 2. Build and Execute All Unit Tests
+
+Compiles all libc unit test executables and executes them in parallel. As each test completes, its test harness writes execution counters to a PID-specific `.profraw` file via direct Linux syscalls:
+
+```bash
+ninja -k 0 -C build-cov libc-unit-tests
+```
+
+:::{note}
+In LLVM-libc, `libc-unit-tests` builds and executes tests in a single invocation. The `-k 0` flag ensures Ninja continues executing all remaining test targets even if an individual edge-case test encounters an error. To only compile test binaries without immediately executing them, use `ninja -C build-cov libc-unit-tests-build`.
+:::
+
+### 3. Merge Profile Counters
+
+Scans the build tree for all generated `.profraw` files and indexes them into a unified, sparse `.profdata` archive using `llvm-profdata`:
+
+```bash
+find . -name "libc_cov_*.profraw" > profraw_list.txt
+llvm-profdata merge -sparse -f profraw_list.txt -o libc_full.profdata
+```
+
+### 4. Generate Coverage Reports
+
+Collects all compiled test binary paths and invokes `llvm-cov` to correlate recorded profile counters against the libc source tree:
+
+```bash
+TEST_BINS=($(find build-cov -type f -executable -name "*__build__"))
+OBJECT_FLAGS=()
+for bin in "${TEST_BINS[@]:1}"; do
+  OBJECT_FLAGS+=("-object=$bin")
+done
+```
+
+Reports can be generated in different formats:
+
+#### Option 1: Terminal Summary Report
+Prints an aggregated terminal summary showing line, region, and branch coverage percentages for each file:
+
+```bash
+llvm-cov report \
+  -instr-profile=libc_full.profdata \
+  "${TEST_BINS[0]}" "${OBJECT_FLAGS[@]}" \
+  --show-branch-summary \
+  -ignore-filename-regex=".*(test|utils).*"
+```
+
+#### Option 2: Interactive HTML Dashboard
+Generates an interactive HTML dashboard containing sortable directory metrics and syntax-highlighted source views:
+
+```bash
+llvm-cov show \
+  -format=html \
+  -output-dir=coverage_html \
+  -instr-profile=libc_full.profdata \
+  "${TEST_BINS[0]}" "${OBJECT_FLAGS[@]}" \
+  --show-directory-coverage \
+  --show-branches=count \
+  -ignore-filename-regex=".*(test|utils).*"
+
+# Open dashboard in browser
+xdg-open coverage_html/index.html
+```
+
+---
+
+## Modified Condition / Decision Coverage (MC/DC)
+
+MC/DC evaluates boolean sub-conditions within compound logical expressions (such as `if (A && B)`). It verifies that each individual sub-condition evaluates to both true and false and independently affects the outcome of the enclosing decision.
+
+### 1. CMake Configuration
+
+Configures CMake with `-fcoverage-mcdc` alongside profiling flags, enabling the compiler frontend to generate boolean condition bitmaps for compound decisions:
+
+```bash
+cmake -G Ninja -S runtimes -B build-cov-mcdc \
+  -DCMAKE_C_COMPILER=clang \
+  -DCMAKE_CXX_COMPILER=clang++ \
+  -DCMAKE_BUILD_TYPE=Debug \
+  -DLLVM_ENABLE_RUNTIMES="libc" \
+  -DLLVM_LIBC_FULL_BUILD=OFF \
+  -DLIBC_ENABLE_COVERAGE=ON \
+  -DCMAKE_C_FLAGS="-fprofile-instr-generate -fcoverage-mapping -fcoverage-mcdc" \
+  -DCMAKE_CXX_FLAGS="-fprofile-instr-generate -fcoverage-mapping -fcoverage-mcdc"
+```
+
+### 2. Build and Execute Tests
+
+Compiles and executes all unit tests with MC/DC instrumentation enabled, saving condition evaluation bitmasks into raw profile files upon completion:
+
+```bash
+ninja -k 0 -C build-cov-mcdc libc-unit-tests
----------------
tapiwagonga wrote:

I investigated the failure and filed an issue here: https://github.com/llvm/llvm-project/issues/221207

Casting `unsigned long _Fract` to integer emits an out-of-bounds shift (`lshr i32 %x, 32`) in `FixedPointBuilder`, producing undefined behavior in LLVM IR. When `10 <= static_cast<int>(max)` is evaluated in `CountlsTest.h`, the branch condition becomes undefined and tests uninitialized register state, which led to the intermittent `-4 == 0` assertion failure in `countlsulr_test`.

https://github.com/llvm/llvm-project/pull/214692