[libc-commits] [libc] [libc][utils] Add a binary size checker script (PR #233196)
Prabhu Rajasekaran via libc-commits
libc-commits at lists.llvm.org
Sat Oct 17 23:45:04 PDT 2026
https://github.com/Prabhuk created https://github.com/llvm/llvm-project/pull/233196
Add libc/utils/size_check/libc_size_check.py, a checker for the binary
size of libc entrypoint objects. It reads the symbol table
(llvm-nm -S -C) and the section sizes (llvm-size -A) of each object
listed in a manifest and evaluates the rules of a JSON rules file
against them, e.g.:
* a function must not define a given table or multi-precision helper,
* a function must not reference soft-float double precision helpers,
* the largest named data object must stay below a limit,
* a function must not have writable data.
Rules are selected by the LIBC_MATH_* flags in
LIBC_CONF_MATH_OPTIMIZATIONS and by the predefined macros of the options
each object was compiled with (e.g. hardware double precision or FMA),
so a single rules file serves every ELF target and configuration.
Symbol names are matched after removing the versioned libc namespace.
Known exceptions let a rule land before every function complies; the
checker reports exceptions that are no longer needed so the rules can be
tightened.
The checks only inspect objects and never execute target code, so they
also work for cross and baremetal builds. The unit tests run on canned
tool output and need no toolchain:
python3 libc/utils/size_check/libc_size_check_test.py
This change only adds the script; follow-up changes add the CMake
integration and the rules for the math functions.
Assisted-by: Gemini
---
<sub>Stack created with <a href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
>From ae7c738c6c1620500a3cc580a6574d508b98e0ff Mon Sep 17 00:00:00 2001
From: prabhukr <prabhukr at google.com>
Date: Sat, 17 Oct 2026 17:10:19 +0000
Subject: [PATCH] [libc][utils] Add a binary size checker script
Add libc/utils/size_check/libc_size_check.py, a checker for the binary
size of libc entrypoint objects. It reads the symbol table
(llvm-nm -S -C) and the section sizes (llvm-size -A) of each object
listed in a manifest and evaluates the rules of a JSON rules file
against them, e.g.:
* a function must not define a given table or multi-precision helper,
* a function must not reference soft-float double precision helpers,
* the largest named data object must stay below a limit,
* a function must not have writable data.
Rules are selected by the LIBC_MATH_* flags in
LIBC_CONF_MATH_OPTIMIZATIONS and by the predefined macros of the options
each object was compiled with (e.g. hardware double precision or FMA),
so a single rules file serves every ELF target and configuration.
Symbol names are matched after removing the versioned libc namespace.
Known exceptions let a rule land before every function complies; the
checker reports exceptions that are no longer needed so the rules can be
tightened.
The checks only inspect objects and never execute target code, so they
also work for cross and baremetal builds. The unit tests run on canned
tool output and need no toolchain:
python3 libc/utils/size_check/libc_size_check_test.py
This change only adds the script; follow-up changes add the CMake
integration and the rules for the math functions.
Assisted-by: Gemini
---
libc/utils/size_check/README.md | 79 ++
libc/utils/size_check/libc_size_check.py | 804 ++++++++++++++++++
libc/utils/size_check/libc_size_check_test.py | 469 ++++++++++
3 files changed, 1352 insertions(+)
create mode 100644 libc/utils/size_check/README.md
create mode 100644 libc/utils/size_check/libc_size_check.py
create mode 100644 libc/utils/size_check/libc_size_check_test.py
diff --git a/libc/utils/size_check/README.md b/libc/utils/size_check/README.md
new file mode 100644
index 00000000000000..f5fb04bf0efa1f
--- /dev/null
+++ b/libc/utils/size_check/README.md
@@ -0,0 +1,79 @@
+# libc binary size checks
+
+This directory contains a checker for the binary size of LLVM libc entrypoints.
+It catches size regressions that the functional tests cannot see, for example a
+large lookup table or the multi-precision accurate pass creeping back into a
+configuration that asked for `LIBC_MATH_SMALL_TABLES` or
+`LIBC_MATH_SKIP_ACCURATE_PASS`.
+
+The checks never execute target code. They only inspect objects with
+`llvm-nm` and `llvm-size`, so they work the same way for host, cross and
+baremetal builds.
+
+## The checker
+
+`libc_size_check.py objects` reads a manifest of entrypoint objects, runs
+`llvm-nm -S -C` and `llvm-size -A` on each object and evaluates the rules of a
+rules file against them:
+
+```
+libc_size_check.py objects --manifest <file> --rules rules/math.json \
+ --math-optimizations "<value of LIBC_CONF_MATH_OPTIMIZATIONS>" \
+ --nm <llvm-nm> --size <llvm-size> [--json <report>] [--markdown <report>] \
+ [--report-only] [--verbose]
+```
+
+Each manifest line has three tab separated fields: the entrypoint name, a file
+with the `-dM -E` output of the compiler for the options the object was built
+with (may be empty), and a `;` separated list of object files.
+
+The checker prints every rule violation and a summary line per rule, and exits
+with a non-zero status if a rule is violated (unless `--report-only` is given).
+Known exceptions and advisory notes (for example an exception that is no longer
+needed) are printed with `--verbose` and are always part of the JSON and
+markdown reports.
+
+The unit tests run on canned tool output and need no toolchain:
+
+```
+python3 libc/utils/size_check/libc_size_check_test.py
+```
+
+## Rules files
+
+A rules file is a JSON object:
+
+```json
+{
+ "version": 1,
+ "symbol_prefix_regex": "^__llvm_libc[0-9A-Za-z_]*::",
+ "regex_aliases": {"soft_double_helpers": "__aeabi_dadd|..."},
+ "rules": [ ... ]
+}
+```
+
+Symbol names are demangled, and the versioned libc namespace prefix matched by
+`symbol_prefix_regex` is removed, so rules can name
+`common_constants_internal::LOG_R` instead of
+`__llvm_libc_24_0_0_git::common_constants_internal::LOG_R`. A pattern of the
+form `@name` refers to an entry of `regex_aliases`.
+
+Each rule has an `id`, an optional `comment`, and these optional fields:
+
+| field | meaning |
+|---|---|
+| `when_flags` | `LIBC_MATH_*` flags that must all be set in `LIBC_CONF_MATH_OPTIMIZATIONS` (composite flags like `LIBC_MATH_FAST` are expanded). Otherwise the rule is inactive. |
+| `when_macros` | Conditions on the predefined macros of each object's compile options, all of which must hold. A condition is a `\|\|` separated list of `NAME` (defined and not 0) or `NAME&MASK` (value has a bit of MASK set), each optionally negated with `!`. Example: `"!__ARM_FP&0x8"` (no hardware double precision). |
+| `functions` | Regular expressions; the rule applies to the entrypoints fully matching one of them. Defaults to all entrypoints. |
+| `exclude_functions` | Regular expressions of entrypoints to skip. |
+| `exclude_variants` | `f128` skips `*f128` entrypoints, `long_double` skips `<name>l` when `<name>` is also an entrypoint. |
+| `exceptions` | Entrypoints the rule does not apply to. |
+| `forbid_defined` | Symbol patterns that must not be defined. A pattern matches trailing `::` separated components of the symbol name: `common_constants_internal::LOG_R` matches `math::common_constants_internal::LOG_R` but not `common_constants_internal::LOG_R_DD`. |
+| `forbid_defined_regex` | Regular expressions searched in the defined symbol names. |
+| `forbid_undefined` | Regular expressions that must not fully match an undefined symbol, e.g. soft-float helpers. |
+| `require_defined` | Symbol patterns that must be defined. They detect rules that silently stopped matching because a table was renamed. |
+| `max_named_data_object_bytes` | Limit for the largest named data object (table) of an object. |
+| `max_named_data_object_exceptions` | Per-entrypoint limits, either a number of bytes or `{"bytes": N, "note": "..."}`. |
+| `max_writable_data_bytes` | Limit for the size of the `.data`, `.bss`, `.tdata` and `.tbss` sections. |
+| `allow_writable_sections` | Regular expressions of writable sections not counted against the limit. |
+| `known_exceptions` | `{"<entrypoint>": {"symbols": [...], "note": "...", "when_macros": [...]}}`. Violations caused by the listed symbols are reported as known exceptions instead of failures. Use them for regressions that are already being worked on; the checker reports exceptions that no longer trigger so they can be removed. |
diff --git a/libc/utils/size_check/libc_size_check.py b/libc/utils/size_check/libc_size_check.py
new file mode 100644
index 00000000000000..9ced35863bae19
--- /dev/null
+++ b/libc/utils/size_check/libc_size_check.py
@@ -0,0 +1,804 @@
+#!/usr/bin/env python3
+#
+# ===- libc_size_check.py - Binary size checks for LLVM libc ---*- python -*--==#
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+# ==-------------------------------------------------------------------------==#
+"""Structural binary size checks for LLVM libc entrypoint objects.
+
+The checker reads the symbol table (``llvm-nm -S -C``) and the section sizes
+(``llvm-size -A``) of each entrypoint object listed in a manifest and evaluates
+the rules from a JSON rules file against them. Rules are selected by the active
+``LIBC_CONF_MATH_OPTIMIZATIONS`` flags and by the target's predefined macros, so
+the same rules file can be used for every ELF target and configuration.
+
+See README.md in this directory for the rules file format.
+"""
+
+import argparse
+import concurrent.futures
+import json
+import os
+import re
+import subprocess
+import sys
+from typing import Dict, Iterable, List, Optional, Set, Tuple
+
+# The default regular expression matching the versioned libc namespace prefix
+# of a demangled symbol, e.g. "__llvm_libc_24_0_0_git::".
+DEFAULT_SYMBOL_PREFIX_REGEX = r"^__llvm_libc[0-9A-Za-z_]*::"
+
+# Composite LIBC_MATH_* flags and the individual flags they stand for. Keep in
+# sync with libc/src/__support/macros/optimization.h.
+COMPOSITE_MATH_FLAGS = {
+ "LIBC_MATH_FAST": [
+ "LIBC_MATH_SKIP_ACCURATE_PASS",
+ "LIBC_MATH_SMALL_TABLES",
+ "LIBC_MATH_NO_ERRNO",
+ "LIBC_MATH_NO_EXCEPT",
+ "LIBC_MATH_ASSUME_ROUND_NEAREST_ONLY",
+ ],
+}
+
+# nm symbol types that denote named data objects (tables, constants, writable
+# variables). Lower case letters are local symbols.
+DATA_SYMBOL_TYPES = set("VvRrDdBbGgSsu")
+
+# Section name prefixes that hold writable data.
+WRITABLE_SECTION_PREFIXES = (".data", ".bss", ".sdata", ".sbss", ".tdata", ".tbss")
+
+STATUS_FAIL = "fail"
+STATUS_KNOWN = "known"
+STATUS_NOTE = "note"
+
+
+class Symbol:
+ """A defined symbol of an object file."""
+
+ def __init__(self, name: str, raw_name: str, sym_type: str, size: int):
+ self.name = name # Demangled, with the libc namespace prefix removed.
+ self.raw_name = raw_name # Demangled as printed by llvm-nm.
+ self.type = sym_type
+ self.size = size
+
+ def is_data(self) -> bool:
+ return self.type in DATA_SYMBOL_TYPES and not self.raw_name.startswith(".")
+
+ def __repr__(self):
+ return "Symbol(%r, %r, %d)" % (self.name, self.type, self.size)
+
+
+class ObjectInfo:
+ """Symbols and section sizes of one entrypoint object."""
+
+ def __init__(
+ self,
+ function: str,
+ paths: List[str],
+ defined: List[Symbol],
+ undefined: Set[str],
+ sections: Dict[str, int],
+ macros: Optional[Dict[str, str]] = None,
+ ):
+ self.function = function
+ self.paths = paths
+ self.defined = defined
+ self.undefined = undefined
+ self.sections = sections
+ # Predefined macros of the options the object was compiled with.
+ self.macros = macros or {}
+
+ def data_symbols(self) -> List[Symbol]:
+ return [s for s in self.defined if s.is_data()]
+
+ def largest_data_symbol(self) -> Optional[Symbol]:
+ data = self.data_symbols()
+ if not data:
+ return None
+ return max(data, key=lambda s: (s.size, s.name))
+
+ def writable_bytes(self, allow: Iterable["re.Pattern"] = ()) -> int:
+ return sum(size for _, size in self.writable_sections(allow))
+
+ def writable_sections(
+ self, allow: Iterable["re.Pattern"] = ()
+ ) -> List[Tuple[str, int]]:
+ """Returns the non-empty writable sections not matched by `allow`."""
+ allow = list(allow)
+ return sorted(
+ (name, size)
+ for name, size in self.sections.items()
+ if size
+ and name.startswith(WRITABLE_SECTION_PREFIXES)
+ and not any(a.search(name) for a in allow)
+ )
+
+ def total_bytes(self) -> int:
+ """Size of all allocatable sections (code, rodata and data)."""
+ total = 0
+ for name, size in self.sections.items():
+ if name.startswith((".text", ".rodata")) or name.startswith(
+ WRITABLE_SECTION_PREFIXES
+ ):
+ total += size
+ return total
+
+
+class Finding:
+ """The result of a rule violation, known exception or advisory note."""
+
+ def __init__(self, rule_id: str, function: str, status: str, message: str):
+ self.rule_id = rule_id
+ self.function = function
+ self.status = status
+ self.message = message
+
+ def to_json(self) -> dict:
+ return {
+ "rule": self.rule_id,
+ "function": self.function,
+ "status": self.status,
+ "message": self.message,
+ }
+
+ def __repr__(self):
+ return "Finding(%r, %r, %r, %r)" % (
+ self.rule_id,
+ self.function,
+ self.status,
+ self.message,
+ )
+
+
+# ===----------------------------------------------------------------------===#
+# Parsing of tool output.
+# ===----------------------------------------------------------------------===#
+
+_NM_SIZED_RE = re.compile(r"^([0-9a-fA-F]+)\s+([0-9a-fA-F]+)\s+(\S)\s+(.+)$")
+_NM_UNSIZED_RE = re.compile(r"^([0-9a-fA-F]+)\s+(\S)\s+(.+)$")
+_NM_UNDEFINED_RE = re.compile(r"^\s*([Uwv])\s+(.+)$")
+
+
+def strip_prefix(name: str, prefix_re: "re.Pattern") -> str:
+ return prefix_re.sub("", name, count=1)
+
+
+def parse_nm(text: str, prefix_re: "re.Pattern") -> Tuple[List[Symbol], Set[str]]:
+ """Parses ``llvm-nm -S -C`` output into defined symbols and undefined names.
+
+ Lines that name the input file (as printed for archives or multiple inputs)
+ and empty lines are ignored.
+ """
+ defined = []
+ undefined = set()
+ for line in text.splitlines():
+ if not line.strip() or line.rstrip().endswith(":"):
+ continue
+ m = _NM_UNDEFINED_RE.match(line)
+ if m:
+ undefined.add(m.group(2).strip())
+ continue
+ m = _NM_SIZED_RE.match(line)
+ if m:
+ raw = m.group(4).strip()
+ defined.append(
+ Symbol(
+ strip_prefix(raw, prefix_re), raw, m.group(3), int(m.group(2), 16)
+ )
+ )
+ continue
+ m = _NM_UNSIZED_RE.match(line)
+ if m:
+ raw = m.group(3).strip()
+ defined.append(Symbol(strip_prefix(raw, prefix_re), raw, m.group(2), 0))
+ return defined, undefined
+
+
+def parse_size_sysv(text: str) -> Dict[str, int]:
+ """Parses ``llvm-size -A`` output into a map of section name to size.
+
+ Sizes of sections with the same name (e.g. COMDAT groups) are summed.
+ """
+ sections: Dict[str, int] = {}
+ for line in text.splitlines():
+ parts = line.split()
+ if len(parts) != 3 or not parts[0].startswith("."):
+ continue
+ try:
+ size = int(parts[1], 0)
+ except ValueError:
+ continue
+ sections[parts[0]] = sections.get(parts[0], 0) + size
+ return sections
+
+
+def parse_math_flags(value: str) -> Set[str]:
+ """Returns the set of LIBC_MATH_* flags named in a configuration value.
+
+ ``value`` is the value of LIBC_CONF_MATH_OPTIMIZATIONS, for example
+ ``(LIBC_MATH_SKIP_ACCURATE_PASS | LIBC_MATH_SMALL_TABLES)``. Composite flags
+ such as LIBC_MATH_FAST are expanded.
+ """
+ flags = set(re.findall(r"LIBC_MATH_[A-Z0-9_]+", value or ""))
+ for composite, parts in COMPOSITE_MATH_FLAGS.items():
+ if composite in flags:
+ flags.update(parts)
+ return flags
+
+
+def parse_macros(text: str) -> Dict[str, str]:
+ """Parses ``-dM -E`` output into a map of macro name to value."""
+ macros = {}
+ for line in text.splitlines():
+ m = re.match(r"^#define\s+([A-Za-z_][A-Za-z0-9_]*)(\([^)]*\))?\s*(.*)$", line)
+ if m and not m.group(2):
+ macros[m.group(1)] = m.group(3).strip()
+ return macros
+
+
+class ManifestEntry:
+ def __init__(self, function: str, macros_file: str, objects: List[str]):
+ self.function = function
+ self.macros_file = macros_file
+ self.objects = objects
+
+
+def parse_manifest(text: str) -> List[ManifestEntry]:
+ """Parses a manifest of tab separated lines.
+
+ Each line is ``<function>\\t<macros file>\\t<object>[;<object>...]``. The
+ macros file holds the ``-dM -E`` output for the options the object was
+ compiled with and may be empty.
+ """
+ entries = []
+ for line in text.splitlines():
+ if not line.strip() or line.startswith("#"):
+ continue
+ fields = line.rstrip("\r\n").split("\t")
+ if len(fields) != 3:
+ raise ValueError("malformed manifest line: %r" % line)
+ objs = [p for p in fields[2].split(";") if p]
+ if objs:
+ entries.append(ManifestEntry(fields[0], fields[1], objs))
+ return entries
+
+
+# ===----------------------------------------------------------------------===#
+# Rule evaluation.
+# ===----------------------------------------------------------------------===#
+
+
+def _macro_int(value: str) -> Optional[int]:
+ value = value.strip().rstrip("uUlL")
+ try:
+ return int(value, 0)
+ except ValueError:
+ return None
+
+
+def eval_macro_condition(cond: str, macros: Dict[str, str]) -> bool:
+ """Evaluates a macro condition.
+
+ A condition is a ``||`` separated list of alternatives. Each alternative is
+ ``NAME`` (defined and not 0), ``NAME&MASK`` (defined and value & MASK is
+ non-zero), optionally negated with a leading ``!``.
+ """
+ for alt in cond.split("||"):
+ alt = alt.strip()
+ negate = alt.startswith("!")
+ if negate:
+ alt = alt[1:].strip()
+ name, _, mask = alt.partition("&")
+ name = name.strip()
+ result = False
+ if name in macros:
+ value = _macro_int(macros[name])
+ if mask:
+ result = value is not None and (value & int(mask.strip(), 0)) != 0
+ else:
+ result = value is None or value != 0
+ if result != negate:
+ return True
+ return False
+
+
+class RuleContext:
+ def __init__(
+ self,
+ flags: Set[str],
+ functions: Set[str],
+ aliases: Optional[Dict[str, str]] = None,
+ ):
+ self.flags = flags
+ self.functions = functions
+ self.aliases = aliases or {}
+
+
+def rule_is_active(rule: dict, ctx: RuleContext) -> bool:
+ """Returns whether the configuration flags select the rule."""
+ for flag in rule.get("when_flags", []):
+ if flag not in ctx.flags:
+ return False
+ return True
+
+
+def rule_macros_match(rule: dict, macros: Dict[str, str]) -> bool:
+ """Returns whether an object's compile macros select the rule."""
+ for cond in rule.get("when_macros", []):
+ if not eval_macro_condition(cond, macros):
+ return False
+ return True
+
+
+def _is_variant(function: str, variant: str, functions: Set[str]) -> bool:
+ if variant == "f128":
+ return function.endswith("f128")
+ if variant == "long_double":
+ return function.endswith("l") and function[:-1] in functions
+ raise ValueError("unknown function variant '%s'" % variant)
+
+
+def rule_applies_to(rule: dict, function: str, ctx: RuleContext) -> bool:
+ patterns = rule.get("functions", [".*"])
+ if not any(re.fullmatch(p, function) for p in patterns):
+ return False
+ if any(re.fullmatch(p, function) for p in rule.get("exclude_functions", [])):
+ return False
+ for variant in rule.get("exclude_variants", []):
+ if _is_variant(function, variant, ctx.functions):
+ return False
+ return True
+
+
+def _expand(pattern: str, ctx: RuleContext) -> str:
+ if pattern.startswith("@"):
+ if pattern[1:] not in ctx.aliases:
+ raise ValueError("unknown regex alias '%s'" % pattern)
+ return ctx.aliases[pattern[1:]]
+ return pattern
+
+
+def compile_symbol_pattern(pattern: str) -> "re.Pattern":
+ """Compiles a namespace-anchored symbol pattern.
+
+ ``pattern`` must match a trailing sequence of ``::``-separated components of
+ the (prefix stripped) symbol name, e.g. ``common_constants_internal::LOG_R``
+ matches ``math::common_constants_internal::LOG_R`` but neither
+ ``common_constants_internal::LOG_R_DD`` nor ``xcommon_constants_internal::LOG_R``.
+ """
+ return re.compile(r"(?:^|::)(?:%s)$" % pattern)
+
+
+def _known_exception(rule: dict, obj: ObjectInfo) -> dict:
+ """Returns the known exception entry of the rule for the object, if any.
+
+ An entry only applies when its optional ``when_macros`` conditions hold for
+ the object.
+ """
+ entry = rule.get("known_exceptions", {}).get(obj.function)
+ if entry is None:
+ return {}
+ for cond in entry.get("when_macros", []):
+ if not eval_macro_condition(cond, obj.macros):
+ return {}
+ return entry
+
+
+def _max_object_limit(rule: dict, function: str) -> Tuple[int, bool]:
+ """Returns the named data object limit for `function` and whether it is an
+ exception. Exception values are either a byte count or an object with a
+ "bytes" field (and an optional "note")."""
+ value = rule.get("max_named_data_object_exceptions", {}).get(function)
+ if value is None:
+ return rule["max_named_data_object_bytes"], False
+ if isinstance(value, dict):
+ value = value["bytes"]
+ return value, True
+
+
+def evaluate_rule(rule: dict, obj: ObjectInfo, ctx: RuleContext) -> List[Finding]:
+ """Evaluates one active rule against one object it applies to."""
+ rid = rule["id"]
+ fn = obj.function
+ findings: List[Finding] = []
+ where = " in %s" % ", ".join(obj.paths)
+ known_entry = _known_exception(rule, obj)
+ known_patterns = known_entry.get("symbols", [])
+ known = [compile_symbol_pattern(_expand(p, ctx)) for p in known_patterns]
+ known_used = set()
+
+ def violation(message: str, symbol: Optional[str] = None):
+ if symbol is not None:
+ for i, k in enumerate(known):
+ if k.search(symbol):
+ known_used.add(i)
+ note = known_entry.get("note", "")
+ findings.append(
+ Finding(
+ rid,
+ fn,
+ STATUS_KNOWN,
+ message + (" (known exception: %s)" % note if note else ""),
+ )
+ )
+ return
+ findings.append(Finding(rid, fn, STATUS_FAIL, message + where))
+
+ for pattern in rule.get("forbid_defined", []):
+ regex = compile_symbol_pattern(_expand(pattern, ctx))
+ for sym in obj.defined:
+ if regex.search(sym.name):
+ violation(
+ "defines forbidden symbol '%s' (%d bytes)" % (sym.name, sym.size),
+ sym.name,
+ )
+
+ for pattern in rule.get("forbid_defined_regex", []):
+ regex = re.compile(_expand(pattern, ctx))
+ for sym in obj.defined:
+ if regex.search(sym.name):
+ violation(
+ "defines forbidden symbol '%s' (matches '%s')"
+ % (sym.name, pattern),
+ sym.name,
+ )
+
+ for pattern in rule.get("forbid_undefined", []):
+ regex = re.compile(_expand(pattern, ctx))
+ hits = sorted(u for u in obj.undefined if regex.fullmatch(u))
+ for u in hits:
+ violation("references forbidden helper '%s'" % u, u)
+
+ for pattern in rule.get("require_defined", []):
+ regex = compile_symbol_pattern(_expand(pattern, ctx))
+ if not any(regex.search(s.name) for s in obj.defined):
+ findings.append(
+ Finding(
+ rid,
+ fn,
+ STATUS_FAIL,
+ "does not define a symbol matching '%s'; the rule may be stale"
+ % pattern
+ + where,
+ )
+ )
+
+ if "max_named_data_object_bytes" in rule:
+ limit, is_exception = _max_object_limit(rule, fn)
+ largest = obj.largest_data_symbol()
+ size = largest.size if largest else 0
+ if size > limit:
+ findings.append(
+ Finding(
+ rid,
+ fn,
+ STATUS_FAIL,
+ "largest named data object '%s' is %d bytes, limit is %d bytes"
+ % (largest.name, size, limit)
+ + where,
+ )
+ )
+ elif is_exception and size < limit:
+ findings.append(
+ Finding(
+ rid,
+ fn,
+ STATUS_NOTE,
+ "exception of %d bytes is larger than needed for this build "
+ "(largest named data object: %d bytes)" % (limit, size),
+ )
+ )
+
+ if "max_writable_data_bytes" in rule:
+ limit = rule["max_writable_data_bytes"]
+ allow = [re.compile(p) for p in rule.get("allow_writable_sections", [])]
+ sections = obj.writable_sections(allow)
+ size = sum(s for _, s in sections)
+ if size > limit:
+ violation(
+ "has %d bytes of writable data, limit is %d bytes: %s"
+ % (size, limit, ", ".join("%s (%d)" % s for s in sections))
+ )
+
+ for i, pattern in enumerate(known_patterns):
+ if i not in known_used:
+ findings.append(
+ Finding(
+ rid,
+ fn,
+ STATUS_NOTE,
+ "known exception '%s' no longer triggers; remove it from the rules"
+ % pattern,
+ )
+ )
+ return findings
+
+
+def evaluate(rules_doc: dict, objects: List[ObjectInfo], ctx: RuleContext):
+ """Evaluates all rules. Returns (findings, summary)."""
+ findings: List[Finding] = []
+ summary = []
+ for rule in rules_doc.get("rules", []):
+ rid = rule["id"]
+ if not rule_is_active(rule, ctx):
+ summary.append({"rule": rid, "active": False, "checked": 0})
+ continue
+ exempt = set(rule.get("exceptions", []))
+ checked = 0
+ skipped = 0
+ for obj in objects:
+ if not rule_applies_to(rule, obj.function, ctx):
+ continue
+ if obj.function in exempt:
+ continue
+ if not rule_macros_match(rule, obj.macros):
+ skipped += 1
+ continue
+ checked += 1
+ findings.extend(evaluate_rule(rule, obj, ctx))
+ summary.append(
+ {"rule": rid, "active": True, "checked": checked, "target_skipped": skipped}
+ )
+ return findings, summary
+
+
+# ===----------------------------------------------------------------------===#
+# Driver.
+# ===----------------------------------------------------------------------===#
+
+
+def run_tool(cmd: List[str]) -> str:
+ result = subprocess.run(
+ cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True
+ )
+ if result.returncode != 0:
+ raise RuntimeError(
+ "command failed (%d): %s\n%s"
+ % (result.returncode, " ".join(cmd), result.stderr)
+ )
+ return result.stdout
+
+
+def load_object(
+ entry: ManifestEntry,
+ nm: str,
+ size: str,
+ prefix_re: "re.Pattern",
+ macros: Dict[str, str],
+) -> ObjectInfo:
+ defined: List[Symbol] = []
+ undefined: Set[str] = set()
+ sections: Dict[str, int] = {}
+ for path in entry.objects:
+ d, u = parse_nm(run_tool([nm, "-S", "-C", path]), prefix_re)
+ defined.extend(d)
+ undefined.update(u)
+ for name, sz in parse_size_sysv(run_tool([size, "-A", path])).items():
+ sections[name] = sections.get(name, 0) + sz
+ # A symbol defined by one of the objects is not an external reference.
+ undefined -= {s.raw_name for s in defined}
+ return ObjectInfo(
+ entry.function, entry.objects, defined, undefined, sections, macros
+ )
+
+
+def load_objects(
+ manifest: List[ManifestEntry], nm: str, size: str, prefix_re, jobs: int
+) -> List[ObjectInfo]:
+ macro_cache: Dict[str, Dict[str, str]] = {}
+ for entry in manifest:
+ if entry.macros_file and entry.macros_file not in macro_cache:
+ macro_cache[entry.macros_file] = parse_macros(read_file(entry.macros_file))
+ with concurrent.futures.ThreadPoolExecutor(max_workers=jobs) as pool:
+ futures = [
+ pool.submit(
+ load_object,
+ entry,
+ nm,
+ size,
+ prefix_re,
+ macro_cache.get(entry.macros_file, {}),
+ )
+ for entry in manifest
+ ]
+ return [f.result() for f in futures]
+
+
+def _count(findings: List[Finding], rule_id: str, status: str) -> int:
+ return sum(1 for f in findings if f.rule_id == rule_id and f.status == status)
+
+
+def format_text(findings: List[Finding], summary: List[dict], verbose=False) -> str:
+ """Formats the findings for the terminal.
+
+ Failures are always listed; known exceptions and notes only when `verbose`.
+ """
+ lines = []
+ for f in findings:
+ if f.status == STATUS_FAIL or verbose:
+ lines.append(
+ "%s: [%s] %s: %s" % (f.status.upper(), f.rule_id, f.function, f.message)
+ )
+ active = [s for s in summary if s["active"]]
+ inactive = [s["rule"] for s in summary if not s["active"]]
+ for s in active:
+ rid = s["rule"]
+ lines.append(
+ "rule %-36s checked %4d objects: %d failures, %d known exceptions, "
+ "%d notes%s"
+ % (
+ rid,
+ s["checked"],
+ _count(findings, rid, STATUS_FAIL),
+ _count(findings, rid, STATUS_KNOWN),
+ _count(findings, rid, STATUS_NOTE),
+ (
+ " (%d skipped by target macros)" % s["target_skipped"]
+ if s.get("target_skipped")
+ else ""
+ ),
+ )
+ )
+ if inactive:
+ lines.append("inactive rules (flags not set): %s" % ", ".join(inactive))
+ return "\n".join(lines)
+
+
+def format_markdown(
+ findings: List[Finding], summary: List[dict], objects: List[ObjectInfo]
+) -> str:
+ out = ["# libc size check report", ""]
+ out.append("| rule | active | checked | failures | known | notes |")
+ out.append("|---|---|---:|---:|---:|---:|")
+ for s in summary:
+ rid = s["rule"]
+ out.append(
+ "| %s | %s | %d | %d | %d | %d |"
+ % (
+ rid,
+ "yes" if s["active"] else "no",
+ s["checked"],
+ _count(findings, rid, STATUS_FAIL),
+ _count(findings, rid, STATUS_KNOWN),
+ _count(findings, rid, STATUS_NOTE),
+ )
+ )
+ if findings:
+ out += ["", "## Findings", "", "| status | rule | function | message |"]
+ out.append("|---|---|---|---|")
+ for f in findings:
+ out.append(
+ "| %s | %s | %s | %s |"
+ % (f.status, f.rule_id, f.function, f.message.replace("|", "\\|"))
+ )
+ out += ["", "## Objects", "", "| function | bytes | largest data object | bytes |"]
+ out.append("|---|---:|---|---:|")
+ for obj in sorted(objects, key=lambda o: o.function):
+ largest = obj.largest_data_symbol()
+ out.append(
+ "| %s | %d | %s | %d |"
+ % (
+ obj.function,
+ obj.total_bytes(),
+ largest.name if largest else "",
+ largest.size if largest else 0,
+ )
+ )
+ return "\n".join(out) + "\n"
+
+
+def objects_report_json(objects: List[ObjectInfo]) -> dict:
+ result = {}
+ for obj in objects:
+ largest = obj.largest_data_symbol()
+ result[obj.function] = {
+ "bytes": obj.total_bytes(),
+ "writable_bytes": obj.writable_bytes(),
+ "named_data_bytes": sum(s.size for s in obj.data_symbols()),
+ "largest_data_object": largest.name if largest else None,
+ "largest_data_object_bytes": largest.size if largest else 0,
+ }
+ return result
+
+
+def read_file(path: str) -> str:
+ with open(path, encoding="utf-8") as f:
+ return f.read()
+
+
+def write_file(path: str, content: str):
+ os.makedirs(os.path.dirname(os.path.abspath(path)), exist_ok=True)
+ with open(path, "w", encoding="utf-8", newline="\n") as f:
+ f.write(content)
+
+
+def cmd_objects(args) -> int:
+ rules_doc = json.loads(read_file(args.rules))
+ prefix_re = re.compile(
+ rules_doc.get("symbol_prefix_regex", DEFAULT_SYMBOL_PREFIX_REGEX)
+ )
+ manifest = parse_manifest(read_file(args.manifest))
+ flags = parse_math_flags(args.math_optimizations)
+ ctx = RuleContext(
+ flags, {e.function for e in manifest}, rules_doc.get("regex_aliases", {})
+ )
+ objects = load_objects(manifest, args.nm, args.size, prefix_re, args.jobs)
+ findings, summary = evaluate(rules_doc, objects, ctx)
+
+ print(
+ "libc size check: %d objects, LIBC_MATH flags: %s"
+ % (len(objects), " ".join(sorted(flags)) or "(none)")
+ )
+ print(format_text(findings, summary, args.verbose))
+ if args.json:
+ write_file(
+ args.json,
+ json.dumps(
+ {
+ "math_flags": sorted(flags),
+ "summary": summary,
+ "findings": [f.to_json() for f in findings],
+ "objects": objects_report_json(objects),
+ },
+ indent=2,
+ sort_keys=True,
+ )
+ + "\n",
+ )
+ if args.markdown:
+ write_file(args.markdown, format_markdown(findings, summary, objects))
+
+ failures = [f for f in findings if f.status == STATUS_FAIL]
+ if failures:
+ print(
+ "libc size check: %d rule violation(s)%s. See %s for the rules."
+ % (
+ len(failures),
+ " (report only, not failing)" if args.report_only else "",
+ args.rules,
+ )
+ )
+ return 0 if args.report_only else 1
+ print("libc size check: all rules passed.")
+ return 0
+
+
+def build_parser() -> argparse.ArgumentParser:
+ parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+ sub = parser.add_subparsers(dest="command")
+ sub.required = True
+
+ p = sub.add_parser("objects", help="check entrypoint objects against rules")
+ p.add_argument("--manifest", required=True, help="see parse_manifest()")
+ p.add_argument("--rules", required=True, help="rules JSON file")
+ p.add_argument(
+ "--math-optimizations",
+ default="",
+ help="value of LIBC_CONF_MATH_OPTIMIZATIONS",
+ )
+ p.add_argument("--nm", default="llvm-nm", help="path to llvm-nm")
+ p.add_argument("--size", default="llvm-size", help="path to llvm-size")
+ p.add_argument("--json", help="write a JSON report to this file")
+ p.add_argument("--markdown", help="write a markdown report to this file")
+ p.add_argument("--report-only", action="store_true", help="never fail")
+ p.add_argument(
+ "-v",
+ "--verbose",
+ action="store_true",
+ help="also list known exceptions and notes",
+ )
+ p.add_argument("-j", "--jobs", type=int, default=os.cpu_count() or 1)
+ p.set_defaults(func=cmd_objects)
+ return parser
+
+
+def main(argv=None) -> int:
+ args = build_parser().parse_args(argv)
+ return args.func(args)
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/libc/utils/size_check/libc_size_check_test.py b/libc/utils/size_check/libc_size_check_test.py
new file mode 100644
index 00000000000000..416e47f65de5a3
--- /dev/null
+++ b/libc/utils/size_check/libc_size_check_test.py
@@ -0,0 +1,469 @@
+#!/usr/bin/env python3
+#
+# ===- libc_size_check_test.py - Tests for libc_size_check.py --*- python -*--==#
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+# ==-------------------------------------------------------------------------==#
+"""Unit tests for libc_size_check.py on canned tool output.
+
+No toolchain is needed: run with `python3 libc_size_check_test.py`.
+"""
+
+import os
+import re
+import sys
+import unittest
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+import libc_size_check as lsc
+
+PREFIX_RE = re.compile(lsc.DEFAULT_SYMBOL_PREFIX_REGEX)
+
+# `llvm-nm -S -C` output of a logf object for an ARM soft-float target.
+NM_LOGF_ARM = """\
+00000000 00000468 T logf
+00000000 00000080 V __llvm_libc_24_0_0_git::common_constants_internal::LOG_R
+00000000 00000004 r __llvm_libc_24_0_0_git::math::logf()::SMALL
+ U __aeabi_dadd
+ U __aeabi_f2d
+ U __aeabi_fmul
+ w __llvm_libc_24_0_0_git::weak_hook()
+00000000 T __llvm_libc_24_0_0_git::logf(float)
+"""
+
+# `llvm-nm -S -C` output of a pow object for x86-64 in the default config.
+NM_POW_X86 = """\
+0000000000000000 0000000000000a2c T pow
+0000000000000000 0000000000001000 V __llvm_libc_24_0_0_git::pow_internal::LOG2_RD_FRAC256
+0000000000000000 0000000000000120 W __llvm_libc_24_0_0_git::fputil::DyadicFloat<128ul>::operator*(__llvm_libc_24_0_0_git::fputil::DyadicFloat<128ul> const&) const
+0000000000000000 0000000000000004 d __llvm_libc_24_0_0_git::fputil::generic::quick_get_round()::x
+ U __llvm_libc_errno
+"""
+
+# `llvm-size -A` output.
+SIZE_POW_X86 = """\
+pow.cpp.o :
+section size addr
+.text 0 0
+.text.pow 2604 0
+.rodata._ZN22__llvm_libc_24_0_0_git12pow_internal15LOG2_RD_FRAC256E 4096 0
+.rodata.cst8 48 0
+.rodata.cst8 16 0
+.data._ZZN22__llvm_libc_24_0_0_git6fputil7generic15quick_get_roundEvE1x 4 0
+.bss 0 0
+.comment 102 0
+.note.GNU-stack 0 0
+.llvm_addrsig 4 0
+Total 6874
+"""
+
+MACROS_ARMV7EM = """\
+#define __arm__ 1
+#define __ARM_ARCH 7
+#define __ARM_FP 0x4
+#define __ARM_ARCH_PROFILE 'M'
+#define __SIZEOF_DOUBLE__ 8
+#define __GNUC_PREREQ(a, b) 1
+"""
+
+MACROS_ARMV81M = """\
+#define __arm__ 1
+#define __ARM_FP 0xe
+#define __ARM_FEATURE_FMA 1
+"""
+
+
+def make_obj(function, nm_text, size_text="", macros=None):
+ defined, undefined = lsc.parse_nm(nm_text, PREFIX_RE)
+ return lsc.ObjectInfo(
+ function,
+ ["%s.cpp.o" % function],
+ defined,
+ undefined - {s.raw_name for s in defined},
+ lsc.parse_size_sysv(size_text),
+ macros,
+ )
+
+
+def statuses(findings, status=lsc.STATUS_FAIL):
+ return [(f.rule_id, f.function) for f in findings if f.status == status]
+
+
+class ParseTest(unittest.TestCase):
+ def test_parse_nm(self):
+ defined, undefined = lsc.parse_nm(NM_LOGF_ARM, PREFIX_RE)
+ names = {s.name: s for s in defined}
+ self.assertIn("common_constants_internal::LOG_R", names)
+ self.assertEqual(names["common_constants_internal::LOG_R"].size, 128)
+ self.assertEqual(names["common_constants_internal::LOG_R"].type, "V")
+ self.assertEqual(names["math::logf()::SMALL"].size, 4)
+ # Symbols without a size are still recorded.
+ self.assertEqual(names["logf(float)"].size, 0)
+ self.assertEqual(
+ undefined,
+ {
+ "__aeabi_dadd",
+ "__aeabi_f2d",
+ "__aeabi_fmul",
+ "__llvm_libc_24_0_0_git::weak_hook()",
+ },
+ )
+
+ def test_parse_nm_names_with_spaces(self):
+ defined, _ = lsc.parse_nm(NM_POW_X86, PREFIX_RE)
+ names = [s.name for s in defined]
+ self.assertIn(
+ "fputil::DyadicFloat<128ul>::operator*(__llvm_libc_24_0_0_git::fputil::"
+ "DyadicFloat<128ul> const&) const",
+ names,
+ )
+
+ def test_parse_nm_ignores_file_headers(self):
+ defined, undefined = lsc.parse_nm(
+ "\nfoo.o:\n00000000 00000004 R x\n", PREFIX_RE
+ )
+ self.assertEqual([s.name for s in defined], ["x"])
+ self.assertEqual(undefined, set())
+
+ def test_parse_size(self):
+ sections = lsc.parse_size_sysv(SIZE_POW_X86)
+ self.assertEqual(sections[".text.pow"], 2604)
+ self.assertEqual(sections[".rodata.cst8"], 64) # Summed.
+ self.assertNotIn("Total", sections)
+ obj = make_obj("pow", NM_POW_X86, SIZE_POW_X86)
+ self.assertEqual(obj.writable_bytes(), 4)
+ self.assertEqual(obj.total_bytes(), 2604 + 4096 + 64 + 4)
+
+ def test_largest_data_symbol(self):
+ obj = make_obj("pow", NM_POW_X86)
+ largest = obj.largest_data_symbol()
+ self.assertEqual(largest.name, "pow_internal::LOG2_RD_FRAC256")
+ self.assertEqual(largest.size, 4096)
+ # Weak functions (W) are not data.
+ self.assertEqual(len(obj.data_symbols()), 2)
+
+ def test_parse_math_flags(self):
+ self.assertEqual(lsc.parse_math_flags(""), set())
+ self.assertEqual(
+ lsc.parse_math_flags(
+ "(LIBC_MATH_SKIP_ACCURATE_PASS | LIBC_MATH_SMALL_TABLES)"
+ ),
+ {"LIBC_MATH_SKIP_ACCURATE_PASS", "LIBC_MATH_SMALL_TABLES"},
+ )
+ fast = lsc.parse_math_flags("LIBC_MATH_FAST")
+ self.assertIn("LIBC_MATH_SMALL_TABLES", fast)
+ self.assertIn("LIBC_MATH_ASSUME_ROUND_NEAREST_ONLY", fast)
+ self.assertNotIn("LIBC_MATH_INTERMEDIATE_COMP_IN_FLOAT", fast)
+
+ def test_parse_macros(self):
+ macros = lsc.parse_macros(MACROS_ARMV7EM)
+ self.assertEqual(macros["__ARM_FP"], "0x4")
+ self.assertEqual(macros["__ARM_ARCH_PROFILE"], "'M'")
+ self.assertNotIn("__GNUC_PREREQ", macros) # Function-like macro.
+
+ def test_parse_manifest(self):
+ entries = lsc.parse_manifest(
+ "# comment\nlogf\t/m/a.txt\t/o/logf.o\n"
+ "powf\t\t/o/powf1.o;/o/powf2.o\nempty\t/m/a.txt\t\n"
+ )
+ self.assertEqual([e.function for e in entries], ["logf", "powf"])
+ self.assertEqual(entries[0].macros_file, "/m/a.txt")
+ self.assertEqual(entries[1].objects, ["/o/powf1.o", "/o/powf2.o"])
+ with self.assertRaises(ValueError):
+ lsc.parse_manifest("logf /o/logf.o\n")
+
+
+class MacroConditionTest(unittest.TestCase):
+ def setUp(self):
+ self.v7em = lsc.parse_macros(MACROS_ARMV7EM)
+ self.v81m = lsc.parse_macros(MACROS_ARMV81M)
+
+ def test_defined(self):
+ self.assertTrue(lsc.eval_macro_condition("__arm__", self.v7em))
+ self.assertFalse(lsc.eval_macro_condition("__x86_64__", self.v7em))
+ self.assertTrue(lsc.eval_macro_condition("!__x86_64__", self.v7em))
+ # Non-integer values count as defined.
+ self.assertTrue(lsc.eval_macro_condition("__ARM_ARCH_PROFILE", self.v7em))
+
+ def test_zero_is_false(self):
+ self.assertFalse(lsc.eval_macro_condition("ZERO", {"ZERO": "0"}))
+
+ def test_mask(self):
+ # __ARM_FP bit 3 (0x8) means hardware double precision.
+ self.assertFalse(lsc.eval_macro_condition("__ARM_FP&0x8", self.v7em))
+ self.assertTrue(lsc.eval_macro_condition("__ARM_FP&0x8", self.v81m))
+ self.assertTrue(lsc.eval_macro_condition("!__ARM_FP&0x8", self.v7em))
+ self.assertTrue(lsc.eval_macro_condition("!__ARM_FP&0x8", {}))
+
+ def test_or(self):
+ cond = "__FMA__ || __ARM_FEATURE_FMA"
+ self.assertFalse(lsc.eval_macro_condition(cond, self.v7em))
+ self.assertTrue(lsc.eval_macro_condition(cond, self.v81m))
+ self.assertTrue(lsc.eval_macro_condition(cond, {"__FMA__": "1"}))
+
+
+class SymbolPatternTest(unittest.TestCase):
+ def test_namespace_anchored(self):
+ p = lsc.compile_symbol_pattern("common_constants_internal::LOG_R")
+ self.assertTrue(p.search("common_constants_internal::LOG_R"))
+ self.assertTrue(p.search("math::common_constants_internal::LOG_R"))
+ self.assertFalse(p.search("common_constants_internal::LOG_R_DD"))
+ self.assertFalse(p.search("xcommon_constants_internal::LOG_R"))
+
+ def test_alternation(self):
+ p = lsc.compile_symbol_pattern("EXP2_MID1|EXP2_MID2")
+ self.assertTrue(p.search("EXP2_MID1"))
+ self.assertTrue(p.search("math::EXP2_MID2"))
+ self.assertFalse(p.search("EXP2_MID"))
+
+
+class RuleTest(unittest.TestCase):
+ ALIASES = {"soft_double": "__aeabi_(dadd|dmul|f2d|d2f)"}
+
+ def evaluate(self, rules, objects, flags=()):
+ ctx = lsc.RuleContext(set(flags), {o.function for o in objects}, self.ALIASES)
+ return lsc.evaluate({"rules": rules}, objects, ctx)
+
+ def test_when_flags(self):
+ rule = {"id": "r", "when_flags": ["LIBC_MATH_SMALL_TABLES"]}
+ rule["forbid_defined"] = ["common_constants_internal::LOG_R"]
+ obj = make_obj("logf", NM_LOGF_ARM)
+ findings, summary = self.evaluate([rule], [obj])
+ self.assertEqual(findings, [])
+ self.assertFalse(summary[0]["active"])
+ findings, summary = self.evaluate([rule], [obj], ["LIBC_MATH_SMALL_TABLES"])
+ self.assertEqual(statuses(findings), [("r", "logf")])
+ self.assertIn("common_constants_internal::LOG_R", findings[0].message)
+ self.assertIn("128 bytes", findings[0].message)
+ self.assertIn("logf.cpp.o", findings[0].message)
+
+ def test_when_macros_per_object(self):
+ rule = {
+ "id": "r",
+ "when_macros": ["!__ARM_FP&0x8"],
+ "forbid_undefined": ["@soft_double"],
+ }
+ v7em = make_obj("logf", NM_LOGF_ARM, macros=lsc.parse_macros(MACROS_ARMV7EM))
+ findings, summary = self.evaluate([rule], [v7em])
+ self.assertEqual(
+ sorted(f.message.split("'")[1] for f in findings),
+ ["__aeabi_dadd", "__aeabi_f2d"],
+ )
+ v81m = make_obj("logf", NM_LOGF_ARM, macros=lsc.parse_macros(MACROS_ARMV81M))
+ findings, summary = self.evaluate([rule], [v81m])
+ self.assertEqual(findings, [])
+ self.assertEqual(summary[0]["checked"], 0)
+ self.assertEqual(summary[0]["target_skipped"], 1)
+
+ def test_functions_and_exceptions(self):
+ rule = {
+ "id": "r",
+ "functions": ["log.*"],
+ "exceptions": ["log2f"],
+ "forbid_defined": ["common_constants_internal::LOG_R"],
+ }
+ objs = [
+ make_obj("logf", NM_LOGF_ARM),
+ make_obj("log2f", NM_LOGF_ARM),
+ make_obj("expf", NM_LOGF_ARM),
+ ]
+ findings, summary = self.evaluate([rule], objs)
+ self.assertEqual(statuses(findings), [("r", "logf")])
+ self.assertEqual(summary[0]["checked"], 1)
+
+ def test_exclude_variants(self):
+ rule = {
+ "id": "r",
+ "exclude_variants": ["f128", "long_double"],
+ "forbid_defined_regex": ["DyadicFloat<128"],
+ }
+ objs = [
+ make_obj("pow", NM_POW_X86),
+ make_obj("powl", NM_POW_X86),
+ make_obj("powf128", NM_POW_X86),
+ # "ceil" ends in 'l' but "cei" is not a function.
+ make_obj("ceil", NM_POW_X86),
+ ]
+ findings, _ = self.evaluate([rule], objs)
+ self.assertEqual(statuses(findings), [("r", "pow"), ("r", "ceil")])
+
+ def test_known_exceptions(self):
+ rule = {
+ "id": "r",
+ "forbid_defined": ["common_constants_internal::LOG_R", "UNUSED_TABLE"],
+ "known_exceptions": {
+ "logf": {
+ "symbols": ["common_constants_internal::LOG_R", "UNUSED_TABLE"],
+ "note": "being fixed",
+ }
+ },
+ }
+ findings, _ = self.evaluate([rule], [make_obj("logf", NM_LOGF_ARM)])
+ self.assertEqual(statuses(findings), [])
+ self.assertEqual(statuses(findings, lsc.STATUS_KNOWN), [("r", "logf")])
+ self.assertIn("being fixed", findings[0].message)
+ notes = [f.message for f in findings if f.status == lsc.STATUS_NOTE]
+ self.assertEqual(len(notes), 1)
+ self.assertIn("UNUSED_TABLE", notes[0])
+
+ def test_require_defined(self):
+ rule = {"id": "r", "require_defined": ["log2f_float_internal::R_16"]}
+ findings, _ = self.evaluate([rule], [make_obj("logf", NM_LOGF_ARM)])
+ self.assertEqual(statuses(findings), [("r", "logf")])
+ self.assertIn("stale", findings[0].message)
+
+ def test_max_named_data_object(self):
+ rule = {
+ "id": "r",
+ "max_named_data_object_bytes": 512,
+ "max_named_data_object_exceptions": {
+ "pow": 4096,
+ "logf": {"bytes": 1024, "note": "being fixed"},
+ },
+ }
+ objs = [
+ make_obj("pow", NM_POW_X86),
+ make_obj("logf", NM_LOGF_ARM),
+ make_obj("powx", NM_POW_X86),
+ ]
+ findings, _ = self.evaluate([rule], objs)
+ self.assertEqual(statuses(findings), [("r", "powx")])
+ fails = [f for f in findings if f.status == lsc.STATUS_FAIL]
+ self.assertIn("4096 bytes, limit is 512", fails[0].message)
+ # The logf exception is larger than needed for this build.
+ notes = [f for f in findings if f.status == lsc.STATUS_NOTE]
+ self.assertEqual([(f.function) for f in notes], ["logf"])
+ self.assertIn("exception of 1024 bytes is larger than needed", notes[0].message)
+
+ def test_max_writable_data(self):
+ rule = {"id": "r", "max_writable_data_bytes": 0}
+ findings, _ = self.evaluate([rule], [make_obj("pow", NM_POW_X86, SIZE_POW_X86)])
+ self.assertEqual(statuses(findings), [("r", "pow")])
+ self.assertIn("4 bytes of writable data", findings[0].message)
+ rule["allow_writable_sections"] = ["quick_get_round"]
+ findings, _ = self.evaluate([rule], [make_obj("pow", NM_POW_X86, SIZE_POW_X86)])
+ self.assertEqual(findings, [])
+
+ def test_known_exception_when_macros(self):
+ rule = {
+ "id": "r",
+ "forbid_undefined": ["@soft_double"],
+ "known_exceptions": {
+ "logf": {
+ "symbols": ["@soft_double"],
+ "when_macros": ["!__ARM_FP&0x8"],
+ "note": "being fixed",
+ }
+ },
+ }
+ v7em = make_obj("logf", NM_LOGF_ARM, macros=lsc.parse_macros(MACROS_ARMV7EM))
+ findings, _ = self.evaluate([rule], [v7em])
+ self.assertEqual(statuses(findings), [])
+ self.assertEqual(len(statuses(findings, lsc.STATUS_KNOWN)), 2)
+ # On a target with hardware double the exception does not apply, so a
+ # soft-double reference is a failure.
+ v81m = make_obj("logf", NM_LOGF_ARM, macros=lsc.parse_macros(MACROS_ARMV81M))
+ findings, _ = self.evaluate([rule], [v81m])
+ self.assertEqual(len(statuses(findings)), 2)
+ # A clean object on a target the exception does not apply to gets no
+ # stale exception note; on a target it applies to, it does.
+ clean = "00000000 00000010 T logf\n"
+ v81m = make_obj("logf", clean, macros=lsc.parse_macros(MACROS_ARMV81M))
+ findings, _ = self.evaluate([rule], [v81m])
+ self.assertEqual(findings, [])
+ v7em = make_obj("logf", clean, macros=lsc.parse_macros(MACROS_ARMV7EM))
+ findings, _ = self.evaluate([rule], [v7em])
+ self.assertEqual(statuses(findings, lsc.STATUS_NOTE), [("r", "logf")])
+
+ def test_unknown_alias(self):
+ rule = {"id": "r", "forbid_undefined": ["@missing"]}
+ with self.assertRaises(ValueError):
+ self.evaluate([rule], [make_obj("logf", NM_LOGF_ARM)])
+
+
+class ReportTest(unittest.TestCase):
+ def test_text_and_markdown(self):
+ rule = {"id": "r", "forbid_defined": ["common_constants_internal::LOG_R"]}
+ objs = [make_obj("logf", NM_LOGF_ARM)]
+ ctx = lsc.RuleContext(set(), {"logf"})
+ findings, summary = lsc.evaluate({"rules": [rule]}, objs, ctx)
+ text = lsc.format_text(findings, summary)
+ self.assertIn("FAIL: [r] logf: defines forbidden symbol", text)
+ self.assertIn("1 failures", text)
+ md = lsc.format_markdown(findings, summary, objs)
+ self.assertIn("| r | yes | 1 | 1 | 0 | 0 |", md)
+ self.assertIn("| logf |", md)
+
+
+class RulesFileTest(unittest.TestCase):
+ """Sanity checks of the rules files in this directory."""
+
+ KNOWN_KEYS = {
+ "id",
+ "comment",
+ "functions",
+ "exclude_functions",
+ "exclude_variants",
+ "exceptions",
+ "known_exceptions",
+ "when_flags",
+ "when_macros",
+ "forbid_defined",
+ "forbid_defined_regex",
+ "forbid_undefined",
+ "require_defined",
+ "max_named_data_object_bytes",
+ "max_named_data_object_exceptions",
+ "max_writable_data_bytes",
+ "allow_writable_sections",
+ }
+ KNOWN_EXCEPTION_KEYS = {"symbols", "note", "when_macros"}
+
+ def test_rules_files(self):
+ import json
+
+ rules_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "rules")
+ if not os.path.isdir(rules_dir):
+ self.skipTest("no rules directory")
+ for name in sorted(os.listdir(rules_dir)):
+ if not name.endswith(".json"):
+ continue
+ with self.subTest(rules=name):
+ with open(os.path.join(rules_dir, name), encoding="utf-8") as f:
+ doc = json.load(f)
+ ids = [r["id"] for r in doc["rules"]]
+ self.assertEqual(len(ids), len(set(ids)), "duplicate rule ids")
+ aliases = doc.get("regex_aliases", {})
+ for alias in aliases.values():
+ re.compile(alias)
+ for rule in doc["rules"]:
+ unknown = set(rule) - self.KNOWN_KEYS
+ self.assertFalse(unknown, "%s: unknown keys" % rule["id"])
+ for entry in rule.get("known_exceptions", {}).values():
+ unknown = set(entry) - self.KNOWN_EXCEPTION_KEYS
+ self.assertFalse(unknown, "%s: unknown keys" % rule["id"])
+ for p in rule.get("functions", []):
+ re.compile(p)
+ for key in ("forbid_defined", "forbid_defined_regex"):
+ for p in rule.get(key, []):
+ re.compile(aliases.get(p[1:], p) if p[0] == "@" else p)
+ for p in rule.get("forbid_undefined", []):
+ if p.startswith("@"):
+ self.assertIn(p[1:], aliases)
+ # Exercise every rule on an empty object to catch typos in
+ # patterns and aliases.
+ ctx = lsc.RuleContext(
+ lsc.parse_math_flags("LIBC_MATH_FAST|LIBC_MATH_SMALL_TABLES"),
+ set(),
+ aliases,
+ )
+ for rule in doc["rules"]:
+ lsc.evaluate_rule(rule, lsc.ObjectInfo("x", [], [], set(), {}), ctx)
+
+
+if __name__ == "__main__":
+ unittest.main()
More information about the libc-commits
mailing list