[llvm] [Testing] Allow custom markers in llvm::Annotations (PR #195570)
Harlen Batagelo via llvm-commits
llvm-commits at lists.llvm.org
Sun May 3 18:22:19 PDT 2026
https://github.com/hbatagelo created https://github.com/llvm/llvm-project/pull/195570
The current annotation markers can conflict with several language constructs. Notably, `[[ ]]` and `^` collide with C++ attributes (e.g., `[[nodiscard]]`), the C++26 reflection operator (`^^int`), and Objective-C blocks (`void (^foo)(void)`). Similarly, `$` can conflict with identifiers that also use `$` with `-fdollars-in-identifiers`, as well as with C++26 code that uses `$` as raw-string delimiters or as a preprocessing token ([P2558R2](https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2023/p2558r2.html)).
Because the markers are currently hardcoded in `llvm::Annotations`, existing workarounds have to rely on digraphs or macro substitution via implicit `#define`s. These approaches reduce readability and make tests cumbersome to write. This PR alleviates these issues by adding support for custom markers. It adds an overloaded `Annotations` constructor that accepts a new `Annotations::Markers` struct. For example, to use `~` for point, `@` for name, and `{{`/`}}` for range:
```cpp
Annotations Example(R"cpp(
@name(payload){{[[nodiscard]] int foo(int x);}}~
)cpp", {"~", "@", "{{", "}}"});
```
Using longer markers:
```cpp
Annotations Example(R"cpp(
$$name(payload)[[[[[nodiscard]] int foo(int x);]]]^^
)cpp", {"^^", "$$", "[[[", "]]]"});
```
Using multi-byte characters:
```cpp
Annotations Example(R"cpp(
🏷️name(payload)👉[[nodiscard]] int foo(int x);👈🎯
)cpp", {"🎯", "🏷️", "👉", "👈"});
```
The existing single-argument constructor delegates to the new overload, preserving backward compatibility.
PS: The original code has a FIXME comment mentioning alternative approaches, such as escaping and changing the default syntax. While valid, escaping would increase visual noise, and changing the default syntax would break existing tests. The approach proposed here provides the flexibility to choose a syntax that is clean for a specific context and is backward compatible.
>From 0a474c3a8d4fac3a486bc521f678ab14fcb62da0 Mon Sep 17 00:00:00 2001
From: Harlen Batagelo <hbatagelo at gmail.com>
Date: Sun, 3 May 2026 21:53:07 -0300
Subject: [PATCH] Add support for custom annotation syntax
---
.../llvm/Testing/Annotations/Annotations.h | 32 +++++++---
llvm/lib/Testing/Annotations/Annotations.cpp | 50 +++++++++++----
.../Testing/Annotations/AnnotationsTest.cpp | 61 ++++++++++++++++++-
3 files changed, 122 insertions(+), 21 deletions(-)
diff --git a/llvm/include/llvm/Testing/Annotations/Annotations.h b/llvm/include/llvm/Testing/Annotations/Annotations.h
index 4d38002f029de..ec5ee506ee1e3 100644
--- a/llvm/include/llvm/Testing/Annotations/Annotations.h
+++ b/llvm/include/llvm/Testing/Annotations/Annotations.h
@@ -26,7 +26,8 @@ class raw_ostream;
/// $definition^class Foo{}; // points can be named: "definition"
/// $(foo)^class Foo{}; // ...or have a payload: "foo"
/// $definition(foo)^class Foo{}; // ...or both
-/// $fail(runtime)[[assert(false)]] // ranges can have names/payloads too
+/// $fail(runtime)[[assert(false)]] // ranges can have names/payloads
+/// // too
/// )cpp");
///
/// StringRef Code = Example.code(); // annotations stripped.
@@ -38,18 +39,19 @@ class raw_ostream;
/// annotations.
///
/// Names consist of only alphanumeric characters or '_'.
-/// Payloads can contain any character expect '(' and ')'.
+/// Payloads can contain any character except '(' and ')'.
///
/// Ranges may be nested (and points can be inside ranges), but there's no way
/// to define general overlapping ranges.
///
-/// FIXME: the choice of the marking syntax makes it impossible to represent
-/// some of the C++ and Objective C constructs (including common ones
-/// like C++ attributes). We can fix this by:
-/// 1. introducing an escaping mechanism for the special characters,
-/// 2. making characters for marking points and ranges configurable,
-/// 3. changing the syntax to something less commonly used,
-/// 4. ...
+/// The markers for points, names and ranges can be customized. For example, to
+/// use "~" as the point marker, "@@" as the name/payload marker, and "{{" /
+/// "}}" as range delimiters:
+///
+/// Annotations Example("~point @@name{{range}}", {"~", "@@", "{{", "}}"});
+///
+/// This is useful when the default markers would otherwise conflict with the
+/// underlying syntax (e.g. C++ attributes or the reflection operator).
class Annotations {
public:
/// Two offsets pointing to a continuous substring. End is not included, i.e.
@@ -64,8 +66,20 @@ class Annotations {
friend bool operator!=(const Range &L, const Range &R) { return !(L == R); }
};
+ /// Markers used to denote points, names/payloads and ranges in the annotated
+ /// text.
+ struct Markers {
+ llvm::StringRef Point = "^";
+ llvm::StringRef Name = "$";
+ llvm::StringRef RangeBegin = "[[";
+ llvm::StringRef RangeEnd = "]]";
+ };
+
/// Parses the annotations from Text. Crashes if it's malformed.
Annotations(llvm::StringRef Text);
+ /// Parses the annotations from Text using custom markers.
+ /// Markers must be non-empty and unambiguous.
+ Annotations(llvm::StringRef Text, const Markers &Markers);
/// The input text with all annotations stripped.
/// All points and ranges are relative to this stripped text.
diff --git a/llvm/lib/Testing/Annotations/Annotations.cpp b/llvm/lib/Testing/Annotations/Annotations.cpp
index 1e6852619a874..55bdec5d13aa4 100644
--- a/llvm/lib/Testing/Annotations/Annotations.cpp
+++ b/llvm/lib/Testing/Annotations/Annotations.cpp
@@ -16,15 +16,42 @@ using namespace llvm;
// Crash if the assertion fails, printing the message and testcase.
// More elegant error handling isn't needed for unit tests.
-static void require(bool Assertion, const char *Msg, llvm::StringRef Code) {
+static void require(bool Assertion, const llvm::Twine &Msg,
+ llvm::StringRef Code) {
if (!Assertion) {
llvm::errs() << "Annotated testcase: " << Msg << "\n" << Code << "\n";
llvm_unreachable("Annotated testcase assertion failed!");
}
}
-Annotations::Annotations(llvm::StringRef Text) {
- auto Require = [Text](bool Assertion, const char *Msg) {
+Annotations::Annotations(llvm::StringRef Text) : Annotations(Text, {}) {}
+
+Annotations::Annotations(llvm::StringRef Text, const Markers &Markers) {
+ assert(!Markers.Point.empty() && "point marker cannot be empty");
+ assert(!Markers.Name.empty() && "name marker cannot be empty");
+ assert(!Markers.RangeBegin.empty() && "range begin marker cannot be empty");
+ assert(!Markers.RangeEnd.empty() && "range end marker cannot be empty");
+
+ assert(!Markers.Point.starts_with(Markers.RangeBegin) &&
+ !Markers.RangeBegin.starts_with(Markers.Point) &&
+ "point and range begin markers cannot be prefixes of each other");
+ assert(!Markers.Point.starts_with(Markers.RangeEnd) &&
+ !Markers.RangeEnd.starts_with(Markers.Point) &&
+ "point and range end markers cannot be prefixes of each other");
+ assert(!Markers.Point.starts_with(Markers.Name) &&
+ !Markers.Name.starts_with(Markers.Point) &&
+ "point and name markers cannot be prefixes of each other");
+ assert(!Markers.RangeBegin.starts_with(Markers.RangeEnd) &&
+ !Markers.RangeEnd.starts_with(Markers.RangeBegin) &&
+ "range begin and range end markers cannot be prefixes of each other");
+ assert(!Markers.RangeBegin.starts_with(Markers.Name) &&
+ !Markers.Name.starts_with(Markers.RangeBegin) &&
+ "range begin and name markers cannot be prefixes of each other");
+ assert(!Markers.RangeEnd.starts_with(Markers.Name) &&
+ !Markers.Name.starts_with(Markers.RangeEnd) &&
+ "range end and name markers cannot be prefixes of each other");
+
+ auto Require = [Text](bool Assertion, const llvm::Twine &Msg) {
require(Assertion, Msg, Text);
};
std::optional<llvm::StringRef> Name;
@@ -33,7 +60,7 @@ Annotations::Annotations(llvm::StringRef Text) {
Code.reserve(Text.size());
while (!Text.empty()) {
- if (Text.consume_front("^")) {
+ if (Text.consume_front(Markers.Point)) {
All.push_back(
{Code.size(), size_t(-1), Name.value_or(""), Payload.value_or("")});
Points[Name.value_or("")].push_back(All.size() - 1);
@@ -41,16 +68,17 @@ Annotations::Annotations(llvm::StringRef Text) {
Payload = std::nullopt;
continue;
}
- if (Text.consume_front("[[")) {
+ if (Text.consume_front(Markers.RangeBegin)) {
OpenRanges.push_back(
{Code.size(), size_t(-1), Name.value_or(""), Payload.value_or("")});
Name = std::nullopt;
Payload = std::nullopt;
continue;
}
- Require(!Name, "$name should be followed by ^ or [[");
- if (Text.consume_front("]]")) {
- Require(!OpenRanges.empty(), "unmatched ]]");
+ Require(!Name, Markers.Name + "name should be followed by " +
+ Markers.Point + " or " + Markers.RangeBegin);
+ if (Text.consume_front(Markers.RangeEnd)) {
+ Require(!OpenRanges.empty(), "unmatched " + Markers.RangeEnd);
const Annotation &NewRange = OpenRanges.back();
All.push_back(
@@ -60,7 +88,7 @@ Annotations::Annotations(llvm::StringRef Text) {
OpenRanges.pop_back();
continue;
}
- if (Text.consume_front("$")) {
+ if (Text.consume_front(Markers.Name)) {
Name =
Text.take_while([](char C) { return llvm::isAlnum(C) || C == '_'; });
Text = Text.drop_front(Name->size());
@@ -76,8 +104,8 @@ Annotations::Annotations(llvm::StringRef Text) {
Code.push_back(Text.front());
Text = Text.drop_front();
}
- Require(!Name, "unterminated $name");
- Require(OpenRanges.empty(), "unmatched [[");
+ Require(!Name, "unterminated " + Markers.Name + "name");
+ Require(OpenRanges.empty(), "unmatched " + Markers.RangeBegin);
}
size_t Annotations::point(llvm::StringRef Name) const {
diff --git a/llvm/unittests/Testing/Annotations/AnnotationsTest.cpp b/llvm/unittests/Testing/Annotations/AnnotationsTest.cpp
index d8c42888e5547..7db43ff7424aa 100644
--- a/llvm/unittests/Testing/Annotations/AnnotationsTest.cpp
+++ b/llvm/unittests/Testing/Annotations/AnnotationsTest.cpp
@@ -107,7 +107,7 @@ TEST(AnnotationsTest, Nested) {
}
TEST(AnnotationsTest, Payload) {
- // // A single unnamed point or range with unspecified payload
+ // A single unnamed point or range with unspecified payload
EXPECT_THAT(llvm::Annotations("a$^b").pointWithPayload(), Pair(1u, ""));
EXPECT_THAT(llvm::Annotations("a$[[b]]cdef").rangeWithPayload(),
Pair(range(1, 2), ""));
@@ -154,6 +154,58 @@ TEST(AnnotationsTest, Named) {
EXPECT_EQ(Annotated.point("p2"), 4u);
}
+TEST(AnnotationsTest, CustomMarkers) {
+ llvm::Annotations::Markers Custom{"~", "@", "{{", "}}"};
+
+ // Cleaned code.
+ EXPECT_EQ(llvm::Annotations("foo~bar at nnn{{baz@~{{qux}}}}", Custom).code(),
+ "foobarbazqux");
+
+ // A single point.
+ EXPECT_EQ(llvm::Annotations("~ab", Custom).point(), 0u);
+ EXPECT_EQ(llvm::Annotations("a~b", Custom).point(), 1u);
+
+ // Multiple points.
+ EXPECT_THAT(llvm::Annotations("~a~bc~d~", Custom).points(),
+ ElementsAre(0u, 1u, 3u, 4u));
+
+ // A single range.
+ EXPECT_EQ(llvm::Annotations("{{a}}bc", Custom).range(), range(0, 1));
+ EXPECT_EQ(llvm::Annotations("a{{bc}}d", Custom).range(), range(1, 3));
+
+ // Multiple ranges.
+ EXPECT_THAT(llvm::Annotations("{{a}}{{b}}cd{{ef}}ef", Custom).ranges(),
+ ElementsAre(range(0, 1), range(1, 2), range(4, 6)));
+
+ // Named and payloads.
+ EXPECT_EQ(llvm::Annotations("a at foo~b", Custom).point("foo"), 1u);
+ EXPECT_THAT(llvm::Annotations("a at name(foo){{b}}cdef", Custom)
+ .rangeWithPayload("name"),
+ Pair(range(1, 2), "foo"));
+
+ // Custom markers with longer tokens.
+ llvm::Annotations::Markers Multi{"-->", "###", "<<<", ">>>"};
+
+ // Cleaned code.
+ EXPECT_EQ(llvm::Annotations("foo-->bar###nnn<<<baz###--><<<qux>>>>>>", Multi)
+ .code(),
+ "foobarbazqux");
+
+ // A single point.
+ EXPECT_EQ(llvm::Annotations("-->ab", Multi).point(), 0u);
+ EXPECT_EQ(llvm::Annotations("a-->b", Multi).point(), 1u);
+
+ // A single range.
+ EXPECT_EQ(llvm::Annotations("<<<a>>>bc", Multi).range(), range(0, 1));
+ EXPECT_EQ(llvm::Annotations("a<<<bc>>>d", Multi).range(), range(1, 3));
+
+ // Named and payloads.
+ EXPECT_EQ(llvm::Annotations("a###foo-->b", Multi).point("foo"), 1u);
+ EXPECT_THAT(llvm::Annotations("a###name(foo)<<<b>>>cdef", Multi)
+ .rangeWithPayload("name"),
+ Pair(range(1, 2), "foo"));
+}
+
TEST(AnnotationsTest, Errors) {
// Annotations use llvm_unreachable, it will only crash in debug mode.
#ifndef NDEBUG
@@ -179,6 +231,13 @@ TEST(AnnotationsTest, Errors) {
EXPECT_DEATH(llvm::Annotations("ff$fdsfd"), "unterminated \\$name");
EXPECT_DEATH(llvm::Annotations("ff$("), "unterminated payload");
EXPECT_DEATH(llvm::Annotations("ff$name("), "unterminated payload");
+
+ // Parsing failures with custom markers.
+ llvm::Annotations::Markers Custom{"~", "@", "{{", "}}"};
+ EXPECT_DEATH(llvm::Annotations("ff{{fdfd", Custom), "unmatched \\{\\{");
+ EXPECT_DEATH(llvm::Annotations("ff{{fdjs}}xx}}", Custom), "unmatched \\}\\}");
+ EXPECT_DEATH(llvm::Annotations("ff at name^", Custom),
+ "should be followed by ~ or \\{\\{");
#endif
}
} // namespace
More information about the llvm-commits
mailing list