[Lldb-commits] [lldb] [lldb] Fix highlight of digit separators (#224902) (PR #226363)

[via lldb-commits]

llvmorg-github-actions[bot] wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-lldb

Author: José Guilherme (zeguilherme1)

<details>
<summary>Changes</summary>

### [lldb] Fix syntax highlighting for C++ literal digit separators

The main goal of this PR is to fix a syntax highlighting bug in LLDB where C++ literal digit separators (e.g., `0b00'00` or `1'000'000`) were incorrectly parsed and colored as character literals.

Currently, `ClangHighlighter::Highlight` initializes `LangOptions` manually. While it enables the `CPlusPlus14` version flag, it doesn't use  `LangOptions::setLangDefaults()`. As a result, underlying feature flags are left in their default `false` state.

**Implementation Notes**
As discussed in the related issue #<!-- -->224902, we opted for explicitly setting `Opts.AllowLiteralDigitSeparator = true;`. We tried to use `LangOptions::setLangDefaults()`, but avoided it because, in the isolated context of the Highlighter, it caused unexpected regressions in other token tests.

**Testing**
* Added a new unit test (`ClangDigitSeparator`) to validate the fix and prevent future regressions.
* Verified the change by running the LLDB test suite (`ninja check-lldb`), with all tests passing successfully.

Co-authored-by: Luis Antoniosi <luisgustavo_antoniosi@<!-- -->hotmail.com>

---
Full diff: https://github.com/llvm/llvm-project/pull/226363.diff


2 Files Affected:

- (modified) lldb/source/Plugins/Highlighter/Clang/ClangHighlighter.cpp (+2-1) 
- (modified) lldb/unittests/Highlighter/HighlighterTest.cpp (+14) 


``````````diff
diff --git a/lldb/source/Plugins/Highlighter/Clang/ClangHighlighter.cpp b/lldb/source/Plugins/Highlighter/Clang/ClangHighlighter.cpp
index 3889a0483d85a8..38289e0286f307 100644
--- a/lldb/source/Plugins/Highlighter/Clang/ClangHighlighter.cpp
+++ b/lldb/source/Plugins/Highlighter/Clang/ClangHighlighter.cpp
@@ -178,6 +178,7 @@ void ClangHighlighter::Highlight(const HighlightStyle &options,
   Opts.ObjC = true;
   // FIXME: This should probably set CPlusPlus, CPlusPlus11, ... too
   Opts.CPlusPlus17 = true;
+  Opts.AllowLiteralDigitSeparator = true;
   Opts.LineComment = true;
 
   Lexer lex(FID, buf->getMemBufferRef(), SM, Opts);
@@ -272,4 +273,4 @@ void ClangHighlighter::Initialize() {
 
 void ClangHighlighter::Terminate() {
   PluginManager::UnregisterPlugin(CreateInstance);
-}
+}
\ No newline at end of file
diff --git a/lldb/unittests/Highlighter/HighlighterTest.cpp b/lldb/unittests/Highlighter/HighlighterTest.cpp
index 95d77f4f1303ef..4ef4948e0b180b 100644
--- a/lldb/unittests/Highlighter/HighlighterTest.cpp
+++ b/lldb/unittests/Highlighter/HighlighterTest.cpp
@@ -338,6 +338,20 @@ TEST_F(HighlighterTest, ClangCursorPosInOtherToken) {
             highlightC(" foo c = bar(); return 1;", s, 3));
 }
 
+TEST_F(HighlighterTest, ClangDigitSeparator) {
+  HighlightStyle s;
+  s.scalar_literal.Set("<scalar>", "</scalar>");
+
+  EXPECT_EQ(" int i = <scalar>0b00'00</scalar>;",
+            highlightC(" int i = 0b00'00;", s));
+  
+  EXPECT_EQ(" x = <scalar>1'000'000</scalar>;",
+            highlightC(" x = 1'000'000;", s));
+  
+  EXPECT_EQ("<scalar>0b00'00</scalar> == <scalar>0b00'00</scalar>",
+            highlightC("0b00'00 == 0b00'00", s));
+}
+
 #if LLDB_ENABLE_TREESITTER
 static std::string
 highlightSwift(llvm::StringRef code, HighlightStyle style,

``````````

</details>


https://github.com/llvm/llvm-project/pull/226363