[clang-tools-extra] bug fix for pp-trace (PR #124376)

[via cfe-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-clang-tools-extra

Author: seekamoon (seekamoon)

<details>
<summary>Changes</summary>

When using `pp-trace` in practice, I noticed that the YAML files it generated could contain grammar errors, leading to failed parsing. Specifically, it is the `Args` of `MacroExpands`.

E.g., 
```YAML
- Callback: MacroExpands
  MacroNameTok: EXPORT_TEMPLATE_STYLE
  MacroDefinition: [(local)]
  Range: [(nonfile), (nonfile)]
  Args: [, ]
- Callback: MacroExpands
  MacroNameTok: IS_TYPE_FUNCTION_DECL
  MacroDefinition: [(local)]
  Range: ["../../src/objects/objects.h:660:1", "../../src/objects/objects.h:660:54"]
  Args: [NullOrUndefined, , ]
- Callback: MacroExpands
  MacroNameTok: ABSL_INTERNAL_ANY_INVOCABLE_IMPL
  MacroDefinition: [(local)]
  Range: ["../../third_party/abseil-cpp/absl/functional/internal/any_invocable.h:868:1", "../../third_party/abseil-cpp/absl/functional/internal/any_invocable.h:868:39"]
  Args: [, , <amp>]
```

The rootcause is the `PPCallbacksTracker::appendArgument(const char *Name, const MacroArgs *Value)`. When iterating `Value`, sometimes the Token is eof, and the previous implementation did not take it into consideration.

I fixed this bug and the output YAML code will be
```YAML
- Callback: MacroExpands
  MacroNameTok: EXPORT_TEMPLATE_STYLE
  MacroDefinition: [(local)]
  Range: [(nonfile), (nonfile)]
  Args: []
- Callback: MacroExpands
  MacroNameTok: IS_TYPE_FUNCTION_DECL
  MacroDefinition: [(local)]
  Range: ["../../src/objects/objects.h:660:1", "../../src/objects/objects.h:660:54"]
  Args: [NullOrUndefined]
- Callback: MacroExpands
  MacroNameTok: ABSL_INTERNAL_ANY_INVOCABLE_IMPL
  MacroDefinition: [(local)]
  Range: ["../../third_party/abseil-cpp/absl/functional/internal/any_invocable.h:868:1", "../../third_party/abseil-cpp/absl/functional/internal/any_invocable.h:868:39"]
  Args: [<amp>]
```

---
Full diff: https://github.com/llvm/llvm-project/pull/124376.diff


1 Files Affected:

- (modified) clang-tools-extra/pp-trace/PPCallbacksTracker.cpp (+6-1) 


``````````diff
diff --git a/clang-tools-extra/pp-trace/PPCallbacksTracker.cpp b/clang-tools-extra/pp-trace/PPCallbacksTracker.cpp
index 3bb30fd15b2e1d..9c393ae3227097 100644
--- a/clang-tools-extra/pp-trace/PPCallbacksTracker.cpp
+++ b/clang-tools-extra/pp-trace/PPCallbacksTracker.cpp
@@ -604,11 +604,15 @@ void PPCallbacksTracker::appendArgument(const char *Name,
   llvm::raw_string_ostream SS(Str);
   SS << "[";
 
+  // Output ", " before outputing next argument
+  bool Comma = false;
   // Each argument is a series of contiguous Tokens, terminated by a eof.
   // Go through each argument printing tokens until we reach eof.
   for (unsigned I = 0; I < Value->getNumMacroArguments(); ++I) {
     const Token *Current = Value->getUnexpArgument(I);
-    if (I)
+    if (Current->is(tok::eof))
+      continue;
+    if (Comma)
       SS << ", ";
     bool First = true;
     while (Current->isNot(tok::eof)) {
@@ -625,6 +629,7 @@ void PPCallbacksTracker::appendArgument(const char *Name,
       ++Current;
       First = false;
     }
+    Comma = true;
   }
   SS << "]";
   appendArgument(Name, SS.str());

``````````

</details>


https://github.com/llvm/llvm-project/pull/124376