[clang] [clang] Make -dump-tokens option align tokens (PR #164894)

[via cfe-commits]

================
@@ -240,14 +241,59 @@ void Preprocessor::FinalizeForModelFile() {
 }
 
 void Preprocessor::DumpToken(const Token &Tok, bool DumpFlags) const {
-  llvm::errs() << tok::getTokenName(Tok.getKind());
+  std::string TokenStr;
+  llvm::raw_string_ostream OS(TokenStr);
+
+  // The alignment of 16 is chosen to comfortably fit most identifiers.
+  OS << llvm::formatv("{0,-16} ", tok::getTokenName(Tok.getKind()));
+
+  // Annotation tokens are just markers that don't have a spelling -- they
----------------
alexpaniman wrote:

I didn't know, thanks for the feedback. I'll follow up with changes to include module spelling in a second PR.

https://github.com/llvm/llvm-project/pull/164894