[clang] [SourceManager] Expose max usage of source location space as a Statistic (PR #96292)

[Ilya Biryukov via cfe-commits]

https://github.com/ilya-biryukov created https://github.com/llvm/llvm-project/pull/96292

We have been running into source location exhaustion recently and want to use the statistics to monitor the usage in various files to be able to anticipate where the next problem will happen.

I picked `Statistic` because it can be written into a structured JSON file and is easier to consume by further automation.

This commit does not change any existing per-source-manager metrics exposed via `SourceManager::PrintStats()`. This does create some redundancy, but I also expect to be non-controversial because it aligns with the intended use of `Statistic`.

>From ded4cdee02b56b0284b5e44fd24de6e07c57e6bf Mon Sep 17 00:00:00 2001
From: Ilya Biryukov <ibiryukov at google.com>
Date: Tue, 11 Jun 2024 19:07:07 +0200
Subject: [PATCH] [SourceManager] Expose max usage of source location space as
 a Statistic

We have been running into source location exhaustion recently and want
to use the statistics to monitor the usage in various files to be able
to anticipate where the next problem will happen.

I picked `Statistic` because it can be written into a structured JSON
file and is easier to consume by further automation.

This commit does not change any existing per-source-manager metrics
exposed via `SourceManager::PrintStats()`. This does create some
redundancy, but I also expect to be non-controversial because it aligns
with the intended use of `Statistic`.
---
 clang/include/clang/Basic/SourceManager.h |  1 +
 clang/lib/Basic/SourceManager.cpp         | 16 ++++++++++++++++
 2 files changed, 17 insertions(+)

diff --git a/clang/include/clang/Basic/SourceManager.h b/clang/include/clang/Basic/SourceManager.h
index d2e2e914327f2..d3ccc7ef81c07 100644
--- a/clang/include/clang/Basic/SourceManager.h
+++ b/clang/include/clang/Basic/SourceManager.h
@@ -1981,6 +1981,7 @@ class SourceManager : public RefCountedBase<SourceManager> {
                                          SourceLocation SpellLoc,
                                          SourceLocation ExpansionLoc,
                                          unsigned ExpansionLength) const;
+  void updateSlocUsageStats() const;
 };
 
 /// Comparison function object.
diff --git a/clang/lib/Basic/SourceManager.cpp b/clang/lib/Basic/SourceManager.cpp
index f0af1a3e3a38b..e430e8974ff5c 100644
--- a/clang/lib/Basic/SourceManager.cpp
+++ b/clang/lib/Basic/SourceManager.cpp
@@ -20,6 +20,7 @@
 #include "llvm/ADT/MapVector.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/Statistic.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/Support/Allocator.h"
@@ -46,6 +47,12 @@ using namespace clang;
 using namespace SrcMgr;
 using llvm::MemoryBuffer;
 
+#define DEBUG_TYPE "source-manager"
+
+STATISTIC(
+    MaxUsedSLocBytes,
+    "Maximum number of bytes used by source locations (both loaded and local)");
+
 //===----------------------------------------------------------------------===//
 // SourceManager Helper Classes
 //===----------------------------------------------------------------------===//
@@ -466,6 +473,7 @@ SourceManager::AllocateLoadedSLocEntries(unsigned NumSLocEntries,
   SLocEntryLoaded.resize(LoadedSLocEntryTable.size());
   SLocEntryOffsetLoaded.resize(LoadedSLocEntryTable.size());
   CurrentLoadedOffset -= TotalSize;
+  updateSlocUsageStats();
   int BaseID = -int(LoadedSLocEntryTable.size()) - 1;
   LoadedSLocEntryAllocBegin.push_back(FileID::get(BaseID));
   return std::make_pair(BaseID, CurrentLoadedOffset);
@@ -619,6 +627,7 @@ FileID SourceManager::createFileIDImpl(ContentCache &File, StringRef Filename,
   // We do a +1 here because we want a SourceLocation that means "the end of the
   // file", e.g. for the "no newline at the end of the file" diagnostic.
   NextLocalOffset += FileSize + 1;
+  updateSlocUsageStats();
 
   // Set LastFileIDLookup to the newly created file.  The next getFileID call is
   // almost guaranteed to be from that file.
@@ -679,6 +688,7 @@ SourceManager::createExpansionLocImpl(const ExpansionInfo &Info,
   }
   // See createFileID for that +1.
   NextLocalOffset += Length + 1;
+  updateSlocUsageStats();
   return SourceLocation::getMacroLoc(NextLocalOffset - (Length + 1));
 }
 
@@ -1843,6 +1853,12 @@ void SourceManager::associateFileChunkWithMacroArgExp(
   MacroArgsCache[EndOffs] = EndOffsMappedLoc;
 }
 
+void SourceManager::updateSlocUsageStats() const {
+  SourceLocation::UIntTy UsedBytes =
+      NextLocalOffset + (MaxLoadedOffset - CurrentLoadedOffset);
+  MaxUsedSLocBytes.updateMax(UsedBytes);
+}
+
 /// If \arg Loc points inside a function macro argument, the returned
 /// location will be the macro location in which the argument was expanded.
 /// If a macro argument is used multiple times, the expanded location will