[llvm] r356813 - [pdb] Add -type-stats and sort stats by descending size
Reid Kleckner via llvm-commits
llvm-commits at lists.llvm.org
Fri Mar 22 14:22:13 PDT 2019
Author: rnk
Date: Fri Mar 22 14:22:13 2019
New Revision: 356813
URL: http://llvm.org/viewvc/llvm-project?rev=356813&view=rev
Log:
[pdb] Add -type-stats and sort stats by descending size
Summary:
It prints this on chromium browser_tests.exe.pdb:
Types
Total: 5647475 entries ( 371,897,512 bytes, 65.85 avg)
--------------------------------------------------------------------------
LF_CLASS: 397894 entries ( 119,537,780 bytes, 300.43 avg)
LF_STRUCTURE: 236351 entries ( 83,208,084 bytes, 352.05 avg)
LF_FIELDLIST: 291003 entries ( 66,087,920 bytes, 227.10 avg)
LF_MFUNCTION: 1884176 entries ( 52,756,928 bytes, 28.00 avg)
LF_POINTER: 1149030 entries ( 13,877,344 bytes, 12.08 avg)
LF_ARGLIST: 789980 entries ( 12,436,752 bytes, 15.74 avg)
LF_METHODLIST: 361498 entries ( 8,351,008 bytes, 23.10 avg)
LF_ENUM: 16069 entries ( 6,108,340 bytes, 380.13 avg)
LF_PROCEDURE: 269374 entries ( 4,309,984 bytes, 16.00 avg)
LF_MODIFIER: 235602 entries ( 2,827,224 bytes, 12.00 avg)
LF_UNION: 9131 entries ( 2,072,168 bytes, 226.94 avg)
LF_VFTABLE: 323 entries ( 207,784 bytes, 643.29 avg)
LF_ARRAY: 6639 entries ( 106,380 bytes, 16.02 avg)
LF_VTSHAPE: 126 entries ( 6,472 bytes, 51.37 avg)
LF_BITFIELD: 278 entries ( 3,336 bytes, 12.00 avg)
LF_LABEL: 1 entries ( 8 bytes, 8.00 avg)
The PDB is overall 1.9GB, so the LF_CLASS and LF_STRUCTURE declarations
account for about 10% of the overall file size. I was surprised to find
that on average LF_FIELDLIST records are short. Maybe this is because
there are many more types with short member lists than there are
instantiations with lots of members, like std::vector.
Reviewers: aganea, zturner
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D59672
Added:
llvm/trunk/test/DebugInfo/PDB/type-stats.test
Modified:
llvm/trunk/test/DebugInfo/PDB/module-stats.test
llvm/trunk/test/DebugInfo/PDB/udt-stats.test
llvm/trunk/tools/llvm-pdbutil/DumpOutputStyle.cpp
llvm/trunk/tools/llvm-pdbutil/DumpOutputStyle.h
llvm/trunk/tools/llvm-pdbutil/llvm-pdbutil.cpp
llvm/trunk/tools/llvm-pdbutil/llvm-pdbutil.h
Modified: llvm/trunk/test/DebugInfo/PDB/module-stats.test
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/DebugInfo/PDB/module-stats.test?rev=356813&r1=356812&r2=356813&view=diff
==============================================================================
--- llvm/trunk/test/DebugInfo/PDB/module-stats.test (original)
+++ llvm/trunk/test/DebugInfo/PDB/module-stats.test Fri Mar 22 14:22:13 2019
@@ -6,28 +6,28 @@ ALL-NEXT: Stream 12, 308 bytes
ALL: Symbols
ALL-NEXT: Total: 6 entries ( 204 bytes)
ALL-NEXT: --------------------------------------------------------------------------
+ALL-NEXT: S_COMPILE3: 1 entries ( 60 bytes)
+ALL-NEXT: S_OBJNAME: 1 entries ( 56 bytes)
ALL-NEXT: S_GPROC32: 1 entries ( 44 bytes)
ALL-NEXT: S_FRAMEPROC: 1 entries ( 32 bytes)
-ALL-NEXT: S_END: 1 entries ( 4 bytes)
-ALL-NEXT: S_OBJNAME: 1 entries ( 56 bytes)
-ALL-NEXT: S_COMPILE3: 1 entries ( 60 bytes)
ALL-NEXT: S_BUILDINFO: 1 entries ( 8 bytes)
+ALL-NEXT: S_END: 1 entries ( 4 bytes)
ALL: Chunks
ALL-NEXT: Total: 2 entries ( 88 bytes)
ALL-NEXT: --------------------------------------------------------------------------
-ALL-NEXT: DEBUG_S_FILECHKSMS: 1 entries ( 32 bytes)
ALL-NEXT: DEBUG_S_LINES: 1 entries ( 56 bytes)
+ALL-NEXT: DEBUG_S_FILECHKSMS: 1 entries ( 32 bytes)
ALL-NEXT: Mod 0001 | `* Linker *`:
ALL-NEXT: Stream 14, 520 bytes
ALL: Symbols
ALL-NEXT: Total: 13 entries ( 512 bytes)
ALL-NEXT: --------------------------------------------------------------------------
-ALL-NEXT: S_SECTION: 4 entries ( 112 bytes)
ALL-NEXT: S_ENVBLOCK: 1 entries ( 172 bytes)
+ALL-NEXT: S_COFFGROUP: 5 entries ( 140 bytes)
+ALL-NEXT: S_SECTION: 4 entries ( 112 bytes)
+ALL-NEXT: S_COMPILE3: 1 entries ( 48 bytes)
ALL-NEXT: S_TRAMPOLINE: 1 entries ( 20 bytes)
ALL-NEXT: S_OBJNAME: 1 entries ( 20 bytes)
-ALL-NEXT: S_COMPILE3: 1 entries ( 48 bytes)
-ALL-NEXT: S_COFFGROUP: 5 entries ( 140 bytes)
ALL: Chunks
ALL-NEXT: Total: 0 entries ( 0 bytes)
ALL-NEXT: --------------------------------------------------------------------------
@@ -35,21 +35,21 @@ ALL-NEXT: Summary |
ALL: Symbols
ALL-NEXT: Total: 19 entries ( 716 bytes)
ALL-NEXT: --------------------------------------------------------------------------
+ALL-NEXT: S_ENVBLOCK: 1 entries ( 172 bytes)
+ALL-NEXT: S_COFFGROUP: 5 entries ( 140 bytes)
ALL-NEXT: S_SECTION: 4 entries ( 112 bytes)
+ALL-NEXT: S_COMPILE3: 2 entries ( 108 bytes)
+ALL-NEXT: S_OBJNAME: 2 entries ( 76 bytes)
ALL-NEXT: S_GPROC32: 1 entries ( 44 bytes)
-ALL-NEXT: S_ENVBLOCK: 1 entries ( 172 bytes)
ALL-NEXT: S_FRAMEPROC: 1 entries ( 32 bytes)
ALL-NEXT: S_TRAMPOLINE: 1 entries ( 20 bytes)
-ALL-NEXT: S_END: 1 entries ( 4 bytes)
-ALL-NEXT: S_OBJNAME: 2 entries ( 76 bytes)
-ALL-NEXT: S_COMPILE3: 2 entries ( 108 bytes)
-ALL-NEXT: S_COFFGROUP: 5 entries ( 140 bytes)
ALL-NEXT: S_BUILDINFO: 1 entries ( 8 bytes)
+ALL-NEXT: S_END: 1 entries ( 4 bytes)
ALL: Chunks
ALL-NEXT: Total: 2 entries ( 88 bytes)
ALL-NEXT: --------------------------------------------------------------------------
-ALL-NEXT: DEBUG_S_FILECHKSMS: 1 entries ( 32 bytes)
ALL-NEXT: DEBUG_S_LINES: 1 entries ( 56 bytes)
+ALL-NEXT: DEBUG_S_FILECHKSMS: 1 entries ( 32 bytes)
ONE-NOT: Mod 0000
ONE: Mod 0001 | `* Linker *`:
@@ -57,12 +57,12 @@ ONE-NEXT: Stream 14, 520 bytes
ONE: Symbols
ONE-NEXT: Total: 13 entries ( 512 bytes)
ONE-NEXT: --------------------------------------------------------------------------
-ONE-NEXT: S_SECTION: 4 entries ( 112 bytes)
ONE-NEXT: S_ENVBLOCK: 1 entries ( 172 bytes)
+ONE-NEXT: S_COFFGROUP: 5 entries ( 140 bytes)
+ONE-NEXT: S_SECTION: 4 entries ( 112 bytes)
+ONE-NEXT: S_COMPILE3: 1 entries ( 48 bytes)
ONE-NEXT: S_TRAMPOLINE: 1 entries ( 20 bytes)
ONE-NEXT: S_OBJNAME: 1 entries ( 20 bytes)
-ONE-NEXT: S_COMPILE3: 1 entries ( 48 bytes)
-ONE-NEXT: S_COFFGROUP: 5 entries ( 140 bytes)
ONE: Chunks
ONE-NEXT: Total: 0 entries ( 0 bytes)
ONE-NEXT: --------------------------------------------------------------------------
@@ -70,12 +70,12 @@ ONE-NEXT: Summary |
ONE: Symbols
ONE-NEXT: Total: 13 entries ( 512 bytes)
ONE-NEXT: --------------------------------------------------------------------------
-ONE-NEXT: S_SECTION: 4 entries ( 112 bytes)
ONE-NEXT: S_ENVBLOCK: 1 entries ( 172 bytes)
+ONE-NEXT: S_COFFGROUP: 5 entries ( 140 bytes)
+ONE-NEXT: S_SECTION: 4 entries ( 112 bytes)
+ONE-NEXT: S_COMPILE3: 1 entries ( 48 bytes)
ONE-NEXT: S_TRAMPOLINE: 1 entries ( 20 bytes)
ONE-NEXT: S_OBJNAME: 1 entries ( 20 bytes)
-ONE-NEXT: S_COMPILE3: 1 entries ( 48 bytes)
-ONE-NEXT: S_COFFGROUP: 5 entries ( 140 bytes)
ONE: Chunks
ONE-NEXT: Total: 0 entries ( 0 bytes)
ONE-NEXT: --------------------------------------------------------------------------
Added: llvm/trunk/test/DebugInfo/PDB/type-stats.test
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/DebugInfo/PDB/type-stats.test?rev=356813&view=auto
==============================================================================
--- llvm/trunk/test/DebugInfo/PDB/type-stats.test (added)
+++ llvm/trunk/test/DebugInfo/PDB/type-stats.test Fri Mar 22 14:22:13 2019
@@ -0,0 +1,17 @@
+; RUN: llvm-pdbutil dump -type-stats %p/Inputs/empty.pdb | FileCheck --check-prefix=ALL %s
+
+ALL: Type Record Stats
+ALL-NEXT: ============================================================
+
+ALL: Types
+ALL-NEXT: Total: 75 entries ( 5,336 bytes, 71.15 avg)
+ALL-NEXT: --------------------------------------------------------------------------
+ALL-NEXT: LF_FIELDLIST: 14 entries ( 1,956 bytes, 139.71 avg)
+ALL-NEXT: LF_STRUCTURE: 14 entries ( 1,584 bytes, 113.14 avg)
+ALL-NEXT: LF_ENUM: 8 entries ( 988 bytes, 123.50 avg)
+ALL-NEXT: LF_MFUNCTION: 14 entries ( 392 bytes, 28.00 avg)
+ALL-NEXT: LF_ARGLIST: 10 entries ( 176 bytes, 17.60 avg)
+ALL-NEXT: LF_METHODLIST: 5 entries ( 116 bytes, 23.20 avg)
+ALL-NEXT: LF_POINTER: 8 entries ( 96 bytes, 12.00 avg)
+ALL-NEXT: LF_PROCEDURE: 1 entries ( 16 bytes, 16.00 avg)
+ALL-NEXT: LF_MODIFIER: 1 entries ( 12 bytes, 12.00 avg)
Modified: llvm/trunk/test/DebugInfo/PDB/udt-stats.test
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/DebugInfo/PDB/udt-stats.test?rev=356813&r1=356812&r2=356813&view=diff
==============================================================================
--- llvm/trunk/test/DebugInfo/PDB/udt-stats.test (original)
+++ llvm/trunk/test/DebugInfo/PDB/udt-stats.test Fri Mar 22 14:22:13 2019
@@ -4,12 +4,12 @@ CHECK: S_UDT Record
CHECK-NEXT: ============================================================
CHECK: Record Kind | Count Size
CHECK-NEXT: -----------------------------
-CHECK-NEXT: LF_ENUM | 3 188
+CHECK-NEXT: LF_STRUCTURE | 27 1,788
CHECK-NEXT: LF_POINTER | 39 468
+CHECK-NEXT: LF_ENUM | 3 188
CHECK-NEXT: LF_UNION | 1 52
-CHECK-NEXT: <simple type> | 43 0
CHECK-NEXT: LF_PROCEDURE | 1 16
-CHECK-NEXT: LF_STRUCTURE | 27 1,788
+CHECK-NEXT: <simple type> | 43 0
CHECK-NEXT: -----------------------------
CHECK-NEXT: Total (S_UDT) | 114 2,604
CHECK-NEXT: -----------------------------
Modified: llvm/trunk/tools/llvm-pdbutil/DumpOutputStyle.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-pdbutil/DumpOutputStyle.cpp?rev=356813&r1=356812&r2=356813&view=diff
==============================================================================
--- llvm/trunk/tools/llvm-pdbutil/DumpOutputStyle.cpp (original)
+++ llvm/trunk/tools/llvm-pdbutil/DumpOutputStyle.cpp Fri Mar 22 14:22:13 2019
@@ -110,6 +110,12 @@ Error DumpOutputStyle::dump() {
P.NewLine();
}
+ if (opts::dump::DumpTypeStats) {
+ if (auto EC = dumpTypeStats())
+ return EC;
+ P.NewLine();
+ }
+
if (opts::dump::DumpNamedStreams) {
if (auto EC = dumpNamedStreams())
return EC;
@@ -307,18 +313,30 @@ static inline std::string formatModuleDe
return formatSymbolKind(K);
}
+// Get the stats sorted by size, descending.
+std::vector<StatCollection::KindAndStat>
+StatCollection::getStatsSortedBySize() const {
+ std::vector<KindAndStat> SortedStats(Individual.begin(), Individual.end());
+ std::stable_sort(SortedStats.begin(), SortedStats.end(),
+ [](const KindAndStat &LHS, const KindAndStat &RHS) {
+ return LHS.second.Size > RHS.second.Size;
+ });
+ return SortedStats;
+}
+
template <typename Kind>
static void printModuleDetailStats(LinePrinter &P, StringRef Label,
const StatCollection &Stats) {
P.NewLine();
P.formatLine(" {0}", Label);
AutoIndent Indent(P);
- P.formatLine("{0,40}: {1,7} entries ({2,8} bytes)", "Total",
+ P.formatLine("{0,40}: {1,7} entries ({2,12:N} bytes)", "Total",
Stats.Totals.Count, Stats.Totals.Size);
P.formatLine("{0}", fmt_repeat('-', 74));
- for (const auto &K : Stats.Individual) {
+
+ for (const auto &K : Stats.getStatsSortedBySize()) {
std::string KindName = formatModuleDetailKind(Kind(K.first));
- P.formatLine("{0,40}: {1,7} entries ({2,8} bytes)", KindName,
+ P.formatLine("{0,40}: {1,7} entries ({2,12:N} bytes)", KindName,
K.second.Count, K.second.Size);
}
}
@@ -676,6 +694,35 @@ Error DumpOutputStyle::dumpSymbolStats()
return Error::success();
}
+Error DumpOutputStyle::dumpTypeStats() {
+ printHeader(P, "Type Record Stats");
+
+ // Iterate the types, categorize by kind, accumulate size stats.
+ StatCollection TypeStats;
+ LazyRandomTypeCollection &Types = File.types();
+ for (Optional<TypeIndex> TI = Types.getFirst(); TI; TI = Types.getNext(*TI)) {
+ CVType Type = Types.getType(*TI);
+ TypeStats.update(uint32_t(Type.kind()), Type.length());
+ }
+
+ P.NewLine();
+ P.formatLine(" Types");
+ AutoIndent Indent(P);
+ P.formatLine("{0,14}: {1,7} entries ({2,12:N} bytes, {3,7} avg)", "Total",
+ TypeStats.Totals.Count, TypeStats.Totals.Size,
+ (double)TypeStats.Totals.Size / TypeStats.Totals.Count);
+ P.formatLine("{0}", fmt_repeat('-', 74));
+
+ for (const auto &K : TypeStats.getStatsSortedBySize()) {
+ P.formatLine("{0,14}: {1,7} entries ({2,12:N} bytes, {3,7} avg)",
+ formatTypeLeafKind(TypeLeafKind(K.first)), K.second.Count,
+ K.second.Size, (double)K.second.Size / K.second.Count);
+ }
+
+
+ return Error::success();
+}
+
static bool isValidNamespaceIdentifier(StringRef S) {
if (S.empty())
return false;
@@ -820,7 +867,7 @@ Error DumpOutputStyle::dumpUdtStats() {
fmt_align(SizeHeader, AlignStyle::Right, SD));
P.formatLine("{0}", fmt_repeat('-', TableWidth));
- for (const auto &Stat : UdtTargetStats.Individual) {
+ for (const auto &Stat : UdtTargetStats.getStatsSortedBySize()) {
StringRef Label = getUdtStatLabel(Stat.first);
P.formatLine("{0} | {1:N} {2:N}",
fmt_align(Label, AlignStyle::Right, FieldWidth),
@@ -833,12 +880,25 @@ Error DumpOutputStyle::dumpUdtStats() {
fmt_align(UdtStats.Totals.Count, AlignStyle::Right, CD),
fmt_align(UdtStats.Totals.Size, AlignStyle::Right, SD));
P.formatLine("{0}", fmt_repeat('-', TableWidth));
- for (const auto &Stat : NamespacedStats) {
- std::string Label = formatv("namespace '{0}'", Stat.getKey());
+ struct StrAndStat {
+ StringRef Key;
+ StatCollection::Stat Stat;
+ };
+
+ // Print namespace stats in descending order of size.
+ std::vector<StrAndStat> NamespacedStatsSorted;
+ for (const auto &Stat : NamespacedStats)
+ NamespacedStatsSorted.push_back({Stat.getKey(), Stat.second});
+ std::stable_sort(NamespacedStatsSorted.begin(), NamespacedStatsSorted.end(),
+ [](const StrAndStat &L, const StrAndStat &R) {
+ return L.Stat.Size > R.Stat.Size;
+ });
+ for (const auto &Stat : NamespacedStatsSorted) {
+ std::string Label = formatv("namespace '{0}'", Stat.Key);
P.formatLine("{0} | {1:N} {2:N}",
fmt_align(Label, AlignStyle::Right, FieldWidth),
- fmt_align(Stat.second.Count, AlignStyle::Right, CD),
- fmt_align(Stat.second.Size, AlignStyle::Right, SD));
+ fmt_align(Stat.Stat.Count, AlignStyle::Right, CD),
+ fmt_align(Stat.Stat.Size, AlignStyle::Right, SD));
}
return Error::success();
}
Modified: llvm/trunk/tools/llvm-pdbutil/DumpOutputStyle.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-pdbutil/DumpOutputStyle.h?rev=356813&r1=356812&r2=356813&view=diff
==============================================================================
--- llvm/trunk/tools/llvm-pdbutil/DumpOutputStyle.h (original)
+++ llvm/trunk/tools/llvm-pdbutil/DumpOutputStyle.h Fri Mar 22 14:22:13 2019
@@ -49,6 +49,8 @@ struct StatCollection {
}
};
+ using KindAndStat = std::pair<uint32_t, Stat>;
+
void update(uint32_t Kind, uint32_t RecordSize) {
Totals.update(RecordSize);
auto Iter = Individual.try_emplace(Kind, 1, RecordSize);
@@ -57,6 +59,8 @@ struct StatCollection {
}
Stat Totals;
DenseMap<uint32_t, Stat> Individual;
+
+ std::vector<KindAndStat> getStatsSortedBySize() const;
};
class DumpOutputStyle : public OutputStyle {
@@ -78,6 +82,7 @@ private:
Error dumpStreamSummary();
Error dumpSymbolStats();
Error dumpUdtStats();
+ Error dumpTypeStats();
Error dumpNamedStreams();
Error dumpStringTable();
Error dumpStringTableFromPdb();
Modified: llvm/trunk/tools/llvm-pdbutil/llvm-pdbutil.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-pdbutil/llvm-pdbutil.cpp?rev=356813&r1=356812&r2=356813&view=diff
==============================================================================
--- llvm/trunk/tools/llvm-pdbutil/llvm-pdbutil.cpp (original)
+++ llvm/trunk/tools/llvm-pdbutil/llvm-pdbutil.cpp Fri Mar 22 14:22:13 2019
@@ -462,7 +462,10 @@ cl::opt<bool> DumpSymbolStats(
"sym-stats",
cl::desc("Dump a detailed breakdown of symbol usage/size for each module"),
cl::cat(MsfOptions), cl::sub(DumpSubcommand));
-
+cl::opt<bool> DumpTypeStats(
+ "type-stats",
+ cl::desc("Dump a detailed breakdown of type usage/size"),
+ cl::cat(MsfOptions), cl::sub(DumpSubcommand));
cl::opt<bool> DumpUdtStats(
"udt-stats",
cl::desc("Dump a detailed breakdown of S_UDT record usage / stats"),
Modified: llvm/trunk/tools/llvm-pdbutil/llvm-pdbutil.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-pdbutil/llvm-pdbutil.h?rev=356813&r1=356812&r2=356813&view=diff
==============================================================================
--- llvm/trunk/tools/llvm-pdbutil/llvm-pdbutil.h (original)
+++ llvm/trunk/tools/llvm-pdbutil/llvm-pdbutil.h Fri Mar 22 14:22:13 2019
@@ -140,6 +140,7 @@ extern llvm::cl::opt<bool> DumpSummary;
extern llvm::cl::opt<bool> DumpFpm;
extern llvm::cl::opt<bool> DumpStreams;
extern llvm::cl::opt<bool> DumpSymbolStats;
+extern llvm::cl::opt<bool> DumpTypeStats;
extern llvm::cl::opt<bool> DumpUdtStats;
extern llvm::cl::opt<bool> DumpStreamBlocks;
More information about the llvm-commits
mailing list