[compiler-rt] [llvm] [docs][IRPGO]Document two binary formats for IRPGO profiles (PR #76105)
Mingming Liu via llvm-commits
llvm-commits at lists.llvm.org
Wed Dec 20 17:44:30 PST 2023
https://github.com/minglotus-6 updated https://github.com/llvm/llvm-project/pull/76105
>From e3bfecf57ff7440ed50b51114df0d726c0e16b61 Mon Sep 17 00:00:00 2001
From: mingmingl <mingmingl at google.com>
Date: Tue, 19 Dec 2023 20:57:56 -0800
Subject: [PATCH 1/4] [compiler-rt][test]Mark thinlto icp test as UNSUPPORTED.
Test failed when building instrumented binary
---
.../profile/instrprof-thinlto-indirect-call-promotion.cpp | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/compiler-rt/test/profile/instrprof-thinlto-indirect-call-promotion.cpp b/compiler-rt/test/profile/instrprof-thinlto-indirect-call-promotion.cpp
index 08efa42167e94d..adba5b24253a42 100644
--- a/compiler-rt/test/profile/instrprof-thinlto-indirect-call-promotion.cpp
+++ b/compiler-rt/test/profile/instrprof-thinlto-indirect-call-promotion.cpp
@@ -13,8 +13,13 @@
// - Generate ThinLTO summary file with LLVM bitcodes, and run `function-import` pass.
// - Run `pgo-icall-prom` pass for the IR module which needs to import callees.
+
// REQUIRES: windows || linux || darwin
+// The test failed on ppc when building the instrumented binary.
+// ld.lld: error: /lib/../lib64/Scrt1.o: ABI version 1 is not supported
+// UNSUPPORTED: ppc
+
// This test and IR test llvm/test/Transforms/PGOProfile/thinlto_indirect_call_promotion.ll
// are complementary to each other; a compiler-rt test has better test coverage
// on different platforms, and the IR test is less restrictive in terms of
>From 1b3a6b9d4e21e7355e086d2527b1a33835cba551 Mon Sep 17 00:00:00 2001
From: mingmingl <mingmingl at google.com>
Date: Tue, 19 Dec 2023 20:58:51 -0800
Subject: [PATCH 2/4] Revert "[compiler-rt][test]Mark thinlto icp test as
UNSUPPORTED. Test failed when building instrumented binary"
This reverts commit e3bfecf57ff7440ed50b51114df0d726c0e16b61.
---
.../profile/instrprof-thinlto-indirect-call-promotion.cpp | 5 -----
1 file changed, 5 deletions(-)
diff --git a/compiler-rt/test/profile/instrprof-thinlto-indirect-call-promotion.cpp b/compiler-rt/test/profile/instrprof-thinlto-indirect-call-promotion.cpp
index adba5b24253a42..08efa42167e94d 100644
--- a/compiler-rt/test/profile/instrprof-thinlto-indirect-call-promotion.cpp
+++ b/compiler-rt/test/profile/instrprof-thinlto-indirect-call-promotion.cpp
@@ -13,13 +13,8 @@
// - Generate ThinLTO summary file with LLVM bitcodes, and run `function-import` pass.
// - Run `pgo-icall-prom` pass for the IR module which needs to import callees.
-
// REQUIRES: windows || linux || darwin
-// The test failed on ppc when building the instrumented binary.
-// ld.lld: error: /lib/../lib64/Scrt1.o: ABI version 1 is not supported
-// UNSUPPORTED: ppc
-
// This test and IR test llvm/test/Transforms/PGOProfile/thinlto_indirect_call_promotion.ll
// are complementary to each other; a compiler-rt test has better test coverage
// on different platforms, and the IR test is less restrictive in terms of
>From ad7fe43817a0aa77f5d56e9c0813f81f5f0b18c4 Mon Sep 17 00:00:00 2001
From: mingmingl <mingmingl at google.com>
Date: Wed, 20 Dec 2023 14:17:12 -0800
Subject: [PATCH 3/4] [docs][IRPGO]Document two binary formats for IRPGO
profiles
---
llvm/docs/PGOProfileFormat.rst | 387 +++++++++++++++++++++++++++++++++
llvm/docs/UserGuides.rst | 4 +
2 files changed, 391 insertions(+)
create mode 100644 llvm/docs/PGOProfileFormat.rst
diff --git a/llvm/docs/PGOProfileFormat.rst b/llvm/docs/PGOProfileFormat.rst
new file mode 100644
index 00000000000000..5602172e147f00
--- /dev/null
+++ b/llvm/docs/PGOProfileFormat.rst
@@ -0,0 +1,387 @@
+=====================
+IRPGO Profile Format
+=====================
+
+.. contents::
+ :local:
+
+
+Overview
+==========
+
+IR-based instrumentation (IRPGO) and its context-sensitive variant (CS-IRPGO)
+inserts `llvm.instrprof.*` `code generator intrinsics <https://llvm.org/docs/LangRef.html#code-generator-intrinsics>`_
+in LLVM IR to generate profiles. This document describes two binary profile
+formats (raw and indexed) used by IR-based instrumentation.
+
+.. note::
+
+ Both the compiler-rt profiling infrastructure and profile format are general
+ and could support other use cases (e.g., coverage and temporal profiling).
+ This document will focus on IRPGO while briefly introducing other use cases
+ with pointers.
+
+Raw PGO Profile Format
+========================
+
+The raw PGO profile is generated by running the instrumented binary. It is a
+memory dump of the profile data.
+
+Two kinds of frequently used profile information are function's basic block
+counters and its (various flavors of) value profiles. A function's profiled
+information span across several sections in the profile.
+
+General Storage Layout
+-----------------------
+
+A raw profile for an executable [1]_ consists of a profile header and several
+sections. The storage layout is illustrated below. Generally, when raw profile
+is read into an memory buffer, the actual byte offset of a section is inferred
+from the section's order in the layout and size information of all sections
+ahead of it.
+
+::
+
+ +----+-----------------------+
+ | | Magic |
+ | +-----------------------+
+ | | Version |
+ | +-----------------------+
+ H | Size Info for |
+ E | Section 1 |
+ A +-----------------------+
+ D | Size Info for |
+ E | Section 2 |
+ R +-----------------------+
+ | | ... |
+ | +-----------------------+
+ | | Size Info for |
+ | | Section N |
+ +----+-----------------------+
+ P | Section 1 |
+ A +-----------------------+
+ Y | Section 2 |
+ L +-----------------------+
+ O | ... |
+ A +-----------------------+
+ D | Section N |
+ +----+-----------------------+
+
+
+.. note::
+ Sections might be padded to meet platform-specific alignment requirements.
+ For simplicity, header fields and data sections solely for padding purpose
+ are omitted in the data layout graph above and the rest of this document.
+
+Header
+-------
+
+``Magic``
+ With the magic number, data consumer could detect profile format and
+ endianness of the data, and quickly tells whether/how to continue reading.
+
+``Version``
+ The lower 32 bits specifies the actual version and the most significant 32
+ bits specify the variant types of the profile. IRPGO and CS-IRPGO are two
+ variant types.
+
+``BinaryIdsSize``
+ The byte size of binary id section.
+
+``NumData``
+ The number of per-function profile data control structures. The byte size of
+ profile data section could be computed with this field.
+
+``NumCounter``
+ The number of entries in the profile counter section. The byte size of counter
+ section could be computed with this field.
+
+``NumBitmapBytes``
+ The number of bytes in the profile bitmap section.
+
+``NamesSize``
+ The number of bytes in the name section.
+
+``CountersDelta``
+ Records the in-memory address difference between the data and counter section,
+ i.e., `start(__llvm_prf_cnts) - start(__llvm_prf_data)`. It's used jointly
+ with the in-memory address difference of profile data record and its counter
+ to find the counter of a profile data record. Check out calculation-of-counter-offset_
+ for details.
+
+``BitmapDelta``
+ Records the in-memory address difference between the data and bitmap section,
+ i.e., `start(__llvm_prf_bits) - start(__llvm_prf_data)`. It's used jointly
+ with the in-memory address difference of a profile data record and its bitmap
+ to find the bitmap of a profile data record, in a similar to how counters are
+ referenced as explained by calculation-of-counter-offset_ .
+
+``NamesDelta``
+ Records the in-memory address of compressed name section. Not used except for
+ raw profile reader error checking.
+
+``ValueKindLast``
+ Records the number of value kinds. As of writing, two kinds of value profiles
+ are supported. `IndirectCallTarget` is to profile the frequent callees of
+ indirect call instructions and `MemOPSize` is for memory intrinsic function
+ size profiling.
+
+ The number of value kinds affects the byte size of per function profile data
+ control structure.
+
+Payload Sections
+------------------
+
+Binary Ids
+^^^^^^^^^^^
+Stores the binary ids of the instrumented binaries to associate binaries with
+profiles for source code coverage. See `Binary Id RFC`_ for introduction.
+
+.. _`Binary Id RFC`: https://lists.llvm.org/pipermail/llvm-dev/2021-June/151154.html
+
+Profile Data
+^^^^^^^^^^^^^
+
+This section stores per-function profile data control structure. The in-memory
+representation of the control structure is `__llvm_profile_data` and the fields
+are defined by `INSTRPROFDATA` macro. Some fields are used to reference data
+from other sections in the profile. The fields are documented as follows:
+
+``NameRef``
+ The MD5 of the function's IRPGO name. IRPGO name has the format
+ `[<filepath>;]<linkage-name>` where `<filepath>;` is provided for local-linkage
+ functions to tell possibly identical function names.
+
+``FuncHash``
+ A fingerprint of the function's control flow graph.
+
+``CounterPtr``
+ The in-memory address difference between profile data and its corresponding counters.
+
+``BitmapPtr``
+ The in-memory address difference between profile data and its bitmap.
+
+``FunctionPointer``
+ Records the function address when instrumented binary runs. This is used to
+ map the profiled callee address of indirect calls to the `NameRef` during
+ conversion from raw to indexed profiles.
+
+``Values``
+ Represents value profiles in a two dimensional array. The number of elements
+ in the first dimension is the number of instrumented value sites across all
+ kinds. Each element in the first dimension is the head of a linked list, and
+ the each element in the second dimension is linked list element, carrying
+ `<profiled-value, count>` as payload. This is used by compiler runtime when
+ writing out value profiles.
+
+``NumCounters``
+ The number of counters for the instrumented function.
+
+``NumValueSites``
+ This is an array of counters, and each counter represents the number of
+ instrumented sites for a kind of value in the function.
+
+``NumBitmapBytes``
+ The number of bitmap bytes for the function.
+
+Profile Counters
+^^^^^^^^^^^^^^^^^
+
+For IRPGO [2]_, the counters within an instrumented function are stored contiguously
+and in an order that is consistent with basic block selection in the instrumentation
+pass.
+
+.. _calculation-of-counter-offset:
+
+So how are function counters associated with a function?
+
+Basically, the profile reader iterates per-function control structure (from the
+profile data section) and makes use of the recorded relative distances, as
+illustrated below.
+
+::
+
+ + --> start(__llvm_prf_data) --> +---------------------+ ------------+
+ | | Data 1 | |
+ | +---------------------+ =====|| |
+ | | Data 2 | || |
+ | +---------------------+ || |
+ | | ... | || |
+ Counter| +---------------------+ || |
+ Delta | | Data N | || |
+ | +---------------------+ || | CounterPtr1
+ | || |
+ | CounterPtr2 || |
+ | || |
+ | || |
+ + --> start(__llvm_prf_cnts) --> +---------------------+ || |
+ | ... | || |
+ +---------------------+ -----||----+
+ | Counter 1 | ||
+ +---------------------+ ||
+ | ... | ||
+ +---------------------+ =====||
+ | Counter 2 |
+ +---------------------+
+ | ... |
+ +---------------------+
+ | Counter N |
+ +---------------------+
+
+
+In the graph,
+
+* The profile header records `CounterDelta` with the value as `start(__llvm_prf_cnts) - start(__llvm_prf_data)`.
+ We will call it `CounterDeltaInitVal` below for convenience.
+* For each profile data record, `CounterPtrN` is recorded as `start(Counter) - start(ProfileData)`.
+
+Each time the reader advances to the next data record, it updates `CounterDelta` to minus the size of one `ProfileData`.
+
+For the counter corresponding to the first data record, the byte offset
+relative to the start of the counter section is calculated as `CounterPtr1 - CounterDeltaInitVal`.
+When profile reader advances to the second data record, note `CounterDelta` is now `CounterDeltaInitVal - sizeof(ProfileData)`.
+Thus the byte offset relative to the start of the counter section is calculated as `CounterPtr2 - (CounterDeltaInitVal - sizeof(ProfileData))`.
+
+Bitmap
+^^^^^^^
+This section is used for source-based MC/DC code coverage. Check out `Bitmap RFC`_
+if interested.
+
+.. _`Bitmap RFC`: https://discourse.llvm.org/t/rfc-source-based-mc-dc-code-coverage/59244
+
+Names
+^^^^^^
+
+This section contains the concatenated string of function IRPGO names. If
+compressed, zlib compression algorithm is used.
+
+Function names serve as keys in the PGO data hash table when raw profiles are
+converted into indexed profiles. They are also crucial for `llvm-profdata` to
+show the profiles in a human-readable way.
+
+Value Profile Data
+^^^^^^^^^^^^^^^^^^^^
+
+This section contains the profile data for value profiling.
+
+The value profiles corresponding to a profile data are serialized contiguously
+as one record, and value profile records are stored in the same order as the
+respective profile data, such that a raw profile reader advances the pointer to
+profile data and the pointer to value profile records simutaneously [3]_ to find
+value profiles for a per function, per cfg fingerprint profile data.
+
+Indexed PGO Profile Format
+===========================
+
+General Storage Layout
+-----------------------
+
+::
+
+ +-----------------------+---+
+ | Magic | |
+ +-----------------------+ |
+ | Version | |
+ +-----------------------+ |
+ | HashType | H
+ +-----------------------+ E
+ +-------| HashOffset | A
+ | +-----------------------+ D
+ +-----------| MemProfOffset | E
+ | | +-----------------------+ R
+ | | | BinaryIdOffset | |
+ | | +-----------------------+ |
+ +---------------| TemporalProf- | |
+ | | | | TracesOffset | |
+ | | | +-----------------------+---+
+ | | | | Profile Summary | |
+ | | | +-----------------------+ P
+ | | +------>| Function PGO data | A
+ | | +-----------------------+ Y
+ | +---------- | MemProf profile data | L
+ | +-----------------------+ O
+ | | Binary Ids | A
+ | +-----------------------+ D
+ +-------------->| Temporal profiles | |
+ +-----------------------+---+
+
+Header
+--------
+
+``Magic``
+ The purpose of the magic number is to be able to quickly tell if the profile
+ is an indexed profile.
+
+``Version``
+ Similar to raw profile version, the lower 32 bits specifies the version of the
+ indexed profile and the most significant 32 bits are reserved to specify the
+ variant types of the profile.
+
+``HashType``
+ The hashing scheme for on-disk hash table keys. Only MD5 hashing is used as of
+ writing.
+
+``HashOffset``
+ An on-disk hash table stores the per-function profile records.
+ Precisely speaking, `HashOffset` records the offset of this hash table's
+ metadata (i.e., the number of buckets and entries), which follows right after
+ the payload of the entire hash table.
+
+``MemProfOffset``
+ Records the byte offset of MemProf profiling data.
+
+``BinaryIdOffset``
+ Records the byte offset of binary id sections.
+
+``TemporalProfTracesOffset``
+ Records the byte offset of temporal profiles.
+
+Payload Sections
+------------------
+
+(CS) Profile Summary
+^^^^^^^^^^^^^^^^^^^^^
+This section is right after profile header. It stores the serialized profile
+summary. For context-sensitive IRPGO, this section stores an additional profile
+summary corresponding to the context-sensitive profiles.
+
+Function PGO data
+^^^^^^^^^^^^^^^^^^
+This section stores functions and their PGO profiling data as an on-disk hash
+table. The key of a hash table entry is function's PGO name, and the in-memory
+representation of value is a map. The key of this map is CFG hash, and the value
+is C++ struct `llvm::InstrProfRecord`. The C++ struct collects the profiling
+information like counters and value profiles.
+
+MemProf Profile data
+^^^^^^^^^^^^^^^^^^^^^^
+This section stores function's memory profiling data. See
+`MemProf binary serialization format RFC`_ for the design.
+
+.. _`MemProf binary serialization format RFC`: https://lists.llvm.org/pipermail/llvm-dev/2021-September/153007.html
+
+Binary Ids
+^^^^^^^^^^^^^^^^^^^^^^
+The section to carry on binary-id information from raw profiles.
+
+Temporal Profile Traces
+^^^^^^^^^^^^^^^^^^^^^^^^
+The section to carry on temporal profile information from raw profiles.
+See `Temporal profiling RFC`_ for an overview.
+
+.. _`Temporal profiling RFC`: https://discourse.llvm.org/t/rfc-temporal-profiling-extension-for-irpgo/68068
+
+Profile Data Usage
+=======================================
+
+`llvm-profdata` is the command line tool to display and process profile data.
+For supported usages, check out its `documentation <https://llvm.org/docs/CommandGuide/llvm-profdata.html>`_.
+
+
+.. [1] A raw profile file could contain multiple raw profiles. Raw profile
+ reader could parse all raw profiles from the file correctly.
+.. [2] The counter section is used by a few variant types (like coverage and
+ temporal profiling) and might have different semantics there.
+.. [3] The step size of data pointer is the `sizeof(ProfileData)`, and the step
+ size of value profile pointer is calcuated based on the number of collected
+ values.
diff --git a/llvm/docs/UserGuides.rst b/llvm/docs/UserGuides.rst
index 006df613bc5e7d..14a2e161ea54cb 100644
--- a/llvm/docs/UserGuides.rst
+++ b/llvm/docs/UserGuides.rst
@@ -58,6 +58,7 @@ intermediate LLVM representation.
NVPTXUsage
Phabricator
Passes
+ PGOProfileFormat
ReportingGuide
ResponseGuide
Remarks
@@ -177,6 +178,9 @@ Optimizations
referencing, to determine variable locations for debug info in the final
stages of compilation.
+:doc:`PGOProfileFormat`
+ This document explains two binary formats of IRPGO profiles.
+
Code Generation
---------------
>From ac0e5507946c1d25ec4c1eb1c10ea4298083c3c7 Mon Sep 17 00:00:00 2001
From: mingmingl <mingmingl at google.com>
Date: Wed, 20 Dec 2023 17:36:28 -0800
Subject: [PATCH 4/4] resolve feedback
---
llvm/docs/PGOProfileFormat.rst | 148 +++++++++++++++++----------------
1 file changed, 78 insertions(+), 70 deletions(-)
diff --git a/llvm/docs/PGOProfileFormat.rst b/llvm/docs/PGOProfileFormat.rst
index 5602172e147f00..37b1049d84d761 100644
--- a/llvm/docs/PGOProfileFormat.rst
+++ b/llvm/docs/PGOProfileFormat.rst
@@ -1,44 +1,46 @@
-=====================
-IRPGO Profile Format
-=====================
+===================================
+Instrumentation PGO Profile Format
+===================================
.. contents::
:local:
Overview
-==========
+=========
-IR-based instrumentation (IRPGO) and its context-sensitive variant (CS-IRPGO)
-inserts `llvm.instrprof.*` `code generator intrinsics <https://llvm.org/docs/LangRef.html#code-generator-intrinsics>`_
-in LLVM IR to generate profiles. This document describes two binary profile
-formats (raw and indexed) used by IR-based instrumentation.
+Instrumentation PGO inserts `llvm.instrprof.*` `code generator intrinsics`_
+in the code to generate profiles. This document describes two binary profile
+formats (raw and indexed) used by instrumentation.
+
+.. _`code generator intrinsics`: https://llvm.org/docs/LangRef.html#code-generator-intrinsics
.. note::
+ The instrumentation profile format supports non-PGO use cases (e.g., temporal
+ profiling). This document will focus on PGO. Source coverage uses both
+ frontend instrumentation profiles and coverage mapping. The format for
+ coverage mapping has its own `documentation`_.
- Both the compiler-rt profiling infrastructure and profile format are general
- and could support other use cases (e.g., coverage and temporal profiling).
- This document will focus on IRPGO while briefly introducing other use cases
- with pointers.
+.. _`documentation`: https://llvm.org/docs/CoverageMappingFormat.html
-Raw PGO Profile Format
-========================
+Raw Profile Format
+===================
-The raw PGO profile is generated by running the instrumented binary. It is a
-memory dump of the profile data.
+The raw profile is generated by running the instrumented binary. It is a memory
+dump of the profile data.
-Two kinds of frequently used profile information are function's basic block
-counters and its (various flavors of) value profiles. A function's profiled
-information span across several sections in the profile.
+The instrumented binary currently collects two kinds of profile data, counters
+to profile branch probability and (various flavors of) value profiles. The
+profile data for a function span across several sections in the profile.
General Storage Layout
-----------------------
-A raw profile for an executable [1]_ consists of a profile header and several
-sections. The storage layout is illustrated below. Generally, when raw profile
-is read into an memory buffer, the actual byte offset of a section is inferred
-from the section's order in the layout and size information of all sections
-ahead of it.
+A raw profile from an executable or a shared library [1]_ consists of a profile
+header and several sections. The storage layout is illustrated below. Generally,
+when the raw profile is read into an memory buffer, the actual byte offset of a
+section is inferred from the section's order in the layout and size information
+of all the sections ahead of it.
::
@@ -78,12 +80,12 @@ Header
``Magic``
With the magic number, data consumer could detect profile format and
- endianness of the data, and quickly tells whether/how to continue reading.
+ endianness of the data, and tells whether/how to continue reading.
``Version``
The lower 32 bits specifies the actual version and the most significant 32
- bits specify the variant types of the profile. IRPGO and CS-IRPGO are two
- variant types.
+ bits specify the variant types of the profile. IR-based instrumentation PGO
+ and context-sensitive IR-based instrumentation PGO are two variant types.
``BinaryIdsSize``
The byte size of binary id section.
@@ -113,12 +115,12 @@ Header
Records the in-memory address difference between the data and bitmap section,
i.e., `start(__llvm_prf_bits) - start(__llvm_prf_data)`. It's used jointly
with the in-memory address difference of a profile data record and its bitmap
- to find the bitmap of a profile data record, in a similar to how counters are
- referenced as explained by calculation-of-counter-offset_ .
+ to find the bitmap of a profile data record, in a similar way to how counters
+ are referenced as explained by calculation-of-counter-offset_ .
``NamesDelta``
- Records the in-memory address of compressed name section. Not used except for
- raw profile reader error checking.
+ Records the in-memory address of name section. Not used except for raw profile
+ reader error checking.
``ValueKindLast``
Records the number of value kinds. As of writing, two kinds of value profiles
@@ -148,15 +150,17 @@ are defined by `INSTRPROFDATA` macro. Some fields are used to reference data
from other sections in the profile. The fields are documented as follows:
``NameRef``
- The MD5 of the function's IRPGO name. IRPGO name has the format
- `[<filepath>;]<linkage-name>` where `<filepath>;` is provided for local-linkage
- functions to tell possibly identical function names.
+ The MD5 of the function's PGO name. PGO name has the format
+ `[<filepath><delimiter>]<linkage-or-mangled-name>` where `<filepath>` and
+ `<delimiter>` is provided for local-linkage functions to tell possibly
+ identical functions.
``FuncHash``
A fingerprint of the function's control flow graph.
``CounterPtr``
- The in-memory address difference between profile data and its corresponding counters.
+ The in-memory address difference between profile data and its corresponding
+ counters.
``BitmapPtr``
The in-memory address difference between profile data and its bitmap.
@@ -187,9 +191,9 @@ from other sections in the profile. The fields are documented as follows:
Profile Counters
^^^^^^^^^^^^^^^^^
-For IRPGO [2]_, the counters within an instrumented function are stored contiguously
-and in an order that is consistent with basic block selection in the instrumentation
-pass.
+For PGO [2]_, the counters within an instrumented function are stored contiguously
+and in an order that is consistent with instrumentation points selection in the
+instrumentation pass.
.. _calculation-of-counter-offset:
@@ -235,12 +239,15 @@ In the graph,
We will call it `CounterDeltaInitVal` below for convenience.
* For each profile data record, `CounterPtrN` is recorded as `start(Counter) - start(ProfileData)`.
-Each time the reader advances to the next data record, it updates `CounterDelta` to minus the size of one `ProfileData`.
+Each time the reader advances to the next data record, it updates `CounterDelta`
+to minus the size of one `ProfileData`.
For the counter corresponding to the first data record, the byte offset
relative to the start of the counter section is calculated as `CounterPtr1 - CounterDeltaInitVal`.
-When profile reader advances to the second data record, note `CounterDelta` is now `CounterDeltaInitVal - sizeof(ProfileData)`.
-Thus the byte offset relative to the start of the counter section is calculated as `CounterPtr2 - (CounterDeltaInitVal - sizeof(ProfileData))`.
+When profile reader advances to the second data record, note `CounterDelta`
+is updated to `CounterDeltaInitVal - sizeof(ProfileData)`.
+Thus the byte offset relative to the start of the counter section is calculated
+as `CounterPtr2 - (CounterDeltaInitVal - sizeof(ProfileData))`.
Bitmap
^^^^^^^
@@ -252,8 +259,8 @@ if interested.
Names
^^^^^^
-This section contains the concatenated string of function IRPGO names. If
-compressed, zlib compression algorithm is used.
+This section contains possibly compressed concatenated string of functions' PGO
+names. If compressed, zlib compression algorithm is used.
Function names serve as keys in the PGO data hash table when raw profiles are
converted into indexed profiles. They are also crucial for `llvm-profdata` to
@@ -289,18 +296,18 @@ General Storage Layout
| +-----------------------+ D
+-----------| MemProfOffset | E
| | +-----------------------+ R
- | | | BinaryIdOffset | |
- | | +-----------------------+ |
+ | | +--| BinaryIdOffset | |
+ | | | +-----------------------+ |
+---------------| TemporalProf- | |
- | | | | TracesOffset | |
- | | | +-----------------------+---+
- | | | | Profile Summary | |
- | | | +-----------------------+ P
+ | | | | | TracesOffset | |
+ | | | | +-----------------------+---+
+ | | | | | Profile Summary | |
+ | | | | +-----------------------+ P
| | +------>| Function PGO data | A
- | | +-----------------------+ Y
+ | | | +-----------------------+ Y
| +---------- | MemProf profile data | L
- | +-----------------------+ O
- | | Binary Ids | A
+ | | +-----------------------+ O
+ | +--| Binary Ids | A
| +-----------------------+ D
+-------------->| Temporal profiles | |
+-----------------------+---+
@@ -322,10 +329,10 @@ Header
writing.
``HashOffset``
- An on-disk hash table stores the per-function profile records.
- Precisely speaking, `HashOffset` records the offset of this hash table's
- metadata (i.e., the number of buckets and entries), which follows right after
- the payload of the entire hash table.
+ An on-disk hash table stores the per-function profile records. It records the
+ offset of this hash table's metadata (i.e., the number of buckets and entries),
+ which follows right after the payload of the entire hash table for
+ deserialization.
``MemProfOffset``
Records the byte offset of MemProf profiling data.
@@ -342,16 +349,16 @@ Payload Sections
(CS) Profile Summary
^^^^^^^^^^^^^^^^^^^^^
This section is right after profile header. It stores the serialized profile
-summary. For context-sensitive IRPGO, this section stores an additional profile
-summary corresponding to the context-sensitive profiles.
+summary. For context-sensitive IR-based instrumentation PGO, this section stores
+an additional profile summary corresponding to the context-sensitive profiles.
Function PGO data
^^^^^^^^^^^^^^^^^^
This section stores functions and their PGO profiling data as an on-disk hash
table. The key of a hash table entry is function's PGO name, and the in-memory
-representation of value is a map. The key of this map is CFG hash, and the value
-is C++ struct `llvm::InstrProfRecord`. The C++ struct collects the profiling
-information like counters and value profiles.
+representation of value is a map. The key of this map is the fingerprint of CFG,
+and the value is a C++ struct named `llvm::InstrProfRecord`. The C++ struct
+collects the profiling information like counters and value profiles.
MemProf Profile data
^^^^^^^^^^^^^^^^^^^^^^
@@ -362,26 +369,27 @@ This section stores function's memory profiling data. See
Binary Ids
^^^^^^^^^^^^^^^^^^^^^^
-The section to carry on binary-id information from raw profiles.
+The section is used to carry on binary-id information from raw profiles.
Temporal Profile Traces
^^^^^^^^^^^^^^^^^^^^^^^^
-The section to carry on temporal profile information from raw profiles.
-See `Temporal profiling RFC`_ for an overview.
+The section is used to carry on temporal profile information from raw profiles.
+See `Temporal profiling RFC`_ for the design.
.. _`Temporal profiling RFC`: https://discourse.llvm.org/t/rfc-temporal-profiling-extension-for-irpgo/68068
Profile Data Usage
=======================================
-`llvm-profdata` is the command line tool to display and process profile data.
-For supported usages, check out its `documentation <https://llvm.org/docs/CommandGuide/llvm-profdata.html>`_.
+`llvm-profdata` is the command line tool to display and process instrumentation-
+based profile data. For supported usages, check out `llvm-profdata documentation <https://llvm.org/docs/CommandGuide/llvm-profdata.html>`_.
-.. [1] A raw profile file could contain multiple raw profiles. Raw profile
- reader could parse all raw profiles from the file correctly.
-.. [2] The counter section is used by a few variant types (like coverage and
- temporal profiling) and might have different semantics there.
+.. [1] A raw profile file could contain the concatenation of multiple raw
+ profiles. Raw profile reader could parse all raw profiles from the file
+ correctly.
+.. [2] The counter section is used by a few variant types (like temporal
+ profiling) and might have different semantics there.
.. [3] The step size of data pointer is the `sizeof(ProfileData)`, and the step
size of value profile pointer is calcuated based on the number of collected
values.
More information about the llvm-commits
mailing list