[llvm-branch-commits] [Remarks] Restructure bitstream remarks to be fully standalone (PR #156715)
via llvm-branch-commits
llvm-branch-commits at lists.llvm.org
Wed Sep 3 10:03:08 PDT 2025
llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-debuginfo
Author: Tobias Stadler (tobias-stadler)
<details>
<summary>Changes</summary>
Currently there are two serialization modes for bitstream Remarks:
standalone and separate. The separate mode splits remark metadata (e.g.
the string table) from actual remark data. The metadata is written into
the object file by the AsmPrinter, while the remark data is stored in a
separate remarks file. This means we can't use bitstream remarks with
tools like opt that don't generate an object file. Also, it is confusing
to post-process bitstream remarks files, because only the standalone
files can be read by llvm-remarkutil. We always need to use dsymutil
to convert the separate files to standalone files, which only works for
MachO. It is not possible for clang/opt to directly emit bitstream
remark files in standalone mode, because the string table can only be
serialized after all remarks were emitted.
Therefore, this change completely removes the separate serialization
mode. Instead, the remark string table is now always written to the end
of the remarks file. This requires us to tell the serializer when to
finalize remark serialization. This automatically happens when the
serializer goes out of scope. However, often the remark file goes out of
scope before the serializer is destroyed. To diagnose this, I have added
an assert to alert users that they need to explicitly call
finalizeLLVMOptimizationRemarks.
---
Patch is 118.86 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/156715.diff
80 Files Affected:
- (modified) clang/lib/CodeGen/CodeGenAction.cpp (+5)
- (modified) flang/lib/Frontend/FrontendActions.cpp (+2)
- (modified) llvm/docs/Remarks.rst (+22-101)
- (modified) llvm/include/llvm/IR/LLVMRemarkStreamer.h (+9-3)
- (modified) llvm/include/llvm/LTO/LTOBackend.h (+2-2)
- (modified) llvm/include/llvm/Remarks/BitstreamRemarkContainer.h (+24-24)
- (modified) llvm/include/llvm/Remarks/BitstreamRemarkSerializer.h (+38-77)
- (modified) llvm/include/llvm/Remarks/RemarkSerializer.h (+14-22)
- (modified) llvm/include/llvm/Remarks/RemarkStreamer.h (+9)
- (modified) llvm/include/llvm/Remarks/YAMLRemarkSerializer.h (+6-12)
- (modified) llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp (+9-10)
- (modified) llvm/lib/IR/LLVMRemarkStreamer.cpp (+10-4)
- (modified) llvm/lib/LTO/LTO.cpp (+9-6)
- (modified) llvm/lib/LTO/LTOBackend.cpp (+16-8)
- (modified) llvm/lib/LTO/LTOCodeGenerator.cpp (+1)
- (modified) llvm/lib/LTO/ThinLTOCodeGenerator.cpp (+2)
- (modified) llvm/lib/Remarks/BitstreamRemarkParser.cpp (+99-86)
- (modified) llvm/lib/Remarks/BitstreamRemarkParser.h (+26-22)
- (modified) llvm/lib/Remarks/BitstreamRemarkSerializer.cpp (+49-87)
- (modified) llvm/lib/Remarks/RemarkLinker.cpp (+1-1)
- (modified) llvm/lib/Remarks/RemarkSerializer.cpp (+7-9)
- (modified) llvm/lib/Remarks/RemarkStreamer.cpp (+12-12)
- (modified) llvm/lib/Remarks/YAMLRemarkParser.cpp (+5-1)
- (modified) llvm/lib/Remarks/YAMLRemarkSerializer.cpp (+11-10)
- (modified) llvm/test/Transforms/Inline/optimization-remarks-passed-yaml.ll (+5)
- (added) llvm/test/tools/dsymutil/ARM/remarks-linking-bundle-empty.test (+14)
- (renamed) llvm/test/tools/dsymutil/ARM/remarks-linking-bundle.test (+20-9)
- (modified) llvm/test/tools/dsymutil/Inputs/basic1.c (+2-2)
- (added) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/basic1.macho.remarks.arm64.o ()
- (added) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/basic1.macho.remarks.arm64.opt.bitstream ()
- (added) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/basic1.macho.remarks.empty.arm64.o ()
- (renamed) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/basic1.macho.remarks.empty.arm64.opt.bitstream ()
- (added) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/basic2.macho.remarks.arm64.o ()
- (added) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/basic2.macho.remarks.arm64.opt.bitstream ()
- (added) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/basic2.macho.remarks.empty.arm64.o ()
- (renamed) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/basic2.macho.remarks.empty.arm64.opt.bitstream ()
- (added) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/basic3.macho.remarks.arm64.o ()
- (added) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/basic3.macho.remarks.arm64.opt.bitstream ()
- (added) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/basic3.macho.remarks.empty.arm64.o ()
- (renamed) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/basic3.macho.remarks.empty.arm64.opt.bitstream ()
- (added) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/fat.macho.remarks.x86.o ()
- (added) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/fat.macho.remarks.x86_64.opt.bitstream ()
- (added) llvm/test/tools/dsymutil/Inputs/private/tmp/remarks/fat.macho.remarks.x86_64h.opt.bitstream ()
- (added) llvm/test/tools/dsymutil/Inputs/remarks/basic.macho.remarks.arm64 ()
- (added) llvm/test/tools/dsymutil/Inputs/remarks/basic.macho.remarks.empty.arm64 ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/basic.macho.remarks.empty.x86_64 ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/basic.macho.remarks.x86_64 ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/basic1.macho.remarks.empty.x86_64.o ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/basic1.macho.remarks.x86_64.o ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/basic1.macho.remarks.x86_64.opt.bitstream ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/basic2.macho.remarks.empty.x86_64.o ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/basic2.macho.remarks.x86_64.o ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/basic2.macho.remarks.x86_64.opt.bitstream ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/basic3.macho.remarks.empty.x86_64.o ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/basic3.macho.remarks.x86_64.o ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/basic3.macho.remarks.x86_64.opt.bitstream ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/fat.macho.remarks.i386.opt.bitstream ()
- (modified) llvm/test/tools/dsymutil/Inputs/remarks/fat.macho.remarks.x86 ()
- (modified) llvm/test/tools/dsymutil/Inputs/remarks/fat.macho.remarks.x86.c (+3-3)
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/fat.macho.remarks.x86.o ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/fat.macho.remarks.x86.opt.bitstream ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/fat.macho.remarks.x86_64.opt.bitstream ()
- (removed) llvm/test/tools/dsymutil/Inputs/remarks/fat.macho.remarks.x86_64h.opt.bitstream ()
- (removed) llvm/test/tools/dsymutil/X86/remarks-linking-bundle-empty.test (-13)
- (modified) llvm/test/tools/dsymutil/X86/remarks-linking-fat-bundle.test (+2-3)
- (modified) llvm/test/tools/llvm-remarkutil/Inputs/two-remarks.bitstream ()
- (added) llvm/test/tools/llvm-remarkutil/Inputs/two-remarks.v0.bitstream ()
- (modified) llvm/test/tools/llvm-remarkutil/convert.test (+3)
- (modified) llvm/tools/dsymutil/DwarfLinkerForBinary.cpp (+1)
- (modified) llvm/tools/llc/llc.cpp (+2)
- (modified) llvm/tools/llvm-remarkutil/RemarkConvert.cpp (+3-4)
- (modified) llvm/tools/opt/optdriver.cpp (+3)
- (modified) llvm/unittests/Remarks/BitstreamRemarksFormatTest.cpp (+1-1)
- (modified) llvm/unittests/Remarks/BitstreamRemarksParsingTest.cpp (+7-7)
- (modified) llvm/unittests/Remarks/BitstreamRemarksSerializerTest.cpp (+88-84)
- (modified) llvm/unittests/Remarks/RemarksLinkingTest.cpp (+28-20)
- (modified) llvm/unittests/Remarks/YAMLRemarksSerializerTest.cpp (+7-9)
- (modified) mlir/include/mlir/Remark/RemarkStreamer.h (+2-1)
- (modified) mlir/lib/Remark/RemarkStreamer.cpp (+1-2)
- (modified) offload/plugins-nextgen/common/src/JIT.cpp (+2)
``````````diff
diff --git a/clang/lib/CodeGen/CodeGenAction.cpp b/clang/lib/CodeGen/CodeGenAction.cpp
index dc54c97eeae8e..c559c6ec3eb22 100644
--- a/clang/lib/CodeGen/CodeGenAction.cpp
+++ b/clang/lib/CodeGen/CodeGenAction.cpp
@@ -29,6 +29,7 @@
#include "clang/Lex/Preprocessor.h"
#include "clang/Serialization/ASTWriter.h"
#include "llvm/ADT/Hashing.h"
+#include "llvm/ADT/ScopeExit.h"
#include "llvm/Bitcode/BitcodeReader.h"
#include "llvm/CodeGen/MachineOptimizationRemarkEmitter.h"
#include "llvm/Demangle/Demangle.h"
@@ -272,6 +273,8 @@ void BackendConsumer::HandleTranslationUnit(ASTContext &C) {
std::unique_ptr<llvm::ToolOutputFile> OptRecordFile =
std::move(*OptRecordFileOrErr);
+ auto FinalizeRemarks =
+ make_scope_exit([&]() { llvm::finalizeLLVMOptimizationRemarks(Ctx); });
if (OptRecordFile && CodeGenOpts.getProfileUse() !=
llvm::driver::ProfileInstrKind::ProfileNone)
@@ -1185,6 +1188,8 @@ void CodeGenAction::ExecuteAction() {
}
std::unique_ptr<llvm::ToolOutputFile> OptRecordFile =
std::move(*OptRecordFileOrErr);
+ auto FinalizeRemarks =
+ make_scope_exit([&]() { llvm::finalizeLLVMOptimizationRemarks(Ctx); });
emitBackendOutput(CI, CI.getCodeGenOpts(),
CI.getTarget().getDataLayoutString(), TheModule.get(), BA,
diff --git a/flang/lib/Frontend/FrontendActions.cpp b/flang/lib/Frontend/FrontendActions.cpp
index 3bef6b1c31825..c39d81ad5ee56 100644
--- a/flang/lib/Frontend/FrontendActions.cpp
+++ b/flang/lib/Frontend/FrontendActions.cpp
@@ -1356,6 +1356,8 @@ void CodeGenAction::executeAction() {
std::unique_ptr<llvm::ToolOutputFile> optRecordFile =
std::move(*optRecordFileOrErr);
+ auto FinalizeRemarks =
+ make_scope_exit([&]() { llvm::finalizeLLVMOptimizationRemarks(Ctx); });
if (optRecordFile) {
optRecordFile->keep();
diff --git a/llvm/docs/Remarks.rst b/llvm/docs/Remarks.rst
index c89940f9ff4d5..d51f8e71e0366 100644
--- a/llvm/docs/Remarks.rst
+++ b/llvm/docs/Remarks.rst
@@ -152,28 +152,6 @@ Other tools that support remarks:
.. option:: -opt-remarks-format=<format>
.. option:: -opt-remarks-with-hotness
-Serialization modes
-===================
-
-There are two modes available for serializing remarks:
-
-``Separate``
-
- In this mode, the remarks and the metadata are serialized separately. The
- client is responsible for parsing the metadata first, then use the metadata
- to correctly parse the remarks.
-
-``Standalone``
-
- In this mode, the remarks and the metadata are serialized to the same
- stream. The metadata will always come before the remarks.
-
- The compiler does not support emitting standalone remarks. This mode is
- more suited for post-processing tools like linkers, that can merge the
- remarks for one whole project.
-
-.. _yamlremarks:
-
YAML remarks
============
@@ -374,27 +352,10 @@ This block can contain the following records:
The remark container
--------------------
-Bitstream remarks are designed to be used in two different modes:
-
-``The separate mode``
-
- The separate mode is the mode that is typically used during compilation. It
- provides a way to serialize the remark entries to a stream while some
- metadata is kept in memory to be emitted in the product of the compilation
- (typically, an object file).
+The bitstream remark container supports multiple types:
-``The standalone mode``
-
- The standalone mode is typically stored and used after the distribution of
- a program. It contains all the information that allows the parsing of all
- the remarks without having any external dependencies.
-
-In order to support multiple modes, the format introduces the concept of a
-bitstream remark container type.
-
-.. _bitstreamremarksseparateremarksmeta:
-
-``SeparateRemarksMeta: the metadata emitted separately``
+.. _bitstreamremarksfileexternal:
+``RemarksFileExternal: a link to an external remarks file``
This container type expects only a :ref:`META_BLOCK <bitstreamremarksmetablock>` containing only:
@@ -406,28 +367,32 @@ bitstream remark container type.
clients to retrieve remarks and their associated metadata directly from
intermediate products.
-``SeparateRemarksFile: the remark entries emitted separately``
+ The container versions of the external separate container should match in order to
+ have a well-formed file.
- This container type expects only a :ref:`META_BLOCK <bitstreamremarksmetablock>` containing only:
+.. _bitstreamremarksfile:
+``RemarksFile: a standalone remarks file``
+
+ This container type expects a :ref:`META_BLOCK <bitstreamremarksmetablock>` containing only:
* :ref:`RECORD_META_CONTAINER_INFO <bitstreamremarksrecordmetacontainerinfo>`
* :ref:`RECORD_META_REMARK_VERSION <bitstreamremarksrecordmetaremarkversion>`
- This container type expects 0 or more :ref:`REMARK_BLOCK <bitstreamremarksremarkblock>`.
+ This container type expects 1 or more :ref:`REMARK_BLOCK <bitstreamremarksremarkblock>`.
+ If no remarks are emitted, the meta blocks are also not emitted, so the file is empty.
+
+ After the remark blocks, another :ref:`META_BLOCK <bitstreamremarksmetablock>` is emitted, containing:
+ * :ref:`RECORD_META_STRTAB <bitstreamremarksrecordmetastrtab>`
+
+ When the parser reads this container type, it jumps to the end of the file
+ to read the string table before parsing the individual remarks.
- Typically, this is emitted in a side-file alongside an object file, and is
- made to be able to stream to without increasing the memory consumption of
- the compiler. This is referenced by the :ref:`RECORD_META_EXTERNAL_FILE
- <bitstreamremarksrecordmetaexternalfile>` entry in the
- :ref:`SeparateRemarksMeta <bitstreamremarksseparateremarksmeta>` container.
+ Standalone remarks files can be referenced by the
+ :ref:`RECORD_META_EXTERNAL_FILE <bitstreamremarksrecordmetaexternalfile>`
+ entry in the :ref:`RemarksFileExternal
+ <_bitstreamremarksfileexternal>` container.
-When the parser tries to parse a container that contains the metadata for the
-separate remarks, it should parse the version and type, then keep the string
-table in memory while opening the external file, validating its metadata and
-parsing the remark entries.
-The container versions from the separate container should match in order to
-have a well-formed file.
``Standalone: the metadata and the remark entries emitted together``
@@ -439,51 +404,7 @@ have a well-formed file.
This container type expects 0 or more :ref:`REMARK_BLOCK <bitstreamremarksremarkblock>`.
-A complete output of :program:`llvm-bcanalyzer` on the different container types:
-
-``SeparateRemarksMeta``
-
-.. code-block:: none
-
- <BLOCKINFO_BLOCK/>
- <Meta BlockID=8 NumWords=13 BlockCodeSize=3>
- <Container info codeid=1 abbrevid=4 op0=5 op1=0/>
- <String table codeid=3 abbrevid=5/> blob data = 'pass\\x00key\\x00value\\x00'
- <External File codeid=4 abbrevid=6/> blob data = '/path/to/file/name'
- </Meta>
-
-``SeparateRemarksFile``
-
-.. code-block:: none
-
- <BLOCKINFO_BLOCK/>
- <Meta BlockID=8 NumWords=3 BlockCodeSize=3>
- <Container info codeid=1 abbrevid=4 op0=0 op1=1/>
- <Remark version codeid=2 abbrevid=5 op0=0/>
- </Meta>
- <Remark BlockID=9 NumWords=8 BlockCodeSize=4>
- <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>
- <Remark debug location codeid=6 abbrevid=5 op0=3 op1=99 op2=55/>
- <Remark hotness codeid=7 abbrevid=6 op0=999999999/>
- <Argument with debug location codeid=8 abbrevid=7 op0=4 op1=5 op2=6 op3=11 op4=66/>
- </Remark>
-
-``Standalone``
-
-.. code-block:: none
-
- <BLOCKINFO_BLOCK/>
- <Meta BlockID=8 NumWords=15 BlockCodeSize=3>
- <Container info codeid=1 abbrevid=4 op0=5 op1=2/>
- <Remark version codeid=2 abbrevid=5 op0=30/>
- <String table codeid=3 abbrevid=6/> blob data = 'pass\\x00remark\\x00function\\x00path\\x00key\\x00value\\x00argpath\\x00'
- </Meta>
- <Remark BlockID=9 NumWords=8 BlockCodeSize=4>
- <Remark header codeid=5 abbrevid=4 op0=2 op1=1 op2=0 op3=2/>
- <Remark debug location codeid=6 abbrevid=5 op0=3 op1=99 op2=55/>
- <Remark hotness codeid=7 abbrevid=6 op0=999999999/>
- <Argument with debug location codeid=8 abbrevid=7 op0=4 op1=5 op2=6 op3=11 op4=66/>
- </Remark>
+.. FIXME: Add complete output of :program:`llvm-bcanalyzer` on the different container types (once format changes are completed)
opt-viewer
==========
diff --git a/llvm/include/llvm/IR/LLVMRemarkStreamer.h b/llvm/include/llvm/IR/LLVMRemarkStreamer.h
index 376acdec49fbb..b4d879ed0cc52 100644
--- a/llvm/include/llvm/IR/LLVMRemarkStreamer.h
+++ b/llvm/include/llvm/IR/LLVMRemarkStreamer.h
@@ -82,20 +82,26 @@ struct LLVMRemarkSetupFormatError
LLVMRemarkSetupFormatError>::LLVMRemarkSetupErrorInfo;
};
-/// Setup optimization remarks that output to a file.
+/// Setup optimization remarks that output to a file. The returned
+/// ToolOutputFile must be kept open for writing until
+/// \ref finalizeLLVMOptimizationRemarks() is called.
LLVM_ABI Expected<std::unique_ptr<ToolOutputFile>> setupLLVMOptimizationRemarks(
LLVMContext &Context, StringRef RemarksFilename, StringRef RemarksPasses,
StringRef RemarksFormat, bool RemarksWithHotness,
std::optional<uint64_t> RemarksHotnessThreshold = 0);
/// Setup optimization remarks that output directly to a raw_ostream.
-/// \p OS is managed by the caller and should be open for writing as long as \p
-/// Context is streaming remarks to it.
+/// \p OS is managed by the caller and must be open for writing until
+/// \ref finalizeLLVMOptimizationRemarks() is called.
LLVM_ABI Error setupLLVMOptimizationRemarks(
LLVMContext &Context, raw_ostream &OS, StringRef RemarksPasses,
StringRef RemarksFormat, bool RemarksWithHotness,
std::optional<uint64_t> RemarksHotnessThreshold = 0);
+/// Finalize optimization remarks. This must be called before closing the
+/// (file) stream that was used to setup the remarks.
+LLVM_ABI void finalizeLLVMOptimizationRemarks(LLVMContext &Context);
+
} // end namespace llvm
#endif // LLVM_IR_LLVMREMARKSTREAMER_H
diff --git a/llvm/include/llvm/LTO/LTOBackend.h b/llvm/include/llvm/LTO/LTOBackend.h
index 86b488c764e06..a9590bf681257 100644
--- a/llvm/include/llvm/LTO/LTOBackend.h
+++ b/llvm/include/llvm/LTO/LTOBackend.h
@@ -65,8 +65,8 @@ thinBackend(const Config &C, unsigned Task, AddStreamFn AddStream, Module &M,
AddStreamFn IRAddStream = nullptr,
const std::vector<uint8_t> &CmdArgs = std::vector<uint8_t>());
-LLVM_ABI Error
-finalizeOptimizationRemarks(std::unique_ptr<ToolOutputFile> DiagOutputFile);
+LLVM_ABI Error finalizeOptimizationRemarks(
+ LLVMContext &Context, std::unique_ptr<ToolOutputFile> DiagOutputFile);
/// Returns the BitcodeModule that is ThinLTO.
LLVM_ABI BitcodeModule *findThinLTOModule(MutableArrayRef<BitcodeModule> BMs);
diff --git a/llvm/include/llvm/Remarks/BitstreamRemarkContainer.h b/llvm/include/llvm/Remarks/BitstreamRemarkContainer.h
index 48a148a3adc13..d4b70e54bf6bc 100644
--- a/llvm/include/llvm/Remarks/BitstreamRemarkContainer.h
+++ b/llvm/include/llvm/Remarks/BitstreamRemarkContainer.h
@@ -23,35 +23,35 @@ namespace remarks {
/// The current version of the remark container.
/// Note: this is different from the version of the remark entry.
-constexpr uint64_t CurrentContainerVersion = 0;
+constexpr uint64_t CurrentContainerVersion = 1;
/// The magic number used for identifying remark blocks.
constexpr StringLiteral ContainerMagic("RMRK");
/// Type of the remark container.
-/// The remark container has two modes:
-/// * separate: the metadata is separate from the remarks and points to the
-/// auxiliary file that contains the remarks.
-/// * standalone: the metadata and the remarks are emitted together.
enum class BitstreamRemarkContainerType {
- /// The metadata emitted separately.
- /// This will contain the following:
- /// * Container version and type
- /// * String table
- /// * External file
- SeparateRemarksMeta,
- /// The remarks emitted separately.
- /// This will contain the following:
- /// * Container version and type
- /// * Remark version
- SeparateRemarksFile,
- /// Everything is emitted together.
- /// This will contain the following:
- /// * Container version and type
- /// * Remark version
- /// * String table
- Standalone,
- First = SeparateRemarksMeta,
- Last = Standalone,
+ /// Emit a link to an external remarks file
+ /// (usually as a section of the object file, to enable discovery of all
+ /// remarks files from the final linked object file)
+ /// RemarksFileExternal:
+ /// | Meta:
+ /// | | Container info
+ /// | | External file
+ RemarksFileExternal,
+ /// Emit metadata and remarks into a file
+ /// RemarksFile:
+ /// | Meta:
+ /// | | Container info
+ /// | | Remark version
+ /// | Remarks:
+ /// | | Remark0
+ /// | | Remark1
+ /// | | Remark2
+ /// | | ...
+ /// | Late Meta:
+ /// | | String table
+ RemarksFile,
+ First = RemarksFileExternal,
+ Last = RemarksFile
};
/// The possible blocks that will be encountered in a bitstream remark
diff --git a/llvm/include/llvm/Remarks/BitstreamRemarkSerializer.h b/llvm/include/llvm/Remarks/BitstreamRemarkSerializer.h
index 6236800337508..76e2d5b4fd3bc 100644
--- a/llvm/include/llvm/Remarks/BitstreamRemarkSerializer.h
+++ b/llvm/include/llvm/Remarks/BitstreamRemarkSerializer.h
@@ -27,31 +27,7 @@ struct Remarks;
/// Serialize the remarks to LLVM bitstream.
/// This class provides ways to emit remarks in the LLVM bitstream format and
/// its associated metadata.
-///
-/// * The separate model:
-/// Separate meta: | Container info
-/// | String table
-/// | External file
-///
-/// Separate remarks: | Container info
-/// | Remark version
-/// | Remark0
-/// | Remark1
-/// | Remark2
-/// | ...
-///
-/// * The standalone model: | Container info
-/// | String table
-/// | Remark version
-/// | Remark0
-/// | Remark1
-/// | Remark2
-/// | ...
-///
struct BitstreamRemarkSerializerHelper {
- /// Buffer used for encoding the bitstream before writing it to the final
- /// stream.
- SmallVector<char, 1024> Encoded;
/// Buffer used to construct records and pass to the bitstream writer.
SmallVector<uint64_t, 64> R;
/// The Bitstream writer.
@@ -73,7 +49,8 @@ struct BitstreamRemarkSerializerHelper {
uint64_t RecordRemarkArgWithDebugLocAbbrevID = 0;
uint64_t RecordRemarkArgWithoutDebugLocAbbrevID = 0;
- BitstreamRemarkSerializerHelper(BitstreamRemarkContainerType ContainerType);
+ BitstreamRemarkSerializerHelper(BitstreamRemarkContainerType ContainerType,
+ raw_ostream &OS);
// Disable copy and move: Bitstream points to Encoded, which needs special
// handling during copy/move, but moving the vectors is probably useless
@@ -104,20 +81,15 @@ struct BitstreamRemarkSerializerHelper {
/// The block info for the remarks block.
void setupRemarkBlockInfo();
- /// Emit the metadata for the remarks.
- void emitMetaBlock(uint64_t ContainerVersion,
- std::optional<uint64_t> RemarkVersion,
- std::optional<const StringTable *> StrTab = std::nullopt,
- std::optional<StringRef> Filename = std::nullopt);
+ /// Emit the main metadata at the beginning of the file
+ void emitMetaBlock(std::optional<StringRef> Filename = std::nullopt);
+
+ /// Emit the remaining metadata at the end of the file. Here we emit metadata
+ /// that is only known once all remarks were emitted.
+ void emitLateMetaBlock(const StringTable &StrTab);
/// Emit a remark block. The string table is required.
- void emitRemarkBlock(const Remark &Remark, StringTable &StrTab);
- /// Finalize the writing to \p OS.
- void flushToStream(raw_ostream &OS);
- /// Finalize the writing to a buffer.
- /// The contents of the buffer remain valid for the lifetime of the object.
- /// Any call to any other function in this class will invalidate the buffer.
- StringRef getBuffer();
+ void emitRemark(const Remark &Remark, StringTable &StrTab);
};
/// Implementation of the remark serializer using LLVM bitstream.
@@ -127,68 +99,57 @@ struct BitstreamRemarkSerializer : public RemarkSerializer {
/// 2) The metadata block that contains various information about the remarks
/// in the file.
/// 3) A number of remark blocks.
+ /// 4) Another metadata block for metadata that is only finalized once all
+ /// remarks were emitted (e.g. StrTab)
- /// We need to set up 1) and 2) first, so that we can emit 3) after. This flag
- /// is used to emit the first two blocks only once.
- bool DidSetUp = false;
- /// The helper to emit bitstream.
- BitstreamRemarkSerializerHelper Helper;
+ /// The helper to emit bitstream. This is nullopt when the Serializer has not
+ /// been setup yet.
+ std::optional<BitstreamRemarkSerializerHelper> Helper;
/// Construct a serializer that will create its own string table.
- BitstreamRemarkSerializer(raw_ostream &OS, SerializerMode Mode);
+ BitstreamRemarkSerializer(raw_ostream &OS);
/// Construct a serializer with a pre-filled string table.
- BitstreamRemarkSerializer(raw_ostream &OS, SerializerMode Mode,
- StringTable StrTab);
+ BitstreamRemarkSerializer(raw_ostream &OS, StringTable StrTab);
+
+ ~BitstreamRemarkSerializer() override;
/// Emit a remark to the stream. This also emits the metadata associated to
- /// the remarks based on the SerializerMode specified at construction.
- /// This writes the serialized output to the provided stream.
+ /// the remarks. This writes the serialized output to the provided stream.
void emit(const Remark &Remark) override;
+
+ /// Finalize emission of remarks. This emits the late metadata block and
+ /// flushes internal buffers. It is safe to call this function multiple times,
+ /// and it is automatically executed on destruction of the Serializer.
+ void finalize() override;
+
/// The metadata serializer associated to this remark serializer. Based on the
/// container type of the current serializer, the container type of the
/// metadata serializer will change.
- std::unique_ptr<MetaSerializer> metaSerializer(
- raw_ostream &OS,
- std::optional<StringRef> ExternalFilename = std::nullopt) override;
+ std::unique_ptr<MetaSerializer>
+ metaSerializer(raw_ostream &OS, StringRef ExternalFilename) override;
static bool classof(const RemarkSerializer *S) {
return S->SerializerFormat == Format::Bitstream;
}
+
+private:
+ void setup();
};
/// Serializer of metadata for bitstream remarks.
struct BitstreamMetaSerializer : public MetaSerializer {
- /// This class can be used with [1] a pre-constructed
- /// BitstreamRemarkSerializerHelper, or with [2] one that is owned by the meta
- /// serializer. In case of [1], we need to be able to store a reference to the
- /// object, while in case of [2] we need to store the whole object.
- std::optional<BitstreamRemarkSerializerHelper> TmpHelper;
- /// The actual helper, that can point to \p TmpHelper or to an external helper
- /// object.
- BitstreamRemarkSerializerHelper *Helper = nullptr;
-
- std::optional<const StringTable *> StrTab;
- std::optional<StringRef> ExternalFilename;
+ std::optional<BitstreamRemarkSerializerHelper> Helper;
+
+ StringRef ExternalFilename;
/// Create a new meta serializer based on \p ContainerType.
- BitstreamMetaSerializer(
- raw_ostream &OS, BitstreamRemarkContainerType ContainerType,
- std::optional<const StringTable *> StrTab = std::nullopt,
- std::optional<StringRef> ExternalFilename = std::nullopt)
- : MetaSerializer(OS), TmpHelper(std::nullopt), Helper(nullptr),
- StrTab(StrTab), ExternalFilename(ExternalFilename) {
- TmpHelper.emplace(ContainerType);
- Helper = &*TmpHelper;
+ BitstreamMetaSerialize...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/156715
More information about the llvm-branch-commits
mailing list