[llvm] r367364 - [Remarks] Add an LLVM-bitstream-based remark serializer

Francis Visoiu Mistrih via llvm-commits llvm-commits at lists.llvm.org
Tue Jul 30 16:11:58 PDT 2019


Author: thegameg
Date: Tue Jul 30 16:11:57 2019
New Revision: 367364

URL: http://llvm.org/viewvc/llvm-project?rev=367364&view=rev
Log:
[Remarks] Add an LLVM-bitstream-based remark serializer

Add a new serializer, using a binary format based on the LLVM bitstream
format.

This format provides a way to serialize the remarks in two modes:

1) Separate mode: the metadata is separate from the remark entries.
2) Standalone mode: the metadata and the remark entries are in the same
file.

The format contains:

* a meta block: container version, container type, string table,
external file path, remark version
* a remark block: type, remark name, pass name, function name, debug
file, debug line, debug column, hotness, arguments (key, value, debug
file, debug line, debug column)

A string table is required for this format, which will be dumped in the
meta block to be consumed before parsing the remark blocks.

On clang itself, we noticed a size reduction of 13.4x compared to YAML,
and a compile-time reduction of between 1.7% and 3.5% on CTMark.

Differential Revision: https://reviews.llvm.org/D63466

Added:
    llvm/trunk/include/llvm/Remarks/BitstreamRemarkContainer.h
    llvm/trunk/include/llvm/Remarks/BitstreamRemarkSerializer.h
    llvm/trunk/lib/Remarks/BitstreamRemarkSerializer.cpp
    llvm/trunk/test/Bitcode/stream-types.c.opt.bitstream   (with props)
    llvm/trunk/unittests/Remarks/BitstreamRemarksFormatTest.cpp
    llvm/trunk/unittests/Remarks/BitstreamRemarksSerializerTest.cpp
Modified:
    llvm/trunk/docs/Remarks.rst
    llvm/trunk/include/llvm/Bitcode/BitcodeAnalyzer.h
    llvm/trunk/include/llvm/Remarks/Remark.h
    llvm/trunk/include/llvm/Remarks/RemarkFormat.h
    llvm/trunk/lib/Bitcode/Reader/BitcodeAnalyzer.cpp
    llvm/trunk/lib/IR/RemarkStreamer.cpp
    llvm/trunk/lib/Remarks/CMakeLists.txt
    llvm/trunk/lib/Remarks/RemarkFormat.cpp
    llvm/trunk/lib/Remarks/RemarkParser.cpp
    llvm/trunk/lib/Remarks/RemarkSerializer.cpp
    llvm/trunk/lib/Remarks/YAMLRemarkParser.cpp
    llvm/trunk/lib/Remarks/YAMLRemarkSerializer.cpp
    llvm/trunk/test/Bitcode/stream-types.c
    llvm/trunk/unittests/Remarks/CMakeLists.txt

Modified: llvm/trunk/docs/Remarks.rst
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/Remarks.rst?rev=367364&r1=367363&r2=367364&view=diff
==============================================================================
--- llvm/trunk/docs/Remarks.rst (original)
+++ llvm/trunk/docs/Remarks.rst Tue Jul 30 16:11:57 2019
@@ -113,6 +113,7 @@ following options:
 
       * :ref:`yaml <yamlremarks>` (default)
       * :ref:`yaml-strtab <yamlstrtabremarks>`
+      * :ref:`bitstream <bitstreamremarks>`
 
 ``Content configuration``
 
@@ -260,6 +261,250 @@ should be present and point to the file
 In case the metadata only acts as a header to the remarks, the file path can be
 omitted.
 
+.. _bitstreamremarks:
+
+LLVM bitstream remarks
+======================
+
+This format is using :doc:`LLVM bitstream <BitCodeFormat>` to serialize remarks
+and their associated metadata.
+
+A bitstream remark stream can be identified by the magic number ``"RMRK"`` that
+is placed at the very beginning.
+
+The format for serializing remarks is composed of two different block types:
+
+.. _bitstreamremarksmetablock:
+
+META_BLOCK
+----------
+
+The block providing information about the rest of the content in the stream.
+
+Exactly one block is expected. Having multiple metadata blocks is an error.
+
+This block can contain the following records:
+
+.. _bitstreamremarksrecordmetacontainerinfo:
+
+``RECORD_META_CONTAINER_INFO``
+
+    The container version and type.
+
+    Version: u32
+
+    Type:    u2
+
+.. _bitstreamremarksrecordmetaremarkversion:
+
+``RECORD_META_REMARK_VERSION``
+
+    The version of the remark entries. This can change independently from the
+    container version.
+
+    Version: u32
+
+.. _bitstreamremarksrecordmetastrtab:
+
+``RECORD_META_STRTAB``
+
+    The string table used by the remark entries. The format of the string table
+    is a sequence of strings separated by ``\0``.
+
+.. _bitstreamremarksrecordmetaexternalfile:
+
+``RECORD_META_EXTERNAL_FILE``
+
+    The external remark file path that contains the remark blocks associated
+    with this metadata. This is an absolute path.
+
+.. _bitstreamremarksremarkblock:
+
+REMARK_BLOCK
+------------
+
+The block describing a remark entry.
+
+0 or more blocks per file are allowed. Each block will depend on the
+:ref:`META_BLOCK <bitstreamremarksmetablock>` in order to be parsed correctly.
+
+This block can contain the following records:
+
+``RECORD_REMARK_HEADER``
+
+    The header of the remark. This contains all the mandatory information about
+    a remark.
+
+    +---------------+---------------------------+
+    | Type          | u3                        |
+    +---------------+---------------------------+
+    | Remark name   | VBR6 (string table index) |
+    +---------------+---------------------------+
+    | Pass name     | VBR6 (string table index) |
+    +---------------+---------------------------+
+    | Function name | VBR6 (string table index) |
+    +---------------+---------------------------+
+
+``RECORD_REMARK_DEBUG_LOC``
+
+    The source location for the corresponding remark. This record is optional.
+
+    +--------+---------------------------+
+    | File   | VBR7 (string table index) |
+    +--------+---------------------------+
+    | Line   | u32                       |
+    +--------+---------------------------+
+    | Column | u32                       |
+    +--------+---------------------------+
+
+``RECORD_REMARK_HOTNESS``
+
+    The hotness of the remark. This record is optional.
+
+    +---------------+---------------------+
+    | Hotness | VBR8 (string table index) |
+    +---------------+---------------------+
+
+``RECORD_REMARK_ARG_WITH_DEBUGLOC``
+
+    A remark argument with an associated debug location.
+
+    +--------+---------------------------+
+    | Key    | VBR7 (string table index) |
+    +--------+---------------------------+
+    | Value  | VBR7 (string table index) |
+    +--------+---------------------------+
+    | File   | VBR7 (string table index) |
+    +--------+---------------------------+
+    | Line   | u32                       |
+    +--------+---------------------------+
+    | Column | u32                       |
+    +--------+---------------------------+
+
+``RECORD_REMARK_ARG_WITHOUT_DEBUGLOC``
+
+    A remark argument with an associated debug location.
+
+    +--------+---------------------------+
+    | Key    | VBR7 (string table index) |
+    +--------+---------------------------+
+    | Value  | VBR7 (string table index) |
+    +--------+---------------------------+
+
+The remark container
+--------------------
+
+Bitstream remarks are designed to be used in two different modes:
+
+``The separate mode``
+
+    The separate mode is the mode that is typically used during compilation. It
+    provides a way to serialize the remark entries to a stream while some
+    metadata is kept in memory to be emitted in the product of the compilation
+    (typically, an object file).
+
+``The standalone mode``
+
+    The standalone mode is typically stored and used after the distribution of
+    a program. It contains all the information that allows the parsing of all
+    the remarks without having any external dependencies.
+
+In order to support multiple modes, the format introduces the concept of a
+bitstream remark container type.
+
+.. _bitstreamremarksseparateremarksmeta:
+
+``SeparateRemarksMeta: the metadata emitted separately``
+
+    This container type expects only a :ref:`META_BLOCK <bitstreamremarksmetablock>` containing only:
+
+    * :ref:`RECORD_META_CONTAINER_INFO <bitstreamremarksrecordmetacontainerinfo>`
+    * :ref:`RECORD_META_STRTAB <bitstreamremarksrecordmetastrtab>`
+    * :ref:`RECORD_META_EXTERNAL_FILE <bitstreamremarksrecordmetaexternalfile>`
+
+    Typically, this is emitted in a section in the object files, allowing
+    clients to retrieve remarks and their associated metadata directly from
+    intermediate products.
+
+``SeparateRemarksFile: the remark entries emitted separately``
+
+    This container type expects only a :ref:`META_BLOCK <bitstreamremarksmetablock>` containing only:
+
+    * :ref:`RECORD_META_CONTAINER_INFO <bitstreamremarksrecordmetacontainerinfo>`
+    * :ref:`RECORD_META_REMARK_VERSION <bitstreamremarksrecordmetaremarkversion>`
+
+    This container type expects 0 or more :ref:`REMARK_BLOCK <bitstreamremarksremarkblock>`.
+
+    Typically, this is emitted in a side-file alongside an object file, and is
+    made to be able to stream to without increasing the memory consumption of
+    the compiler. This is referenced by the :ref:`RECORD_META_EXTERNAL_FILE
+    <bitstreamremarksrecordmetaexternalfile>` entry in the
+    :ref:`SeparateRemarksMeta <bitstreamremarksseparateremarksmeta>` container.
+
+When the parser tries to parse a container that contains the metadata for the
+separate remarks, it should parse the version and type, then keep the string
+table in memory while opening the external file, validating its metadata and
+parsing the remark entries.
+
+The container versions from the separate container should match in order to
+have a well-formed file.
+
+``Standalone: the metadata and the remark entries emitted together``
+
+    This container type expects only a :ref:`META_BLOCK <bitstreamremarksmetablock>` containing only:
+
+    * :ref:`RECORD_META_CONTAINER_INFO <bitstreamremarksrecordmetacontainerinfo>`
+    * :ref:`RECORD_META_REMARK_VERSION <bitstreamremarksrecordmetaremarkversion>`
+    * :ref:`RECORD_META_STRTAB <bitstreamremarksrecordmetastrtab>`
+
+    This container type expects 0 or more :ref:`REMARK_BLOCK <bitstreamremarksremarkblock>`.
+
+A complete output of :program:`llvm-bcanalyzer` on the different container types:
+
+``SeparateRemarksMeta``
+
+.. code-block:: none
+
+    <BLOCKINFO_BLOCK/>
+    <Meta BlockID=8 NumWords=13 BlockCodeSize=3>
+      <Container info codeid=1 abbrevid=4 op0=5 op1=0/>
+      <String table codeid=3 abbrevid=5/> blob data = 'pass\\x00key\\x00value\\x00'
+      <External File codeid=4 abbrevid=6/> blob data = '/path/to/file/name'
+    </Meta>
+
+``SeparateRemarksFile``
+
+.. code-block:: none
+
+    <BLOCKINFO_BLOCK/>
+    <Meta BlockID=8 NumWords=3 BlockCodeSize=3>
+      <Container info codeid=1 abbrevid=4 op0=0 op1=1/>
+      <Remark version codeid=2 abbrevid=5 op0=0/>
+    </Meta>
+    <Remark BlockID=9 NumWords=8 BlockCodeSize=4>
+      <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>
+      <Remark debug location codeid=6 abbrevid=5 op0=3 op1=99 op2=55/>
+      <Remark hotness codeid=7 abbrevid=6 op0=999999999/>
+      <Argument with debug location codeid=8 abbrevid=7 op0=4 op1=5 op2=6 op3=11 op4=66/>
+    </Remark>
+
+``Standalone``
+
+.. code-block:: none
+
+    <BLOCKINFO_BLOCK/>
+    <Meta BlockID=8 NumWords=15 BlockCodeSize=3>
+      <Container info codeid=1 abbrevid=4 op0=5 op1=2/>
+      <Remark version codeid=2 abbrevid=5 op0=30/>
+      <String table codeid=3 abbrevid=6/> blob data = 'pass\\x00remark\\x00function\\x00path\\x00key\\x00value\\x00argpath\\x00'
+    </Meta>
+    <Remark BlockID=9 NumWords=8 BlockCodeSize=4>
+      <Remark header codeid=5 abbrevid=4 op0=2 op1=1 op2=0 op3=2/>
+      <Remark debug location codeid=6 abbrevid=5 op0=3 op1=99 op2=55/>
+      <Remark hotness codeid=7 abbrevid=6 op0=999999999/>
+      <Argument with debug location codeid=8 abbrevid=7 op0=4 op1=5 op2=6 op3=11 op4=66/>
+    </Remark>
+
 opt-viewer
 ==========
 

Modified: llvm/trunk/include/llvm/Bitcode/BitcodeAnalyzer.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Bitcode/BitcodeAnalyzer.h?rev=367364&r1=367363&r2=367364&view=diff
==============================================================================
--- llvm/trunk/include/llvm/Bitcode/BitcodeAnalyzer.h (original)
+++ llvm/trunk/include/llvm/Bitcode/BitcodeAnalyzer.h Tue Jul 30 16:11:57 2019
@@ -30,6 +30,7 @@ enum CurStreamTypeType {
   LLVMIRBitstream,
   ClangSerializedASTBitstream,
   ClangSerializedDiagnosticsBitstream,
+  LLVMBitstreamRemarks
 };
 
 struct BCDumpOptions {

Added: llvm/trunk/include/llvm/Remarks/BitstreamRemarkContainer.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Remarks/BitstreamRemarkContainer.h?rev=367364&view=auto
==============================================================================
--- llvm/trunk/include/llvm/Remarks/BitstreamRemarkContainer.h (added)
+++ llvm/trunk/include/llvm/Remarks/BitstreamRemarkContainer.h Tue Jul 30 16:11:57 2019
@@ -0,0 +1,106 @@
+//===-- BitstreamRemarkContainer.h - Container for remarks --------------*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file provides declarations for things used in the various types of
+// remark containers.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_REMARKS_REMARK_CONTAINER_H
+#define LLVM_REMARKS_REMARK_CONTAINER_H
+
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Bitstream/BitCodes.h"
+#include <cstdint>
+
+namespace llvm {
+namespace remarks {
+
+/// The current version of the remark container.
+/// Note: this is different from the version of the remark entry.
+constexpr uint64_t CurrentContainerVersion = 0;
+/// The magic number used for identifying remark blocks.
+constexpr StringRef ContainerMagic("RMRK", 4);
+
+/// Type of the remark container.
+/// The remark container has two modes:
+/// * separate: the metadata is separate from the remarks and points to the
+///             auxiliary file that contains the remarks.
+/// * standalone: the metadata and the remarks are emitted together.
+enum class BitstreamRemarkContainerType {
+  /// The metadata emitted separately.
+  /// This will contain the following:
+  /// * Container version and type
+  /// * String table
+  /// * External file
+  SeparateRemarksMeta,
+  /// The remarks emitted separately.
+  /// This will contain the following:
+  /// * Container version and type
+  /// * Remark version
+  SeparateRemarksFile,
+  /// Everything is emitted together.
+  /// This will contain the following:
+  /// * Container version and type
+  /// * Remark version
+  /// * String table
+  Standalone,
+  First = SeparateRemarksMeta,
+  Last = Standalone,
+};
+
+/// The possible blocks that will be encountered in a bitstream remark
+/// container.
+enum BlockIDs {
+  /// The metadata block is mandatory. It should always come after the
+  /// BLOCKINFO_BLOCK, and contains metadata that should be used when parsing
+  /// REMARK_BLOCKs.
+  /// There should always be only one META_BLOCK.
+  META_BLOCK_ID = bitc::FIRST_APPLICATION_BLOCKID,
+  /// One remark entry is represented using a REMARK_BLOCK. There can be
+  /// multiple REMARK_BLOCKs in the same file.
+  REMARK_BLOCK_ID
+};
+
+constexpr StringRef MetaBlockName = StringRef("Meta", 4);
+constexpr StringRef RemarkBlockName = StringRef("Remark", 6);
+
+/// The possible records that can be encountered in the previously described
+/// blocks.
+enum RecordIDs {
+  // Meta block records.
+  RECORD_META_CONTAINER_INFO = 1,
+  RECORD_META_REMARK_VERSION,
+  RECORD_META_STRTAB,
+  RECORD_META_EXTERNAL_FILE,
+  // Remark block records.
+  RECORD_REMARK_HEADER,
+  RECORD_REMARK_DEBUG_LOC,
+  RECORD_REMARK_HOTNESS,
+  RECORD_REMARK_ARG_WITH_DEBUGLOC,
+  RECORD_REMARK_ARG_WITHOUT_DEBUGLOC,
+  // Helpers.
+  RECORD_FIRST = RECORD_META_CONTAINER_INFO,
+  RECORD_LAST = RECORD_REMARK_ARG_WITHOUT_DEBUGLOC
+};
+
+constexpr StringRef MetaContainerInfoName = StringRef("Container info", 14);
+constexpr StringRef MetaRemarkVersionName = StringRef("Remark version", 14);
+constexpr StringRef MetaStrTabName = StringRef("String table", 12);
+constexpr StringRef MetaExternalFileName = StringRef("External File", 13);
+constexpr StringRef RemarkHeaderName = StringRef("Remark header", 13);
+constexpr StringRef RemarkDebugLocName = StringRef("Remark debug location", 21);
+constexpr StringRef RemarkHotnessName = StringRef("Remark hotness", 14);
+constexpr StringRef RemarkArgWithDebugLocName =
+    StringRef("Argument with debug location", 28);
+constexpr StringRef RemarkArgWithoutDebugLocName = StringRef("Argument", 8);
+
+} // end namespace remarks
+} // end namespace llvm
+
+#endif /* LLVM_REMARKS_REMARK_CONTAINER_H */

Added: llvm/trunk/include/llvm/Remarks/BitstreamRemarkSerializer.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Remarks/BitstreamRemarkSerializer.h?rev=367364&view=auto
==============================================================================
--- llvm/trunk/include/llvm/Remarks/BitstreamRemarkSerializer.h (added)
+++ llvm/trunk/include/llvm/Remarks/BitstreamRemarkSerializer.h Tue Jul 30 16:11:57 2019
@@ -0,0 +1,188 @@
+//===-- BitstreamRemarkSerializer.h - Bitstream serializer ------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file provides an implementation of the serializer using the LLVM
+// Bitstream format.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_REMARKS_BITSTREAM_REMARK_SERIALIZER_H
+#define LLVM_REMARKS_BITSTREAM_REMARK_SERIALIZER_H
+
+#include "llvm/Bitstream/BitstreamWriter.h"
+#include "llvm/Remarks/BitstreamRemarkContainer.h"
+#include "llvm/Remarks/RemarkSerializer.h"
+#include "llvm/Support/raw_ostream.h"
+
+namespace llvm {
+namespace remarks {
+
+/// Serialize the remarks to LLVM bitstream.
+/// This class provides ways to emit remarks in the LLVM bitstream format and
+/// its associated metadata.
+///
+/// * The separate model:
+///   Separate meta:        | Container info
+///                         | String table
+///                         | External file
+///
+///   Separate remarks:     | Container info
+///                         | Remark version
+///                         | Remark0
+///                         | Remark1
+///                         | Remark2
+///                         | ...
+///
+/// * The standalone model: | Container info
+///                         | String table
+///                         | Remark version
+///                         | Remark0
+///                         | Remark1
+///                         | Remark2
+///                         | ...
+///
+struct BitstreamSerializerHelper {
+  /// Buffer used for encoding the bitstream before writing it to the final
+  /// stream.
+  SmallVector<char, 1024> Encoded;
+  /// Buffer used to construct records and pass to the bitstream writer.
+  SmallVector<uint64_t, 64> R;
+  /// The Bitstream writer.
+  BitstreamWriter Bitstream;
+  /// The type of the container we are serializing.
+  BitstreamRemarkContainerType ContainerType;
+
+  /// Abbrev IDs initialized in the block info block.
+  /// Note: depending on the container type, some IDs might be uninitialized.
+  /// Warning: When adding more abbrev IDs, make sure to update the
+  /// BlockCodeSize (in the call to EnterSubblock).
+  uint64_t RecordMetaContainerInfoAbbrevID = 0;
+  uint64_t RecordMetaRemarkVersionAbbrevID = 0;
+  uint64_t RecordMetaStrTabAbbrevID = 0;
+  uint64_t RecordMetaExternalFileAbbrevID = 0;
+  uint64_t RecordRemarkHeaderAbbrevID = 0;
+  uint64_t RecordRemarkDebugLocAbbrevID = 0;
+  uint64_t RecordRemarkHotnessAbbrevID = 0;
+  uint64_t RecordRemarkArgWithDebugLocAbbrevID = 0;
+  uint64_t RecordRemarkArgWithoutDebugLocAbbrevID = 0;
+
+  BitstreamSerializerHelper(BitstreamRemarkContainerType ContainerType);
+
+  // Disable copy and move: Bitstream points to Encoded, which needs special
+  // handling during copy/move, but moving the vectors is probably useless
+  // anyway.
+  BitstreamSerializerHelper(const BitstreamSerializerHelper &) = delete;
+  BitstreamSerializerHelper &
+  operator=(const BitstreamSerializerHelper &) = delete;
+  BitstreamSerializerHelper(BitstreamSerializerHelper &&) = delete;
+  BitstreamSerializerHelper &operator=(BitstreamSerializerHelper &&) = delete;
+
+  /// Set up the necessary block info entries according to the container type.
+  void setupBlockInfo();
+
+  /// Set up the block info for the metadata block.
+  void setupMetaBlockInfo();
+  /// The remark version in the metadata block.
+  void setupMetaRemarkVersion();
+  void emitMetaRemarkVersion(uint64_t RemarkVersion);
+  /// The strtab in the metadata block.
+  void setupMetaStrTab();
+  void emitMetaStrTab(const StringTable &StrTab);
+  /// The external file in the metadata block.
+  void setupMetaExternalFile();
+  void emitMetaExternalFile(StringRef Filename);
+
+  /// The block info for the remarks block.
+  void setupRemarkBlockInfo();
+
+  /// Emit the metadata for the remarks.
+  void emitMetaBlock(uint64_t ContainerVersion,
+                     Optional<uint64_t> RemarkVersion,
+                     Optional<const StringTable *> StrTab = None,
+                     Optional<StringRef> Filename = None);
+
+  /// Emit a remark block. The string table is required.
+  void emitRemarkBlock(const Remark &Remark, StringTable &StrTab);
+  /// Finalize the writing to \p OS.
+  void flushToStream(raw_ostream &OS);
+  /// Finalize the writing to a buffer.
+  /// The contents of the buffer remain valid for the lifetime of the object.
+  /// Any call to any other function in this class will invalidate the buffer.
+  StringRef getBuffer();
+};
+
+/// Implementation of the remark serializer using LLVM bitstream.
+struct BitstreamSerializer : public RemarkSerializer {
+  /// The file should contain:
+  /// 1) The block info block that describes how to read the blocks.
+  /// 2) The metadata block that contains various information about the remarks
+  ///    in the file.
+  /// 3) A number of remark blocks.
+
+  /// We need to set up 1) and 2) first, so that we can emit 3) after. This flag
+  /// is used to emit the first two blocks only once.
+  bool DidSetUp = false;
+  /// The helper to emit bitstream.
+  BitstreamSerializerHelper Helper;
+
+  /// Construct a serializer that will create its own string table.
+  BitstreamSerializer(raw_ostream &OS, SerializerMode Mode);
+  /// Construct a serializer with a pre-filled string table.
+  BitstreamSerializer(raw_ostream &OS, SerializerMode Mode, StringTable StrTab);
+
+  /// Emit a remark to the stream. This also emits the metadata associated to
+  /// the remarks based on the SerializerMode specified at construction.
+  /// This writes the serialized output to the provided stream.
+  void emit(const Remark &Remark) override;
+  /// The metadata serializer associated to this remark serializer. Based on the
+  /// container type of the current serializer, the container type of the
+  /// metadata serializer will change.
+  std::unique_ptr<MetaSerializer>
+  metaSerializer(raw_ostream &OS,
+                 Optional<StringRef> ExternalFilename = None) override;
+};
+
+/// Serializer of metadata for bitstream remarks.
+struct BitstreamMetaSerializer : public MetaSerializer {
+  /// This class can be used with [1] a pre-constructed
+  /// BitstreamSerializerHelper, or with [2] one that is owned by the meta
+  /// serializer. In case of [1], we need to be able to store a reference to the
+  /// object, while in case of [2] we need to store the whole object.
+  Optional<BitstreamSerializerHelper> TmpHelper;
+  /// The actual helper, that can point to \p TmpHelper or to an external helper
+  /// object.
+  BitstreamSerializerHelper *Helper = nullptr;
+
+  Optional<const StringTable *> StrTab;
+  Optional<StringRef> ExternalFilename;
+
+  /// Create a new meta serializer based on \p ContainerType.
+  BitstreamMetaSerializer(raw_ostream &OS,
+                          BitstreamRemarkContainerType ContainerType,
+                          Optional<const StringTable *> StrTab = None,
+                          Optional<StringRef> ExternalFilename = None)
+      : MetaSerializer(OS), TmpHelper(None), Helper(nullptr), StrTab(StrTab),
+        ExternalFilename(ExternalFilename) {
+    TmpHelper.emplace(ContainerType);
+    Helper = &*TmpHelper;
+  }
+
+  /// Create a new meta serializer based on a previously built \p Helper.
+  BitstreamMetaSerializer(raw_ostream &OS, BitstreamSerializerHelper &Helper,
+                          Optional<const StringTable *> StrTab = None,
+                          Optional<StringRef> ExternalFilename = None)
+      : MetaSerializer(OS), TmpHelper(None), Helper(&Helper), StrTab(StrTab),
+        ExternalFilename(ExternalFilename) {}
+
+  void emit() override;
+};
+
+} // end namespace remarks
+} // end namespace llvm
+
+#endif /* LLVM_REMARKS_BITSTREAM_REMARK_SERIALIZER_H */

Modified: llvm/trunk/include/llvm/Remarks/Remark.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Remarks/Remark.h?rev=367364&r1=367363&r2=367364&view=diff
==============================================================================
--- llvm/trunk/include/llvm/Remarks/Remark.h (original)
+++ llvm/trunk/include/llvm/Remarks/Remark.h Tue Jul 30 16:11:57 2019
@@ -23,7 +23,8 @@
 namespace llvm {
 namespace remarks {
 
-constexpr uint64_t Version = 0;
+/// The current version of the remark entry.
+constexpr uint64_t CurrentRemarkVersion = 0;
 
 /// The debug location used to track a remark back to the source file.
 struct RemarkLocation {
@@ -58,7 +59,8 @@ enum class Type {
   AnalysisFPCommute,
   AnalysisAliasing,
   Failure,
-  LastTypeValue = Failure
+  First = Unknown,
+  Last = Failure
 };
 
 /// A remark type used for both emission and parsing.

Modified: llvm/trunk/include/llvm/Remarks/RemarkFormat.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Remarks/RemarkFormat.h?rev=367364&r1=367363&r2=367364&view=diff
==============================================================================
--- llvm/trunk/include/llvm/Remarks/RemarkFormat.h (original)
+++ llvm/trunk/include/llvm/Remarks/RemarkFormat.h Tue Jul 30 16:11:57 2019
@@ -22,7 +22,7 @@ namespace remarks {
 constexpr StringRef Magic("REMARKS", 7);
 
 /// The format used for serializing/deserializing remarks.
-enum class Format { Unknown, YAML, YAMLStrTab };
+enum class Format { Unknown, YAML, YAMLStrTab, Bitstream };
 
 /// Parse and validate a string for the remark format.
 Expected<Format> parseFormat(StringRef FormatStr);

Modified: llvm/trunk/lib/Bitcode/Reader/BitcodeAnalyzer.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Bitcode/Reader/BitcodeAnalyzer.cpp?rev=367364&r1=367363&r2=367364&view=diff
==============================================================================
--- llvm/trunk/lib/Bitcode/Reader/BitcodeAnalyzer.cpp (original)
+++ llvm/trunk/lib/Bitcode/Reader/BitcodeAnalyzer.cpp Tue Jul 30 16:11:57 2019
@@ -434,6 +434,13 @@ static Expected<CurStreamTypeType> ReadS
       return std::move(Err);
     if (Signature[2] == 'A' && Signature[3] == 'G')
       return ClangSerializedDiagnosticsBitstream;
+  } else if (Signature[0] == 'R' && Signature[1] == 'M') {
+    if (Error Err = tryRead(Signature[2], 8))
+      return std::move(Err);
+    if (Error Err = tryRead(Signature[3], 8))
+      return std::move(Err);
+    if (Signature[2] == 'R' && Signature[3] == 'K')
+      return LLVMBitstreamRemarks;
   } else {
     if (Error Err = tryRead(Signature[2], 4))
       return std::move(Err);
@@ -627,6 +634,9 @@ void BitcodeAnalyzer::printStats(BCDumpO
   case ClangSerializedDiagnosticsBitstream:
     O.OS << "Clang Serialized Diagnostics\n";
     break;
+  case LLVMBitstreamRemarks:
+    O.OS << "LLVM Remarks\n";
+    break;
   }
   O.OS << "  # Toplevel Blocks: " << NumTopBlocks << "\n";
   O.OS << "\n";

Modified: llvm/trunk/lib/IR/RemarkStreamer.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/RemarkStreamer.cpp?rev=367364&r1=367363&r2=367364&view=diff
==============================================================================
--- llvm/trunk/lib/IR/RemarkStreamer.cpp (original)
+++ llvm/trunk/lib/IR/RemarkStreamer.cpp Tue Jul 30 16:11:57 2019
@@ -15,6 +15,7 @@
 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/GlobalValue.h"
+#include "llvm/Remarks/BitstreamRemarkSerializer.h"
 #include "llvm/Remarks/RemarkFormat.h"
 #include "llvm/Remarks/RemarkSerializer.h"
 

Added: llvm/trunk/lib/Remarks/BitstreamRemarkSerializer.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Remarks/BitstreamRemarkSerializer.cpp?rev=367364&view=auto
==============================================================================
--- llvm/trunk/lib/Remarks/BitstreamRemarkSerializer.cpp (added)
+++ llvm/trunk/lib/Remarks/BitstreamRemarkSerializer.cpp Tue Jul 30 16:11:57 2019
@@ -0,0 +1,383 @@
+//===- BitstreamRemarkSerializer.cpp --------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file provides the implementation of the LLVM bitstream remark serializer
+// using LLVM's bitstream writer.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Remarks/BitstreamRemarkSerializer.h"
+
+using namespace llvm;
+using namespace llvm::remarks;
+
+BitstreamSerializerHelper::BitstreamSerializerHelper(
+    BitstreamRemarkContainerType ContainerType)
+    : Encoded(), R(), Bitstream(Encoded), ContainerType(ContainerType) {}
+
+static void push(SmallVectorImpl<uint64_t> &R, StringRef Str) {
+  for (const char C : Str)
+    R.push_back(C);
+}
+
+static void setRecordName(unsigned RecordID, BitstreamWriter &Bitstream,
+                          SmallVectorImpl<uint64_t> &R, StringRef Str) {
+  R.clear();
+  R.push_back(RecordID);
+  push(R, Str);
+  Bitstream.EmitRecord(bitc::BLOCKINFO_CODE_SETRECORDNAME, R);
+}
+
+static void initBlock(unsigned BlockID, BitstreamWriter &Bitstream,
+                      SmallVectorImpl<uint64_t> &R, StringRef Str) {
+  R.clear();
+  R.push_back(BlockID);
+  Bitstream.EmitRecord(bitc::BLOCKINFO_CODE_SETBID, R);
+
+  R.clear();
+  push(R, Str);
+  Bitstream.EmitRecord(bitc::BLOCKINFO_CODE_BLOCKNAME, R);
+}
+
+void BitstreamSerializerHelper::setupMetaBlockInfo() {
+  // Setup the metadata block.
+  initBlock(META_BLOCK_ID, Bitstream, R, MetaBlockName);
+
+  // The container information.
+  setRecordName(RECORD_META_CONTAINER_INFO, Bitstream, R,
+                MetaContainerInfoName);
+
+  auto Abbrev = std::make_shared<BitCodeAbbrev>();
+  Abbrev->Add(BitCodeAbbrevOp(RECORD_META_CONTAINER_INFO));
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Version.
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 2));  // Type.
+  RecordMetaContainerInfoAbbrevID =
+      Bitstream.EmitBlockInfoAbbrev(META_BLOCK_ID, Abbrev);
+}
+
+void BitstreamSerializerHelper::setupMetaRemarkVersion() {
+  setRecordName(RECORD_META_REMARK_VERSION, Bitstream, R,
+                MetaRemarkVersionName);
+
+  auto Abbrev = std::make_shared<BitCodeAbbrev>();
+  Abbrev->Add(BitCodeAbbrevOp(RECORD_META_REMARK_VERSION));
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Version.
+  RecordMetaRemarkVersionAbbrevID =
+      Bitstream.EmitBlockInfoAbbrev(META_BLOCK_ID, Abbrev);
+}
+
+void BitstreamSerializerHelper::emitMetaRemarkVersion(uint64_t RemarkVersion) {
+  // The remark version is emitted only if we emit remarks.
+  R.clear();
+  R.push_back(RECORD_META_REMARK_VERSION);
+  R.push_back(RemarkVersion);
+  Bitstream.EmitRecordWithAbbrev(RecordMetaRemarkVersionAbbrevID, R);
+}
+
+void BitstreamSerializerHelper::setupMetaStrTab() {
+  setRecordName(RECORD_META_STRTAB, Bitstream, R, MetaStrTabName);
+
+  auto Abbrev = std::make_shared<BitCodeAbbrev>();
+  Abbrev->Add(BitCodeAbbrevOp(RECORD_META_STRTAB));
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // Raw table.
+  RecordMetaStrTabAbbrevID =
+      Bitstream.EmitBlockInfoAbbrev(META_BLOCK_ID, Abbrev);
+}
+
+void BitstreamSerializerHelper::emitMetaStrTab(const StringTable &StrTab) {
+  // The string table is not emitted if we emit remarks separately.
+  R.clear();
+  R.push_back(RECORD_META_STRTAB);
+
+  // Serialize to a blob.
+  std::string Buf;
+  raw_string_ostream OS(Buf);
+  StrTab.serialize(OS);
+  StringRef Blob = OS.str();
+  Bitstream.EmitRecordWithBlob(RecordMetaStrTabAbbrevID, R, Blob);
+}
+
+void BitstreamSerializerHelper::setupMetaExternalFile() {
+  setRecordName(RECORD_META_EXTERNAL_FILE, Bitstream, R, MetaExternalFileName);
+
+  auto Abbrev = std::make_shared<BitCodeAbbrev>();
+  Abbrev->Add(BitCodeAbbrevOp(RECORD_META_EXTERNAL_FILE));
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // Filename.
+  RecordMetaExternalFileAbbrevID =
+      Bitstream.EmitBlockInfoAbbrev(META_BLOCK_ID, Abbrev);
+}
+
+void BitstreamSerializerHelper::emitMetaExternalFile(StringRef Filename) {
+  // The external file is emitted only if we emit the separate metadata.
+  R.clear();
+  R.push_back(RECORD_META_EXTERNAL_FILE);
+  Bitstream.EmitRecordWithBlob(RecordMetaExternalFileAbbrevID, R, Filename);
+}
+
+void BitstreamSerializerHelper::setupRemarkBlockInfo() {
+  // Setup the remark block.
+  initBlock(REMARK_BLOCK_ID, Bitstream, R, RemarkBlockName);
+
+  // The header of a remark.
+  {
+    setRecordName(RECORD_REMARK_HEADER, Bitstream, R, RemarkHeaderName);
+
+    auto Abbrev = std::make_shared<BitCodeAbbrev>();
+    Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_HEADER));
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 3)); // Type
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 6));   // Remark Name
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 6));   // Pass name
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 8));   // Function name
+    RecordRemarkHeaderAbbrevID =
+        Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev);
+  }
+
+  // The location of a remark.
+  {
+    setRecordName(RECORD_REMARK_DEBUG_LOC, Bitstream, R, RemarkDebugLocName);
+
+    auto Abbrev = std::make_shared<BitCodeAbbrev>();
+    Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_DEBUG_LOC));
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7));    // File
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Line
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Column
+    RecordRemarkDebugLocAbbrevID =
+        Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev);
+  }
+
+  // The hotness of a remark.
+  {
+    setRecordName(RECORD_REMARK_HOTNESS, Bitstream, R, RemarkHotnessName);
+
+    auto Abbrev = std::make_shared<BitCodeAbbrev>();
+    Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_HOTNESS));
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 8)); // Hotness
+    RecordRemarkHotnessAbbrevID =
+        Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev);
+  }
+
+  // An argument entry with a debug location attached.
+  {
+    setRecordName(RECORD_REMARK_ARG_WITH_DEBUGLOC, Bitstream, R,
+                  RemarkArgWithDebugLocName);
+
+    auto Abbrev = std::make_shared<BitCodeAbbrev>();
+    Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_ARG_WITH_DEBUGLOC));
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7));    // Key
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7));    // Value
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7));    // File
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Line
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Column
+    RecordRemarkArgWithDebugLocAbbrevID =
+        Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev);
+  }
+
+  // An argument entry with no debug location attached.
+  {
+    setRecordName(RECORD_REMARK_ARG_WITHOUT_DEBUGLOC, Bitstream, R,
+                  RemarkArgWithoutDebugLocName);
+
+    auto Abbrev = std::make_shared<BitCodeAbbrev>();
+    Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_ARG_WITHOUT_DEBUGLOC));
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7)); // Key
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7)); // Value
+    RecordRemarkArgWithoutDebugLocAbbrevID =
+        Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev);
+  }
+}
+
+void BitstreamSerializerHelper::setupBlockInfo() {
+  // Emit magic number.
+  for (const char C : ContainerMagic)
+    Bitstream.Emit(static_cast<unsigned>(C), 8);
+
+  Bitstream.EnterBlockInfoBlock();
+
+  // Setup the main metadata. Depending on the container type, we'll setup the
+  // required records next.
+  setupMetaBlockInfo();
+
+  switch (ContainerType) {
+  case BitstreamRemarkContainerType::SeparateRemarksMeta:
+    // Needs a string table that the separate remark file is using.
+    setupMetaStrTab();
+    // Needs to know where the external remarks file is.
+    setupMetaExternalFile();
+    break;
+  case BitstreamRemarkContainerType::SeparateRemarksFile:
+    // Contains remarks: emit the version.
+    setupMetaRemarkVersion();
+    // Contains remarks: emit the remark abbrevs.
+    setupRemarkBlockInfo();
+    break;
+  case BitstreamRemarkContainerType::Standalone:
+    // Contains remarks: emit the version.
+    setupMetaRemarkVersion();
+    // Needs a string table.
+    setupMetaStrTab();
+    // Contains remarks: emit the remark abbrevs.
+    setupRemarkBlockInfo();
+    break;
+  }
+
+  Bitstream.ExitBlock();
+}
+
+void BitstreamSerializerHelper::emitMetaBlock(
+    uint64_t ContainerVersion, Optional<uint64_t> RemarkVersion,
+    Optional<const StringTable *> StrTab, Optional<StringRef> Filename) {
+  // Emit the meta block
+  Bitstream.EnterSubblock(META_BLOCK_ID, 3);
+
+  // The container version and type.
+  R.clear();
+  R.push_back(RECORD_META_CONTAINER_INFO);
+  R.push_back(ContainerVersion);
+  R.push_back(static_cast<uint64_t>(ContainerType));
+  Bitstream.EmitRecordWithAbbrev(RecordMetaContainerInfoAbbrevID, R);
+
+  switch (ContainerType) {
+  case BitstreamRemarkContainerType::SeparateRemarksMeta:
+    assert(StrTab != None && *StrTab != nullptr);
+    emitMetaStrTab(**StrTab);
+    assert(Filename != None);
+    emitMetaExternalFile(*Filename);
+    break;
+  case BitstreamRemarkContainerType::SeparateRemarksFile:
+    assert(RemarkVersion != None);
+    emitMetaRemarkVersion(*RemarkVersion);
+    break;
+  case BitstreamRemarkContainerType::Standalone:
+    assert(RemarkVersion != None);
+    emitMetaRemarkVersion(*RemarkVersion);
+    assert(StrTab != None && *StrTab != nullptr);
+    emitMetaStrTab(**StrTab);
+    break;
+  }
+
+  Bitstream.ExitBlock();
+}
+
+void BitstreamSerializerHelper::emitRemarkBlock(const Remark &Remark,
+                                                StringTable &StrTab) {
+  Bitstream.EnterSubblock(REMARK_BLOCK_ID, 4);
+
+  R.clear();
+  R.push_back(RECORD_REMARK_HEADER);
+  R.push_back(static_cast<uint64_t>(Remark.RemarkType));
+  R.push_back(StrTab.add(Remark.RemarkName).first);
+  R.push_back(StrTab.add(Remark.PassName).first);
+  R.push_back(StrTab.add(Remark.FunctionName).first);
+  Bitstream.EmitRecordWithAbbrev(RecordRemarkHeaderAbbrevID, R);
+
+  if (const Optional<RemarkLocation> &Loc = Remark.Loc) {
+    R.clear();
+    R.push_back(RECORD_REMARK_DEBUG_LOC);
+    R.push_back(StrTab.add(Loc->SourceFilePath).first);
+    R.push_back(Loc->SourceLine);
+    R.push_back(Loc->SourceColumn);
+    Bitstream.EmitRecordWithAbbrev(RecordRemarkDebugLocAbbrevID, R);
+  }
+
+  if (Optional<uint64_t> Hotness = Remark.Hotness) {
+    R.clear();
+    R.push_back(RECORD_REMARK_HOTNESS);
+    R.push_back(*Hotness);
+    Bitstream.EmitRecordWithAbbrev(RecordRemarkHotnessAbbrevID, R);
+  }
+
+  for (const Argument &Arg : Remark.Args) {
+    R.clear();
+    unsigned Key = StrTab.add(Arg.Key).first;
+    unsigned Val = StrTab.add(Arg.Val).first;
+    bool HasDebugLoc = Arg.Loc != None;
+    R.push_back(HasDebugLoc ? RECORD_REMARK_ARG_WITH_DEBUGLOC
+                            : RECORD_REMARK_ARG_WITHOUT_DEBUGLOC);
+    R.push_back(Key);
+    R.push_back(Val);
+    if (HasDebugLoc) {
+      R.push_back(StrTab.add(Arg.Loc->SourceFilePath).first);
+      R.push_back(Arg.Loc->SourceLine);
+      R.push_back(Arg.Loc->SourceColumn);
+    }
+    Bitstream.EmitRecordWithAbbrev(HasDebugLoc
+                                       ? RecordRemarkArgWithDebugLocAbbrevID
+                                       : RecordRemarkArgWithoutDebugLocAbbrevID,
+                                   R);
+  }
+  Bitstream.ExitBlock();
+}
+
+void BitstreamSerializerHelper::flushToStream(raw_ostream &OS) {
+  OS.write(Encoded.data(), Encoded.size());
+  Encoded.clear();
+}
+
+StringRef BitstreamSerializerHelper::getBuffer() {
+  return StringRef(Encoded.data(), Encoded.size());
+}
+
+BitstreamSerializer::BitstreamSerializer(raw_ostream &OS, SerializerMode Mode)
+    : RemarkSerializer(OS, Mode),
+      Helper(BitstreamRemarkContainerType::SeparateRemarksFile) {
+  assert(Mode == SerializerMode::Separate &&
+         "For SerializerMode::Standalone, a pre-filled string table needs to "
+         "be provided.");
+  // We always use a string table with bitstream.
+  StrTab.emplace();
+}
+
+BitstreamSerializer::BitstreamSerializer(raw_ostream &OS, SerializerMode Mode,
+                                         StringTable StrTabIn)
+    : RemarkSerializer(OS, Mode),
+      Helper(Mode == SerializerMode::Separate
+                 ? BitstreamRemarkContainerType::SeparateRemarksFile
+                 : BitstreamRemarkContainerType::Standalone) {
+  StrTab = std::move(StrTabIn);
+}
+
+void BitstreamSerializer::emit(const Remark &Remark) {
+  if (!DidSetUp) {
+    // Emit the metadata that is embedded in the remark file.
+    // If we're in standalone mode, serialize the string table as well.
+    bool IsStandalone =
+        Helper.ContainerType == BitstreamRemarkContainerType::Standalone;
+    BitstreamMetaSerializer MetaSerializer(
+        OS, Helper,
+        IsStandalone ? &*StrTab : Optional<const StringTable *>(None));
+    MetaSerializer.emit();
+    DidSetUp = true;
+  }
+
+  assert(DidSetUp &&
+         "The Block info block and the meta block were not emitted yet.");
+  Helper.emitRemarkBlock(Remark, *StrTab);
+
+  Helper.flushToStream(OS);
+}
+
+std::unique_ptr<MetaSerializer>
+BitstreamSerializer::metaSerializer(raw_ostream &OS,
+                                    Optional<StringRef> ExternalFilename) {
+  assert(Helper.ContainerType !=
+         BitstreamRemarkContainerType::SeparateRemarksMeta);
+  bool IsStandalone =
+      Helper.ContainerType == BitstreamRemarkContainerType::Standalone;
+  return llvm::make_unique<BitstreamMetaSerializer>(
+      OS,
+      IsStandalone ? BitstreamRemarkContainerType::Standalone
+                   : BitstreamRemarkContainerType::SeparateRemarksMeta,
+      &*StrTab, ExternalFilename);
+}
+
+void BitstreamMetaSerializer::emit() {
+  Helper->setupBlockInfo();
+  Helper->emitMetaBlock(CurrentContainerVersion, CurrentRemarkVersion, StrTab,
+                       ExternalFilename);
+  Helper->flushToStream(OS);
+}

Modified: llvm/trunk/lib/Remarks/CMakeLists.txt
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Remarks/CMakeLists.txt?rev=367364&r1=367363&r2=367364&view=diff
==============================================================================
--- llvm/trunk/lib/Remarks/CMakeLists.txt (original)
+++ llvm/trunk/lib/Remarks/CMakeLists.txt Tue Jul 30 16:11:57 2019
@@ -1,4 +1,5 @@
 add_llvm_library(LLVMRemarks
+  BitstreamRemarkSerializer.cpp
   Remark.cpp
   RemarkFormat.cpp
   RemarkParser.cpp

Modified: llvm/trunk/lib/Remarks/RemarkFormat.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Remarks/RemarkFormat.cpp?rev=367364&r1=367363&r2=367364&view=diff
==============================================================================
--- llvm/trunk/lib/Remarks/RemarkFormat.cpp (original)
+++ llvm/trunk/lib/Remarks/RemarkFormat.cpp Tue Jul 30 16:11:57 2019
@@ -19,7 +19,8 @@ using namespace llvm::remarks;
 Expected<Format> llvm::remarks::parseFormat(StringRef FormatStr) {
   auto Result = StringSwitch<Format>(FormatStr)
                     .Cases("", "yaml", Format::YAML)
-                    .Cases("", "yaml-strtab", Format::YAMLStrTab)
+                    .Case("yaml-strtab", Format::YAMLStrTab)
+                    .Case("bitstream", Format::Bitstream)
                     .Default(Format::Unknown);
 
   if (Result == Format::Unknown)

Modified: llvm/trunk/lib/Remarks/RemarkParser.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Remarks/RemarkParser.cpp?rev=367364&r1=367363&r2=367364&view=diff
==============================================================================
--- llvm/trunk/lib/Remarks/RemarkParser.cpp (original)
+++ llvm/trunk/lib/Remarks/RemarkParser.cpp Tue Jul 30 16:11:57 2019
@@ -56,6 +56,9 @@ llvm::remarks::createRemarkParser(Format
     return createStringError(
         std::make_error_code(std::errc::invalid_argument),
         "The YAML with string table format requires a parsed string table.");
+  case Format::Bitstream:
+    return createStringError(std::make_error_code(std::errc::invalid_argument),
+                             "Parsing bitstream remarks is not supported.");
   case Format::Unknown:
     return createStringError(std::make_error_code(std::errc::invalid_argument),
                              "Unknown remark parser format.");
@@ -73,6 +76,9 @@ llvm::remarks::createRemarkParser(Format
                              "table. Use yaml-strtab instead.");
   case Format::YAMLStrTab:
     return llvm::make_unique<YAMLStrTabRemarkParser>(Buf, std::move(StrTab));
+  case Format::Bitstream:
+    return createStringError(std::make_error_code(std::errc::invalid_argument),
+                             "Parsing bitstream remarks is not supported.");
   case Format::Unknown:
     return createStringError(std::make_error_code(std::errc::invalid_argument),
                              "Unknown remark parser format.");
@@ -89,6 +95,9 @@ llvm::remarks::createRemarkParserFromMet
   case Format::YAML:
   case Format::YAMLStrTab:
     return createYAMLParserFromMeta(Buf, std::move(StrTab));
+  case Format::Bitstream:
+    return createStringError(std::make_error_code(std::errc::invalid_argument),
+                             "Parsing bitstream remarks is not supported.");
   case Format::Unknown:
     return createStringError(std::make_error_code(std::errc::invalid_argument),
                              "Unknown remark parser format.");

Modified: llvm/trunk/lib/Remarks/RemarkSerializer.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Remarks/RemarkSerializer.cpp?rev=367364&r1=367363&r2=367364&view=diff
==============================================================================
--- llvm/trunk/lib/Remarks/RemarkSerializer.cpp (original)
+++ llvm/trunk/lib/Remarks/RemarkSerializer.cpp Tue Jul 30 16:11:57 2019
@@ -11,6 +11,7 @@
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Remarks/RemarkSerializer.h"
+#include "llvm/Remarks/BitstreamRemarkSerializer.h"
 #include "llvm/Remarks/YAMLRemarkSerializer.h"
 
 using namespace llvm;
@@ -27,6 +28,8 @@ remarks::createRemarkSerializer(Format R
     return llvm::make_unique<YAMLRemarkSerializer>(OS, Mode);
   case Format::YAMLStrTab:
     return llvm::make_unique<YAMLStrTabRemarkSerializer>(OS, Mode);
+  case Format::Bitstream:
+    return llvm::make_unique<BitstreamSerializer>(OS, Mode);
   }
   llvm_unreachable("Unknown remarks::Format enum");
 }
@@ -45,6 +48,8 @@ remarks::createRemarkSerializer(Format R
   case Format::YAMLStrTab:
     return llvm::make_unique<YAMLStrTabRemarkSerializer>(OS, Mode,
                                                          std::move(StrTab));
+  case Format::Bitstream:
+    return llvm::make_unique<BitstreamSerializer>(OS, Mode, std::move(StrTab));
   }
   llvm_unreachable("Unknown remarks::Format enum");
 }

Modified: llvm/trunk/lib/Remarks/YAMLRemarkParser.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Remarks/YAMLRemarkParser.cpp?rev=367364&r1=367363&r2=367364&view=diff
==============================================================================
--- llvm/trunk/lib/Remarks/YAMLRemarkParser.cpp (original)
+++ llvm/trunk/lib/Remarks/YAMLRemarkParser.cpp Tue Jul 30 16:11:57 2019
@@ -75,11 +75,11 @@ static Expected<uint64_t> parseVersion(S
   uint64_t Version =
       support::endian::read<uint64_t, support::little, support::unaligned>(
           Buf.data());
-  if (Version != remarks::Version)
+  if (Version != remarks::CurrentRemarkVersion)
     return createStringError(std::errc::illegal_byte_sequence,
                              "Mismatching remark version. Got %" PRId64
                              ", expected %" PRId64 ".",
-                             Version, remarks::Version);
+                             Version, remarks::CurrentRemarkVersion);
   Buf = Buf.drop_front(sizeof(uint64_t));
   return Version;
 }

Modified: llvm/trunk/lib/Remarks/YAMLRemarkSerializer.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Remarks/YAMLRemarkSerializer.cpp?rev=367364&r1=367363&r2=367364&view=diff
==============================================================================
--- llvm/trunk/lib/Remarks/YAMLRemarkSerializer.cpp (original)
+++ llvm/trunk/lib/Remarks/YAMLRemarkSerializer.cpp Tue Jul 30 16:11:57 2019
@@ -191,7 +191,7 @@ static void emitMagic(raw_ostream &OS) {
 static void emitVersion(raw_ostream &OS) {
   // Emit the version number: little-endian uint64_t.
   std::array<char, 8> Version;
-  support::endian::write64le(Version.data(), remarks::Version);
+  support::endian::write64le(Version.data(), remarks::CurrentRemarkVersion);
   OS.write(Version.data(), Version.size());
 }
 

Modified: llvm/trunk/test/Bitcode/stream-types.c
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Bitcode/stream-types.c?rev=367364&r1=367363&r2=367364&view=diff
==============================================================================
--- llvm/trunk/test/Bitcode/stream-types.c (original)
+++ llvm/trunk/test/Bitcode/stream-types.c Tue Jul 30 16:11:57 2019
@@ -10,3 +10,6 @@
 // RUN: not llvm-bcanalyzer -dump %s.ast.incomplete 2>&1 | FileCheck %s -check-prefix=CHECK-INCOMPLETE
 // RUN: not llvm-bcanalyzer -dump %s.dia.incomplete 2>&1 | FileCheck %s -check-prefix=CHECK-INCOMPLETE
 // CHECK-INCOMPLETE: Bitcode stream should be a multiple of 4 bytes in length
+
+// RUN: llvm-bcanalyzer -dump %s.opt.bitstream | FileCheck %s -check-prefix=CHECK-REMARKS
+// CHECK-REMARKS: Stream type: LLVM Remarks

Added: llvm/trunk/test/Bitcode/stream-types.c.opt.bitstream
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Bitcode/stream-types.c.opt.bitstream?rev=367364&view=auto
==============================================================================
Binary file - no diff available.

Propchange: llvm/trunk/test/Bitcode/stream-types.c.opt.bitstream
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Added: llvm/trunk/unittests/Remarks/BitstreamRemarksFormatTest.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/Remarks/BitstreamRemarksFormatTest.cpp?rev=367364&view=auto
==============================================================================
--- llvm/trunk/unittests/Remarks/BitstreamRemarksFormatTest.cpp (added)
+++ llvm/trunk/unittests/Remarks/BitstreamRemarksFormatTest.cpp Tue Jul 30 16:11:57 2019
@@ -0,0 +1,48 @@
+//===- unittest/Support/BitstreamRemarksFormatTest.cpp - BitCodes tests ---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Remarks/BitstreamRemarkContainer.h"
+#include "gtest/gtest.h"
+
+using namespace llvm;
+
+// The goal for this test is to observe test failures and carefully update the
+// constants when they change.
+
+// This should not change over time.
+TEST(BitstreamRemarksFormat, Magic) {
+  EXPECT_EQ(remarks::ContainerMagic, "RMRK");
+}
+
+// This should be updated whenever any of the tests below are modified.
+TEST(BitstreamRemarksFormat, ContainerVersion) {
+  EXPECT_EQ(remarks::CurrentContainerVersion, 0UL);
+}
+
+// The values of the current blocks should not change over time.
+// When adding new blocks, make sure to append them to the enum.
+TEST(BitstreamRemarksFormat, BlockIDs) {
+  EXPECT_EQ(remarks::META_BLOCK_ID, 8);
+  EXPECT_EQ(remarks::REMARK_BLOCK_ID, 9);
+}
+
+// The values of the current records should not change over time.
+// When adding new records, make sure to append them to the enum.
+TEST(BitstreamRemarksFormat, RecordIDs) {
+  EXPECT_EQ(remarks::RECORD_FIRST, 1);
+  EXPECT_EQ(remarks::RECORD_META_CONTAINER_INFO, 1);
+  EXPECT_EQ(remarks::RECORD_META_REMARK_VERSION, 2);
+  EXPECT_EQ(remarks::RECORD_META_STRTAB, 3);
+  EXPECT_EQ(remarks::RECORD_META_EXTERNAL_FILE, 4);
+  EXPECT_EQ(remarks::RECORD_REMARK_HEADER, 5);
+  EXPECT_EQ(remarks::RECORD_REMARK_DEBUG_LOC, 6);
+  EXPECT_EQ(remarks::RECORD_REMARK_HOTNESS, 7);
+  EXPECT_EQ(remarks::RECORD_REMARK_ARG_WITH_DEBUGLOC, 8);
+  EXPECT_EQ(remarks::RECORD_REMARK_ARG_WITHOUT_DEBUGLOC, 9);
+  EXPECT_EQ(remarks::RECORD_LAST, 9);
+}

Added: llvm/trunk/unittests/Remarks/BitstreamRemarksSerializerTest.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/Remarks/BitstreamRemarksSerializerTest.cpp?rev=367364&view=auto
==============================================================================
--- llvm/trunk/unittests/Remarks/BitstreamRemarksSerializerTest.cpp (added)
+++ llvm/trunk/unittests/Remarks/BitstreamRemarksSerializerTest.cpp Tue Jul 30 16:11:57 2019
@@ -0,0 +1,341 @@
+//===- unittest/Support/BitstreamRemarksSerializerTest.cpp ----------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Bitcode/BitcodeAnalyzer.h"
+#include "llvm/Remarks/BitstreamRemarkSerializer.h"
+#include "llvm/Support/raw_ostream.h"
+#include "gtest/gtest.h"
+#include <string>
+
+// We need to supprt Windows paths as well. In order to have paths with the same
+// length, use a different path according to the platform.
+#ifdef _WIN32
+#define EXTERNALFILETESTPATH "C:/externalfi"
+#else
+#define EXTERNALFILETESTPATH "/externalfile"
+#endif
+
+using namespace llvm;
+
+static void checkAnalyze(StringRef Input, StringRef Expected) {
+  std::string OutputBuf;
+  raw_string_ostream OutputOS(OutputBuf);
+  BCDumpOptions O(OutputOS);
+  O.ShowBinaryBlobs = true;
+  BitcodeAnalyzer BA(Input);
+  EXPECT_FALSE(BA.analyze(O)); // Expect no errors.
+  EXPECT_EQ(OutputOS.str(), Expected);
+}
+
+static void check(remarks::SerializerMode Mode, const remarks::Remark &R,
+                  StringRef ExpectedR, Optional<StringRef> ExpectedMeta,
+                  Optional<remarks::StringTable> StrTab) {
+  // Emit the remark.
+  std::string InputBuf;
+  raw_string_ostream InputOS(InputBuf);
+  Expected<std::unique_ptr<remarks::RemarkSerializer>> MaybeSerializer = [&] {
+    if (StrTab)
+      return createRemarkSerializer(remarks::Format::Bitstream, Mode, InputOS,
+                                    std::move(*StrTab));
+    else
+      return createRemarkSerializer(remarks::Format::Bitstream, Mode, InputOS);
+  }();
+  EXPECT_FALSE(errorToBool(MaybeSerializer.takeError()));
+  std::unique_ptr<remarks::RemarkSerializer> Serializer =
+      std::move(*MaybeSerializer);
+  Serializer->emit(R);
+
+  // Analyze the serialized remark.
+  checkAnalyze(InputOS.str(), ExpectedR);
+
+  // Analyze the serialized metadata if it's not in standalone mode.
+  if (ExpectedMeta) {
+    std::string MetaBuf;
+    raw_string_ostream MetaOS(MetaBuf);
+    std::unique_ptr<remarks::MetaSerializer> MetaSerializer =
+        Serializer->metaSerializer(MetaOS, StringRef(EXTERNALFILETESTPATH));
+    MetaSerializer->emit();
+    checkAnalyze(MetaOS.str(), *ExpectedMeta);
+  }
+}
+
+static void check(const remarks::Remark &R, StringRef ExpectedR,
+                  StringRef ExpectedMeta,
+                  Optional<remarks::StringTable> StrTab = None) {
+  return check(remarks::SerializerMode::Separate, R, ExpectedR, ExpectedMeta,
+               std::move(StrTab));
+}
+
+static void checkStandalone(const remarks::Remark &R, StringRef ExpectedR,
+                            Optional<remarks::StringTable> StrTab = None) {
+  return check(remarks::SerializerMode::Standalone, R, ExpectedR,
+               /*ExpectedMeta=*/None, std::move(StrTab));
+}
+
+TEST(BitstreamRemarkSerializer, SeparateRemarkFileNoOptionals) {
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  check(R,
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=3 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=1/>\n"
+        "  <Remark version codeid=2 abbrevid=5 op0=0/>\n"
+        "</Meta>\n"
+        "<Remark BlockID=9 NumWords=1 BlockCodeSize=4>\n"
+        "  <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>\n"
+        "</Remark>\n",
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=14 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=0/>\n"
+        "  <String table codeid=3 abbrevid=5/> blob data = "
+        "'remark\\x00pass\\x00function\\x00'\n"
+        "  <External File codeid=4 abbrevid=6/> blob data = "
+        "'" EXTERNALFILETESTPATH"'\n"
+        "</Meta>\n");
+}
+
+TEST(BitstreamRemarkSerializer, SeparateRemarkFileNoOptionalsSeparateStrTab) {
+  remarks::StringTable StrTab;
+  StrTab.add("function");
+  StrTab.add("pass");
+  StrTab.add("remark");
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  check(R,
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=3 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=1/>\n"
+        "  <Remark version codeid=2 abbrevid=5 op0=0/>\n"
+        "</Meta>\n"
+        "<Remark BlockID=9 NumWords=1 BlockCodeSize=4>\n"
+        "  <Remark header codeid=5 abbrevid=4 op0=2 op1=2 op2=1 op3=0/>\n"
+        "</Remark>\n",
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=14 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=0/>\n"
+        "  <String table codeid=3 abbrevid=5/> blob data = "
+        "'function\\x00pass\\x00remark\\x00'\n"
+        "  <External File codeid=4 abbrevid=6/> blob data = "
+        "'" EXTERNALFILETESTPATH"'\n"
+        "</Meta>\n",
+        std::move(StrTab));
+}
+
+TEST(BitstreamRemarkSerializer, SeparateRemarkFileDebugLoc) {
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  R.Loc.emplace();
+  R.Loc->SourceFilePath = "path";
+  R.Loc->SourceLine = 99;
+  R.Loc->SourceColumn = 55;
+  check(R,
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=3 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=1/>\n"
+        "  <Remark version codeid=2 abbrevid=5 op0=0/>\n"
+        "</Meta>\n"
+        "<Remark BlockID=9 NumWords=4 BlockCodeSize=4>\n"
+        "  <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>\n"
+        "  <Remark debug location codeid=6 abbrevid=5 op0=3 op1=99 op2=55/>\n"
+        "</Remark>\n",
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=15 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=0/>\n"
+        "  <String table codeid=3 abbrevid=5/> blob data = "
+        "'remark\\x00pass\\x00function\\x00path\\x00'\n"
+        "  <External File codeid=4 abbrevid=6/> blob data = "
+        "'" EXTERNALFILETESTPATH"'\n"
+        "</Meta>\n");
+}
+
+TEST(BitstreamRemarkSerializer, SeparateRemarkFileHotness) {
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  R.Hotness.emplace(999999999);
+  check(R,
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=3 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=1/>\n"
+        "  <Remark version codeid=2 abbrevid=5 op0=0/>\n"
+        "</Meta>\n"
+        "<Remark BlockID=9 NumWords=3 BlockCodeSize=4>\n"
+        "  <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>\n"
+        "  <Remark hotness codeid=7 abbrevid=6 op0=999999999/>\n"
+        "</Remark>\n",
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=14 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=0/>\n"
+        "  <String table codeid=3 abbrevid=5/> blob data = "
+        "'remark\\x00pass\\x00function\\x00'\n"
+        "  <External File codeid=4 abbrevid=6/> blob data = "
+        "'" EXTERNALFILETESTPATH"'\n"
+        "</Meta>\n");
+}
+
+TEST(BitstreamRemarkSerializer, SeparateRemarkFileArgNoDebugLoc) {
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  R.Args.emplace_back();
+  R.Args.back().Key = "key";
+  R.Args.back().Val = "value";
+  check(R,
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=3 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=1/>\n"
+        "  <Remark version codeid=2 abbrevid=5 op0=0/>\n"
+        "</Meta>\n"
+        "<Remark BlockID=9 NumWords=2 BlockCodeSize=4>\n"
+        "  <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>\n"
+        "  <Argument codeid=9 abbrevid=8 op0=3 op1=4/>\n"
+        "</Remark>\n",
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=16 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=0/>\n"
+        "  <String table codeid=3 abbrevid=5/> blob data = "
+        "'remark\\x00pass\\x00function\\x00key\\x00value\\x00'\n"
+        "  <External File codeid=4 abbrevid=6/> blob data = "
+        "'" EXTERNALFILETESTPATH"'\n"
+        "</Meta>\n");
+}
+
+TEST(BitstreamRemarkSerializer, SeparateRemarkFileArgDebugLoc) {
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  R.Args.emplace_back();
+  R.Args.back().Key = "key";
+  R.Args.back().Val = "value";
+  R.Args.back().Loc.emplace();
+  R.Args.back().Loc->SourceFilePath = "path";
+  R.Args.back().Loc->SourceLine = 99;
+  R.Args.back().Loc->SourceColumn = 55;
+  check(R,
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=3 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=1/>\n"
+        "  <Remark version codeid=2 abbrevid=5 op0=0/>\n"
+        "</Meta>\n"
+        "<Remark BlockID=9 NumWords=4 BlockCodeSize=4>\n"
+        "  <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>\n"
+        "  <Argument with debug location codeid=8 abbrevid=7 op0=3 op1=4 op2=5 "
+        "op3=99 op4=55/>\n"
+        "</Remark>\n",
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=17 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=0/>\n"
+        "  <String table codeid=3 abbrevid=5/> blob data = "
+        "'remark\\x00pass\\x00function\\x00key\\x00value\\x00path\\x00'\n"
+        "  <External File codeid=4 abbrevid=6/> blob data = "
+        "'" EXTERNALFILETESTPATH"'\n"
+        "</Meta>\n");
+}
+
+TEST(BitstreamRemarkSerializer, SeparateRemarkFileAll) {
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  R.Loc.emplace();
+  R.Loc->SourceFilePath = "path";
+  R.Loc->SourceLine = 99;
+  R.Loc->SourceColumn = 55;
+  R.Hotness.emplace(999999999);
+  R.Args.emplace_back();
+  R.Args.back().Key = "key";
+  R.Args.back().Val = "value";
+  R.Args.back().Loc.emplace();
+  R.Args.back().Loc->SourceFilePath = "argpath";
+  R.Args.back().Loc->SourceLine = 11;
+  R.Args.back().Loc->SourceColumn = 66;
+  check(R,
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=3 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=1/>\n"
+        "  <Remark version codeid=2 abbrevid=5 op0=0/>\n"
+        "</Meta>\n"
+        "<Remark BlockID=9 NumWords=8 BlockCodeSize=4>\n"
+        "  <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>\n"
+        "  <Remark debug location codeid=6 abbrevid=5 op0=3 op1=99 op2=55/>\n"
+        "  <Remark hotness codeid=7 abbrevid=6 op0=999999999/>\n"
+        "  <Argument with debug location codeid=8 abbrevid=7 op0=4 op1=5 op2=6 "
+        "op3=11 op4=66/>\n"
+        "</Remark>\n",
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=19 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=0/>\n"
+        "  <String table codeid=3 abbrevid=5/> blob data = "
+        "'remark\\x00pass\\x00function\\x00path\\x00key\\x00value\\x00argpa"
+        "th\\x00'\n  <External File codeid=4 abbrevid=6/> blob data = "
+        "'" EXTERNALFILETESTPATH"'\n"
+        "</Meta>\n");
+}
+
+TEST(BitstreamRemarkSerializer, Standalone) {
+  // Pre-populate the string table.
+  remarks::StringTable StrTab;
+  StrTab.add("pass");
+  StrTab.add("remark");
+  StrTab.add("function");
+  StrTab.add("path");
+  StrTab.add("key");
+  StrTab.add("value");
+  StrTab.add("argpath");
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  R.Loc.emplace();
+  R.Loc->SourceFilePath = "path";
+  R.Loc->SourceLine = 99;
+  R.Loc->SourceColumn = 55;
+  R.Hotness.emplace(999999999);
+  R.Args.emplace_back();
+  R.Args.back().Key = "key";
+  R.Args.back().Val = "value";
+  R.Args.back().Loc.emplace();
+  R.Args.back().Loc->SourceFilePath = "argpath";
+  R.Args.back().Loc->SourceLine = 11;
+  R.Args.back().Loc->SourceColumn = 66;
+  checkStandalone(
+      R,
+      "<BLOCKINFO_BLOCK/>\n"
+      "<Meta BlockID=8 NumWords=15 BlockCodeSize=3>\n"
+      "  <Container info codeid=1 abbrevid=4 op0=0 op1=2/>\n"
+      "  <Remark version codeid=2 abbrevid=5 op0=0/>\n"
+      "  <String table codeid=3 abbrevid=6/> blob data = "
+      "'pass\\x00remark\\x00function\\x00path\\x00key\\x00value\\x00argpath\\x0"
+      "0'\n"
+      "</Meta>\n"
+      "<Remark BlockID=9 NumWords=8 BlockCodeSize=4>\n"
+      "  <Remark header codeid=5 abbrevid=4 op0=2 op1=1 op2=0 op3=2/>\n"
+      "  <Remark debug location codeid=6 abbrevid=5 op0=3 op1=99 op2=55/>\n"
+      "  <Remark hotness codeid=7 abbrevid=6 op0=999999999/>\n"
+      "  <Argument with debug location codeid=8 abbrevid=7 op0=4 op1=5 op2=6 "
+      "op3=11 op4=66/>\n"
+      "</Remark>\n",
+      std::move(StrTab));
+}

Modified: llvm/trunk/unittests/Remarks/CMakeLists.txt
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/Remarks/CMakeLists.txt?rev=367364&r1=367363&r2=367364&view=diff
==============================================================================
--- llvm/trunk/unittests/Remarks/CMakeLists.txt (original)
+++ llvm/trunk/unittests/Remarks/CMakeLists.txt Tue Jul 30 16:11:57 2019
@@ -1,9 +1,12 @@
 set(LLVM_LINK_COMPONENTS
+  BitReader
   Remarks
   Support
   )
 
 add_llvm_unittest(RemarksTests
+  BitstreamRemarksFormatTest.cpp
+  BitstreamRemarksSerializerTest.cpp
   RemarksStrTabParsingTest.cpp
   YAMLRemarksParsingTest.cpp
   YAMLRemarksSerializerTest.cpp




More information about the llvm-commits mailing list