[llvm] Implements PGOBBAddrMap in Object and ObjectYAML with tests [1/5] (PR #71750)
Micah Weston via llvm-commits
llvm-commits at lists.llvm.org
Wed Nov 8 16:40:25 PST 2023
https://github.com/red1bluelost created https://github.com/llvm/llvm-project/pull/71750
A part of [RFC - PGO Accuracy Metrics: Emitting and Evaluating Branch and Block Analysis](https://discourse.llvm.org/t/rfc-pgo-accuracy-metrics-emitting-and-evaluating-branch-and-block-analysis/73902).
This PR adds the PGOBBAddrMap data structure and implements encoding and decoding through Object and ObjectYAML along with associated tests.
## Design Alternative
The current implementation makes an effort to avoid modifying the original BBAddrMap data structure. This keeps the API the same as before for all pre-existing code and avoids unnecessary fields for regular BBAddrMap usage. It comes with the cost of some complexity in sharing encoding/decoding code.
Alternatively, we could just place all new fields into BBAddrMap. This makes the encoding/decoding code simpler. It has the cost of changing some behavior in the original API. It also adds unnecessary fields for anyone wanting to just use the regular BBAddrMap.
Please let us know if you feel that the alternate design of placing new fields into BBAddrMap is preferable.
## PR Series
The current code for PGOBBAddrMap to be upstreamed is split into five PRs:
1. Object and ObjectYAML - (this one)
2. AsmPrinter - TODO LINK
3. llvm-readobj - TODO LINK
4. llvm-objdump - TODO LINK
5. llvm obj2yaml - TODO LINK
If you would like to try testing PGOBBAddrMap locally on a program, PR-3 llvm-readobj is likely the minimum code needed to meaningfully use this feature.
>From 746ac2ec140b9775f8438a72a48987bf04b1f228 Mon Sep 17 00:00:00 2001
From: Micah Weston <micahsweston at gmail.com>
Date: Tue, 24 Oct 2023 22:31:15 -0400
Subject: [PATCH] Implements PGOBBAddrMap in Object and ObjectYAML with enough
support for tests.
Removes public and adds check for number of bits.
---
llvm/include/llvm/BinaryFormat/ELF.h | 1 +
llvm/include/llvm/Object/ELF.h | 6 +
llvm/include/llvm/Object/ELFObjectFile.h | 3 +
llvm/include/llvm/Object/ELFTypes.h | 108 +++++
llvm/include/llvm/ObjectYAML/ELFYAML.h | 60 ++-
llvm/lib/MC/MCParser/ELFAsmParser.cpp | 2 +
llvm/lib/MC/MCSectionELF.cpp | 2 +
llvm/lib/Object/ELF.cpp | 263 +++++++++---
llvm/lib/Object/ELFObjectFile.cpp | 52 ++-
llvm/lib/ObjectYAML/ELFEmitter.cpp | 130 +++++-
llvm/lib/ObjectYAML/ELFYAML.cpp | 45 +-
llvm/test/MC/AsmParser/llvm_section_types.s | 4 +
llvm/tools/obj2yaml/elf2yaml.cpp | 2 +-
llvm/unittests/Object/ELFObjectFileTest.cpp | 440 ++++++++++++++++++++
llvm/unittests/Object/ELFTypesTest.cpp | 53 ++-
15 files changed, 1080 insertions(+), 91 deletions(-)
diff --git a/llvm/include/llvm/BinaryFormat/ELF.h b/llvm/include/llvm/BinaryFormat/ELF.h
index 3596174f74dde80..299d7e8265317b3 100644
--- a/llvm/include/llvm/BinaryFormat/ELF.h
+++ b/llvm/include/llvm/BinaryFormat/ELF.h
@@ -1037,6 +1037,7 @@ enum : unsigned {
SHT_LLVM_BB_ADDR_MAP = 0x6fff4c0a, // LLVM Basic Block Address Map.
SHT_LLVM_OFFLOADING = 0x6fff4c0b, // LLVM device offloading data.
SHT_LLVM_LTO = 0x6fff4c0c, // .llvm.lto for fat LTO.
+ SHT_LLVM_PGO_BB_ADDR_MAP = 0x6fff4c0d, // LLVM PGO extended BB Addr Map.
// Android's experimental support for SHT_RELR sections.
// https://android.googlesource.com/platform/bionic/+/b7feec74547f84559a1467aca02708ff61346d2a/libc/include/elf.h#512
SHT_ANDROID_RELR = 0x6fffff00, // Relocation entries; only offsets.
diff --git a/llvm/include/llvm/Object/ELF.h b/llvm/include/llvm/Object/ELF.h
index a1cf47a1c4a6173..3768347ee59582b 100644
--- a/llvm/include/llvm/Object/ELF.h
+++ b/llvm/include/llvm/Object/ELF.h
@@ -417,6 +417,12 @@ class ELFFile {
Expected<std::vector<BBAddrMap>>
decodeBBAddrMap(const Elf_Shdr &Sec, const Elf_Shdr *RelaSec = nullptr) const;
+ /// Decodes same as decodeBBAddrMap but also decodes extra information from
+ /// features which are enabled.
+ Expected<std::vector<PGOBBAddrMap>>
+ decodePGOBBAddrMap(const Elf_Shdr &Sec,
+ const Elf_Shdr *RelaSec = nullptr) const;
+
/// Returns a map from every section matching \p IsMatch to its relocation
/// section, or \p nullptr if it has no relocation section. This function
/// returns an error if any of the \p IsMatch calls fail or if it fails to
diff --git a/llvm/include/llvm/Object/ELFObjectFile.h b/llvm/include/llvm/Object/ELFObjectFile.h
index d7947d85739eb3b..d6dd62d487affbe 100644
--- a/llvm/include/llvm/Object/ELFObjectFile.h
+++ b/llvm/include/llvm/Object/ELFObjectFile.h
@@ -114,6 +114,9 @@ class ELFObjectFileBase : public ObjectFile {
// corresponding to the section with that index.
Expected<std::vector<BBAddrMap>>
readBBAddrMap(std::optional<unsigned> TextSectionIndex = std::nullopt) const;
+
+ Expected<std::vector<PGOBBAddrMap>> readPGOBBAddrMap(
+ std::optional<unsigned> TextSectionIndex = std::nullopt) const;
};
class ELFSectionRef : public SectionRef {
diff --git a/llvm/include/llvm/Object/ELFTypes.h b/llvm/include/llvm/Object/ELFTypes.h
index 45fc52288bdd4c0..66880bdfac99581 100644
--- a/llvm/include/llvm/Object/ELFTypes.h
+++ b/llvm/include/llvm/Object/ELFTypes.h
@@ -10,9 +10,12 @@
#define LLVM_OBJECT_ELFTYPES_H
#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/BitmaskEnum.h"
#include "llvm/ADT/StringRef.h"
#include "llvm/BinaryFormat/ELF.h"
#include "llvm/Object/Error.h"
+#include "llvm/Support/BlockFrequency.h"
+#include "llvm/Support/BranchProbability.h"
#include "llvm/Support/Endian.h"
#include "llvm/Support/Error.h"
#include "llvm/Support/MathExtras.h"
@@ -806,6 +809,11 @@ struct BBAddrMap {
bool HasIndirectBranch : 1; // If this block ends with an indirect branch
// (branch via a register).
+ // Number of bits used when encoding Metadata, that way an extension
+ // can pack into the extra space if possible. This must be updated when
+ // new bits are added here.
+ static constexpr uint32_t NumberOfBits = 5;
+
bool operator==(const Metadata &Other) const {
return HasReturn == Other.HasReturn &&
HasTailCall == Other.HasTailCall && IsEHPad == Other.IsEHPad &&
@@ -865,6 +873,106 @@ struct BBAddrMap {
}
};
+/// An extension of BBAddrMap that holds information relevant to PGO.
+struct PGOBBAddrMap {
+ /// Bitmask of optional features to include in the PGO extended map.
+ enum class Features {
+ None = 0,
+ FuncEntryCnt = (1 << 0),
+ BBFreq = (1 << 1),
+ BrProb = (1 << 2),
+ LLVM_MARK_AS_BITMASK_ENUM(/*LargestValue=*/BrProb),
+ };
+
+ /// Super-set of BBAddrMap::BBEntry with additional fields for block frequency
+ /// and branch probability.
+ struct BBEntry {
+ using BaseMetadata = BBAddrMap::BBEntry::Metadata;
+
+ /// Enum indicating the how many successors a block has. This enum must fit
+ /// into two bits.
+ enum class SuccessorsType {
+ /// None should be present if PGOBBAddrMap has disabled branch
+ /// probability.
+ None = 0,
+ /// Single successor blocks are not present in the successor entries.
+ One = 1,
+ /// Common case for conditional branches to avoid encoding size.
+ Two = 2,
+ /// Uncommon case which needs successor size to be encoded.
+ Multiple = 3,
+ };
+
+ /// Single successor of a given basic block that contains the tag and branch
+ /// probability associated with it.
+ struct SuccessorEntry {
+ /// Unique ID of this successor basic block.
+ uint32_t ID;
+ /// Branch Probability of the edge to this successor taken from MBPI
+ BranchProbability Prob;
+
+ bool operator==(const SuccessorEntry &Other) const {
+ return std::tie(ID, Prob) == std::tie(Other.ID, Other.Prob);
+ }
+ };
+
+ /// Reuse of the fields provided by regular BBAddrMap
+ BBAddrMap::BBEntry Base;
+ /// Block frequency taken from MBFI
+ BlockFrequency BlockFreq;
+ /// List of successors of the current block
+ llvm::SmallVector<SuccessorEntry, 2> Successors;
+
+ /// Converts number of successors into a SuccessorsType.
+ static SuccessorsType getSuccessorsType(unsigned SuccessorsCount) {
+ return SuccessorsCount == 0 ? SuccessorsType::None
+ : SuccessorsCount == 1 ? SuccessorsType::One
+ : SuccessorsCount == 2 ? SuccessorsType::Two
+ : SuccessorsType::Multiple;
+ }
+
+ /// Encodes extra information in the free bits of the base metadata
+ static uint32_t encodeMD(BaseMetadata MD, SuccessorsType SuccType) {
+ return MD.encode() | static_cast<uint32_t>(SuccType)
+ << BaseMetadata::NumberOfBits;
+ }
+
+ /// Extracts successors type then defers all errors to the base metadata
+ static Expected<std::pair<BaseMetadata, SuccessorsType>>
+ decodeMD(uint32_t V) {
+ auto SuccType = SuccessorsType((V >> BaseMetadata::NumberOfBits) & 0b11);
+ V &= ~(0b11 << BaseMetadata::NumberOfBits); // Clear extra bits
+ BaseMetadata MD;
+ if (llvm::Error E = BaseMetadata::decode(V).moveInto(MD))
+ return std::move(E);
+ return std::make_pair(MD, SuccType);
+ }
+
+ bool operator==(const BBEntry &Other) const {
+ return std::tie(Base, BlockFreq, Successors) ==
+ std::tie(Other.Base, Other.BlockFreq, Other.Successors);
+ }
+ };
+ // This field is duplicated from BBAddrMap since this class needs a different
+ // type for the vector of entries.
+ uint64_t Addr; // Function address
+ std::vector<BBEntry> BBEntries; // Extended basic block entries
+ uint64_t FuncEntryCount; // Prof count from IR function
+
+ // Flags to indicate if each PGO related info was enabled in this function
+ bool FuncEntryCountEnabled : 1;
+ bool BBFreqEnabled : 1;
+ bool BBSuccProbEnabled : 1;
+
+ bool operator==(const PGOBBAddrMap &Other) const {
+ return std::tie(Addr, FuncEntryCount, BBEntries, FuncEntryCountEnabled,
+ BBFreqEnabled, BBSuccProbEnabled) ==
+ std::tie(Other.Addr, Other.FuncEntryCount, Other.BBEntries,
+ Other.FuncEntryCountEnabled, Other.BBFreqEnabled,
+ Other.BBSuccProbEnabled);
+ }
+};
+
} // end namespace object.
} // end namespace llvm.
diff --git a/llvm/include/llvm/ObjectYAML/ELFYAML.h b/llvm/include/llvm/ObjectYAML/ELFYAML.h
index 1ba41232f552e3a..3f18d189b7dc870 100644
--- a/llvm/include/llvm/ObjectYAML/ELFYAML.h
+++ b/llvm/include/llvm/ObjectYAML/ELFYAML.h
@@ -156,6 +156,13 @@ struct DynamicEntry {
llvm::yaml::Hex64 Val;
};
+struct BBAddrMapCommonBase {
+ uint8_t Version;
+ llvm::yaml::Hex8 Feature;
+ llvm::yaml::Hex64 Address;
+ std::optional<uint64_t> NumBlocks;
+};
+
struct BBAddrMapEntry {
struct BBEntry {
uint32_t ID;
@@ -163,10 +170,22 @@ struct BBAddrMapEntry {
llvm::yaml::Hex64 Size;
llvm::yaml::Hex64 Metadata;
};
- uint8_t Version;
- llvm::yaml::Hex8 Feature;
- llvm::yaml::Hex64 Address;
- std::optional<uint64_t> NumBlocks;
+ BBAddrMapCommonBase Common;
+ std::optional<std::vector<BBEntry>> BBEntries;
+};
+
+struct PGOBBAddrMapEntry {
+ struct BBEntry {
+ struct SuccessorEntry {
+ uint32_t ID;
+ llvm::yaml::Hex32 BrProb;
+ };
+ BBAddrMapEntry::BBEntry Base;
+ std::optional<uint64_t> BBFreq;
+ std::optional<std::vector<SuccessorEntry>> Successors;
+ };
+ BBAddrMapCommonBase Common;
+ std::optional<uint64_t> FuncEntryCount;
std::optional<std::vector<BBEntry>> BBEntries;
};
@@ -204,6 +223,7 @@ struct Chunk {
DependentLibraries,
CallGraphProfile,
BBAddrMap,
+ PGOBBAddrMap,
// Special chunks.
SpecialChunksStart,
@@ -329,6 +349,20 @@ struct BBAddrMapSection : Section {
}
};
+struct PGOBBAddrMapSection : Section {
+ std::optional<std::vector<PGOBBAddrMapEntry>> Entries;
+
+ PGOBBAddrMapSection() : Section(ChunkKind::PGOBBAddrMap) {}
+
+ std::vector<std::pair<StringRef, bool>> getEntries() const override {
+ return {{"Entries", Entries.has_value()}};
+ };
+
+ static bool classof(const Chunk *S) {
+ return S->Kind == ChunkKind::PGOBBAddrMap;
+ }
+};
+
struct StackSizesSection : Section {
std::optional<std::vector<StackSizeEntry>> Entries;
@@ -737,6 +771,10 @@ bool shouldAllocateFileSpace(ArrayRef<ProgramHeader> Phdrs,
LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::ELFYAML::StackSizeEntry)
LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::ELFYAML::BBAddrMapEntry)
LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::ELFYAML::BBAddrMapEntry::BBEntry)
+LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::ELFYAML::PGOBBAddrMapEntry)
+LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::ELFYAML::PGOBBAddrMapEntry::BBEntry)
+LLVM_YAML_IS_SEQUENCE_VECTOR(
+ llvm::ELFYAML::PGOBBAddrMapEntry::BBEntry::SuccessorEntry)
LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::ELFYAML::DynamicEntry)
LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::ELFYAML::LinkerOption)
LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::ELFYAML::CallGraphEntryWeight)
@@ -905,6 +943,20 @@ template <> struct MappingTraits<ELFYAML::BBAddrMapEntry::BBEntry> {
static void mapping(IO &IO, ELFYAML::BBAddrMapEntry::BBEntry &Rel);
};
+template <> struct MappingTraits<ELFYAML::PGOBBAddrMapEntry> {
+ static void mapping(IO &IO, ELFYAML::PGOBBAddrMapEntry &Rel);
+};
+
+template <> struct MappingTraits<ELFYAML::PGOBBAddrMapEntry::BBEntry> {
+ static void mapping(IO &IO, ELFYAML::PGOBBAddrMapEntry::BBEntry &Rel);
+};
+
+template <>
+struct MappingTraits<ELFYAML::PGOBBAddrMapEntry::BBEntry::SuccessorEntry> {
+ static void mapping(IO &IO,
+ ELFYAML::PGOBBAddrMapEntry::BBEntry::SuccessorEntry &Rel);
+};
+
template <> struct MappingTraits<ELFYAML::GnuHashHeader> {
static void mapping(IO &IO, ELFYAML::GnuHashHeader &Rel);
};
diff --git a/llvm/lib/MC/MCParser/ELFAsmParser.cpp b/llvm/lib/MC/MCParser/ELFAsmParser.cpp
index dbfe0d83e1b29b1..e4c1cc7763f7804 100644
--- a/llvm/lib/MC/MCParser/ELFAsmParser.cpp
+++ b/llvm/lib/MC/MCParser/ELFAsmParser.cpp
@@ -673,6 +673,8 @@ bool ELFAsmParser::ParseSectionArguments(bool IsPush, SMLoc loc) {
Type = ELF::SHT_LLVM_SYMPART;
else if (TypeName == "llvm_bb_addr_map")
Type = ELF::SHT_LLVM_BB_ADDR_MAP;
+ else if (TypeName == "llvm_pgo_bb_addr_map")
+ Type = ELF::SHT_LLVM_PGO_BB_ADDR_MAP;
else if (TypeName == "llvm_offloading")
Type = ELF::SHT_LLVM_OFFLOADING;
else if (TypeName == "llvm_lto")
diff --git a/llvm/lib/MC/MCSectionELF.cpp b/llvm/lib/MC/MCSectionELF.cpp
index 95fdf33522076e4..02e63f7b03919a8 100644
--- a/llvm/lib/MC/MCSectionELF.cpp
+++ b/llvm/lib/MC/MCSectionELF.cpp
@@ -170,6 +170,8 @@ void MCSectionELF::printSwitchToSection(const MCAsmInfo &MAI, const Triple &T,
OS << "llvm_bb_addr_map";
else if (Type == ELF::SHT_LLVM_BB_ADDR_MAP_V0)
OS << "llvm_bb_addr_map_v0";
+ else if (Type == ELF::SHT_LLVM_PGO_BB_ADDR_MAP)
+ OS << "llvm_pgo_bb_addr_map";
else if (Type == ELF::SHT_LLVM_OFFLOADING)
OS << "llvm_offloading";
else if (Type == ELF::SHT_LLVM_LTO)
diff --git a/llvm/lib/Object/ELF.cpp b/llvm/lib/Object/ELF.cpp
index 0d1862e573712f5..e5b2b38048b9382 100644
--- a/llvm/lib/Object/ELF.cpp
+++ b/llvm/lib/Object/ELF.cpp
@@ -312,6 +312,7 @@ StringRef llvm::object::getELFSectionTypeName(uint32_t Machine, unsigned Type) {
STRINGIFY_ENUM_CASE(ELF, SHT_LLVM_PART_PHDR);
STRINGIFY_ENUM_CASE(ELF, SHT_LLVM_BB_ADDR_MAP_V0);
STRINGIFY_ENUM_CASE(ELF, SHT_LLVM_BB_ADDR_MAP);
+ STRINGIFY_ENUM_CASE(ELF, SHT_LLVM_PGO_BB_ADDR_MAP);
STRINGIFY_ENUM_CASE(ELF, SHT_LLVM_OFFLOADING);
STRINGIFY_ENUM_CASE(ELF, SHT_LLVM_LTO);
STRINGIFY_ENUM_CASE(ELF, SHT_GNU_ATTRIBUTES);
@@ -645,11 +646,179 @@ ELFFile<ELFT>::toMappedAddr(uint64_t VAddr, WarningHandler WarnHandler) const {
return base() + Offset;
}
-template <class ELFT>
-Expected<std::vector<BBAddrMap>>
-ELFFile<ELFT>::decodeBBAddrMap(const Elf_Shdr &Sec,
- const Elf_Shdr *RelaSec) const {
- bool IsRelocatable = getHeader().e_type == ELF::ET_REL;
+// Helper to extract and decode the next ULEB128 value as unsigned int.
+// Returns zero and sets ULEBSizeErr if the ULEB128 value exceeds the unsigned
+// int limit.
+// Also returns zero if ULEBSizeErr is already in an error state.
+// ULEBSizeErr is an out variable if an error occurs.
+template <typename IntTy>
+static IntTy readULEB128As(DataExtractor &Data, DataExtractor::Cursor &Cur,
+ Error &ULEBSizeErr) {
+ static_assert(std::is_unsigned_v<IntTy> &&
+ (std::numeric_limits<IntTy>::radix == 2),
+ "only use unsigned radix 2");
+ // Bail out and do not extract data if ULEBSizeErr is already set.
+ if (ULEBSizeErr)
+ return 0;
+ uint64_t Offset = Cur.tell();
+ uint64_t Value = Data.getULEB128(Cur);
+ if (Value > std::numeric_limits<IntTy>::max()) {
+ ULEBSizeErr = createError("ULEB128 value at offset 0x" +
+ Twine::utohexstr(Offset) + " exceeds UINT" +
+ Twine(std::numeric_limits<IntTy>::digits) +
+ "_MAX (0x" + Twine::utohexstr(Value) + ")");
+ return 0;
+ }
+ return static_cast<IntTy>(Value);
+}
+
+namespace {
+struct BasicAddrMapBBEntry {
+ uint32_t ID;
+ uint32_t Offset;
+ uint32_t Size;
+ uint32_t MD;
+};
+
+template <typename ELFT, typename AddrMap> struct AddrMapDecodeTrait;
+
+template <typename ELFT> struct AddrMapDecodeTrait<ELFT, BBAddrMap> {
+ // Base addr map has no extra data
+ struct ExtraData {};
+
+ static constexpr unsigned SectionID = ELF::SHT_LLVM_BB_ADDR_MAP;
+
+ DataExtractor &Data;
+ DataExtractor::Cursor &Cur;
+ Error &ULEBSizeErr;
+ Error &MetadataDecodeErr;
+
+ uint32_t PrevBBEndOffset = 0;
+
+ AddrMapDecodeTrait(DataExtractor &Data, DataExtractor::Cursor &Cur,
+ Error &ULEBSizeErr, Error &MetadataDecodeErr)
+ : Data(Data), Cur(Cur), ULEBSizeErr(ULEBSizeErr),
+ MetadataDecodeErr(MetadataDecodeErr) {}
+
+ ExtraData decodeExtraDataHeader(uint8_t, uint8_t) {
+ PrevBBEndOffset = 0;
+ return {};
+ }
+
+ // This helper method avoids decoding the metadata so other AddrMaps can use
+ BasicAddrMapBBEntry decodeBasicInfo(uint8_t Version, uint8_t Feature,
+ uint32_t BlockIndex) {
+ uint32_t ID = Version >= 2 ? readULEB128As<uint32_t>(Data, Cur, ULEBSizeErr)
+ : BlockIndex;
+ uint32_t Offset = readULEB128As<uint32_t>(Data, Cur, ULEBSizeErr);
+ uint32_t Size = readULEB128As<uint32_t>(Data, Cur, ULEBSizeErr);
+ uint32_t MD = readULEB128As<uint32_t>(Data, Cur, ULEBSizeErr);
+ if (Version >= 1) {
+ // Offset is calculated relative to the end of the previous BB.
+ Offset += PrevBBEndOffset;
+ PrevBBEndOffset = Offset + Size;
+ }
+ return {ID, Offset, Size, MD};
+ }
+
+ BBAddrMap::BBEntry decodeBBEntry(uint8_t Version, uint8_t Feature,
+ uint32_t BlockIndex) {
+ auto [ID, Offset, Size, MD] = decodeBasicInfo(Version, Feature, BlockIndex);
+ auto MetadataOrErr = BBAddrMap::BBEntry::Metadata::decode(MD);
+ if (Error E = MetadataOrErr.takeError()) {
+ MetadataDecodeErr = std::move(E);
+ return {{}, {}, {}, {}};
+ }
+ return {ID, Offset, Size, *MetadataOrErr};
+ }
+
+ BBAddrMap makeAddrMap(ExtraData, uint8_t Feature, uint64_t Address,
+ std::vector<BBAddrMap::BBEntry> BBEntries) {
+ return {Address, std::move(BBEntries)};
+ }
+};
+
+template <typename ELFT> struct AddrMapDecodeTrait<ELFT, PGOBBAddrMap> {
+ using Features = PGOBBAddrMap::Features;
+
+ struct ExtraData {
+ uint64_t FuncEntryCount;
+ };
+
+ static constexpr unsigned SectionID = ELF::SHT_LLVM_PGO_BB_ADDR_MAP;
+
+ AddrMapDecodeTrait<ELFT, BBAddrMap> Base;
+
+ AddrMapDecodeTrait(DataExtractor &Data, DataExtractor::Cursor &Cur,
+ Error &ULEBSizeErr, Error &MetadataDecodeErr)
+ : Base(Data, Cur, ULEBSizeErr, MetadataDecodeErr) {}
+
+ ExtraData decodeExtraDataHeader(uint8_t Version, uint8_t Feature) {
+ Base.decodeExtraDataHeader(Version, Feature);
+ if (Version < 2) {
+ // hijack the metadata error if version is too low
+ Base.MetadataDecodeErr =
+ createError("unsupported SHT_LLVM_PGO_BB_ADDR_MAP version: " +
+ Twine(static_cast<int>(Version)));
+ return {};
+ }
+ return {Feature & uint8_t(PGOBBAddrMap::Features::FuncEntryCnt)
+ ? readULEB128As<uint64_t>(Base.Data, Base.Cur, Base.ULEBSizeErr)
+ : 0};
+ }
+
+ PGOBBAddrMap::BBEntry decodeBBEntry(uint8_t Version, uint8_t Feature,
+ uint32_t BlockIndex) {
+ auto [ID, Offset, Size, MD] =
+ Base.decodeBasicInfo(Version, Feature, BlockIndex);
+ auto MetadataOrErr = PGOBBAddrMap::BBEntry::decodeMD(MD);
+ if (Error E = MetadataOrErr.takeError()) {
+ Base.MetadataDecodeErr = std::move(E);
+ return {{{}, {}, {}, {}}, {}, {}};
+ }
+ auto [MetaData, SuccsType] = *MetadataOrErr;
+
+ uint64_t BBF =
+ Feature & uint8_t(PGOBBAddrMap::Features::BBFreq)
+ ? readULEB128As<uint64_t>(Base.Data, Base.Cur, Base.ULEBSizeErr)
+ : 0;
+
+ llvm::SmallVector<PGOBBAddrMap::BBEntry::SuccessorEntry, 2> Successors;
+ if (Feature & uint8_t(PGOBBAddrMap::Features::BrProb)) {
+ auto SuccCount =
+ SuccsType == PGOBBAddrMap::BBEntry::SuccessorsType::Multiple
+ ? readULEB128As<uint64_t>(Base.Data, Base.Cur, Base.ULEBSizeErr)
+ : uint64_t(SuccsType);
+ for (uint64_t I = 0; I < SuccCount; ++I) {
+ uint32_t BBID =
+ readULEB128As<uint32_t>(Base.Data, Base.Cur, Base.ULEBSizeErr);
+ uint32_t BrProb =
+ readULEB128As<uint32_t>(Base.Data, Base.Cur, Base.ULEBSizeErr);
+ Successors.push_back({BBID, BranchProbability::getRaw(BrProb)});
+ }
+ }
+ return PGOBBAddrMap::BBEntry{
+ {ID, Offset, Size, MetaData}, BlockFrequency(BBF), Successors};
+ }
+
+ PGOBBAddrMap makeAddrMap(ExtraData ED, uint8_t Feature, uint64_t Address,
+ std::vector<PGOBBAddrMap::BBEntry> BBEntries) {
+ return {Address,
+ std::move(BBEntries),
+ ED.FuncEntryCount,
+ bool(Feature & uint8_t(Features::FuncEntryCnt)),
+ bool(Feature & uint8_t(Features::BBFreq)),
+ bool(Feature & uint8_t(Features::BrProb))};
+ }
+};
+} // namespace
+
+template <typename AddrMap, typename ELFT>
+static Expected<std::vector<AddrMap>>
+decodeBBAddrMapCommonImpl(const ELFFile<ELFT> &EF,
+ const typename ELFFile<ELFT>::Elf_Shdr &Sec,
+ const typename ELFFile<ELFT>::Elf_Shdr *RelaSec) {
+ bool IsRelocatable = EF.getHeader().e_type == ELF::ET_REL;
// This DenseMap maps the offset of each function (the location of the
// reference to the function in the SHT_LLVM_BB_ADDR_MAP section) to the
@@ -659,57 +828,44 @@ ELFFile<ELFT>::decodeBBAddrMap(const Elf_Shdr &Sec,
assert(RelaSec &&
"Can't read a SHT_LLVM_BB_ADDR_MAP section in a relocatable "
"object file without providing a relocation section.");
- Expected<Elf_Rela_Range> Relas = this->relas(*RelaSec);
+ Expected<typename ELFFile<ELFT>::Elf_Rela_Range> Relas = EF.relas(*RelaSec);
if (!Relas)
return createError("unable to read relocations for section " +
- describe(*this, Sec) + ": " +
+ describe(EF, Sec) + ": " +
toString(Relas.takeError()));
- for (Elf_Rela Rela : *Relas)
+ for (typename ELFFile<ELFT>::Elf_Rela Rela : *Relas)
FunctionOffsetTranslations[Rela.r_offset] = Rela.r_addend;
}
- Expected<ArrayRef<uint8_t>> ContentsOrErr = getSectionContents(Sec);
+ Expected<ArrayRef<uint8_t>> ContentsOrErr = EF.getSectionContents(Sec);
if (!ContentsOrErr)
return ContentsOrErr.takeError();
ArrayRef<uint8_t> Content = *ContentsOrErr;
- DataExtractor Data(Content, isLE(), ELFT::Is64Bits ? 8 : 4);
- std::vector<BBAddrMap> FunctionEntries;
+ DataExtractor Data(Content, EF.isLE(), ELFT::Is64Bits ? 8 : 4);
+ std::vector<AddrMap> FunctionEntries;
DataExtractor::Cursor Cur(0);
Error ULEBSizeErr = Error::success();
Error MetadataDecodeErr = Error::success();
- // Helper to extract and decode the next ULEB128 value as uint32_t.
- // Returns zero and sets ULEBSizeErr if the ULEB128 value exceeds the uint32_t
- // limit.
- // Also returns zero if ULEBSizeErr is already in an error state.
- auto ReadULEB128AsUInt32 = [&Data, &Cur, &ULEBSizeErr]() -> uint32_t {
- // Bail out and do not extract data if ULEBSizeErr is already set.
- if (ULEBSizeErr)
- return 0;
- uint64_t Offset = Cur.tell();
- uint64_t Value = Data.getULEB128(Cur);
- if (Value > UINT32_MAX) {
- ULEBSizeErr = createError(
- "ULEB128 value at offset 0x" + Twine::utohexstr(Offset) +
- " exceeds UINT32_MAX (0x" + Twine::utohexstr(Value) + ")");
- return 0;
- }
- return static_cast<uint32_t>(Value);
- };
+
+ using DecodeTrait = AddrMapDecodeTrait<ELFT, AddrMap>;
+ DecodeTrait DT(Data, Cur, ULEBSizeErr, MetadataDecodeErr);
uint8_t Version = 0;
+ uint8_t Feature = 0;
while (!ULEBSizeErr && !MetadataDecodeErr && Cur &&
Cur.tell() < Content.size()) {
- if (Sec.sh_type == ELF::SHT_LLVM_BB_ADDR_MAP) {
+ if (Sec.sh_type == DecodeTrait::SectionID) {
Version = Data.getU8(Cur);
if (!Cur)
break;
if (Version > 2)
return createError("unsupported SHT_LLVM_BB_ADDR_MAP version: " +
Twine(static_cast<int>(Version)));
- Data.getU8(Cur); // Feature byte
+ Feature = Data.getU8(Cur); // Feature byte
}
uint64_t SectionOffset = Cur.tell();
- uintX_t Address = static_cast<uintX_t>(Data.getAddress(Cur));
+ auto Address =
+ static_cast<typename ELFFile<ELFT>::uintX_t>(Data.getAddress(Cur));
if (!Cur)
return Cur.takeError();
if (IsRelocatable) {
@@ -718,34 +874,23 @@ ELFFile<ELFT>::decodeBBAddrMap(const Elf_Shdr &Sec,
if (FOTIterator == FunctionOffsetTranslations.end()) {
return createError("failed to get relocation data for offset: " +
Twine::utohexstr(SectionOffset) + " in section " +
- describe(*this, Sec));
+ describe(EF, Sec));
}
Address = FOTIterator->second;
}
- uint32_t NumBlocks = ReadULEB128AsUInt32();
- std::vector<BBAddrMap::BBEntry> BBEntries;
- uint32_t PrevBBEndOffset = 0;
+ uint32_t NumBlocks = readULEB128As<uint32_t>(Data, Cur, ULEBSizeErr);
+ auto ExtraData = DT.decodeExtraDataHeader(Version, Feature);
+ std::vector<typename AddrMap::BBEntry> BBEntries;
for (uint32_t BlockIndex = 0;
!MetadataDecodeErr && !ULEBSizeErr && Cur && (BlockIndex < NumBlocks);
++BlockIndex) {
- uint32_t ID = Version >= 2 ? ReadULEB128AsUInt32() : BlockIndex;
- uint32_t Offset = ReadULEB128AsUInt32();
- uint32_t Size = ReadULEB128AsUInt32();
- uint32_t MD = ReadULEB128AsUInt32();
- if (Version >= 1) {
- // Offset is calculated relative to the end of the previous BB.
- Offset += PrevBBEndOffset;
- PrevBBEndOffset = Offset + Size;
- }
- Expected<BBAddrMap::BBEntry::Metadata> MetadataOrErr =
- BBAddrMap::BBEntry::Metadata::decode(MD);
- if (!MetadataOrErr) {
- MetadataDecodeErr = MetadataOrErr.takeError();
+ auto Entry = DT.decodeBBEntry(Version, Feature, BlockIndex);
+ if (MetadataDecodeErr)
break;
- }
- BBEntries.push_back({ID, Offset, Size, *MetadataOrErr});
+ BBEntries.push_back(std::move(Entry));
}
- FunctionEntries.push_back({Address, std::move(BBEntries)});
+ FunctionEntries.push_back(
+ DT.makeAddrMap(ExtraData, Feature, Address, std::move(BBEntries)));
}
// Either Cur is in the error state, or we have an error in ULEBSizeErr or
// MetadataDecodeErr (but not both), but we join all errors here to be safe.
@@ -755,6 +900,20 @@ ELFFile<ELFT>::decodeBBAddrMap(const Elf_Shdr &Sec,
return FunctionEntries;
}
+template <class ELFT>
+Expected<std::vector<BBAddrMap>>
+ELFFile<ELFT>::decodeBBAddrMap(const Elf_Shdr &Sec,
+ const Elf_Shdr *RelaSec) const {
+ return decodeBBAddrMapCommonImpl<BBAddrMap>(*this, Sec, RelaSec);
+}
+
+template <class ELFT>
+Expected<std::vector<PGOBBAddrMap>>
+ELFFile<ELFT>::decodePGOBBAddrMap(const Elf_Shdr &Sec,
+ const Elf_Shdr *RelaSec) const {
+ return decodeBBAddrMapCommonImpl<PGOBBAddrMap>(*this, Sec, RelaSec);
+}
+
template <class ELFT>
Expected<
MapVector<const typename ELFT::Shdr *, const typename ELFT::Shdr *>>
diff --git a/llvm/lib/Object/ELFObjectFile.cpp b/llvm/lib/Object/ELFObjectFile.cpp
index 143f9d37849d238..0c200f090802269 100644
--- a/llvm/lib/Object/ELFObjectFile.cpp
+++ b/llvm/lib/Object/ELFObjectFile.cpp
@@ -708,17 +708,18 @@ std::vector<ELFPltEntry> ELFObjectFileBase::getPltEntries() const {
return Result;
}
-template <class ELFT>
-Expected<std::vector<BBAddrMap>> static readBBAddrMapImpl(
- const ELFFile<ELFT> &EF, std::optional<unsigned> TextSectionIndex) {
+template <typename AddrMap, typename ELFT, typename DecodeAddrMapFn>
+Expected<std::vector<AddrMap>> static readBBAddrMapCommonImpl(
+ const ELFFile<ELFT> &EF, std::optional<unsigned> TextSectionIndex,
+ ArrayRef<unsigned> SectionIDs, DecodeAddrMapFn DecodeAddrMap) {
using Elf_Shdr = typename ELFT::Shdr;
bool IsRelocatable = EF.getHeader().e_type == ELF::ET_REL;
- std::vector<BBAddrMap> BBAddrMaps;
+ std::vector<AddrMap> BBAddrMaps;
const auto &Sections = cantFail(EF.sections());
auto IsMatch = [&](const Elf_Shdr &Sec) -> Expected<bool> {
- if (Sec.sh_type != ELF::SHT_LLVM_BB_ADDR_MAP &&
- Sec.sh_type != ELF::SHT_LLVM_BB_ADDR_MAP_V0)
+ if (llvm::none_of(SectionIDs,
+ [&](unsigned ID) { return ID == Sec.sh_type; }))
return false;
if (!TextSectionIndex)
return true;
@@ -741,8 +742,8 @@ Expected<std::vector<BBAddrMap>> static readBBAddrMapImpl(
if (IsRelocatable && !RelocSec)
return createError("unable to get relocation section for " +
describe(EF, *Sec));
- Expected<std::vector<BBAddrMap>> BBAddrMapOrErr =
- EF.decodeBBAddrMap(*Sec, RelocSec);
+ Expected<std::vector<AddrMap>> BBAddrMapOrErr =
+ DecodeAddrMap(EF, *Sec, RelocSec);
if (!BBAddrMapOrErr)
return createError("unable to read " + describe(EF, *Sec) + ": " +
toString(BBAddrMapOrErr.takeError()));
@@ -752,6 +753,29 @@ Expected<std::vector<BBAddrMap>> static readBBAddrMapImpl(
return BBAddrMaps;
}
+template <class ELFT>
+Expected<std::vector<BBAddrMap>> static readBBAddrMapImpl(
+ const ELFFile<ELFT> &EF, std::optional<unsigned> TextSectionIndex) {
+ return readBBAddrMapCommonImpl<BBAddrMap>(
+ EF, TextSectionIndex,
+ {ELF::SHT_LLVM_BB_ADDR_MAP, ELF::SHT_LLVM_BB_ADDR_MAP_V0},
+ [](const ELFFile<ELFT> &EF, const typename ELFFile<ELFT>::Elf_Shdr &Sec,
+ const typename ELFFile<ELFT>::Elf_Shdr *RelaSec) {
+ return EF.decodeBBAddrMap(Sec, RelaSec);
+ });
+}
+
+template <class ELFT>
+Expected<std::vector<PGOBBAddrMap>> static readPGOBBAddrMapImpl(
+ const ELFFile<ELFT> &EF, std::optional<unsigned> TextSectionIndex) {
+ return readBBAddrMapCommonImpl<PGOBBAddrMap>(
+ EF, TextSectionIndex, {ELF::SHT_LLVM_PGO_BB_ADDR_MAP},
+ [](const ELFFile<ELFT> &EF, const typename ELFFile<ELFT>::Elf_Shdr &Sec,
+ const typename ELFFile<ELFT>::Elf_Shdr *RelaSec) {
+ return EF.decodePGOBBAddrMap(Sec, RelaSec);
+ });
+}
+
template <class ELFT>
static Expected<std::vector<VersionEntry>>
readDynsymVersionsImpl(const ELFFile<ELFT> &EF,
@@ -832,3 +856,15 @@ Expected<std::vector<BBAddrMap>> ELFObjectFileBase::readBBAddrMap(
return readBBAddrMapImpl(cast<ELF64BEObjectFile>(this)->getELFFile(),
TextSectionIndex);
}
+
+Expected<std::vector<PGOBBAddrMap>> ELFObjectFileBase::readPGOBBAddrMap(
+ std::optional<unsigned> TextSectionIndex) const {
+ if (const auto *Obj = dyn_cast<ELF32LEObjectFile>(this))
+ return readPGOBBAddrMapImpl(Obj->getELFFile(), TextSectionIndex);
+ if (const auto *Obj = dyn_cast<ELF64LEObjectFile>(this))
+ return readPGOBBAddrMapImpl(Obj->getELFFile(), TextSectionIndex);
+ if (const auto *Obj = dyn_cast<ELF32BEObjectFile>(this))
+ return readPGOBBAddrMapImpl(Obj->getELFFile(), TextSectionIndex);
+ return readPGOBBAddrMapImpl(cast<ELF64BEObjectFile>(this)->getELFFile(),
+ TextSectionIndex);
+}
diff --git a/llvm/lib/ObjectYAML/ELFEmitter.cpp b/llvm/lib/ObjectYAML/ELFEmitter.cpp
index 40f81f867efa423..4265c4bfd226eac 100644
--- a/llvm/lib/ObjectYAML/ELFEmitter.cpp
+++ b/llvm/lib/ObjectYAML/ELFEmitter.cpp
@@ -175,8 +175,9 @@ struct Fragment {
/// TODO: This class still has a ways to go before it is truly a "single
/// point of truth".
template <class ELFT> class ELFState {
+public:
LLVM_ELF_IMPORT_TYPES_ELFT(ELFT)
-
+private:
enum class SymtabType { Static, Dynamic };
/// The future symbol table string section.
@@ -283,6 +284,9 @@ template <class ELFT> class ELFState {
void writeSectionContent(Elf_Shdr &SHeader,
const ELFYAML::BBAddrMapSection &Section,
ContiguousBlobAccumulator &CBA);
+ void writeSectionContent(Elf_Shdr &SHeader,
+ const ELFYAML::PGOBBAddrMapSection &Section,
+ ContiguousBlobAccumulator &CBA);
void writeSectionContent(Elf_Shdr &SHeader,
const ELFYAML::HashSection &Section,
ContiguousBlobAccumulator &CBA);
@@ -894,6 +898,8 @@ void ELFState<ELFT>::initSectionHeaders(std::vector<Elf_Shdr> &SHeaders,
writeSectionContent(SHeader, *S, CBA);
} else if (auto S = dyn_cast<ELFYAML::BBAddrMapSection>(Sec)) {
writeSectionContent(SHeader, *S, CBA);
+ } else if (auto S = dyn_cast<ELFYAML::PGOBBAddrMapSection>(Sec)) {
+ writeSectionContent(SHeader, *S, CBA);
} else {
llvm_unreachable("Unknown section type");
}
@@ -1386,44 +1392,124 @@ void ELFState<ELFT>::writeSectionContent(
}
}
-template <class ELFT>
-void ELFState<ELFT>::writeSectionContent(
- Elf_Shdr &SHeader, const ELFYAML::BBAddrMapSection &Section,
- ContiguousBlobAccumulator &CBA) {
- if (!Section.Entries)
- return;
-
- for (const ELFYAML::BBAddrMapEntry &E : *Section.Entries) {
+namespace {
+template <typename ELFT, typename AddrMap> struct BBAddrMapWriteTrait;
+
+template <typename ELFT>
+struct BBAddrMapWriteTrait<ELFT, ELFYAML::BBAddrMapSection> {
+ typename ELFState<ELFT>::Elf_Shdr &SHeader;
+ const ELFYAML::Section &Section;
+ ContiguousBlobAccumulator &CBA;
+ const unsigned UpToDateSectionID;
+
+ BBAddrMapWriteTrait(typename ELFState<ELFT>::Elf_Shdr &SHeader,
+ const ELFYAML::Section &Section,
+ ContiguousBlobAccumulator &CBA,
+ unsigned UpToDateSectionID = ELF::SHT_LLVM_BB_ADDR_MAP)
+ : SHeader(SHeader), Section(Section), CBA(CBA),
+ UpToDateSectionID(UpToDateSectionID) {}
+
+ template <typename EntryTy> void writeEntry(const EntryTy &E) {
+ using uintX_t = typename ELFState<ELFT>::uintX_t;
// Write version and feature values.
- if (Section.Type == llvm::ELF::SHT_LLVM_BB_ADDR_MAP) {
- if (E.Version > 2)
+ if (Section.Type == UpToDateSectionID) {
+ if (E.Common.Version > 2)
WithColor::warning() << "unsupported SHT_LLVM_BB_ADDR_MAP version: "
- << static_cast<int>(E.Version)
+ << static_cast<int>(E.Common.Version)
<< "; encoding using the most recent version";
- CBA.write(E.Version);
- CBA.write(E.Feature);
+ CBA.write(E.Common.Version);
+ CBA.write(E.Common.Feature);
SHeader.sh_size += 2;
}
// Write the address of the function.
- CBA.write<uintX_t>(E.Address, ELFT::TargetEndianness);
+ CBA.write<uintX_t>(E.Common.Address, ELFT::TargetEndianness);
// Write number of BBEntries (number of basic blocks in the function). This
// is overridden by the 'NumBlocks' YAML field when specified.
uint64_t NumBlocks =
- E.NumBlocks.value_or(E.BBEntries ? E.BBEntries->size() : 0);
+ E.Common.NumBlocks.value_or(E.BBEntries ? E.BBEntries->size() : 0);
SHeader.sh_size += sizeof(uintX_t) + CBA.writeULEB128(NumBlocks);
+ }
+
+ void writeBBEntry(const ELFYAML::BBAddrMapEntry &E,
+ const ELFYAML::BBAddrMapEntry::BBEntry &BBE) {
+ if (Section.Type == UpToDateSectionID && E.Common.Version > 1)
+ SHeader.sh_size += CBA.writeULEB128(BBE.ID);
+ SHeader.sh_size += CBA.writeULEB128(BBE.AddressOffset) +
+ CBA.writeULEB128(BBE.Size) +
+ CBA.writeULEB128(BBE.Metadata);
+ }
+};
+
+template <typename ELFT>
+struct BBAddrMapWriteTrait<ELFT, ELFYAML::PGOBBAddrMapSection> {
+ using Features = object::PGOBBAddrMap::Features;
+
+ BBAddrMapWriteTrait<ELFT, ELFYAML::BBAddrMapSection> Base;
+
+ BBAddrMapWriteTrait(typename ELFState<ELFT>::Elf_Shdr &SHeader,
+ const ELFYAML::PGOBBAddrMapSection &Section,
+ ContiguousBlobAccumulator &CBA)
+ : Base(SHeader, Section, CBA, ELF::SHT_LLVM_PGO_BB_ADDR_MAP) {}
+
+ void writeEntry(const ELFYAML::PGOBBAddrMapEntry &E) {
+ if (E.Common.Version < 2)
+ WithColor::warning() << "unsupported SHT_LLVM_PGO_BB_ADDR_MAP version: "
+ << static_cast<int>(E.Common.Version)
+ << "; must use version >= 2";
+ Base.writeEntry(E);
+ if (E.FuncEntryCount)
+ Base.SHeader.sh_size += Base.CBA.writeULEB128(*E.FuncEntryCount);
+ }
+
+ void writeBBEntry(const ELFYAML::PGOBBAddrMapEntry &E,
+ const ELFYAML::PGOBBAddrMapEntry::BBEntry &BBE) {
+ Base.writeBBEntry(ELFYAML::BBAddrMapEntry{E.Common, {}}, BBE.Base);
+ if (BBE.BBFreq)
+ Base.SHeader.sh_size += Base.CBA.writeULEB128(*BBE.BBFreq);
+ if (BBE.Successors) {
+ if (BBE.Successors->size() > 2)
+ Base.SHeader.sh_size += Base.CBA.writeULEB128(BBE.Successors->size());
+ for (const auto &Succ : *BBE.Successors)
+ Base.SHeader.sh_size +=
+ Base.CBA.writeULEB128(Succ.ID) + Base.CBA.writeULEB128(Succ.BrProb);
+ }
+ }
+};
+} // namespace
+
+template <typename ELFT, typename SectionTy>
+static void writeAddrMapSectionContent(
+ ELFState<ELFT> &ES, typename ELFState<ELFT>::Elf_Shdr &SHeader,
+ const SectionTy &Section, ContiguousBlobAccumulator &CBA) {
+ if (!Section.Entries)
+ return;
+
+ BBAddrMapWriteTrait<ELFT, SectionTy> AddrTrait(SHeader, Section, CBA);
+ for (const auto &E : *Section.Entries) {
+ AddrTrait.writeEntry(E);
// Write all BBEntries.
if (!E.BBEntries)
continue;
- for (const ELFYAML::BBAddrMapEntry::BBEntry &BBE : *E.BBEntries) {
- if (Section.Type == llvm::ELF::SHT_LLVM_BB_ADDR_MAP && E.Version > 1)
- SHeader.sh_size += CBA.writeULEB128(BBE.ID);
- SHeader.sh_size += CBA.writeULEB128(BBE.AddressOffset) +
- CBA.writeULEB128(BBE.Size) +
- CBA.writeULEB128(BBE.Metadata);
+ for (const auto &BBE : *E.BBEntries) {
+ AddrTrait.writeBBEntry(E, BBE);
}
}
}
+template <class ELFT>
+void ELFState<ELFT>::writeSectionContent(
+ Elf_Shdr &SHeader, const ELFYAML::BBAddrMapSection &Section,
+ ContiguousBlobAccumulator &CBA) {
+ writeAddrMapSectionContent(*this, SHeader, Section, CBA);
+}
+
+template <class ELFT>
+void ELFState<ELFT>::writeSectionContent(
+ Elf_Shdr &SHeader, const ELFYAML::PGOBBAddrMapSection &Section,
+ ContiguousBlobAccumulator &CBA) {
+ writeAddrMapSectionContent(*this, SHeader, Section, CBA);
+}
+
template <class ELFT>
void ELFState<ELFT>::writeSectionContent(
Elf_Shdr &SHeader, const ELFYAML::LinkerOptionsSection &Section,
diff --git a/llvm/lib/ObjectYAML/ELFYAML.cpp b/llvm/lib/ObjectYAML/ELFYAML.cpp
index 872b89420a9e7a7..8aa79b6724e0a07 100644
--- a/llvm/lib/ObjectYAML/ELFYAML.cpp
+++ b/llvm/lib/ObjectYAML/ELFYAML.cpp
@@ -683,6 +683,7 @@ void ScalarEnumerationTraits<ELFYAML::ELF_SHT>::enumeration(
ECase(SHT_LLVM_PART_PHDR);
ECase(SHT_LLVM_BB_ADDR_MAP_V0);
ECase(SHT_LLVM_BB_ADDR_MAP);
+ ECase(SHT_LLVM_PGO_BB_ADDR_MAP);
ECase(SHT_LLVM_OFFLOADING);
ECase(SHT_LLVM_LTO);
ECase(SHT_GNU_ATTRIBUTES);
@@ -1389,6 +1390,12 @@ static void sectionMapping(IO &IO, ELFYAML::BBAddrMapSection &Section) {
IO.mapOptional("Entries", Section.Entries);
}
+static void sectionMapping(IO &IO, ELFYAML::PGOBBAddrMapSection &Section) {
+ commonSectionMapping(IO, Section);
+ IO.mapOptional("Content", Section.Content);
+ IO.mapOptional("Entries", Section.Entries);
+}
+
static void sectionMapping(IO &IO, ELFYAML::StackSizesSection &Section) {
commonSectionMapping(IO, Section);
IO.mapOptional("Entries", Section.Entries);
@@ -1682,6 +1689,11 @@ void MappingTraits<std::unique_ptr<ELFYAML::Chunk>>::mapping(
Section.reset(new ELFYAML::BBAddrMapSection());
sectionMapping(IO, *cast<ELFYAML::BBAddrMapSection>(Section.get()));
break;
+ case ELF::SHT_LLVM_PGO_BB_ADDR_MAP:
+ if (!IO.outputting())
+ Section.reset(new ELFYAML::PGOBBAddrMapSection());
+ sectionMapping(IO, *cast<ELFYAML::PGOBBAddrMapSection>(Section.get()));
+ break;
default:
if (!IO.outputting()) {
StringRef Name;
@@ -1803,13 +1815,17 @@ void MappingTraits<ELFYAML::StackSizeEntry>::mapping(
IO.mapRequired("Size", E.Size);
}
-void MappingTraits<ELFYAML::BBAddrMapEntry>::mapping(
- IO &IO, ELFYAML::BBAddrMapEntry &E) {
- assert(IO.getContext() && "The IO context is not initialized");
+static void mapBBAddrMapCommonBase(IO &IO, ELFYAML::BBAddrMapCommonBase &E) {
IO.mapRequired("Version", E.Version);
IO.mapOptional("Feature", E.Feature, Hex8(0));
IO.mapOptional("Address", E.Address, Hex64(0));
IO.mapOptional("NumBlocks", E.NumBlocks);
+}
+
+void MappingTraits<ELFYAML::BBAddrMapEntry>::mapping(
+ IO &IO, ELFYAML::BBAddrMapEntry &E) {
+ assert(IO.getContext() && "The IO context is not initialized");
+ mapBBAddrMapCommonBase(IO, E.Common);
IO.mapOptional("BBEntries", E.BBEntries);
}
@@ -1822,6 +1838,29 @@ void MappingTraits<ELFYAML::BBAddrMapEntry::BBEntry>::mapping(
IO.mapRequired("Metadata", E.Metadata);
}
+void MappingTraits<ELFYAML::PGOBBAddrMapEntry>::mapping(
+ IO &IO, ELFYAML::PGOBBAddrMapEntry &E) {
+ assert(IO.getContext() && "The IO context is not initialized");
+ mapBBAddrMapCommonBase(IO, E.Common);
+ IO.mapOptional("FuncEntryCount", E.FuncEntryCount);
+ IO.mapOptional("BBEntries", E.BBEntries);
+}
+
+void MappingTraits<ELFYAML::PGOBBAddrMapEntry::BBEntry>::mapping(
+ IO &IO, ELFYAML::PGOBBAddrMapEntry::BBEntry &E) {
+ assert(IO.getContext() && "The IO context is not initialized");
+ MappingTraits<ELFYAML::BBAddrMapEntry::BBEntry>::mapping(IO, E.Base);
+ IO.mapOptional("BBFreq", E.BBFreq);
+ IO.mapOptional("Successors", E.Successors);
+}
+
+void MappingTraits<ELFYAML::PGOBBAddrMapEntry::BBEntry::SuccessorEntry>::
+ mapping(IO &IO, ELFYAML::PGOBBAddrMapEntry::BBEntry::SuccessorEntry &E) {
+ assert(IO.getContext() && "The IO context is not initialized");
+ IO.mapRequired("ID", E.ID);
+ IO.mapRequired("BrProb", E.BrProb);
+}
+
void MappingTraits<ELFYAML::GnuHashHeader>::mapping(IO &IO,
ELFYAML::GnuHashHeader &E) {
assert(IO.getContext() && "The IO context is not initialized");
diff --git a/llvm/test/MC/AsmParser/llvm_section_types.s b/llvm/test/MC/AsmParser/llvm_section_types.s
index 147b1499d2b8883..2d7b650e10432ae 100644
--- a/llvm/test/MC/AsmParser/llvm_section_types.s
+++ b/llvm/test/MC/AsmParser/llvm_section_types.s
@@ -17,6 +17,8 @@
.byte 1
.section .section8,"", at llvm_lto
.byte 1
+.section .section9,"", at llvm_pgo_bb_addr_map
+.byte 1
# CHECK: Name: .section1
# CHECK-NEXT: Type: SHT_LLVM_BB_ADDR_MAP
@@ -34,3 +36,5 @@
# CHECK-NEXT: Type: SHT_LLVM_OFFLOADING
# CHECK: Name: .section8
# CHECK-NEXT: Type: SHT_LLVM_LTO
+# CHECK: Name: .section9
+# CHECK-NEXT: Type: SHT_LLVM_PGO_BB_ADDR_MAP
diff --git a/llvm/tools/obj2yaml/elf2yaml.cpp b/llvm/tools/obj2yaml/elf2yaml.cpp
index 7c4a8853ee2d301..09ad0603c8e3787 100644
--- a/llvm/tools/obj2yaml/elf2yaml.cpp
+++ b/llvm/tools/obj2yaml/elf2yaml.cpp
@@ -916,7 +916,7 @@ ELFDumper<ELFT>::dumpBBAddrMapSection(const Elf_Shdr *Shdr) {
BBEntries.push_back({ID, Offset, Size, Metadata});
}
Entries.push_back(
- {Version, Feature, Address, /*NumBlocks=*/{}, std::move(BBEntries)});
+ {{Version, Feature, Address, /*NumBlocks=*/{}}, std::move(BBEntries)});
}
if (!Cur) {
diff --git a/llvm/unittests/Object/ELFObjectFileTest.cpp b/llvm/unittests/Object/ELFObjectFileTest.cpp
index fe5ce2154dc7444..5725d9c158f3ec3 100644
--- a/llvm/unittests/Object/ELFObjectFileTest.cpp
+++ b/llvm/unittests/Object/ELFObjectFileTest.cpp
@@ -744,6 +744,446 @@ TEST(ELFObjectFileTest, ReadBBAddrMap) {
Section1BBAddrMaps);
}
+// Tests for error paths of the ELFFile::decodePGOBBAddrMap API.
+TEST(ELFObjectFileTest, InvalidDecodePGOBBAddrMap) {
+ if (IsHostWindows())
+ GTEST_SKIP();
+ StringRef CommonYamlString(R"(
+--- !ELF
+FileHeader:
+ Class: ELFCLASS64
+ Data: ELFDATA2LSB
+ Type: ET_EXEC
+Sections:
+ - Type: SHT_LLVM_PGO_BB_ADDR_MAP
+ Name: .llvm_pgo_bb_addr_map
+ Entries:
+ - Address: 0x11111
+)");
+
+ auto DoCheck = [&](StringRef YamlString, const char *ErrMsg) {
+ SmallString<0> Storage;
+ Expected<ELFObjectFile<ELF64LE>> ElfOrErr =
+ toBinary<ELF64LE>(Storage, YamlString);
+ ASSERT_THAT_EXPECTED(ElfOrErr, Succeeded());
+ const ELFFile<ELF64LE> &Elf = ElfOrErr->getELFFile();
+
+ Expected<const typename ELF64LE::Shdr *> PGOBBAddrMapSecOrErr =
+ Elf.getSection(1);
+ ASSERT_THAT_EXPECTED(PGOBBAddrMapSecOrErr, Succeeded());
+ EXPECT_THAT_ERROR(
+ Elf.decodePGOBBAddrMap(**PGOBBAddrMapSecOrErr).takeError(),
+ FailedWithMessage(ErrMsg));
+ };
+
+ // Check that we can detect unsupported versions.
+ SmallString<128> UnsupportedVersionYamlString(CommonYamlString);
+ UnsupportedVersionYamlString += R"(
+ Version: 3
+ BBEntries:
+ - AddressOffset: 0x0
+ Size: 0x1
+ Metadata: 0x2
+)";
+
+ DoCheck(UnsupportedVersionYamlString,
+ "unsupported SHT_LLVM_BB_ADDR_MAP version: 3");
+
+ // Check that we can detect unsupported versions that is too low
+ SmallString<128> UnsupportedLowVersionYamlString(CommonYamlString);
+ UnsupportedLowVersionYamlString += R"(
+ Version: 1
+ BBEntries:
+ - AddressOffset: 0x0
+ Size: 0x1
+ Metadata: 0x2
+)";
+
+ DoCheck(UnsupportedLowVersionYamlString,
+ "unsupported SHT_LLVM_PGO_BB_ADDR_MAP version: 1");
+
+ SmallString<128> CommonVersionedYamlString(CommonYamlString);
+ CommonVersionedYamlString += R"(
+ Version: 2
+ BBEntries:
+ - ID: 1
+ AddressOffset: 0x0
+ Size: 0x1
+ Metadata: 0x2
+)";
+
+ // Check that we can detect the malformed encoding when the section is
+ // truncated.
+ SmallString<128> TruncatedYamlString(CommonVersionedYamlString);
+ TruncatedYamlString += R"(
+ ShSize: 0xb
+)";
+ DoCheck(TruncatedYamlString, "unable to decode LEB128 at offset 0x0000000b: "
+ "malformed uleb128, extends past end");
+
+ // Check that we can detect when the encoded BB entry fields exceed the UINT32
+ // limit.
+ SmallVector<SmallString<128>, 3> OverInt32LimitYamlStrings(
+ 3, CommonVersionedYamlString);
+ OverInt32LimitYamlStrings[0] += R"(
+ - ID: 1
+ AddressOffset: 0x100000000
+ Size: 0xFFFFFFFF
+ Metadata: 0xFFFFFFFF
+)";
+
+ OverInt32LimitYamlStrings[1] += R"(
+ - ID: 2
+ AddressOffset: 0xFFFFFFFF
+ Size: 0x100000000
+ Metadata: 0xFFFFFFFF
+)";
+
+ OverInt32LimitYamlStrings[2] += R"(
+ - ID: 3
+ AddressOffset: 0xFFFFFFFF
+ Size: 0xFFFFFFFF
+ Metadata: 0x100000000
+)";
+
+ DoCheck(OverInt32LimitYamlStrings[0],
+ "ULEB128 value at offset 0x10 exceeds UINT32_MAX (0x100000000)");
+ DoCheck(OverInt32LimitYamlStrings[1],
+ "ULEB128 value at offset 0x15 exceeds UINT32_MAX (0x100000000)");
+ DoCheck(OverInt32LimitYamlStrings[2],
+ "ULEB128 value at offset 0x1a exceeds UINT32_MAX (0x100000000)");
+
+ // Check the proper error handling when the section has fields exceeding
+ // UINT32 and is also truncated. This is for checking that we don't generate
+ // unhandled errors.
+ SmallVector<SmallString<128>, 3> OverInt32LimitAndTruncated(
+ 3, OverInt32LimitYamlStrings[1]);
+ // Truncate before the end of the 5-byte field.
+ OverInt32LimitAndTruncated[0] += R"(
+ ShSize: 0x19
+)";
+ // Truncate at the end of the 5-byte field.
+ OverInt32LimitAndTruncated[1] += R"(
+ ShSize: 0x1a
+)";
+ // Truncate after the end of the 5-byte field.
+ OverInt32LimitAndTruncated[2] += R"(
+ ShSize: 0x1b
+)";
+
+ DoCheck(OverInt32LimitAndTruncated[0],
+ "unable to decode LEB128 at offset 0x00000015: malformed uleb128, "
+ "extends past end");
+ DoCheck(OverInt32LimitAndTruncated[1],
+ "ULEB128 value at offset 0x15 exceeds UINT32_MAX (0x100000000)");
+ DoCheck(OverInt32LimitAndTruncated[2],
+ "ULEB128 value at offset 0x15 exceeds UINT32_MAX (0x100000000)");
+
+ // Check for proper error handling when the 'NumBlocks' field is overridden
+ // with an out-of-range value.
+ SmallString<128> OverLimitNumBlocks(CommonVersionedYamlString);
+ OverLimitNumBlocks += R"(
+ NumBlocks: 0x100000000
+)";
+
+ DoCheck(OverLimitNumBlocks,
+ "ULEB128 value at offset 0xa exceeds UINT32_MAX (0x100000000)");
+
+ // Check that we fail when function entry count is enabled but not provided.
+ SmallString<128> MissingFuncEntryCount(CommonYamlString);
+ MissingFuncEntryCount += R"(
+ Version: 2
+ Feature: 0x01
+)";
+
+ DoCheck(MissingFuncEntryCount,
+ "unable to decode LEB128 at offset 0x0000000b: malformed uleb128, "
+ "extends past end");
+
+ // Check that we fail when basic block frequency is enabled but not provided.
+ SmallString<128> MissingBBFreq(CommonYamlString);
+ MissingBBFreq += R"(
+ Version: 2
+ Feature: 0x02
+ BBEntries:
+ - ID: 1
+ AddressOffset: 0x0
+ Size: 0x1
+ Metadata: 0x2
+)";
+
+ DoCheck(MissingBBFreq, "unable to decode LEB128 at offset 0x0000000f: "
+ "malformed uleb128, extends past end");
+
+ // Check that we fail when branch probability is enabled but not provided.
+ SmallString<128> MissingBrProb(CommonYamlString);
+ MissingBrProb += R"(
+ Version: 2
+ Feature: 0x02
+ BBEntries:
+ - ID: 1
+ AddressOffset: 0x0
+ Size: 0x1
+ Metadata: 0x60
+ - ID: 2
+ AddressOffset: 0x1
+ Size: 0x1
+ Metadata: 0x2
+ - ID: 3
+ AddressOffset: 0x2
+ Size: 0x1
+ Metadata: 0x2
+ - ID: 4
+ AddressOffset: 0x3
+ Size: 0x1
+ Metadata: 0x2
+)";
+
+ DoCheck(MissingBrProb, "unable to decode LEB128 at offset 0x0000001b: "
+ "malformed uleb128, extends past end");
+}
+
+// Test for the ELFObjectFile::readPGOBBAddrMap API.
+TEST(ELFObjectFileTest, ReadPGOBBAddrMap) {
+ if (IsHostWindows())
+ GTEST_SKIP();
+ StringRef CommonYamlString(R"(
+--- !ELF
+FileHeader:
+ Class: ELFCLASS64
+ Data: ELFDATA2LSB
+ Type: ET_EXEC
+Sections:
+ - Name: .llvm_pgo_bb_addr_map_1
+ Type: SHT_LLVM_PGO_BB_ADDR_MAP
+ Link: 1
+ Entries:
+ - Version: 2
+ Address: 0x11111
+ Feature: 0x1
+ FuncEntryCount: 892
+ BBEntries:
+ - ID: 1
+ AddressOffset: 0x0
+ Size: 0x1
+ Metadata: 0x2
+ - Name: .llvm_pgo_bb_addr_map_2
+ Type: SHT_LLVM_PGO_BB_ADDR_MAP
+ Link: 1
+ Entries:
+ - Version: 2
+ Address: 0x22222
+ Feature: 0x2
+ BBEntries:
+ - ID: 2
+ AddressOffset: 0x0
+ Size: 0x2
+ Metadata: 0x4
+ BBFreq: 343
+ - Name: .llvm_bb_addr_map_3
+ Type: SHT_LLVM_PGO_BB_ADDR_MAP
+ Link: 2
+ Entries:
+ - Version: 2
+ Address: 0x33333
+ Feature: 0x4
+ BBEntries:
+ - ID: 0
+ AddressOffset: 0x0
+ Size: 0x3
+ Metadata: 0x46
+ Successors:
+ - ID: 1
+ BrProb: 0x11111111
+ - ID: 2
+ BrProb: 0xeeeeeeee
+ - ID: 1
+ AddressOffset: 0x0
+ Size: 0x3
+ Metadata: 0x24
+ Successors:
+ - ID: 2
+ BrProb: 0xffffffff
+ - ID: 2
+ AddressOffset: 0x0
+ Size: 0x3
+ Metadata: 0x00
+ Successors: []
+ - Name: .llvm_pgo_bb_addr_map_4
+ Type: SHT_LLVM_PGO_BB_ADDR_MAP
+ # Link: 0 (by default, can be overriden)
+ Entries:
+ - Version: 2
+ Address: 0x44444
+ Feature: 0x7
+ FuncEntryCount: 1000
+ BBEntries:
+ - ID: 0
+ AddressOffset: 0x0
+ Size: 0x4
+ Metadata: 0x78
+ BBFreq: 1000
+ Successors:
+ - ID: 1
+ BrProb: 0x22222222
+ - ID: 2
+ BrProb: 0x33333333
+ - ID: 3
+ BrProb: 0xaaaaaaaa
+ - ID: 1
+ AddressOffset: 0x0
+ Size: 0x4
+ Metadata: 0x40
+ BBFreq: 133
+ Successors:
+ - ID: 2
+ BrProb: 0x11111111
+ - ID: 3
+ BrProb: 0xeeeeeeee
+ - ID: 2
+ AddressOffset: 0x0
+ Size: 0x4
+ Metadata: 0x20
+ BBFreq: 18
+ Successors:
+ - ID: 3
+ BrProb: 0xffffffff
+ - ID: 3
+ AddressOffset: 0x0
+ Size: 0x4
+ Metadata: 0x00
+ BBFreq: 1000
+ Successors: []
+)");
+
+ PGOBBAddrMap E1 = {
+ 0x11111, {{{1, 0x0, 0x1, {false, true, false, false, false}}, {}, {}}},
+ 892, true,
+ false, false};
+ PGOBBAddrMap E2 = {0x22222,
+ {{{2, 0x0, 0x2, {false, false, true, false, false}},
+ BlockFrequency(343),
+ {}}},
+ 0,
+ false,
+ true,
+ false};
+ PGOBBAddrMap E3 = {
+ 0x33333,
+ {{{0, 0x0, 0x3, {false, true, true, false, false}},
+ {},
+ {{1, BranchProbability::getRaw(0x1111'1111)},
+ {2, BranchProbability::getRaw(0xeeee'eeee)}}},
+ {{1, 0x3, 0x3, {false, false, true, false, false}},
+ {},
+ {{2, BranchProbability::getRaw(0xffff'ffff)}}},
+ {{2, 0x6, 0x3, {false, false, false, false, false}}, {}, {}}},
+ 0,
+ false,
+ false,
+ true};
+ PGOBBAddrMap E4 = {0x44444,
+ {{{0, 0x0, 0x4, {false, false, false, true, true}},
+ BlockFrequency(1000),
+ {{1, BranchProbability::getRaw(0x2222'2222)},
+ {2, BranchProbability::getRaw(0x3333'3333)},
+ {3, BranchProbability::getRaw(0xaaaa'aaaa)}}},
+ {{1, 0x4, 0x4, {false, false, false, false, false}},
+ BlockFrequency(133),
+ {{2, BranchProbability::getRaw(0x1111'1111)},
+ {3, BranchProbability::getRaw(0xeeee'eeee)}}},
+ {{2, 0x8, 0x4, {false, false, false, false, false}},
+ BlockFrequency(18),
+ {{3, BranchProbability::getRaw(0xffff'ffff)}}},
+ {{3, 0xc, 0x4, {false, false, false, false, false}},
+ BlockFrequency(1000),
+ {}}},
+ 1000,
+ true,
+ true,
+ true};
+
+ std::vector<PGOBBAddrMap> Section0PGOBBAddrMaps = {E4};
+ std::vector<PGOBBAddrMap> Section1PGOBBAddrMaps = {E3};
+ std::vector<PGOBBAddrMap> Section2PGOBBAddrMaps = {E1, E2};
+ std::vector<PGOBBAddrMap> AllPGOBBAddrMaps = {E1, E2, E3, E4};
+
+ auto DoCheckSucceeds = [&](StringRef YamlString,
+ std::optional<unsigned> TextSectionIndex,
+ std::vector<PGOBBAddrMap> ExpectedResult) {
+ SmallString<0> Storage;
+ Expected<ELFObjectFile<ELF64LE>> ElfOrErr =
+ toBinary<ELF64LE>(Storage, YamlString);
+ ASSERT_THAT_EXPECTED(ElfOrErr, Succeeded());
+
+ Expected<const typename ELF64LE::Shdr *> PGOBBAddrMapSecOrErr =
+ ElfOrErr->getELFFile().getSection(1);
+ ASSERT_THAT_EXPECTED(PGOBBAddrMapSecOrErr, Succeeded());
+ auto PGOBBAddrMaps = ElfOrErr->readPGOBBAddrMap(TextSectionIndex);
+ EXPECT_THAT_EXPECTED(PGOBBAddrMaps, Succeeded());
+ EXPECT_EQ(*PGOBBAddrMaps, ExpectedResult);
+ };
+
+ auto DoCheckFails = [&](StringRef YamlString,
+ std::optional<unsigned> TextSectionIndex,
+ const char *ErrMsg) {
+ SmallString<0> Storage;
+ Expected<ELFObjectFile<ELF64LE>> ElfOrErr =
+ toBinary<ELF64LE>(Storage, YamlString);
+ ASSERT_THAT_EXPECTED(ElfOrErr, Succeeded());
+
+ Expected<const typename ELF64LE::Shdr *> BBAddrMapSecOrErr =
+ ElfOrErr->getELFFile().getSection(1);
+ ASSERT_THAT_EXPECTED(BBAddrMapSecOrErr, Succeeded());
+ EXPECT_THAT_ERROR(ElfOrErr->readPGOBBAddrMap(TextSectionIndex).takeError(),
+ FailedWithMessage(ErrMsg));
+ };
+
+ // Check that we can retrieve the data in the normal case.
+ DoCheckSucceeds(CommonYamlString, /*TextSectionIndex=*/std::nullopt,
+ AllPGOBBAddrMaps);
+ DoCheckSucceeds(CommonYamlString, /*TextSectionIndex=*/0,
+ Section0PGOBBAddrMaps);
+ DoCheckSucceeds(CommonYamlString, /*TextSectionIndex=*/2,
+ Section1PGOBBAddrMaps);
+ DoCheckSucceeds(CommonYamlString, /*TextSectionIndex=*/1,
+ Section2PGOBBAddrMaps);
+ // Check that when no bb-address-map section is found for a text section,
+ // we return an empty result.
+ DoCheckSucceeds(CommonYamlString, /*TextSectionIndex=*/3, {});
+
+ // Check that we detect when a bb-addr-map section is linked to an invalid
+ // (not present) section.
+ SmallString<128> InvalidLinkedYamlString(CommonYamlString);
+ InvalidLinkedYamlString += R"(
+ Link: 10
+)";
+
+ DoCheckFails(InvalidLinkedYamlString, /*TextSectionIndex=*/4,
+ "unable to get the linked-to section for "
+ "SHT_LLVM_PGO_BB_ADDR_MAP section with index 4: invalid section "
+ "index: 10");
+ // Linked sections are not checked when we don't target a specific text
+ // section.
+ DoCheckSucceeds(InvalidLinkedYamlString, /*TextSectionIndex=*/std::nullopt,
+ AllPGOBBAddrMaps);
+
+ // Check that we can detect when bb-address-map decoding fails.
+ SmallString<128> TruncatedYamlString(CommonYamlString);
+ TruncatedYamlString += R"(
+ ShSize: 0xa
+)";
+
+ DoCheckFails(TruncatedYamlString, /*TextSectionIndex=*/std::nullopt,
+ "unable to read SHT_LLVM_PGO_BB_ADDR_MAP section with index 4: "
+ "unable to decode LEB128 at offset 0x0000000a: malformed "
+ "uleb128, extends past end");
+ // Check that we can read the other section's bb-address-maps which are
+ // valid.
+ DoCheckSucceeds(TruncatedYamlString, /*TextSectionIndex=*/2,
+ Section1PGOBBAddrMaps);
+}
+
// Test for ObjectFile::getRelocatedSection: check that it returns a relocated
// section for executable and relocatable files.
TEST(ELFObjectFileTest, ExecutableWithRelocs) {
diff --git a/llvm/unittests/Object/ELFTypesTest.cpp b/llvm/unittests/Object/ELFTypesTest.cpp
index 2c9e8b7aebac293..8689f77af805073 100644
--- a/llvm/unittests/Object/ELFTypesTest.cpp
+++ b/llvm/unittests/Object/ELFTypesTest.cpp
@@ -73,8 +73,12 @@ TEST(ELFTypesTest, BBEntryMetadataEncodingTest) {
{false, false, false, false, true},
{true, true, true, true, true}}};
const std::array<uint32_t, 7> Encoded = {{0, 1, 2, 4, 8, 16, 31}};
- for (size_t i = 0; i < Decoded.size(); ++i)
+ for (size_t i = 0; i < Decoded.size(); ++i) {
EXPECT_EQ(Decoded[i].encode(), Encoded[i]);
+ EXPECT_LT(Decoded[i].encode(),
+ uint32_t{1} << BBAddrMap::BBEntry::Metadata::NumberOfBits)
+ << "If a new bit was added, please update NumberOfBits.";
+ }
for (size_t i = 0; i < Encoded.size(); ++i) {
Expected<BBAddrMap::BBEntry::Metadata> MetadataOrError =
BBAddrMap::BBEntry::Metadata::decode(Encoded[i]);
@@ -94,3 +98,50 @@ TEST(ELFTypesTest, BBEntryMetadataInvalidEncodingTest) {
FailedWithMessage(Errors[i]));
}
}
+
+static_assert(
+ std::is_same_v<decltype(PGOBBAddrMap::BBEntry::SuccessorEntry::ID),
+ decltype(BBAddrMap::BBEntry::ID)>,
+ "PGOBBAddrMap should use the same type for basic block ID as BBAddrMap");
+static_assert(BBAddrMap::BBEntry::Metadata::NumberOfBits <
+ (sizeof(uint32_t) * 8) - 2,
+ "currently PGOBBAddrMap relies on having two bits of space to "
+ "encode number of successors, to add more we need increase the "
+ "encoded size of Metadata");
+
+TEST(ELFTypesTest, PGOBBEntryMetadataEncodingTest) {
+ using ST = PGOBBAddrMap::BBEntry::SuccessorsType;
+ const std::array<std::pair<BBAddrMap::BBEntry::Metadata, ST>, 7> Decoded = {
+ {{{false, false, false, false, false}, ST::None},
+ {{true, false, false, false, false}, ST::One},
+ {{false, true, false, false, false}, ST::Two},
+ {{false, false, true, false, false}, ST::Multiple},
+ {{false, false, false, true, false}, ST::One},
+ {{false, false, false, false, true}, ST::Two},
+ {{true, true, true, true, true}, ST::Multiple}}};
+ const std::array<uint32_t, 7> Encoded = {{0b00'00000, 0b01'00001, 0b10'00010,
+ 0b11'00100, 0b01'01000, 0b10'10000,
+ 0b11'11111}};
+ for (auto [Enc, Dec] : llvm::zip(Encoded, Decoded)) {
+ auto [MD, SuccType] = Dec;
+ EXPECT_EQ(PGOBBAddrMap::BBEntry::encodeMD(MD, SuccType), Enc);
+ }
+ for (auto [Enc, Dec] : llvm::zip(Encoded, Decoded)) {
+ Expected<std::pair<BBAddrMap::BBEntry::Metadata, ST>> MetadataOrError =
+ PGOBBAddrMap::BBEntry::decodeMD(Enc);
+ ASSERT_THAT_EXPECTED(MetadataOrError, Succeeded());
+ EXPECT_EQ(*MetadataOrError, Dec);
+ }
+}
+
+TEST(ELFTypesTest, PGOBBEntryMetadataInvalidEncodingTest) {
+ const std::array<std::string, 3> Errors = {
+ "invalid encoding for BBEntry::Metadata: 0xff9f",
+ "invalid encoding for BBEntry::Metadata: 0x100001",
+ "invalid encoding for BBEntry::Metadata: 0x80"};
+ const std::array<uint32_t, 3> Values = {0xFFFF, 0x100001, 0x00c0};
+ for (auto [Val, Err] : llvm::zip(Values, Errors)) {
+ EXPECT_THAT_ERROR(PGOBBAddrMap::BBEntry::decodeMD(Val).takeError(),
+ FailedWithMessage(Err));
+ }
+}
More information about the llvm-commits
mailing list