[llvm] [SampleFDO][TypeProf]Support vtable type profiling for ext-binary and text format (PR #148002)
via llvm-commits
llvm-commits at lists.llvm.org
Thu Jul 10 15:46:02 PDT 2025
llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-pgo
Author: Mingming Liu (mingmingl-llvm)
<details>
<summary>Changes</summary>
This change extends SampleFDO ext-binary and text format to record the vtable symbols and their counts for virtual calls inside a function. The vtable profiles will allow the compiler to annotate vtable types on IR instructions and perform vtable-based indirect call promotion. An RFC is in https://discourse.llvm.org/t/rfc-vtable-type-profiling-for-samplefdo/87283
Given a function below, the before vs after of a function's profile is illustrated in text format in the table:
```
__attribute__((noinline)) int loop_func(int i, int a, int b) {
Base *ptr = createType(i);
int sum = ptr->func(a, b);
delete ptr;
return sum;
}
```
| before | after |
| --- | --- |
| Samples collected in the function's body { <br> 0: 636241 <br> 1: 681458, calls: _Z10createTypei:681458 <br> 3: 543499, calls: _ZN12_GLOBAL__N_18Derived24funcEii:410621 _ZN8Derived14funcEii:132878 <br> 5.1: 602201, calls: _ZN12_GLOBAL__N_18Derived2D0Ev:454635 _ZN8Derived1D0Ev:147566 <br> 7: 511057 <br> } | Samples collected in the function's body { <br> 0: 636241 <br> 1: 681458, calls: _Z10createTypei:681458 <br> 3: 543499, calls: _ZN12_GLOBAL__N_18Derived24funcEii:410621 _ZN8Derived14funcEii:132878 <br> 3: vtables: _ZTV8Derived1:1377 _ZTVN12_GLOBAL__N_18Derived2E:4250 <br> 5.1: 602201, calls: _ZN12_GLOBAL__N_18Derived2D0Ev:454635 _ZN8Derived1D0Ev:147566 <br> 5.1: vtables: _ZTV8Derived1:227 _ZTVN12_GLOBAL__N_18Derived2E:765 <br> 7: 511057 <br> } |
Key points for this change:
1. In-memory representation of vtable profiles
* A field of type `map<LineLocation, map<FunctionId, uint64_t>>` is introduced in a function's in-memory representation [FunctionSamples](https://github.com/llvm/llvm-project/blob/ccc416312ed72e92a885425d9cb9c01f9afa58eb/llvm/include/llvm/ProfileData/SampleProf.h#L749-L754).
2. The vtable counters for one LineLocation represents the relative frequency among vtables for this LineLocation. They are not required to be comparable across LineLocations.
3. For backward compatibility of ext-binary format, we take one bit from ProfSummaryFlag as illustrated in the enum class `SecProfSummaryFlags`. The ext-binary profile reader parses the integer type flag and reads this bit. If it's set, the profile reader will parse vtable profiles.
4. The vtable profiles are optional in ext-binary format, and not serialized out by default, we introduce an LLVM boolean option (named `-extbinary-write-vtable-type-prof`). The ext-binary profile writer reads the boolean option and decide whether to set the section flag bit and serialize the in-memory class members corresponding to vtables.
5. This change doesn't implement `llvm-profdata overlap --sample` for the vtable profiles. A subsequent change will do it to keep this one focused on the profile format change.
We don't plan to add the vtable support to non-extensible format mainly because of the maintenance cost to keep backward compatibility for prior versions of profile data.
* Currently, the [non-extensible binary format](https://github.com/llvm/llvm-project/blob/5c28af409978c19a35021855a29dcaa65e95da00/llvm/lib/ProfileData/SampleProfWriter.cpp#L899-L900) does not have feature parity with extensible binary format today, for instance, the former doesn't support [profile symbol list](https://github.com/llvm/llvm-project/blob/41e22aa31b1905aa3e9d83c0343a96ec0d5187ec/llvm/include/llvm/ProfileData/SampleProf.h#L1518-L1522) or context-sensitive PGO, both of which give measurable performance boost. Presumably the non-extensible format is not in wide use.
---
Patch is 30.51 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/148002.diff
11 Files Affected:
- (modified) llvm/include/llvm/ProfileData/SampleProf.h (+90-6)
- (modified) llvm/include/llvm/ProfileData/SampleProfReader.h (+12)
- (modified) llvm/include/llvm/ProfileData/SampleProfWriter.h (+10-4)
- (modified) llvm/lib/ProfileData/SampleProf.cpp (+43)
- (modified) llvm/lib/ProfileData/SampleProfReader.cpp (+95-3)
- (modified) llvm/lib/ProfileData/SampleProfWriter.cpp (+72-6)
- (added) llvm/test/tools/llvm-profdata/Inputs/profile-symbol-list-ext.expected (+44)
- (added) llvm/test/tools/llvm-profdata/Inputs/sample-profile-ext.proftext (+18)
- (modified) llvm/test/tools/llvm-profdata/profile-symbol-list-compress.test (+6)
- (modified) llvm/test/tools/llvm-profdata/profile-symbol-list.test (+6)
- (modified) llvm/test/tools/llvm-profdata/roundtrip.test (+6)
``````````diff
diff --git a/llvm/include/llvm/ProfileData/SampleProf.h b/llvm/include/llvm/ProfileData/SampleProf.h
index fb2d4d3cc50ed..d15b96db58fef 100644
--- a/llvm/include/llvm/ProfileData/SampleProf.h
+++ b/llvm/include/llvm/ProfileData/SampleProf.h
@@ -62,7 +62,8 @@ enum class sampleprof_error {
uncompress_failed,
zlib_unavailable,
hash_mismatch,
- illegal_line_offset
+ illegal_line_offset,
+ duplicate_vtable_type
};
inline std::error_code make_error_code(sampleprof_error E) {
@@ -91,6 +92,8 @@ struct is_error_code_enum<llvm::sampleprof_error> : std::true_type {};
namespace llvm {
namespace sampleprof {
+constexpr char kVTableProfPrefix[] = "vtables ";
+
enum SampleProfileFormat {
SPF_None = 0,
SPF_Text = 0x1,
@@ -204,6 +207,9 @@ enum class SecProfSummaryFlags : uint32_t {
/// SecFlagIsPreInlined means this profile contains ShouldBeInlined
/// contexts thus this is CS preinliner computed.
SecFlagIsPreInlined = (1 << 4),
+
+ /// SecFlagHasVTableTypeProf means this profile contains vtable type profiles.
+ SecFlagHasVTableTypeProf = (1 << 5),
};
enum class SecFuncMetadataFlags : uint32_t {
@@ -303,7 +309,7 @@ struct LineLocation {
}
uint64_t getHashCode() const {
- return ((uint64_t) Discriminator << 32) | LineOffset;
+ return ((uint64_t)Discriminator << 32) | LineOffset;
}
uint32_t LineOffset;
@@ -318,16 +324,28 @@ struct LineLocationHash {
LLVM_ABI raw_ostream &operator<<(raw_ostream &OS, const LineLocation &Loc);
+/// Key represents the id of a vtable and value represents its count.
+/// TODO: Rename class FunctionId to SymbolId in a separate PR.
+using TypeCountMap = std::map<FunctionId, uint64_t>;
+
+/// Write \p Map to the output stream. Keys are linearized using \p NameTable
+/// and written as ULEB128. Values are written as ULEB128 as well.
+std::error_code
+serializeTypeMap(const TypeCountMap &Map,
+ const MapVector<FunctionId, uint32_t> &NameTable,
+ raw_ostream &OS);
+
/// Representation of a single sample record.
///
/// A sample record is represented by a positive integer value, which
/// indicates how frequently was the associated line location executed.
///
/// Additionally, if the associated location contains a function call,
-/// the record will hold a list of all the possible called targets. For
-/// direct calls, this will be the exact function being invoked. For
-/// indirect calls (function pointers, virtual table dispatch), this
-/// will be a list of one or more functions.
+/// the record will hold a list of all the possible called targets and the types
+/// for virtual table dispatches. For direct calls, this will be the exact
+/// function being invoked. For indirect calls (function pointers, virtual table
+/// dispatch), this will be a list of one or more functions. For virtual table
+/// dispatches, this record will also hold the type of the object.
class SampleRecord {
public:
using CallTarget = std::pair<FunctionId, uint64_t>;
@@ -746,6 +764,7 @@ using BodySampleMap = std::map<LineLocation, SampleRecord>;
// memory, which is *very* significant for large profiles.
using FunctionSamplesMap = std::map<FunctionId, FunctionSamples>;
using CallsiteSampleMap = std::map<LineLocation, FunctionSamplesMap>;
+using CallsiteTypeMap = std::map<LineLocation, TypeCountMap>;
using LocToLocMap =
std::unordered_map<LineLocation, LineLocation, LineLocationHash>;
@@ -928,6 +947,14 @@ class FunctionSamples {
return &Iter->second;
}
+ /// Returns the TypeCountMap for inlined callsites at the given \p Loc.
+ const TypeCountMap *findCallsiteTypeSamplesAt(const LineLocation &Loc) const {
+ auto Iter = VirtualCallsiteTypeCounts.find(mapIRLocToProfileLoc(Loc));
+ if (Iter == VirtualCallsiteTypeCounts.end())
+ return nullptr;
+ return &Iter->second;
+ }
+
/// Returns a pointer to FunctionSamples at the given callsite location
/// \p Loc with callee \p CalleeName. If no callsite can be found, relax
/// the restriction to return the FunctionSamples at callsite location
@@ -989,6 +1016,42 @@ class FunctionSamples {
return CallsiteSamples;
}
+ /// Return all the callsite type samples collected in the body of the
+ /// function.
+ const CallsiteTypeMap &getCallsiteTypeCounts() const {
+ return VirtualCallsiteTypeCounts;
+ }
+
+ /// Returns the type samples for the un-drifted location of \p Loc.
+ TypeCountMap &getTypeSamplesAt(const LineLocation &Loc) {
+ return VirtualCallsiteTypeCounts[mapIRLocToProfileLoc(Loc)];
+ }
+
+ /// Scale \p Other sample counts by \p Weight and add the scaled result to the
+ /// type samples for the undrifted location of \p Loc.
+ template <typename T>
+ sampleprof_error addCallsiteVTableTypeProfAt(const LineLocation &Loc,
+ const T &Other,
+ uint64_t Weight = 1) {
+ static_assert((std::is_same_v<typename T::key_type, StringRef> ||
+ std::is_same_v<typename T::key_type, FunctionId>) &&
+ std::is_same_v<typename T::mapped_type, uint64_t>,
+ "T must be a map with StringRef or FunctionId as key and "
+ "uint64_t as value");
+ TypeCountMap &TypeCounts = getTypeSamplesAt(Loc);
+ bool Overflowed = false;
+
+ for (const auto [Type, Count] : Other) {
+ FunctionId TypeId(Type);
+ bool RowOverflow = false;
+ TypeCounts[TypeId] = SaturatingMultiplyAdd(
+ Count, Weight, TypeCounts[TypeId], &RowOverflow);
+ Overflowed |= RowOverflow;
+ }
+ return Overflowed ? sampleprof_error::counter_overflow
+ : sampleprof_error::success;
+ }
+
/// Return the maximum of sample counts in a function body. When SkipCallSite
/// is false, which is the default, the return count includes samples in the
/// inlined functions. When SkipCallSite is true, the return count only
@@ -1043,6 +1106,10 @@ class FunctionSamples {
mergeSampleProfErrors(Result,
FSMap[Rec.first].merge(Rec.second, Weight));
}
+ for (const auto &[Loc, OtherTypeMap] : Other.getCallsiteTypeCounts())
+ mergeSampleProfErrors(
+ Result, addCallsiteVTableTypeProfAt(Loc, OtherTypeMap, Weight));
+
return Result;
}
@@ -1286,6 +1353,23 @@ class FunctionSamples {
/// collected in the call to baz() at line offset 8.
CallsiteSampleMap CallsiteSamples;
+ /// Map virtual callsites to the vtable from which they are loaded.
+ ///
+ /// Each entry is a mapping from the location to the list of vtables and their
+ /// sampled counts. For example, given:
+ ///
+ /// void foo() {
+ /// ...
+ /// 5 inlined_vcall_bar();
+ /// ...
+ /// 5 inlined_vcall_baz();
+ /// ...
+ /// 200 inlined_vcall_qux();
+ /// }
+ /// This map will contain two entries. One with two types for line offset 5
+ /// and one with one type for line offset 200.
+ CallsiteTypeMap VirtualCallsiteTypeCounts;
+
/// IR to profile location map generated by stale profile matching.
///
/// Each entry is a mapping from the location on current build to the matched
diff --git a/llvm/include/llvm/ProfileData/SampleProfReader.h b/llvm/include/llvm/ProfileData/SampleProfReader.h
index bfe079fbe536f..fe8266247b2db 100644
--- a/llvm/include/llvm/ProfileData/SampleProfReader.h
+++ b/llvm/include/llvm/ProfileData/SampleProfReader.h
@@ -703,6 +703,14 @@ class LLVM_ABI SampleProfileReaderBinary : public SampleProfileReader {
/// otherwise same as readStringFromTable, also return its hash value.
ErrorOr<std::pair<SampleContext, uint64_t>> readSampleContextFromTable();
+ /// Read all virtual functions' vtable access counts for \p FProfile.
+ std::error_code readCallsiteVTableProf(FunctionSamples &FProfile);
+
+ /// Read bytes from the input buffer pointed by `Data` and decode them into
+ /// \p M. `Data` will be advanced to the end of the read bytes when this
+ /// function returns. Returns error if any.
+ std::error_code readVTableTypeCountMap(TypeCountMap &M);
+
/// Points to the current location in the buffer.
const uint8_t *Data = nullptr;
@@ -727,6 +735,10 @@ class LLVM_ABI SampleProfileReaderBinary : public SampleProfileReader {
/// to the start of MD5SampleContextTable.
const uint64_t *MD5SampleContextStart = nullptr;
+ /// If true, the profile has vtable profiles and reader should decode them
+ /// to parse profiles correctly.
+ bool ReadVTableProf = false;
+
private:
std::error_code readSummaryEntry(std::vector<ProfileSummaryEntry> &Entries);
virtual std::error_code verifySPMagic(uint64_t Magic) = 0;
diff --git a/llvm/include/llvm/ProfileData/SampleProfWriter.h b/llvm/include/llvm/ProfileData/SampleProfWriter.h
index e84b2095efd7b..9dbeaf56509b0 100644
--- a/llvm/include/llvm/ProfileData/SampleProfWriter.h
+++ b/llvm/include/llvm/ProfileData/SampleProfWriter.h
@@ -217,13 +217,20 @@ class LLVM_ABI SampleProfileWriterBinary : public SampleProfileWriter {
std::error_code writeBody(const FunctionSamples &S);
inline void stablizeNameTable(MapVector<FunctionId, uint32_t> &NameTable,
std::set<FunctionId> &V);
-
+
MapVector<FunctionId, uint32_t> NameTable;
-
+
void addName(FunctionId FName);
virtual void addContext(const SampleContext &Context);
void addNames(const FunctionSamples &S);
+ /// Write \p CallsiteTypeMap to the output stream \p OS.
+ std::error_code
+ writeCallsiteVTableProf(const CallsiteTypeMap &CallsiteTypeMap,
+ raw_ostream &OS);
+
+ bool WriteVTableProf = false;
+
private:
LLVM_ABI friend ErrorOr<std::unique_ptr<SampleProfileWriter>>
SampleProfileWriter::create(std::unique_ptr<raw_ostream> &OS,
@@ -412,8 +419,7 @@ class LLVM_ABI SampleProfileWriterExtBinaryBase
class LLVM_ABI SampleProfileWriterExtBinary
: public SampleProfileWriterExtBinaryBase {
public:
- SampleProfileWriterExtBinary(std::unique_ptr<raw_ostream> &OS)
- : SampleProfileWriterExtBinaryBase(OS) {}
+ SampleProfileWriterExtBinary(std::unique_ptr<raw_ostream> &OS);
private:
std::error_code writeDefaultLayout(const SampleProfileMap &ProfileMap);
diff --git a/llvm/lib/ProfileData/SampleProf.cpp b/llvm/lib/ProfileData/SampleProf.cpp
index 60c1393616713..f8706186454af 100644
--- a/llvm/lib/ProfileData/SampleProf.cpp
+++ b/llvm/lib/ProfileData/SampleProf.cpp
@@ -47,6 +47,24 @@ bool FunctionSamples::ProfileIsPreInlined = false;
bool FunctionSamples::UseMD5 = false;
bool FunctionSamples::HasUniqSuffix = true;
bool FunctionSamples::ProfileIsFS = false;
+
+std::error_code
+serializeTypeMap(const TypeCountMap &Map,
+ const MapVector<FunctionId, uint32_t> &NameTable,
+ raw_ostream &OS) {
+ encodeULEB128(Map.size(), OS);
+ for (const auto &[TypeName, SampleCount] : Map) {
+ if (auto NameIndexIter = NameTable.find(TypeName);
+ NameIndexIter != NameTable.end()) {
+ encodeULEB128(NameIndexIter->second, OS);
+ } else {
+ // If the type is not in the name table, we cannot serialize it.
+ return sampleprof_error::truncated_name_table;
+ }
+ encodeULEB128(SampleCount, OS);
+ }
+ return sampleprof_error::success;
+}
} // namespace sampleprof
} // namespace llvm
@@ -93,6 +111,8 @@ class SampleProfErrorCategoryType : public std::error_category {
return "Function hash mismatch";
case sampleprof_error::illegal_line_offset:
return "Illegal line offset in sample profile data";
+ case sampleprof_error::duplicate_vtable_type:
+ return "Duplicate vtable type in one map";
}
llvm_unreachable("A value of sampleprof_error has no message.");
}
@@ -126,6 +146,7 @@ sampleprof_error SampleRecord::merge(const SampleRecord &Other,
for (const auto &I : Other.getCallTargets()) {
mergeSampleProfErrors(Result, addCalledTarget(I.first, I.second, Weight));
}
+
return Result;
}
@@ -178,6 +199,17 @@ raw_ostream &llvm::sampleprof::operator<<(raw_ostream &OS,
return OS;
}
+static void printTypeCountMap(raw_ostream &OS, LineLocation Loc,
+ const TypeCountMap &TypeCountMap) {
+ if (TypeCountMap.empty()) {
+ return;
+ }
+ OS << Loc << ": vtables: ";
+ for (const auto &[Type, Count] : TypeCountMap)
+ OS << Type << ":" << Count << " ";
+ OS << "\n";
+}
+
/// Print the samples collected for a function on stream \p OS.
void FunctionSamples::print(raw_ostream &OS, unsigned Indent) const {
if (getFunctionHash())
@@ -192,7 +224,13 @@ void FunctionSamples::print(raw_ostream &OS, unsigned Indent) const {
SampleSorter<LineLocation, SampleRecord> SortedBodySamples(BodySamples);
for (const auto &SI : SortedBodySamples.get()) {
OS.indent(Indent + 2);
+ const auto &Loc = SI->first;
OS << SI->first << ": " << SI->second;
+ if (const TypeCountMap *TypeCountMap =
+ this->findCallsiteTypeSamplesAt(Loc)) {
+ OS.indent(Indent + 2);
+ printTypeCountMap(OS, Loc, *TypeCountMap);
+ }
}
OS.indent(Indent);
OS << "}\n";
@@ -214,6 +252,11 @@ void FunctionSamples::print(raw_ostream &OS, unsigned Indent) const {
OS << Loc << ": inlined callee: " << FuncSample.getFunction() << ": ";
FuncSample.print(OS, Indent + 4);
}
+ auto TypeSamplesIter = VirtualCallsiteTypeCounts.find(Loc);
+ if (TypeSamplesIter != VirtualCallsiteTypeCounts.end()) {
+ OS.indent(Indent + 2);
+ printTypeCountMap(OS, Loc, TypeSamplesIter->second);
+ }
}
OS.indent(Indent);
OS << "}\n";
diff --git a/llvm/lib/ProfileData/SampleProfReader.cpp b/llvm/lib/ProfileData/SampleProfReader.cpp
index cf7874041f3bf..b8b55f72935b5 100644
--- a/llvm/lib/ProfileData/SampleProfReader.cpp
+++ b/llvm/lib/ProfileData/SampleProfReader.cpp
@@ -197,8 +197,30 @@ enum class LineType {
CallSiteProfile,
BodyProfile,
Metadata,
+ VirtualCallTypeProfile,
};
+static bool parseTypeCountMap(StringRef Input,
+ DenseMap<StringRef, uint64_t> &TypeCountMap) {
+ for (size_t Index = Input.find_first_not_of(' '); Index != StringRef::npos;) {
+ size_t n1 = Input.find(':', Index);
+ if (n1 == StringRef::npos)
+ return false; // No colon found, invalid format.
+ StringRef TypeName = Input.substr(Index, n1 - Index);
+ // n2 is the start index of count.
+ size_t n2 = n1 + 1;
+ // n3 is the start index after the 'target:count' pair.
+ size_t n3 = Input.find_first_of(' ', n2);
+ uint64_t Count;
+ if (Input.substr(n2, n3 - n2).getAsInteger(10, Count))
+ return false; // Invalid count.
+ TypeCountMap[TypeName] = Count;
+ Index = (n3 == StringRef::npos) ? StringRef::npos
+ : Input.find_first_not_of(' ', n3);
+ }
+ return true;
+}
+
/// Parse \p Input as line sample.
///
/// \param Input input line.
@@ -215,6 +237,7 @@ static bool ParseLine(const StringRef &Input, LineType &LineTy, uint32_t &Depth,
uint64_t &NumSamples, uint32_t &LineOffset,
uint32_t &Discriminator, StringRef &CalleeName,
DenseMap<StringRef, uint64_t> &TargetCountMap,
+ DenseMap<StringRef, uint64_t> &TypeCountMap,
uint64_t &FunctionHash, uint32_t &Attributes,
bool &IsFlat) {
for (Depth = 0; Input[Depth] == ' '; Depth++)
@@ -289,6 +312,7 @@ static bool ParseLine(const StringRef &Input, LineType &LineTy, uint32_t &Depth,
n4 = AfterColon.find_first_of(' ');
n4 = (n4 != StringRef::npos) ? n3 + n4 + 1 : Rest.size();
StringRef WordAfterColon = Rest.substr(n3 + 1, n4 - n3 - 1);
+ // Break the loop if parsing integer succeeded.
if (!WordAfterColon.getAsInteger(10, count))
break;
@@ -306,6 +330,10 @@ static bool ParseLine(const StringRef &Input, LineType &LineTy, uint32_t &Depth,
// Change n3 to the next blank space after colon + integer pair.
n3 = n4;
}
+ } else if (Rest.starts_with(kVTableProfPrefix)) {
+ LineTy = LineType::VirtualCallTypeProfile;
+ return parseTypeCountMap(Rest.substr(strlen(kVTableProfPrefix)),
+ TypeCountMap);
} else {
LineTy = LineType::CallSiteProfile;
size_t n3 = Rest.find_last_of(':');
@@ -374,14 +402,15 @@ std::error_code SampleProfileReaderText::readImpl() {
uint64_t NumSamples;
StringRef FName;
DenseMap<StringRef, uint64_t> TargetCountMap;
+ DenseMap<StringRef, uint64_t> TypeCountMap;
uint32_t Depth, LineOffset, Discriminator;
LineType LineTy;
uint64_t FunctionHash = 0;
uint32_t Attributes = 0;
bool IsFlat = false;
if (!ParseLine(*LineIt, LineTy, Depth, NumSamples, LineOffset,
- Discriminator, FName, TargetCountMap, FunctionHash,
- Attributes, IsFlat)) {
+ Discriminator, FName, TargetCountMap, TypeCountMap,
+ FunctionHash, Attributes, IsFlat)) {
reportError(LineIt.line_number(),
"Expected 'NUM[.NUM]: NUM[ mangled_name:NUM]*', found " +
*LineIt);
@@ -410,6 +439,14 @@ std::error_code SampleProfileReaderText::readImpl() {
DepthMetadata = 0;
break;
}
+
+ case LineType::VirtualCallTypeProfile: {
+ mergeSampleProfErrors(
+ Result, InlineStack.back()->addCallsiteVTableTypeProfAt(
+ LineLocation(LineOffset, Discriminator), TypeCountMap));
+ break;
+ }
+
case LineType::BodyProfile: {
FunctionSamples &FProfile = *InlineStack.back();
for (const auto &name_count : TargetCountMap) {
@@ -591,6 +628,59 @@ SampleProfileReaderBinary::readSampleContextFromTable() {
return std::make_pair(Context, Hash);
}
+std::error_code
+SampleProfileReaderBinary::readVTableTypeCountMap(TypeCountMap &M) {
+ auto NumVTableTypes = readNumber<uint32_t>();
+ if (std::error_code EC = NumVTableTypes.getError())
+ return EC;
+
+ for (uint32_t I = 0; I < *NumVTableTypes; ++I) {
+ auto VTableType(readStringFromTable());
+ if (std::error_code EC = VTableType.getError())
+ return EC;
+
+ auto VTableSamples = readNumber<uint64_t>();
+ if (std::error_code EC = VTableSamples.getError())
+ return EC;
+
+ if (!M.insert(std::make_pair(*VTableType, *VTableSamples)).second)
+ return sampleprof_error::duplicate_vtable_type;
+ }
+ return sampleprof_error::success;
+}
+
+std::error_code
+SampleProfileReaderBinary::readCallsiteVTableProf(FunctionSamples &FProfile) {
+ if (!ReadVTableProf)
+ return sampleprof_error::success;
+
+ // Read the vtable type profile for the callsite.
+ auto NumCallsites = readNumber<uint32_t>();
+ if (std::error_code EC = NumCallsites.getError())
+ return EC;
+
+ for (uint32_t I = 0; I < *NumCallsites; ++I) {
+ auto LineOffset = readNumber<uint64_t>();
+ if (std::error_code EC = LineOffset.getError())
+ return EC;
+
+ if (!isOffsetLegal(*LineOffset))
+ return sampleprof_error::illegal_line_offset;
+
+ auto Discriminator = readNumber<uint64_t>();
+ if (std::error_code EC = Discriminator.getError())
+ return EC;
+
+ // Here we handle FS discriminators:
+ const uint32_t DiscriminatorVal = (*Discriminator) & getDiscriminatorMask();
+
+ if (std::error_code EC = readVTableTypeCountMap(FProfile.getTypeSamplesAt(
+ LineLocation(*LineOffset, DiscriminatorVal))))
+ return EC;
+ }
+ return sampleprof_error::success;
+}
+
std::error_code
SampleProfileReaderBinary::readProfile(FunctionSamples &FProfile) {
auto NumSamples = readNumber<uint64_t>();
@@ -671,7 +761,7 @@ SampleProfileReaderBinary::readProfile(FunctionSamples &FProfile) {
return EC;
}
- return sampleprof_error::success;
+ return readCallsiteVTableProf(FProfile);
}
std::error_code
@@ -733,6 +823,8 @@ std::error_code SampleProfileReaderExtBinaryBase::readOneSection(
FunctionSamples::ProfileIsPreInlined = ProfileIsPreInlined = true;
if (hasSecFlag(Entry, SecProfSummaryFlags::SecFlagFSDiscriminator))
FunctionSamples::ProfileIsFS = ProfileIsFS = true;
+ if (hasSecFlag(Entry, SecProfSummaryFlags::SecFlagHasVTableTypeProf)...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/148002
More information about the llvm-commits
mailing list