[llvm] r342283 - [SampleFDO] Add FunctionOffsetTable in compact binary format profile.
Wei Mi via llvm-commits
llvm-commits at lists.llvm.org
Fri Sep 14 16:35:13 PDT 2018
There is file system race indeed. I will fix that. Thanks Reid!
Wei.
On Fri, Sep 14, 2018 at 4:09 PM, Reid Kleckner <rnk at google.com> wrote:
> These tests seem flaky when I run them locally. Are there possible
> filesystem race issues here?
>
> FAIL: LLVM-Unit :: ProfileData/./ProfileDataTests.exe/
> SampleProfTest.roundtrip_compact_binary_profile (2329 of 27766)
> ******************** TEST 'LLVM-Unit :: ProfileData/./
> ProfileDataTests.exe/SampleProfTest.roundtrip_compact_binary_profile'
> FAILED ********************
> Note: Google Test filter = SampleProfTest.roundtrip_compact_binary_profile
> [==========] Running 1 test from 1 test case.
> [----------] Global test environment set-up.
> [----------] 1 test from SampleProfTest
> [ RUN ] SampleProfTest.roundtrip_compact_binary_profile
> C:\src\llvm-project\llvm\unittests\ProfileData\SampleProfTest.cpp(59):
> error: Value of: NoError(ReaderOrErr.getError())
> Actual: false (error 6: Unrecognized sample profile encoding format)
> Expected: true
> unknown file: error: SEH exception with code 0x3221225477 thrown in the
> test body.
> unknown file: error: SEH exception with code 0x3221225477 thrown in the
> test fixture's destructor.
> [ FAILED ] SampleProfTest.roundtrip_compact_binary_profile (2 ms)
> [----------] 1 test from SampleProfTest (2 ms total)
>
> [----------] Global test environment tear-down
> [==========] 1 test from 1 test case ran. (2 ms total)
> [ PASSED ] 0 tests.
> [ FAILED ] 1 test, listed below:
> [ FAILED ] SampleProfTest.roundtrip_compact_binary_profile
>
> 1 FAILED TEST
>
> ********************
> Testing: 0 ..
> FAIL: LLVM-Unit :: ProfileData/./ProfileDataTests.exe/
> SampleProfTest.roundtrip_text_profile (2330 of 27766)
> ******************** TEST 'LLVM-Unit :: ProfileData/./
> ProfileDataTests.exe/SampleProfTest.roundtrip_text_profile' FAILED
> ********************
> Note: Google Test filter = SampleProfTest.roundtrip_text_profile
> [==========] Running 1 test from 1 test case.
> [----------] Global test environment set-up.
> [----------] 1 test from SampleProfTest
> [ RUN ] SampleProfTest.roundtrip_text_profile
> C:\src\llvm-project\llvm\unittests\ProfileData\SampleProfTest.cpp(59):
> error: Value of: NoError(ReaderOrErr.getError())
> Actual: false (error 6: Unrecognized sample profile encoding format)
> Expected: true
> unknown file: error: SEH exception with code 0x3221225477 thrown in the
> test body.
> unknown file: error: SEH exception with code 0x3221225477 thrown in the
> test fixture's destructor.
> [ FAILED ] SampleProfTest.roundtrip_text_profile (2 ms)
> [----------] 1 test from SampleProfTest (2 ms total)
>
> [----------] Global test environment tear-down
> [==========] 1 test from 1 test case ran. (2 ms total)
> [ PASSED ] 0 tests.
> [ FAILED ] 1 test, listed below:
> [ FAILED ] SampleProfTest.roundtrip_text_profile
>
> 1 FAILED TEST
>
> ********************
>
> On Fri, Sep 14, 2018 at 1:54 PM Wei Mi via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
>
>> Author: wmi
>> Date: Fri Sep 14 13:52:59 2018
>> New Revision: 342283
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=342283&view=rev
>> Log:
>> [SampleFDO] Add FunctionOffsetTable in compact binary format profile.
>>
>> The patch saves a function offset table which maps function name index to
>> the
>> offset of its function profile to the start of the binary profile. By
>> using
>> the function offset table, for those function profiles which will not be
>> used
>> when compiling a module, the profile reader does't have to read them. For
>> profile size around 10~20M, it saves ~10% compile time.
>>
>> Differential Revision: https://reviews.llvm.org/D51863
>>
>>
>> Modified:
>> llvm/trunk/include/llvm/ProfileData/SampleProf.h
>> llvm/trunk/include/llvm/ProfileData/SampleProfReader.h
>> llvm/trunk/include/llvm/ProfileData/SampleProfWriter.h
>> llvm/trunk/lib/ProfileData/SampleProf.cpp
>> llvm/trunk/lib/ProfileData/SampleProfReader.cpp
>> llvm/trunk/lib/ProfileData/SampleProfWriter.cpp
>> llvm/trunk/lib/Transforms/IPO/SampleProfile.cpp
>> llvm/trunk/test/Transforms/SampleProfile/Inputs/function_
>> metadata.compact.afdo
>> llvm/trunk/test/Transforms/SampleProfile/Inputs/indirect-
>> call.compact.afdo
>> llvm/trunk/test/Transforms/SampleProfile/Inputs/inline.
>> compactbinary.afdo
>> llvm/trunk/unittests/ProfileData/SampleProfTest.cpp
>>
>> Modified: llvm/trunk/include/llvm/ProfileData/SampleProf.h
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/
>> llvm/ProfileData/SampleProf.h?rev=342283&r1=342282&r2=342283&view=diff
>> ============================================================
>> ==================
>> --- llvm/trunk/include/llvm/ProfileData/SampleProf.h (original)
>> +++ llvm/trunk/include/llvm/ProfileData/SampleProf.h Fri Sep 14 13:52:59
>> 2018
>> @@ -49,7 +49,8 @@ enum class sampleprof_error {
>> unsupported_writing_format,
>> truncated_name_table,
>> not_implemented,
>> - counter_overflow
>> + counter_overflow,
>> + ostream_seek_unsupported
>> };
>>
>> inline std::error_code make_error_code(sampleprof_error E) {
>>
>> Modified: llvm/trunk/include/llvm/ProfileData/SampleProfReader.h
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/
>> llvm/ProfileData/SampleProfReader.h?rev=342283&
>> r1=342282&r2=342283&view=diff
>> ============================================================
>> ==================
>> --- llvm/trunk/include/llvm/ProfileData/SampleProfReader.h (original)
>> +++ llvm/trunk/include/llvm/ProfileData/SampleProfReader.h Fri Sep 14
>> 13:52:59 2018
>> @@ -279,6 +279,8 @@ public:
>> /// Print the profile for \p FName on stream \p OS.
>> void dumpFunctionProfile(StringRef FName, raw_ostream &OS = dbgs());
>>
>> + virtual void collectFuncsToUse(const Module &M) {}
>> +
>> /// Print all the profiles on stream \p OS.
>> void dump(raw_ostream &OS = dbgs());
>>
>> @@ -364,7 +366,7 @@ public:
>> : SampleProfileReader(std::move(B), C, Format) {}
>>
>> /// Read and validate the file header.
>> - std::error_code readHeader() override;
>> + virtual std::error_code readHeader() override;
>>
>> /// Read sample profiles from the associated file.
>> std::error_code read() override;
>> @@ -378,6 +380,10 @@ protected:
>> /// \returns the read value.
>> template <typename T> ErrorOr<T> readNumber();
>>
>> + /// Read a numeric value of type T from the profile. The value is saved
>> + /// without encoded.
>> + template <typename T> ErrorOr<T> readUnencodedNumber();
>> +
>> /// Read a string from the profile.
>> ///
>> /// If an error occurs during decoding, a diagnostic message is
>> emitted and
>> @@ -392,6 +398,9 @@ protected:
>> /// Return true if we've reached the end of file.
>> bool at_eof() const { return Data >= End; }
>>
>> + /// Read the next function profile instance.
>> + std::error_code readFuncProfile();
>> +
>> /// Read the contents of the given profile instance.
>> std::error_code readProfile(FunctionSamples &FProfile);
>>
>> @@ -436,10 +445,17 @@ class SampleProfileReaderCompactBinary :
>> private:
>> /// Function name table.
>> std::vector<std::string> NameTable;
>> + /// The table mapping from function name to the offset of its
>> FunctionSample
>> + /// towards file start.
>> + DenseMap<StringRef, uint64_t> FuncOffsetTable;
>> + /// The set containing the functions to use when compiling a module.
>> + DenseSet<StringRef> FuncsToUse;
>> virtual std::error_code verifySPMagic(uint64_t Magic) override;
>> virtual std::error_code readNameTable() override;
>> /// Read a string indirectly via the name table.
>> virtual ErrorOr<StringRef> readStringFromTable() override;
>> + virtual std::error_code readHeader() override;
>> + std::error_code readFuncOffsetTable();
>>
>> public:
>> SampleProfileReaderCompactBinary(std::unique_ptr<MemoryBuffer> B,
>> @@ -448,6 +464,12 @@ public:
>>
>> /// \brief Return true if \p Buffer is in the format supported by this
>> class.
>> static bool hasFormat(const MemoryBuffer &Buffer);
>> +
>> + /// Read samples only for functions to use.
>> + std::error_code read() override;
>> +
>> + /// Collect functions to be used when compiling Module \p M.
>> + void collectFuncsToUse(const Module &M) override;
>> };
>>
>> using InlineCallStack = SmallVector<FunctionSamples *, 10>;
>>
>> Modified: llvm/trunk/include/llvm/ProfileData/SampleProfWriter.h
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/
>> llvm/ProfileData/SampleProfWriter.h?rev=342283&
>> r1=342282&r2=342283&view=diff
>> ============================================================
>> ==================
>> --- llvm/trunk/include/llvm/ProfileData/SampleProfWriter.h (original)
>> +++ llvm/trunk/include/llvm/ProfileData/SampleProfWriter.h Fri Sep 14
>> 13:52:59 2018
>> @@ -42,7 +42,7 @@ public:
>> /// Write all the sample profiles in the given map of samples.
>> ///
>> /// \returns status code of the file update operation.
>> - std::error_code write(const StringMap<FunctionSamples> &ProfileMap);
>> + virtual std::error_code write(const StringMap<FunctionSamples>
>> &ProfileMap);
>>
>> raw_ostream &getOutputStream() { return *OutputStream; }
>>
>> @@ -103,14 +103,15 @@ private:
>> /// Sample-based profile writer (binary format).
>> class SampleProfileWriterBinary : public SampleProfileWriter {
>> public:
>> - std::error_code write(const FunctionSamples &S) override;
>> + virtual std::error_code write(const FunctionSamples &S) override;
>> SampleProfileWriterBinary(std::unique_ptr<raw_ostream> &OS)
>> : SampleProfileWriter(OS) {}
>>
>> protected:
>> virtual std::error_code writeNameTable() = 0;
>> virtual std::error_code writeMagicIdent() = 0;
>> - std::error_code writeHeader(const StringMap<FunctionSamples>
>> &ProfileMap) override;
>> + virtual std::error_code
>> + writeHeader(const StringMap<FunctionSamples> &ProfileMap) override;
>> std::error_code writeSummary();
>> std::error_code writeNameIdx(StringRef FName);
>> std::error_code writeBody(const FunctionSamples &S);
>> @@ -135,12 +136,56 @@ protected:
>> virtual std::error_code writeMagicIdent() override;
>> };
>>
>> +// CompactBinary is a compact format of binary profile which both reduces
>> +// the profile size and the load time needed when compiling. It has two
>> +// major difference with Binary format.
>> +// 1. It represents all the strings in name table using md5 hash.
>> +// 2. It saves a function offset table which maps function name index to
>> +// the offset of its function profile to the start of the binary profile,
>> +// so by using the function offset table, for those function profiles
>> which
>> +// will not be needed when compiling a module, the profile reader does't
>> +// have to read them and it saves compile time if the profile size is
>> huge.
>> +// The layout of the compact format is shown as follows:
>> +//
>> +// Part1: Profile header, the same as binary format, containing magic
>> +// number, version, summary, name table...
>> +// Part2: Function Offset Table Offset, which saves the position of
>> +// Part4.
>> +// Part3: Function profile collection
>> +// function1 profile start
>> +// ....
>> +// function2 profile start
>> +// ....
>> +// function3 profile start
>> +// ....
>> +// ......
>> +// Part4: Function Offset Table
>> +// function1 name index --> function1 profile start
>> +// function2 name index --> function2 profile start
>> +// function3 name index --> function3 profile start
>> +//
>> +// We need Part2 because profile reader can use it to find out and read
>> +// function offset table without reading Part3 first.
>> class SampleProfileWriterCompactBinary : public
>> SampleProfileWriterBinary {
>> using SampleProfileWriterBinary::SampleProfileWriterBinary;
>>
>> +public:
>> + virtual std::error_code write(const FunctionSamples &S) override;
>> + virtual std::error_code
>> + write(const StringMap<FunctionSamples> &ProfileMap) override;
>> +
>> protected:
>> + /// The table mapping from function name to the offset of its
>> FunctionSample
>> + /// towards profile start.
>> + MapVector<StringRef, uint64_t> FuncOffsetTable;
>> + /// The offset of the slot to be filled with the offset of
>> FuncOffsetTable
>> + /// towards profile start.
>> + uint64_t TableOffset;
>> virtual std::error_code writeNameTable() override;
>> virtual std::error_code writeMagicIdent() override;
>> + virtual std::error_code
>> + writeHeader(const StringMap<FunctionSamples> &ProfileMap) override;
>> + std::error_code writeFuncOffsetTable();
>> };
>>
>> } // end namespace sampleprof
>>
>> Modified: llvm/trunk/lib/ProfileData/SampleProf.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/
>> ProfileData/SampleProf.cpp?rev=342283&r1=342282&r2=342283&view=diff
>> ============================================================
>> ==================
>> --- llvm/trunk/lib/ProfileData/SampleProf.cpp (original)
>> +++ llvm/trunk/lib/ProfileData/SampleProf.cpp Fri Sep 14 13:52:59 2018
>> @@ -67,6 +67,8 @@ class SampleProfErrorCategoryType : publ
>> return "Unimplemented feature";
>> case sampleprof_error::counter_overflow:
>> return "Counter overflow";
>> + case sampleprof_error::ostream_seek_unsupported:
>> + return "Ostream does not support seek";
>> }
>> llvm_unreachable("A value of sampleprof_error has no message.");
>> }
>>
>> Modified: llvm/trunk/lib/ProfileData/SampleProfReader.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/
>> ProfileData/SampleProfReader.cpp?rev=342283&r1=342282&r2=342283&view=diff
>> ============================================================
>> ==================
>> --- llvm/trunk/lib/ProfileData/SampleProfReader.cpp (original)
>> +++ llvm/trunk/lib/ProfileData/SampleProfReader.cpp Fri Sep 14 13:52:59
>> 2018
>> @@ -30,6 +30,7 @@
>> #include "llvm/Support/ErrorOr.h"
>> #include "llvm/Support/LEB128.h"
>> #include "llvm/Support/LineIterator.h"
>> +#include "llvm/Support/MD5.h"
>> #include "llvm/Support/MemoryBuffer.h"
>> #include "llvm/Support/raw_ostream.h"
>> #include <algorithm>
>> @@ -320,6 +321,21 @@ ErrorOr<StringRef> SampleProfileReaderBi
>> }
>>
>> template <typename T>
>> +ErrorOr<T> SampleProfileReaderBinary::readUnencodedNumber() {
>> + std::error_code EC;
>> +
>> + if (Data + sizeof(T) > End) {
>> + EC = sampleprof_error::truncated;
>> + reportError(0, EC.message());
>> + return EC;
>> + }
>> +
>> + using namespace support;
>> + T Val = endian::readNext<T, little, unaligned>(Data);
>> + return Val;
>> +}
>> +
>> +template <typename T>
>> inline ErrorOr<uint32_t> SampleProfileReaderBinary::readStringIndex(T
>> &Table) {
>> std::error_code EC;
>> auto Idx = readNumber<uint32_t>();
>> @@ -423,29 +439,51 @@ SampleProfileReaderBinary::readProfile(F
>> return sampleprof_error::success;
>> }
>>
>> -std::error_code SampleProfileReaderBinary::read() {
>> - while (!at_eof()) {
>> - auto NumHeadSamples = readNumber<uint64_t>();
>> - if (std::error_code EC = NumHeadSamples.getError())
>> - return EC;
>> +std::error_code SampleProfileReaderBinary::readFuncProfile() {
>> + auto NumHeadSamples = readNumber<uint64_t>();
>> + if (std::error_code EC = NumHeadSamples.getError())
>> + return EC;
>>
>> - auto FName(readStringFromTable());
>> - if (std::error_code EC = FName.getError())
>> - return EC;
>> + auto FName(readStringFromTable());
>> + if (std::error_code EC = FName.getError())
>> + return EC;
>>
>> - Profiles[*FName] = FunctionSamples();
>> - FunctionSamples &FProfile = Profiles[*FName];
>> - FProfile.setName(*FName);
>> + Profiles[*FName] = FunctionSamples();
>> + FunctionSamples &FProfile = Profiles[*FName];
>> + FProfile.setName(*FName);
>>
>> - FProfile.addHeadSamples(*NumHeadSamples);
>> + FProfile.addHeadSamples(*NumHeadSamples);
>> +
>> + if (std::error_code EC = readProfile(FProfile))
>> + return EC;
>> + return sampleprof_error::success;
>> +}
>>
>> - if (std::error_code EC = readProfile(FProfile))
>> +std::error_code SampleProfileReaderBinary::read() {
>> + while (!at_eof()) {
>> + if (std::error_code EC = readFuncProfile())
>> return EC;
>> }
>>
>> return sampleprof_error::success;
>> }
>>
>> +std::error_code SampleProfileReaderCompactBinary::read() {
>> + for (auto Name : FuncsToUse) {
>> + auto GUID = std::to_string(MD5Hash(Name));
>> + auto iter = FuncOffsetTable.find(StringRef(GUID));
>> + if (iter == FuncOffsetTable.end())
>> + continue;
>> + const uint8_t *SavedData = Data;
>> + Data = reinterpret_cast<const uint8_t *>(Buffer->getBufferStart()) +
>> + iter->second;
>> + if (std::error_code EC = readFuncProfile())
>> + return EC;
>> + Data = SavedData;
>> + }
>> + return sampleprof_error::success;
>> +}
>> +
>> std::error_code SampleProfileReaderRawBinary::verifySPMagic(uint64_t
>> Magic) {
>> if (Magic == SPMagic())
>> return sampleprof_error::success;
>> @@ -514,6 +552,53 @@ std::error_code SampleProfileReaderBinar
>> return sampleprof_error::success;
>> }
>>
>> +std::error_code SampleProfileReaderCompactBinary::readHeader() {
>> + SampleProfileReaderBinary::readHeader();
>> + if (std::error_code EC = readFuncOffsetTable())
>> + return EC;
>> + return sampleprof_error::success;
>> +}
>> +
>> +std::error_code SampleProfileReaderCompactBinary::readFuncOffsetTable()
>> {
>> + auto TableOffset = readUnencodedNumber<uint64_t>();
>> + if (std::error_code EC = TableOffset.getError())
>> + return EC;
>> +
>> + const uint8_t *SavedData = Data;
>> + const uint8_t *TableStart =
>> + reinterpret_cast<const uint8_t *>(Buffer->getBufferStart()) +
>> + *TableOffset;
>> + Data = TableStart;
>> +
>> + auto Size = readNumber<uint64_t>();
>> + if (std::error_code EC = Size.getError())
>> + return EC;
>> +
>> + FuncOffsetTable.reserve(*Size);
>> + for (uint32_t I = 0; I < *Size; ++I) {
>> + auto FName(readStringFromTable());
>> + if (std::error_code EC = FName.getError())
>> + return EC;
>> +
>> + auto Offset = readNumber<uint64_t>();
>> + if (std::error_code EC = Offset.getError())
>> + return EC;
>> +
>> + FuncOffsetTable[*FName] = *Offset;
>> + }
>> + End = TableStart;
>> + Data = SavedData;
>> + return sampleprof_error::success;
>> +}
>> +
>> +void SampleProfileReaderCompactBinary::collectFuncsToUse(const Module
>> &M) {
>> + FuncsToUse.clear();
>> + for (auto &F : M) {
>> + StringRef Fname = F.getName().split('.').first;
>> + FuncsToUse.insert(Fname);
>> + }
>> +}
>> +
>> std::error_code SampleProfileReaderBinary::readSummaryEntry(
>> std::vector<ProfileSummaryEntry> &Entries) {
>> auto Cutoff = readNumber<uint64_t>();
>>
>> Modified: llvm/trunk/lib/ProfileData/SampleProfWriter.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/
>> ProfileData/SampleProfWriter.cpp?rev=342283&r1=342282&r2=342283&view=diff
>> ============================================================
>> ==================
>> --- llvm/trunk/lib/ProfileData/SampleProfWriter.cpp (original)
>> +++ llvm/trunk/lib/ProfileData/SampleProfWriter.cpp Fri Sep 14 13:52:59
>> 2018
>> @@ -22,6 +22,8 @@
>> #include "llvm/ADT/StringRef.h"
>> #include "llvm/ProfileData/ProfileCommon.h"
>> #include "llvm/ProfileData/SampleProf.h"
>> +#include "llvm/Support/Endian.h"
>> +#include "llvm/Support/EndianStream.h"
>> #include "llvm/Support/ErrorOr.h"
>> #include "llvm/Support/FileSystem.h"
>> #include "llvm/Support/LEB128.h"
>> @@ -64,6 +66,15 @@ SampleProfileWriter::write(const StringM
>> return sampleprof_error::success;
>> }
>>
>> +std::error_code SampleProfileWriterCompactBinary::write(
>> + const StringMap<FunctionSamples> &ProfileMap) {
>> + if (std::error_code EC = SampleProfileWriter::write(ProfileMap))
>> + return EC;
>> + if (std::error_code EC = writeFuncOffsetTable())
>> + return EC;
>> + return sampleprof_error::success;
>> +}
>> +
>> /// Write samples to a text file.
>> ///
>> /// Note: it may be tempting to implement this in terms of
>> @@ -168,6 +179,30 @@ std::error_code SampleProfileWriterRawBi
>> return sampleprof_error::success;
>> }
>>
>> +std::error_code SampleProfileWriterCompactBinary::writeFuncOffsetTable()
>> {
>> + auto &OS = *OutputStream;
>> +
>> + // Fill the slot remembered by TableOffset with the offset of
>> FuncOffsetTable.
>> + auto &OFS = static_cast<raw_fd_ostream &>(OS);
>> + uint64_t FuncOffsetTableStart = OS.tell();
>> + if (OFS.seek(TableOffset) == (uint64_t)-1)
>> + return sampleprof_error::ostream_seek_unsupported;
>> + support::endian::Writer Writer(*OutputStream, support::little);
>> + Writer.write(FuncOffsetTableStart);
>> + if (OFS.seek(FuncOffsetTableStart) == (uint64_t)-1)
>> + return sampleprof_error::ostream_seek_unsupported;
>> +
>> + // Write out the table size.
>> + encodeULEB128(FuncOffsetTable.size(), OS);
>> +
>> + // Write out FuncOffsetTable.
>> + for (auto entry : FuncOffsetTable) {
>> + writeNameIdx(entry.first);
>> + encodeULEB128(entry.second, OS);
>> + }
>> + return sampleprof_error::success;
>> +}
>> +
>> std::error_code SampleProfileWriterCompactBinary::writeNameTable() {
>> auto &OS = *OutputStream;
>> std::set<StringRef> V;
>> @@ -215,6 +250,19 @@ std::error_code SampleProfileWriterBinar
>> return sampleprof_error::success;
>> }
>>
>> +std::error_code SampleProfileWriterCompactBinary::writeHeader(
>> + const StringMap<FunctionSamples> &ProfileMap) {
>> + support::endian::Writer Writer(*OutputStream, support::little);
>> + if (auto EC = SampleProfileWriterBinary::writeHeader(ProfileMap))
>> + return EC;
>> +
>> + // Reserve a slot for the offset of function offset table. The slot
>> will
>> + // be populated with the offset of FuncOffsetTable later.
>> + TableOffset = OutputStream->tell();
>> + Writer.write(static_cast<uint64_t>(-2));
>> + return sampleprof_error::success;
>> +}
>> +
>> std::error_code SampleProfileWriterBinary::writeSummary() {
>> auto &OS = *OutputStream;
>> encodeULEB128(Summary->getTotalCount(), OS);
>> @@ -282,6 +330,15 @@ std::error_code SampleProfileWriterBinar
>> encodeULEB128(S.getHeadSamples(), *OutputStream);
>> return writeBody(S);
>> }
>> +
>> +std::error_code
>> +SampleProfileWriterCompactBinary::write(const FunctionSamples &S) {
>> + uint64_t Offset = OutputStream->tell();
>> + StringRef Name = S.getName();
>> + FuncOffsetTable[Name] = Offset;
>> + encodeULEB128(S.getHeadSamples(), *OutputStream);
>> + return writeBody(S);
>> +}
>>
>> /// Create a sample profile file writer based on the specified format.
>> ///
>>
>> Modified: llvm/trunk/lib/Transforms/IPO/SampleProfile.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/
>> Transforms/IPO/SampleProfile.cpp?rev=342283&r1=342282&r2=342283&view=diff
>> ============================================================
>> ==================
>> --- llvm/trunk/lib/Transforms/IPO/SampleProfile.cpp (original)
>> +++ llvm/trunk/lib/Transforms/IPO/SampleProfile.cpp Fri Sep 14 13:52:59
>> 2018
>> @@ -1515,6 +1515,7 @@ bool SampleProfileLoader::doInitializati
>> return false;
>> }
>> Reader = std::move(ReaderOrErr.get());
>> + Reader->collectFuncsToUse(M);
>> ProfileIsValid = (Reader->read() == sampleprof_error::success);
>> return true;
>> }
>>
>> Modified: llvm/trunk/test/Transforms/SampleProfile/Inputs/function_
>> metadata.compact.afdo
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/
>> Transforms/SampleProfile/Inputs/function_metadata.
>> compact.afdo?rev=342283&r1=342282&r2=342283&view=diff
>> ============================================================
>> ==================
>> Binary files - no diff available.
>>
>> Modified: llvm/trunk/test/Transforms/SampleProfile/Inputs/indirect-
>> call.compact.afdo
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/
>> Transforms/SampleProfile/Inputs/indirect-call.compact.
>> afdo?rev=342283&r1=342282&r2=342283&view=diff
>> ============================================================
>> ==================
>> Binary files - no diff available.
>>
>> Modified: llvm/trunk/test/Transforms/SampleProfile/Inputs/inline.
>> compactbinary.afdo
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/
>> Transforms/SampleProfile/Inputs/inline.compactbinary.
>> afdo?rev=342283&r1=342282&r2=342283&view=diff
>> ============================================================
>> ==================
>> Binary files - no diff available.
>>
>> Modified: llvm/trunk/unittests/ProfileData/SampleProfTest.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/
>> ProfileData/SampleProfTest.cpp?rev=342283&r1=342282&r2=342283&view=diff
>> ============================================================
>> ==================
>> --- llvm/trunk/unittests/ProfileData/SampleProfTest.cpp (original)
>> +++ llvm/trunk/unittests/ProfileData/SampleProfTest.cpp Fri Sep 14
>> 13:52:59 2018
>> @@ -36,14 +36,17 @@ static ::testing::AssertionResult NoErro
>> namespace {
>>
>> struct SampleProfTest : ::testing::Test {
>> - std::string Data;
>> LLVMContext Context;
>> + std::string Profile;
>> std::unique_ptr<raw_ostream> OS;
>> std::unique_ptr<SampleProfileWriter> Writer;
>> std::unique_ptr<SampleProfileReader> Reader;
>> + std::error_code EC;
>>
>> SampleProfTest()
>> - : Data(), OS(new raw_string_ostream(Data)), Writer(), Reader() {}
>> + : Profile("profile"),
>> + OS(new raw_fd_ostream(Profile, EC, sys::fs::F_None)), Writer(),
>> + Reader() {}
>>
>> void createWriter(SampleProfileFormat Format) {
>> auto WriterOrErr = SampleProfileWriter::create(OS, Format);
>> @@ -51,10 +54,11 @@ struct SampleProfTest : ::testing::Test
>> Writer = std::move(WriterOrErr.get());
>> }
>>
>> - void readProfile(std::unique_ptr<MemoryBuffer> &Profile) {
>> + void readProfile(const Module &M) {
>> auto ReaderOrErr = SampleProfileReader::create(Profile, Context);
>> ASSERT_TRUE(NoError(ReaderOrErr.getError()));
>> Reader = std::move(ReaderOrErr.get());
>> + Reader->collectFuncsToUse(M);
>> }
>>
>> void testRoundTrip(SampleProfileFormat Format) {
>> @@ -83,6 +87,12 @@ struct SampleProfTest : ::testing::Test
>> BarSamples.addCalledTargetSamples(1, 0, MconstructName, 1000);
>> BarSamples.addCalledTargetSamples(1, 0, StringviewName, 437);
>>
>> + Module M("my_module", Context);
>> + FunctionType *fn_type =
>> + FunctionType::get(Type::getVoidTy(Context), {}, false);
>> + M.getOrInsertFunction(FooName, fn_type);
>> + M.getOrInsertFunction(BarName, fn_type);
>> +
>> StringMap<FunctionSamples> Profiles;
>> Profiles[FooName] = std::move(FooSamples);
>> Profiles[BarName] = std::move(BarSamples);
>> @@ -93,8 +103,7 @@ struct SampleProfTest : ::testing::Test
>>
>> Writer->getOutputStream().flush();
>>
>> - auto Profile = MemoryBuffer::getMemBufferCopy(Data);
>> - readProfile(Profile);
>> + readProfile(M);
>>
>> EC = Reader->read();
>> ASSERT_TRUE(NoError(EC));
>> @@ -164,7 +173,6 @@ struct SampleProfTest : ::testing::Test
>> delete PS;
>>
>> // Test that summary can be attached to and read back from module.
>> - Module M("my_module", Context);
>> M.setProfileSummary(MD);
>> MD = M.getProfileSummary();
>> ASSERT_TRUE(MD);
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20180914/765b25b1/attachment.html>
More information about the llvm-commits
mailing list