[llvm] r342283 - [SampleFDO] Add FunctionOffsetTable in compact binary format profile.
Wei Mi via llvm-commits
llvm-commits at lists.llvm.org
Fri Sep 14 17:06:13 PDT 2018
Should be fixed in rL342301.
Thanks!
Wei.
On Fri, Sep 14, 2018 at 4:35 PM, Wei Mi <wmi at google.com> wrote:
> There is file system race indeed. I will fix that. Thanks Reid!
>
> Wei.
>
> On Fri, Sep 14, 2018 at 4:09 PM, Reid Kleckner <rnk at google.com> wrote:
>
>> These tests seem flaky when I run them locally. Are there possible
>> filesystem race issues here?
>>
>> FAIL: LLVM-Unit :: ProfileData/./ProfileDataTests
>> .exe/SampleProfTest.roundtrip_compact_binary_profile (2329 of 27766)
>> ******************** TEST 'LLVM-Unit :: ProfileData/./ProfileDataTests
>> .exe/SampleProfTest.roundtrip_compact_binary_profile' FAILED
>> ********************
>> Note: Google Test filter = SampleProfTest.roundtrip_compa
>> ct_binary_profile
>> [==========] Running 1 test from 1 test case.
>> [----------] Global test environment set-up.
>> [----------] 1 test from SampleProfTest
>> [ RUN ] SampleProfTest.roundtrip_compact_binary_profile
>> C:\src\llvm-project\llvm\unittests\ProfileData\SampleProfTest.cpp(59):
>> error: Value of: NoError(ReaderOrErr.getError())
>> Actual: false (error 6: Unrecognized sample profile encoding format)
>> Expected: true
>> unknown file: error: SEH exception with code 0x3221225477 thrown in the
>> test body.
>> unknown file: error: SEH exception with code 0x3221225477 thrown in the
>> test fixture's destructor.
>> [ FAILED ] SampleProfTest.roundtrip_compact_binary_profile (2 ms)
>> [----------] 1 test from SampleProfTest (2 ms total)
>>
>> [----------] Global test environment tear-down
>> [==========] 1 test from 1 test case ran. (2 ms total)
>> [ PASSED ] 0 tests.
>> [ FAILED ] 1 test, listed below:
>> [ FAILED ] SampleProfTest.roundtrip_compact_binary_profile
>>
>> 1 FAILED TEST
>>
>> ********************
>> Testing: 0 ..
>> FAIL: LLVM-Unit :: ProfileData/./ProfileDataTests
>> .exe/SampleProfTest.roundtrip_text_profile (2330 of 27766)
>> ******************** TEST 'LLVM-Unit :: ProfileData/./ProfileDataTests
>> .exe/SampleProfTest.roundtrip_text_profile' FAILED ********************
>> Note: Google Test filter = SampleProfTest.roundtrip_text_profile
>> [==========] Running 1 test from 1 test case.
>> [----------] Global test environment set-up.
>> [----------] 1 test from SampleProfTest
>> [ RUN ] SampleProfTest.roundtrip_text_profile
>> C:\src\llvm-project\llvm\unittests\ProfileData\SampleProfTest.cpp(59):
>> error: Value of: NoError(ReaderOrErr.getError())
>> Actual: false (error 6: Unrecognized sample profile encoding format)
>> Expected: true
>> unknown file: error: SEH exception with code 0x3221225477 thrown in the
>> test body.
>> unknown file: error: SEH exception with code 0x3221225477 thrown in the
>> test fixture's destructor.
>> [ FAILED ] SampleProfTest.roundtrip_text_profile (2 ms)
>> [----------] 1 test from SampleProfTest (2 ms total)
>>
>> [----------] Global test environment tear-down
>> [==========] 1 test from 1 test case ran. (2 ms total)
>> [ PASSED ] 0 tests.
>> [ FAILED ] 1 test, listed below:
>> [ FAILED ] SampleProfTest.roundtrip_text_profile
>>
>> 1 FAILED TEST
>>
>> ********************
>>
>> On Fri, Sep 14, 2018 at 1:54 PM Wei Mi via llvm-commits <
>> llvm-commits at lists.llvm.org> wrote:
>>
>>> Author: wmi
>>> Date: Fri Sep 14 13:52:59 2018
>>> New Revision: 342283
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=342283&view=rev
>>> Log:
>>> [SampleFDO] Add FunctionOffsetTable in compact binary format profile.
>>>
>>> The patch saves a function offset table which maps function name index
>>> to the
>>> offset of its function profile to the start of the binary profile. By
>>> using
>>> the function offset table, for those function profiles which will not be
>>> used
>>> when compiling a module, the profile reader does't have to read them. For
>>> profile size around 10~20M, it saves ~10% compile time.
>>>
>>> Differential Revision: https://reviews.llvm.org/D51863
>>>
>>>
>>> Modified:
>>> llvm/trunk/include/llvm/ProfileData/SampleProf.h
>>> llvm/trunk/include/llvm/ProfileData/SampleProfReader.h
>>> llvm/trunk/include/llvm/ProfileData/SampleProfWriter.h
>>> llvm/trunk/lib/ProfileData/SampleProf.cpp
>>> llvm/trunk/lib/ProfileData/SampleProfReader.cpp
>>> llvm/trunk/lib/ProfileData/SampleProfWriter.cpp
>>> llvm/trunk/lib/Transforms/IPO/SampleProfile.cpp
>>> llvm/trunk/test/Transforms/SampleProfile/Inputs/function_met
>>> adata.compact.afdo
>>> llvm/trunk/test/Transforms/SampleProfile/Inputs/indirect-cal
>>> l.compact.afdo
>>> llvm/trunk/test/Transforms/SampleProfile/Inputs/inline.compa
>>> ctbinary.afdo
>>> llvm/trunk/unittests/ProfileData/SampleProfTest.cpp
>>>
>>> Modified: llvm/trunk/include/llvm/ProfileData/SampleProf.h
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/
>>> ProfileData/SampleProf.h?rev=342283&r1=342282&r2=342283&view=diff
>>> ============================================================
>>> ==================
>>> --- llvm/trunk/include/llvm/ProfileData/SampleProf.h (original)
>>> +++ llvm/trunk/include/llvm/ProfileData/SampleProf.h Fri Sep 14
>>> 13:52:59 2018
>>> @@ -49,7 +49,8 @@ enum class sampleprof_error {
>>> unsupported_writing_format,
>>> truncated_name_table,
>>> not_implemented,
>>> - counter_overflow
>>> + counter_overflow,
>>> + ostream_seek_unsupported
>>> };
>>>
>>> inline std::error_code make_error_code(sampleprof_error E) {
>>>
>>> Modified: llvm/trunk/include/llvm/ProfileData/SampleProfReader.h
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/
>>> ProfileData/SampleProfReader.h?rev=342283&r1=342282&r2=342283&view=diff
>>> ============================================================
>>> ==================
>>> --- llvm/trunk/include/llvm/ProfileData/SampleProfReader.h (original)
>>> +++ llvm/trunk/include/llvm/ProfileData/SampleProfReader.h Fri Sep 14
>>> 13:52:59 2018
>>> @@ -279,6 +279,8 @@ public:
>>> /// Print the profile for \p FName on stream \p OS.
>>> void dumpFunctionProfile(StringRef FName, raw_ostream &OS = dbgs());
>>>
>>> + virtual void collectFuncsToUse(const Module &M) {}
>>> +
>>> /// Print all the profiles on stream \p OS.
>>> void dump(raw_ostream &OS = dbgs());
>>>
>>> @@ -364,7 +366,7 @@ public:
>>> : SampleProfileReader(std::move(B), C, Format) {}
>>>
>>> /// Read and validate the file header.
>>> - std::error_code readHeader() override;
>>> + virtual std::error_code readHeader() override;
>>>
>>> /// Read sample profiles from the associated file.
>>> std::error_code read() override;
>>> @@ -378,6 +380,10 @@ protected:
>>> /// \returns the read value.
>>> template <typename T> ErrorOr<T> readNumber();
>>>
>>> + /// Read a numeric value of type T from the profile. The value is
>>> saved
>>> + /// without encoded.
>>> + template <typename T> ErrorOr<T> readUnencodedNumber();
>>> +
>>> /// Read a string from the profile.
>>> ///
>>> /// If an error occurs during decoding, a diagnostic message is
>>> emitted and
>>> @@ -392,6 +398,9 @@ protected:
>>> /// Return true if we've reached the end of file.
>>> bool at_eof() const { return Data >= End; }
>>>
>>> + /// Read the next function profile instance.
>>> + std::error_code readFuncProfile();
>>> +
>>> /// Read the contents of the given profile instance.
>>> std::error_code readProfile(FunctionSamples &FProfile);
>>>
>>> @@ -436,10 +445,17 @@ class SampleProfileReaderCompactBinary :
>>> private:
>>> /// Function name table.
>>> std::vector<std::string> NameTable;
>>> + /// The table mapping from function name to the offset of its
>>> FunctionSample
>>> + /// towards file start.
>>> + DenseMap<StringRef, uint64_t> FuncOffsetTable;
>>> + /// The set containing the functions to use when compiling a module.
>>> + DenseSet<StringRef> FuncsToUse;
>>> virtual std::error_code verifySPMagic(uint64_t Magic) override;
>>> virtual std::error_code readNameTable() override;
>>> /// Read a string indirectly via the name table.
>>> virtual ErrorOr<StringRef> readStringFromTable() override;
>>> + virtual std::error_code readHeader() override;
>>> + std::error_code readFuncOffsetTable();
>>>
>>> public:
>>> SampleProfileReaderCompactBinary(std::unique_ptr<MemoryBuffer> B,
>>> @@ -448,6 +464,12 @@ public:
>>>
>>> /// \brief Return true if \p Buffer is in the format supported by
>>> this class.
>>> static bool hasFormat(const MemoryBuffer &Buffer);
>>> +
>>> + /// Read samples only for functions to use.
>>> + std::error_code read() override;
>>> +
>>> + /// Collect functions to be used when compiling Module \p M.
>>> + void collectFuncsToUse(const Module &M) override;
>>> };
>>>
>>> using InlineCallStack = SmallVector<FunctionSamples *, 10>;
>>>
>>> Modified: llvm/trunk/include/llvm/ProfileData/SampleProfWriter.h
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/
>>> ProfileData/SampleProfWriter.h?rev=342283&r1=342282&r2=342283&view=diff
>>> ============================================================
>>> ==================
>>> --- llvm/trunk/include/llvm/ProfileData/SampleProfWriter.h (original)
>>> +++ llvm/trunk/include/llvm/ProfileData/SampleProfWriter.h Fri Sep 14
>>> 13:52:59 2018
>>> @@ -42,7 +42,7 @@ public:
>>> /// Write all the sample profiles in the given map of samples.
>>> ///
>>> /// \returns status code of the file update operation.
>>> - std::error_code write(const StringMap<FunctionSamples> &ProfileMap);
>>> + virtual std::error_code write(const StringMap<FunctionSamples>
>>> &ProfileMap);
>>>
>>> raw_ostream &getOutputStream() { return *OutputStream; }
>>>
>>> @@ -103,14 +103,15 @@ private:
>>> /// Sample-based profile writer (binary format).
>>> class SampleProfileWriterBinary : public SampleProfileWriter {
>>> public:
>>> - std::error_code write(const FunctionSamples &S) override;
>>> + virtual std::error_code write(const FunctionSamples &S) override;
>>> SampleProfileWriterBinary(std::unique_ptr<raw_ostream> &OS)
>>> : SampleProfileWriter(OS) {}
>>>
>>> protected:
>>> virtual std::error_code writeNameTable() = 0;
>>> virtual std::error_code writeMagicIdent() = 0;
>>> - std::error_code writeHeader(const StringMap<FunctionSamples>
>>> &ProfileMap) override;
>>> + virtual std::error_code
>>> + writeHeader(const StringMap<FunctionSamples> &ProfileMap) override;
>>> std::error_code writeSummary();
>>> std::error_code writeNameIdx(StringRef FName);
>>> std::error_code writeBody(const FunctionSamples &S);
>>> @@ -135,12 +136,56 @@ protected:
>>> virtual std::error_code writeMagicIdent() override;
>>> };
>>>
>>> +// CompactBinary is a compact format of binary profile which both
>>> reduces
>>> +// the profile size and the load time needed when compiling. It has two
>>> +// major difference with Binary format.
>>> +// 1. It represents all the strings in name table using md5 hash.
>>> +// 2. It saves a function offset table which maps function name index to
>>> +// the offset of its function profile to the start of the binary
>>> profile,
>>> +// so by using the function offset table, for those function profiles
>>> which
>>> +// will not be needed when compiling a module, the profile reader does't
>>> +// have to read them and it saves compile time if the profile size is
>>> huge.
>>> +// The layout of the compact format is shown as follows:
>>> +//
>>> +// Part1: Profile header, the same as binary format, containing magic
>>> +// number, version, summary, name table...
>>> +// Part2: Function Offset Table Offset, which saves the position of
>>> +// Part4.
>>> +// Part3: Function profile collection
>>> +// function1 profile start
>>> +// ....
>>> +// function2 profile start
>>> +// ....
>>> +// function3 profile start
>>> +// ....
>>> +// ......
>>> +// Part4: Function Offset Table
>>> +// function1 name index --> function1 profile start
>>> +// function2 name index --> function2 profile start
>>> +// function3 name index --> function3 profile start
>>> +//
>>> +// We need Part2 because profile reader can use it to find out and read
>>> +// function offset table without reading Part3 first.
>>> class SampleProfileWriterCompactBinary : public
>>> SampleProfileWriterBinary {
>>> using SampleProfileWriterBinary::SampleProfileWriterBinary;
>>>
>>> +public:
>>> + virtual std::error_code write(const FunctionSamples &S) override;
>>> + virtual std::error_code
>>> + write(const StringMap<FunctionSamples> &ProfileMap) override;
>>> +
>>> protected:
>>> + /// The table mapping from function name to the offset of its
>>> FunctionSample
>>> + /// towards profile start.
>>> + MapVector<StringRef, uint64_t> FuncOffsetTable;
>>> + /// The offset of the slot to be filled with the offset of
>>> FuncOffsetTable
>>> + /// towards profile start.
>>> + uint64_t TableOffset;
>>> virtual std::error_code writeNameTable() override;
>>> virtual std::error_code writeMagicIdent() override;
>>> + virtual std::error_code
>>> + writeHeader(const StringMap<FunctionSamples> &ProfileMap) override;
>>> + std::error_code writeFuncOffsetTable();
>>> };
>>>
>>> } // end namespace sampleprof
>>>
>>> Modified: llvm/trunk/lib/ProfileData/SampleProf.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ProfileDa
>>> ta/SampleProf.cpp?rev=342283&r1=342282&r2=342283&view=diff
>>> ============================================================
>>> ==================
>>> --- llvm/trunk/lib/ProfileData/SampleProf.cpp (original)
>>> +++ llvm/trunk/lib/ProfileData/SampleProf.cpp Fri Sep 14 13:52:59 2018
>>> @@ -67,6 +67,8 @@ class SampleProfErrorCategoryType : publ
>>> return "Unimplemented feature";
>>> case sampleprof_error::counter_overflow:
>>> return "Counter overflow";
>>> + case sampleprof_error::ostream_seek_unsupported:
>>> + return "Ostream does not support seek";
>>> }
>>> llvm_unreachable("A value of sampleprof_error has no message.");
>>> }
>>>
>>> Modified: llvm/trunk/lib/ProfileData/SampleProfReader.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ProfileDa
>>> ta/SampleProfReader.cpp?rev=342283&r1=342282&r2=342283&view=diff
>>> ============================================================
>>> ==================
>>> --- llvm/trunk/lib/ProfileData/SampleProfReader.cpp (original)
>>> +++ llvm/trunk/lib/ProfileData/SampleProfReader.cpp Fri Sep 14 13:52:59
>>> 2018
>>> @@ -30,6 +30,7 @@
>>> #include "llvm/Support/ErrorOr.h"
>>> #include "llvm/Support/LEB128.h"
>>> #include "llvm/Support/LineIterator.h"
>>> +#include "llvm/Support/MD5.h"
>>> #include "llvm/Support/MemoryBuffer.h"
>>> #include "llvm/Support/raw_ostream.h"
>>> #include <algorithm>
>>> @@ -320,6 +321,21 @@ ErrorOr<StringRef> SampleProfileReaderBi
>>> }
>>>
>>> template <typename T>
>>> +ErrorOr<T> SampleProfileReaderBinary::readUnencodedNumber() {
>>> + std::error_code EC;
>>> +
>>> + if (Data + sizeof(T) > End) {
>>> + EC = sampleprof_error::truncated;
>>> + reportError(0, EC.message());
>>> + return EC;
>>> + }
>>> +
>>> + using namespace support;
>>> + T Val = endian::readNext<T, little, unaligned>(Data);
>>> + return Val;
>>> +}
>>> +
>>> +template <typename T>
>>> inline ErrorOr<uint32_t> SampleProfileReaderBinary::readStringIndex(T
>>> &Table) {
>>> std::error_code EC;
>>> auto Idx = readNumber<uint32_t>();
>>> @@ -423,29 +439,51 @@ SampleProfileReaderBinary::readProfile(F
>>> return sampleprof_error::success;
>>> }
>>>
>>> -std::error_code SampleProfileReaderBinary::read() {
>>> - while (!at_eof()) {
>>> - auto NumHeadSamples = readNumber<uint64_t>();
>>> - if (std::error_code EC = NumHeadSamples.getError())
>>> - return EC;
>>> +std::error_code SampleProfileReaderBinary::readFuncProfile() {
>>> + auto NumHeadSamples = readNumber<uint64_t>();
>>> + if (std::error_code EC = NumHeadSamples.getError())
>>> + return EC;
>>>
>>> - auto FName(readStringFromTable());
>>> - if (std::error_code EC = FName.getError())
>>> - return EC;
>>> + auto FName(readStringFromTable());
>>> + if (std::error_code EC = FName.getError())
>>> + return EC;
>>>
>>> - Profiles[*FName] = FunctionSamples();
>>> - FunctionSamples &FProfile = Profiles[*FName];
>>> - FProfile.setName(*FName);
>>> + Profiles[*FName] = FunctionSamples();
>>> + FunctionSamples &FProfile = Profiles[*FName];
>>> + FProfile.setName(*FName);
>>>
>>> - FProfile.addHeadSamples(*NumHeadSamples);
>>> + FProfile.addHeadSamples(*NumHeadSamples);
>>> +
>>> + if (std::error_code EC = readProfile(FProfile))
>>> + return EC;
>>> + return sampleprof_error::success;
>>> +}
>>>
>>> - if (std::error_code EC = readProfile(FProfile))
>>> +std::error_code SampleProfileReaderBinary::read() {
>>> + while (!at_eof()) {
>>> + if (std::error_code EC = readFuncProfile())
>>> return EC;
>>> }
>>>
>>> return sampleprof_error::success;
>>> }
>>>
>>> +std::error_code SampleProfileReaderCompactBinary::read() {
>>> + for (auto Name : FuncsToUse) {
>>> + auto GUID = std::to_string(MD5Hash(Name));
>>> + auto iter = FuncOffsetTable.find(StringRef(GUID));
>>> + if (iter == FuncOffsetTable.end())
>>> + continue;
>>> + const uint8_t *SavedData = Data;
>>> + Data = reinterpret_cast<const uint8_t *>(Buffer->getBufferStart()) +
>>> + iter->second;
>>> + if (std::error_code EC = readFuncProfile())
>>> + return EC;
>>> + Data = SavedData;
>>> + }
>>> + return sampleprof_error::success;
>>> +}
>>> +
>>> std::error_code SampleProfileReaderRawBinary::verifySPMagic(uint64_t
>>> Magic) {
>>> if (Magic == SPMagic())
>>> return sampleprof_error::success;
>>> @@ -514,6 +552,53 @@ std::error_code SampleProfileReaderBinar
>>> return sampleprof_error::success;
>>> }
>>>
>>> +std::error_code SampleProfileReaderCompactBinary::readHeader() {
>>> + SampleProfileReaderBinary::readHeader();
>>> + if (std::error_code EC = readFuncOffsetTable())
>>> + return EC;
>>> + return sampleprof_error::success;
>>> +}
>>> +
>>> +std::error_code SampleProfileReaderCompactBinary::readFuncOffsetTable()
>>> {
>>> + auto TableOffset = readUnencodedNumber<uint64_t>();
>>> + if (std::error_code EC = TableOffset.getError())
>>> + return EC;
>>> +
>>> + const uint8_t *SavedData = Data;
>>> + const uint8_t *TableStart =
>>> + reinterpret_cast<const uint8_t *>(Buffer->getBufferStart()) +
>>> + *TableOffset;
>>> + Data = TableStart;
>>> +
>>> + auto Size = readNumber<uint64_t>();
>>> + if (std::error_code EC = Size.getError())
>>> + return EC;
>>> +
>>> + FuncOffsetTable.reserve(*Size);
>>> + for (uint32_t I = 0; I < *Size; ++I) {
>>> + auto FName(readStringFromTable());
>>> + if (std::error_code EC = FName.getError())
>>> + return EC;
>>> +
>>> + auto Offset = readNumber<uint64_t>();
>>> + if (std::error_code EC = Offset.getError())
>>> + return EC;
>>> +
>>> + FuncOffsetTable[*FName] = *Offset;
>>> + }
>>> + End = TableStart;
>>> + Data = SavedData;
>>> + return sampleprof_error::success;
>>> +}
>>> +
>>> +void SampleProfileReaderCompactBinary::collectFuncsToUse(const Module
>>> &M) {
>>> + FuncsToUse.clear();
>>> + for (auto &F : M) {
>>> + StringRef Fname = F.getName().split('.').first;
>>> + FuncsToUse.insert(Fname);
>>> + }
>>> +}
>>> +
>>> std::error_code SampleProfileReaderBinary::readSummaryEntry(
>>> std::vector<ProfileSummaryEntry> &Entries) {
>>> auto Cutoff = readNumber<uint64_t>();
>>>
>>> Modified: llvm/trunk/lib/ProfileData/SampleProfWriter.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ProfileDa
>>> ta/SampleProfWriter.cpp?rev=342283&r1=342282&r2=342283&view=diff
>>> ============================================================
>>> ==================
>>> --- llvm/trunk/lib/ProfileData/SampleProfWriter.cpp (original)
>>> +++ llvm/trunk/lib/ProfileData/SampleProfWriter.cpp Fri Sep 14 13:52:59
>>> 2018
>>> @@ -22,6 +22,8 @@
>>> #include "llvm/ADT/StringRef.h"
>>> #include "llvm/ProfileData/ProfileCommon.h"
>>> #include "llvm/ProfileData/SampleProf.h"
>>> +#include "llvm/Support/Endian.h"
>>> +#include "llvm/Support/EndianStream.h"
>>> #include "llvm/Support/ErrorOr.h"
>>> #include "llvm/Support/FileSystem.h"
>>> #include "llvm/Support/LEB128.h"
>>> @@ -64,6 +66,15 @@ SampleProfileWriter::write(const StringM
>>> return sampleprof_error::success;
>>> }
>>>
>>> +std::error_code SampleProfileWriterCompactBinary::write(
>>> + const StringMap<FunctionSamples> &ProfileMap) {
>>> + if (std::error_code EC = SampleProfileWriter::write(ProfileMap))
>>> + return EC;
>>> + if (std::error_code EC = writeFuncOffsetTable())
>>> + return EC;
>>> + return sampleprof_error::success;
>>> +}
>>> +
>>> /// Write samples to a text file.
>>> ///
>>> /// Note: it may be tempting to implement this in terms of
>>> @@ -168,6 +179,30 @@ std::error_code SampleProfileWriterRawBi
>>> return sampleprof_error::success;
>>> }
>>>
>>> +std::error_code SampleProfileWriterCompactBinary::writeFuncOffsetTable()
>>> {
>>> + auto &OS = *OutputStream;
>>> +
>>> + // Fill the slot remembered by TableOffset with the offset of
>>> FuncOffsetTable.
>>> + auto &OFS = static_cast<raw_fd_ostream &>(OS);
>>> + uint64_t FuncOffsetTableStart = OS.tell();
>>> + if (OFS.seek(TableOffset) == (uint64_t)-1)
>>> + return sampleprof_error::ostream_seek_unsupported;
>>> + support::endian::Writer Writer(*OutputStream, support::little);
>>> + Writer.write(FuncOffsetTableStart);
>>> + if (OFS.seek(FuncOffsetTableStart) == (uint64_t)-1)
>>> + return sampleprof_error::ostream_seek_unsupported;
>>> +
>>> + // Write out the table size.
>>> + encodeULEB128(FuncOffsetTable.size(), OS);
>>> +
>>> + // Write out FuncOffsetTable.
>>> + for (auto entry : FuncOffsetTable) {
>>> + writeNameIdx(entry.first);
>>> + encodeULEB128(entry.second, OS);
>>> + }
>>> + return sampleprof_error::success;
>>> +}
>>> +
>>> std::error_code SampleProfileWriterCompactBinary::writeNameTable() {
>>> auto &OS = *OutputStream;
>>> std::set<StringRef> V;
>>> @@ -215,6 +250,19 @@ std::error_code SampleProfileWriterBinar
>>> return sampleprof_error::success;
>>> }
>>>
>>> +std::error_code SampleProfileWriterCompactBinary::writeHeader(
>>> + const StringMap<FunctionSamples> &ProfileMap) {
>>> + support::endian::Writer Writer(*OutputStream, support::little);
>>> + if (auto EC = SampleProfileWriterBinary::writeHeader(ProfileMap))
>>> + return EC;
>>> +
>>> + // Reserve a slot for the offset of function offset table. The slot
>>> will
>>> + // be populated with the offset of FuncOffsetTable later.
>>> + TableOffset = OutputStream->tell();
>>> + Writer.write(static_cast<uint64_t>(-2));
>>> + return sampleprof_error::success;
>>> +}
>>> +
>>> std::error_code SampleProfileWriterBinary::writeSummary() {
>>> auto &OS = *OutputStream;
>>> encodeULEB128(Summary->getTotalCount(), OS);
>>> @@ -282,6 +330,15 @@ std::error_code SampleProfileWriterBinar
>>> encodeULEB128(S.getHeadSamples(), *OutputStream);
>>> return writeBody(S);
>>> }
>>> +
>>> +std::error_code
>>> +SampleProfileWriterCompactBinary::write(const FunctionSamples &S) {
>>> + uint64_t Offset = OutputStream->tell();
>>> + StringRef Name = S.getName();
>>> + FuncOffsetTable[Name] = Offset;
>>> + encodeULEB128(S.getHeadSamples(), *OutputStream);
>>> + return writeBody(S);
>>> +}
>>>
>>> /// Create a sample profile file writer based on the specified format.
>>> ///
>>>
>>> Modified: llvm/trunk/lib/Transforms/IPO/SampleProfile.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transform
>>> s/IPO/SampleProfile.cpp?rev=342283&r1=342282&r2=342283&view=diff
>>> ============================================================
>>> ==================
>>> --- llvm/trunk/lib/Transforms/IPO/SampleProfile.cpp (original)
>>> +++ llvm/trunk/lib/Transforms/IPO/SampleProfile.cpp Fri Sep 14 13:52:59
>>> 2018
>>> @@ -1515,6 +1515,7 @@ bool SampleProfileLoader::doInitializati
>>> return false;
>>> }
>>> Reader = std::move(ReaderOrErr.get());
>>> + Reader->collectFuncsToUse(M);
>>> ProfileIsValid = (Reader->read() == sampleprof_error::success);
>>> return true;
>>> }
>>>
>>> Modified: llvm/trunk/test/Transforms/SampleProfile/Inputs/function_met
>>> adata.compact.afdo
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transfor
>>> ms/SampleProfile/Inputs/function_metadata.compact.
>>> afdo?rev=342283&r1=342282&r2=342283&view=diff
>>> ============================================================
>>> ==================
>>> Binary files - no diff available.
>>>
>>> Modified: llvm/trunk/test/Transforms/SampleProfile/Inputs/indirect-cal
>>> l.compact.afdo
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transfor
>>> ms/SampleProfile/Inputs/indirect-call.compact.afdo?
>>> rev=342283&r1=342282&r2=342283&view=diff
>>> ============================================================
>>> ==================
>>> Binary files - no diff available.
>>>
>>> Modified: llvm/trunk/test/Transforms/SampleProfile/Inputs/inline.compa
>>> ctbinary.afdo
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transfor
>>> ms/SampleProfile/Inputs/inline.compactbinary.afdo?rev=
>>> 342283&r1=342282&r2=342283&view=diff
>>> ============================================================
>>> ==================
>>> Binary files - no diff available.
>>>
>>> Modified: llvm/trunk/unittests/ProfileData/SampleProfTest.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/Pro
>>> fileData/SampleProfTest.cpp?rev=342283&r1=342282&r2=342283&view=diff
>>> ============================================================
>>> ==================
>>> --- llvm/trunk/unittests/ProfileData/SampleProfTest.cpp (original)
>>> +++ llvm/trunk/unittests/ProfileData/SampleProfTest.cpp Fri Sep 14
>>> 13:52:59 2018
>>> @@ -36,14 +36,17 @@ static ::testing::AssertionResult NoErro
>>> namespace {
>>>
>>> struct SampleProfTest : ::testing::Test {
>>> - std::string Data;
>>> LLVMContext Context;
>>> + std::string Profile;
>>> std::unique_ptr<raw_ostream> OS;
>>> std::unique_ptr<SampleProfileWriter> Writer;
>>> std::unique_ptr<SampleProfileReader> Reader;
>>> + std::error_code EC;
>>>
>>> SampleProfTest()
>>> - : Data(), OS(new raw_string_ostream(Data)), Writer(), Reader() {}
>>> + : Profile("profile"),
>>> + OS(new raw_fd_ostream(Profile, EC, sys::fs::F_None)), Writer(),
>>> + Reader() {}
>>>
>>> void createWriter(SampleProfileFormat Format) {
>>> auto WriterOrErr = SampleProfileWriter::create(OS, Format);
>>> @@ -51,10 +54,11 @@ struct SampleProfTest : ::testing::Test
>>> Writer = std::move(WriterOrErr.get());
>>> }
>>>
>>> - void readProfile(std::unique_ptr<MemoryBuffer> &Profile) {
>>> + void readProfile(const Module &M) {
>>> auto ReaderOrErr = SampleProfileReader::create(Profile, Context);
>>> ASSERT_TRUE(NoError(ReaderOrErr.getError()));
>>> Reader = std::move(ReaderOrErr.get());
>>> + Reader->collectFuncsToUse(M);
>>> }
>>>
>>> void testRoundTrip(SampleProfileFormat Format) {
>>> @@ -83,6 +87,12 @@ struct SampleProfTest : ::testing::Test
>>> BarSamples.addCalledTargetSamples(1, 0, MconstructName, 1000);
>>> BarSamples.addCalledTargetSamples(1, 0, StringviewName, 437);
>>>
>>> + Module M("my_module", Context);
>>> + FunctionType *fn_type =
>>> + FunctionType::get(Type::getVoidTy(Context), {}, false);
>>> + M.getOrInsertFunction(FooName, fn_type);
>>> + M.getOrInsertFunction(BarName, fn_type);
>>> +
>>> StringMap<FunctionSamples> Profiles;
>>> Profiles[FooName] = std::move(FooSamples);
>>> Profiles[BarName] = std::move(BarSamples);
>>> @@ -93,8 +103,7 @@ struct SampleProfTest : ::testing::Test
>>>
>>> Writer->getOutputStream().flush();
>>>
>>> - auto Profile = MemoryBuffer::getMemBufferCopy(Data);
>>> - readProfile(Profile);
>>> + readProfile(M);
>>>
>>> EC = Reader->read();
>>> ASSERT_TRUE(NoError(EC));
>>> @@ -164,7 +173,6 @@ struct SampleProfTest : ::testing::Test
>>> delete PS;
>>>
>>> // Test that summary can be attached to and read back from module.
>>> - Module M("my_module", Context);
>>> M.setProfileSummary(MD);
>>> MD = M.getProfileSummary();
>>> ASSERT_TRUE(MD);
>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20180914/0e3e60b4/attachment.html>
More information about the llvm-commits
mailing list