[llvm] r255523 - [PGO] Value profiling text format reader/writer support
Evgenii Stepanov via llvm-commits
llvm-commits at lists.llvm.org
Tue Dec 15 12:58:28 PST 2015
Maybe this would help?
Uninitialized bytes in __interceptor_write at offset 276 inside
[0x74000000e000, 864)
==145927==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x955319 in llvm::raw_fd_ostream::write_impl(char const*,
unsigned long) /code/llvm/lib/Support/raw_ostream.cpp:575:13
#1 0x95562b in flush_nonempty /code/llvm/lib/Support/raw_ostream.cpp:273:3
#2 0x95562b in flush /code/llvm/include/llvm/Support/raw_ostream.h:136
#3 0x95562b in llvm::raw_fd_ostream::seek(unsigned long)
/code/llvm/lib/Support/raw_ostream.cpp:626
#4 0x84e025 in llvm::InstrProfWriter::write(llvm::raw_fd_ostream&)
/code/llvm/lib/ProfileData/InstrProfWriter.cpp:170:3
#5 0x48fea4 in mergeInstrProfile
/code/llvm/tools/llvm-profdata/llvm-profdata.cpp:129:5
#6 0x48fea4 in merge_main(int, char const**)
/code/llvm/tools/llvm-profdata/llvm-profdata.cpp:200
#7 0x489af5 in main /code/llvm/tools/llvm-profdata/llvm-profdata.cpp:379:14
#8 0x7f4c2aec0ec4 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x21ec4)
#9 0x41edf6 in _start (/code/build-llvm-msan/bin/llvm-profdata+0x41edf6)
Uninitialized value was stored to memory at
#0 0x43c077 in __msan_memcpy
/code/llvm/projects/compiler-rt/lib/msan/msan_interceptors.cc:1491:3
#1 0x94f6cc in copy_to_buffer /code/llvm/lib/Support/raw_ostream.cpp:351:5
#2 0x94f6cc in llvm::raw_ostream::write(char const*, unsigned
long) /code/llvm/lib/Support/raw_ostream.cpp:334
#3 0x84ad52 in EmitData /code/llvm/lib/ProfileData/InstrProfWriter.cpp:85:7
#4 0x84ad52 in Emit /code/llvm/include/llvm/Support/OnDiskHashTable.h:183
#5 0x84ad52 in Emit /code/llvm/include/llvm/Support/OnDiskHashTable.h:145
#6 0x84ad52 in
llvm::InstrProfWriter::writeImpl(llvm::raw_ostream&)
/code/llvm/lib/ProfileData/InstrProfWriter.cpp:159
#7 0x84dfd6 in llvm::InstrProfWriter::write(llvm::raw_fd_ostream&)
/code/llvm/lib/ProfileData/InstrProfWriter.cpp:166:21
#8 0x48fea4 in mergeInstrProfile
/code/llvm/tools/llvm-profdata/llvm-profdata.cpp:129:5
#9 0x48fea4 in merge_main(int, char const**)
/code/llvm/tools/llvm-profdata/llvm-profdata.cpp:200
#10 0x489af5 in main /code/llvm/tools/llvm-profdata/llvm-profdata.cpp:379:14
#11 0x7f4c2aec0ec4 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x21ec4)
Uninitialized value was created by a heap allocation
#0 0x488ca2 in operator new(unsigned long)
/code/llvm/projects/compiler-rt/lib/msan/msan_new_delete.cc:35:35
#1 0x8154c9 in llvm::allocValueProfDataInstrProf(unsigned long)
/code/llvm/lib/ProfileData/InstrProf.cpp:205:33
#2 0x8141f1 in
llvm::serializeValueProfDataFrom(llvm::ValueProfRecordClosure*,
llvm::ValueProfData*)
/code/llvm/include/llvm/ProfileData/InstrProfData.inc:504:27
#3 0x81579d in
llvm::ValueProfData::serializeFrom(llvm::InstrProfRecord const&)
/code/llvm/lib/ProfileData/InstrProf.cpp:232:7
#4 0x84ac71 in EmitData /code/llvm/lib/ProfileData/InstrProfWriter.cpp:82:11
#5 0x84ac71 in Emit /code/llvm/include/llvm/Support/OnDiskHashTable.h:183
#6 0x84ac71 in Emit /code/llvm/include/llvm/Support/OnDiskHashTable.h:145
#7 0x84ac71 in
llvm::InstrProfWriter::writeImpl(llvm::raw_ostream&)
/code/llvm/lib/ProfileData/InstrProfWriter.cpp:159
#8 0x84dfd6 in llvm::InstrProfWriter::write(llvm::raw_fd_ostream&)
/code/llvm/lib/ProfileData/InstrProfWriter.cpp:166:21
#9 0x48fea4 in mergeInstrProfile
/code/llvm/tools/llvm-profdata/llvm-profdata.cpp:129:5
#10 0x48fea4 in merge_main(int, char const**)
/code/llvm/tools/llvm-profdata/llvm-profdata.cpp:200
#11 0x489af5 in main /code/llvm/tools/llvm-profdata/llvm-profdata.cpp:379:14
#12 0x7f4c2aec0ec4 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x21ec4)
On Tue, Dec 15, 2015 at 12:03 PM, Xinliang David Li <davidxl at google.com> wrote:
> My change is not directly related to the place where msan complains
> about -- it either triggers an existing bug in raw_ostream
> implementation or a bug in msan (false positive).
>
> The reported uninitialized error is on a buffer to be flushed out
> (triggered by a seek call by the InstrProf writer). I don't see how
> the buffer can be 'uninitialized'. Can you help find the root cause of
> the issue?
>
> thanks,
>
> David
>
> On Tue, Dec 15, 2015 at 11:27 AM, Evgenii Stepanov
> <eugeni.stepanov at gmail.com> wrote:
>> Hi David,
>>
>> I think your change caused this failure:
>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/builds/10349/steps/check-llvm%20msan/logs/stdio
>>
>> On Mon, Dec 14, 2015 at 10:44 AM, Xinliang David Li via llvm-commits
>> <llvm-commits at lists.llvm.org> wrote:
>>> Author: davidxl
>>> Date: Mon Dec 14 12:44:01 2015
>>> New Revision: 255523
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=255523&view=rev
>>> Log:
>>> [PGO] Value profiling text format reader/writer support
>>>
>>> This patch adds the missing functionality in parsable
>>> text format support for value profiling.
>>>
>>> Differential Revision: http://reviews.llvm.org/D15212
>>>
>>>
>>> Added:
>>> llvm/trunk/test/tools/llvm-profdata/Inputs/vp-malform.proftext
>>> llvm/trunk/test/tools/llvm-profdata/Inputs/vp-malform2.proftext
>>> llvm/trunk/test/tools/llvm-profdata/Inputs/vp-truncate.proftext
>>> llvm/trunk/test/tools/llvm-profdata/value-prof.proftext
>>> Modified:
>>> llvm/trunk/include/llvm/ProfileData/InstrProfReader.h
>>> llvm/trunk/lib/ProfileData/InstrProfReader.cpp
>>> llvm/trunk/lib/ProfileData/InstrProfWriter.cpp
>>> llvm/trunk/test/tools/llvm-profdata/text-format-errors.test
>>>
>>> Modified: llvm/trunk/include/llvm/ProfileData/InstrProfReader.h
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ProfileData/InstrProfReader.h?rev=255523&r1=255522&r2=255523&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/include/llvm/ProfileData/InstrProfReader.h (original)
>>> +++ llvm/trunk/include/llvm/ProfileData/InstrProfReader.h Mon Dec 14 12:44:01 2015
>>> @@ -106,8 +106,13 @@ private:
>>> /// Iterator over the profile data.
>>> line_iterator Line;
>>>
>>> + // String table for holding a unique copy of all the strings in the profile.
>>> + InstrProfStringTable StringTable;
>>> +
>>> TextInstrProfReader(const TextInstrProfReader &) = delete;
>>> TextInstrProfReader &operator=(const TextInstrProfReader &) = delete;
>>> + std::error_code readValueProfileData(InstrProfRecord &Record);
>>> +
>>> public:
>>> TextInstrProfReader(std::unique_ptr<MemoryBuffer> DataBuffer_)
>>> : DataBuffer(std::move(DataBuffer_)), Line(*DataBuffer, true, '#') {}
>>>
>>> Modified: llvm/trunk/lib/ProfileData/InstrProfReader.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ProfileData/InstrProfReader.cpp?rev=255523&r1=255522&r2=255523&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/lib/ProfileData/InstrProfReader.cpp (original)
>>> +++ llvm/trunk/lib/ProfileData/InstrProfReader.cpp Mon Dec 14 12:44:01 2015
>>> @@ -109,6 +109,68 @@ bool TextInstrProfReader::hasFormat(cons
>>> [](char c) { return ::isprint(c) || ::isspace(c); });
>>> }
>>>
>>> +std::error_code
>>> +TextInstrProfReader::readValueProfileData(InstrProfRecord &Record) {
>>> +
>>> +#define CHECK_LINE_END(Line) \
>>> + if (Line.is_at_end()) \
>>> + return error(instrprof_error::truncated);
>>> +#define READ_NUM(Str, Dst) \
>>> + if ((Str).getAsInteger(10, (Dst))) \
>>> + return error(instrprof_error::malformed);
>>> +#define VP_READ_ADVANCE(Val) \
>>> + CHECK_LINE_END(Line); \
>>> + uint32_t Val; \
>>> + READ_NUM((*Line), (Val)); \
>>> + Line++;
>>> +
>>> + if (Line.is_at_end())
>>> + return success();
>>> + uint32_t NumValueKinds;
>>> + if (Line->getAsInteger(10, NumValueKinds)) {
>>> + // No value profile data
>>> + return success();
>>> + }
>>> + if (NumValueKinds == 0 || NumValueKinds > IPVK_Last + 1)
>>> + return error(instrprof_error::malformed);
>>> + Line++;
>>> +
>>> + for (uint32_t VK = 0; VK < NumValueKinds; VK++) {
>>> + VP_READ_ADVANCE(ValueKind);
>>> + if (ValueKind > IPVK_Last)
>>> + return error(instrprof_error::malformed);
>>> + VP_READ_ADVANCE(NumValueSites);
>>> + if (!NumValueSites)
>>> + continue;
>>> +
>>> + Record.reserveSites(VK, NumValueSites);
>>> + for (uint32_t S = 0; S < NumValueSites; S++) {
>>> + VP_READ_ADVANCE(NumValueData);
>>> +
>>> + std::vector<InstrProfValueData> CurrentValues;
>>> + for (uint32_t V = 0; V < NumValueData; V++) {
>>> + CHECK_LINE_END(Line);
>>> + std::pair<StringRef, StringRef> VD = Line->split(':');
>>> + uint64_t TakenCount, Value;
>>> + READ_NUM(VD.second, TakenCount);
>>> + if (VK == IPVK_IndirectCallTarget)
>>> + Value = (uint64_t)StringTable.insertString(VD.first);
>>> + else {
>>> + READ_NUM(VD.first, Value);
>>> + }
>>> + CurrentValues.push_back({Value, TakenCount});
>>> + Line++;
>>> + }
>>> + Record.addValueData(VK, S, CurrentValues.data(), NumValueData, nullptr);
>>> + }
>>> + }
>>> + return success();
>>> +
>>> +#undef CHECK_LINE_END
>>> +#undef READ_NUM
>>> +#undef VP_READ_ADVANCE
>>> +}
>>> +
>>> std::error_code TextInstrProfReader::readNextRecord(InstrProfRecord &Record) {
>>> // Skip empty lines and comments.
>>> while (!Line.is_at_end() && (Line->empty() || Line->startswith("#")))
>>> @@ -147,6 +209,10 @@ std::error_code TextInstrProfReader::rea
>>> Record.Counts.push_back(Count);
>>> }
>>>
>>> + // Check if value profile data exists and read it if so.
>>> + if (std::error_code EC = readValueProfileData(Record))
>>> + return EC;
>>> +
>>> return success();
>>> }
>>>
>>>
>>> Modified: llvm/trunk/lib/ProfileData/InstrProfWriter.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ProfileData/InstrProfWriter.cpp?rev=255523&r1=255522&r2=255523&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/lib/ProfileData/InstrProfWriter.cpp (original)
>>> +++ llvm/trunk/lib/ProfileData/InstrProfWriter.cpp Mon Dec 14 12:44:01 2015
>>> @@ -172,15 +172,47 @@ void InstrProfWriter::write(raw_fd_ostre
>>> endian::Writer<little>(OS).write<uint64_t>(TableStart.second);
>>> }
>>>
>>> +static const char *ValueProfKindStr[] = {
>>> +#define VALUE_PROF_KIND(Enumerator, Value) #Enumerator,
>>> +#include "llvm/ProfileData/InstrProfData.inc"
>>> +};
>>> +
>>> void InstrProfWriter::writeRecordInText(const InstrProfRecord &Func,
>>> raw_fd_ostream &OS) {
>>> OS << Func.Name << "\n";
>>> OS << "# Func Hash:\n" << Func.Hash << "\n";
>>> - OS << "# Num Counters:\n" <<Func.Counts.size() << "\n";
>>> + OS << "# Num Counters:\n" << Func.Counts.size() << "\n";
>>> OS << "# Counter Values:\n";
>>> for (uint64_t Count : Func.Counts)
>>> OS << Count << "\n";
>>>
>>> + uint32_t NumValueKinds = Func.getNumValueKinds();
>>> + if (!NumValueKinds) {
>>> + OS << "\n";
>>> + return;
>>> + }
>>> +
>>> + OS << "# Num Value Kinds:\n" << Func.getNumValueKinds() << "\n";
>>> + for (uint32_t VK = 0; VK < IPVK_Last + 1; VK++) {
>>> + uint32_t NS = Func.getNumValueSites(VK);
>>> + if (!NS)
>>> + continue;
>>> + OS << "# ValueKind = " << ValueProfKindStr[VK] << ":\n" << VK << "\n";
>>> + OS << "# NumValueSites:\n" << NS << "\n";
>>> + for (uint32_t S = 0; S < NS; S++) {
>>> + uint32_t ND = Func.getNumValueDataForSite(VK, S);
>>> + OS << ND << "\n";
>>> + std::unique_ptr<InstrProfValueData[]> VD = Func.getValueForSite(VK, S);
>>> + for (uint32_t I = 0; I < ND; I++) {
>>> + if (VK == IPVK_IndirectCallTarget)
>>> + OS << reinterpret_cast<const char *>(VD[I].Value) << ":"
>>> + << VD[I].Count << "\n";
>>> + else
>>> + OS << VD[I].Value << ":" << VD[I].Count << "\n";
>>> + }
>>> + }
>>> + }
>>> +
>>> OS << "\n";
>>> }
>>>
>>>
>>> Added: llvm/trunk/test/tools/llvm-profdata/Inputs/vp-malform.proftext
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-profdata/Inputs/vp-malform.proftext?rev=255523&view=auto
>>> ==============================================================================
>>> --- llvm/trunk/test/tools/llvm-profdata/Inputs/vp-malform.proftext (added)
>>> +++ llvm/trunk/test/tools/llvm-profdata/Inputs/vp-malform.proftext Mon Dec 14 12:44:01 2015
>>> @@ -0,0 +1,42 @@
>>> +foo
>>> +# Func Hash:
>>> +10
>>> +# Num Counters:
>>> +2
>>> +# Counter Values:
>>> +999000
>>> +359800
>>> +
>>> +foo2
>>> +# Func Hash:
>>> +10
>>> +# Num Counters:
>>> +2
>>> +# Counter Values:
>>> +1001000
>>> +360200
>>> +
>>> +main
>>> +# Func Hash:
>>> +16650
>>> +# Num Counters:
>>> +4
>>> +# Counter Values:
>>> +2
>>> +2000
>>> +2000000
>>> +999000
>>> +# NumValueKinds
>>> +1
>>> +# Value Kind IPVK_IndirectCallTarget
>>> +0
>>> +# NumSites
>>> +3
>>> +# Values for each site
>>> +0
>>> +2
>>> +# !!!! Malformed Value/Count pair
>>> +foo+100
>>> +foo2:1000
>>> +1
>>> +foo2:20000
>>>
>>> Added: llvm/trunk/test/tools/llvm-profdata/Inputs/vp-malform2.proftext
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-profdata/Inputs/vp-malform2.proftext?rev=255523&view=auto
>>> ==============================================================================
>>> --- llvm/trunk/test/tools/llvm-profdata/Inputs/vp-malform2.proftext (added)
>>> +++ llvm/trunk/test/tools/llvm-profdata/Inputs/vp-malform2.proftext Mon Dec 14 12:44:01 2015
>>> @@ -0,0 +1,32 @@
>>> +foo
>>> +# Func Hash:
>>> +10
>>> +# Num Counters:
>>> +2
>>> +# Counter Values:
>>> +999000
>>> +359800
>>> +
>>> +main
>>> +# Func Hash:
>>> +16650
>>> +# Num Counters:
>>> +4
>>> +# Counter Values:
>>> +2
>>> +2000
>>> +2000000
>>> +999000
>>> +# NumValueKinds
>>> +1
>>> +# Value Kind IPVK_IndirectCallTarget
>>> +0
>>> +# NumSites
>>> +3
>>> +# Values for each site
>>> +0
>>> +# !! Malformed value site, missing one value
>>> +2
>>> +foo:100
>>> +1
>>> +foo2:20000
>>>
>>> Added: llvm/trunk/test/tools/llvm-profdata/Inputs/vp-truncate.proftext
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-profdata/Inputs/vp-truncate.proftext?rev=255523&view=auto
>>> ==============================================================================
>>> --- llvm/trunk/test/tools/llvm-profdata/Inputs/vp-truncate.proftext (added)
>>> +++ llvm/trunk/test/tools/llvm-profdata/Inputs/vp-truncate.proftext Mon Dec 14 12:44:01 2015
>>> @@ -0,0 +1,36 @@
>>> +foo
>>> +# Func Hash:
>>> +10
>>> +# Num Counters:
>>> +2
>>> +# Counter Values:
>>> +999000
>>> +359800
>>> +
>>> +foo2
>>> +# Func Hash:
>>> +10
>>> +# Num Counters:
>>> +2
>>> +# Counter Values:
>>> +1001000
>>> +360200
>>> +
>>> +main
>>> +# Func Hash:
>>> +16650
>>> +# Num Counters:
>>> +4
>>> +# Counter Values:
>>> +2
>>> +2000
>>> +2000000
>>> +999000
>>> +# NumValueKinds
>>> +1
>>> +# Value Kind IPVK_IndirectCallTarget
>>> +0
>>> +# NumSites
>>> +3
>>> +# Values for each site
>>> +0
>>>
>>> Modified: llvm/trunk/test/tools/llvm-profdata/text-format-errors.test
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-profdata/text-format-errors.test?rev=255523&r1=255522&r2=255523&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/test/tools/llvm-profdata/text-format-errors.test (original)
>>> +++ llvm/trunk/test/tools/llvm-profdata/text-format-errors.test Mon Dec 14 12:44:01 2015
>>> @@ -18,3 +18,12 @@ NO-COUNTS: error: {{.*}}no-counts.profte
>>> RUN: not llvm-profdata show %p/Inputs/text-format-errors.text.bin 2>&1 | FileCheck %s --check-prefix=BINARY
>>> BINARY: error: {{.+}}: Unrecognized instrumentation profile encoding format
>>> BINARY: Perhaps you forgot to use the -sample option?
>>> +
>>> +5- Detect malformed value profile data
>>> +RUN: not llvm-profdata show %p/Inputs/vp-malform.proftext 2>&1 | FileCheck %s --check-prefix=VP
>>> +RUN: not llvm-profdata show %p/Inputs/vp-malform2.proftext 2>&1 | FileCheck %s --check-prefix=VP
>>> +VP: Malformed instrumentation profile data
>>> +
>>> +6- Detect truncated value profile data
>>> +RUN: not llvm-profdata show %p/Inputs/vp-truncate.proftext 2>&1 | FileCheck %s --check-prefix=VPTRUNC
>>> +VPTRUNC: Truncated profile data
>>>
>>> Added: llvm/trunk/test/tools/llvm-profdata/value-prof.proftext
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-profdata/value-prof.proftext?rev=255523&view=auto
>>> ==============================================================================
>>> --- llvm/trunk/test/tools/llvm-profdata/value-prof.proftext (added)
>>> +++ llvm/trunk/test/tools/llvm-profdata/value-prof.proftext Mon Dec 14 12:44:01 2015
>>> @@ -0,0 +1,61 @@
>>> +# RUN: llvm-profdata show -ic-targets -all-functions %s | FileCheck %s --check-prefix=IC
>>> +# RUN: llvm-profdata show -ic-targets -counts -text -all-functions %s | FileCheck %s --check-prefix=ICTEXT
>>> +# RUN: llvm-profdata merge -o %t.profdata %s
>>> +# RUN: llvm-profdata show -ic-targets -all-functions %t.profdata | FileCheck %s --check-prefix=IC
>>> +
>>> +foo
>>> +# Func Hash:
>>> +10
>>> +# Num Counters:
>>> +2
>>> +# Counter Values:
>>> +999000
>>> +359800
>>> +
>>> +foo2
>>> +# Func Hash:
>>> +10
>>> +# Num Counters:
>>> +2
>>> +# Counter Values:
>>> +1001000
>>> +360200
>>> +
>>> +main
>>> +# Func Hash:
>>> +16650
>>> +# Num Counters:
>>> +4
>>> +# Counter Values:
>>> +2
>>> +2000
>>> +2000000
>>> +999000
>>> +# NumValueKinds
>>> +1
>>> +# Value Kind IPVK_IndirectCallTarget
>>> +0
>>> +# NumSites
>>> +3
>>> +# Values for each site
>>> +0
>>> +2
>>> +foo:100
>>> +foo2:1000
>>> +1
>>> +foo2:20000
>>> +
>>> +#IC: Indirect Call Site Count: 3
>>> +#IC-NEXT: Indirect Target Results:
>>> +#IC-NEXT: [ 1, foo, 100 ]
>>> +#IC-NEXT: [ 1, foo2, 1000 ]
>>> +#IC-NEXT: [ 2, foo2, 20000 ]
>>> +
>>> +#ICTEXT: foo:100
>>> +#ICTEXT-NEXT: foo2:1000
>>> +#ICTEXT-NEXT: 1
>>> +#ICTEXT-NEXT: foo2:20000
>>> +# RUN: llvm-profdata show -ic-targets -all-functions %s | FileCheck %s --check-prefix=IC
>>> +# RUN: llvm-profdata show -ic-targets -counts -text -all-functions %s | FileCheck %s --check-prefix=ICTEXT
>>> +# RUN: llvm-profdata merge -o %t.profdata %s
>>> +# RUN: llvm-profdata show -ic-targets -all-functions %t.profdata | FileCheck %s --check-prefix=IC
>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
More information about the llvm-commits
mailing list