[llvm] r252916 - [llvm-profdata] Add check for text profile formats and improve error reporting
Bruno Cardoso Lopes via llvm-commits
llvm-commits at lists.llvm.org
Thu Nov 12 10:21:12 PST 2015
Hi Nathan,
This is failing a test in
http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-incremental_check/
/Users/buildslave/jenkins/sharedspace/incremental at 2/llvm/test/Transforms/SampleProfile/syntax.ll:16:18:
error: expected string not found in input
; BAD-FN-HEADER: error: {{.*}}bad_fn_header.prof:1: Expected
'mangled_name:NUM:NUM', found 3empty:100:BAD
^
<stdin>:1:1: note: scanning from here
error: /Users/buildslave/jenkins/sharedspace/incremental at 2/llvm/test/Transforms/SampleProfile/Inputs/bad_fn_header.prof:
Could not open profile: Unrecognized sample profile encoding format
^
Thanks,
On Thu, Nov 12, 2015 at 10:06 AM, Nathan Slingerland via llvm-commits
<llvm-commits at lists.llvm.org> wrote:
> Author: slingn
> Date: Thu Nov 12 12:06:18 2015
> New Revision: 252916
>
> URL: http://llvm.org/viewvc/llvm-project?rev=252916&view=rev
> Log:
> [llvm-profdata] Add check for text profile formats and improve error reporting
>
> Summary:
> This change addresses two possible instances of user error / confusion when
> merging sampled profile data.
>
> Previously any input that didn't match the raw or processed instrumented format
> would automatically be interpreted as instrumented profile text format data.
> No error would be reported during the merge.
>
> Example:
> If foo-sampled.profdata and bar-sampled.profdata are binary sampled profiles:
>
> Old behavior:
> $ llvm-profdata merge foo-sampled.profdata bar-sampled.profdata -output foobar-sampled.profdata
> $ llvm-profdata show -sample foobar-sampled.profdata
> error: foobar-sampled.profdata:1: Expected 'mangled_name:NUM:NUM', found lprofi
>
> This change adds basic checks for valid input data when assuming text input.
> It also makes error messages related to file format validity more specific about
> the assumbed profile data type.
>
> New behavior:
> $ llvm-profdata merge foo-sampled.profdata bar-sampled.profdata -o foobar-sampled.profdata
> error: foo.profdata: Unrecognized instrumentation profile encoding format
> Perhaps you forgot to use the -sample option?
>
> Reviewers: bogner, davidxl, dnovillo
>
> Subscribers: davidxl, llvm-commits
>
> Differential Revision: http://reviews.llvm.org/D14558
>
> Modified:
> llvm/trunk/include/llvm/ProfileData/InstrProf.h
> llvm/trunk/include/llvm/ProfileData/InstrProfReader.h
> llvm/trunk/include/llvm/ProfileData/SampleProfReader.h
> llvm/trunk/lib/ProfileData/InstrProf.cpp
> llvm/trunk/lib/ProfileData/InstrProfReader.cpp
> llvm/trunk/lib/ProfileData/SampleProf.cpp
> llvm/trunk/lib/ProfileData/SampleProfReader.cpp
> llvm/trunk/test/tools/llvm-profdata/raw-magic-but-no-header.test
> llvm/trunk/test/tools/llvm-profdata/sample-profile-basic.test
> llvm/trunk/test/tools/llvm-profdata/text-format-errors.test
> llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp
>
> Modified: llvm/trunk/include/llvm/ProfileData/InstrProf.h
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ProfileData/InstrProf.h?rev=252916&r1=252915&r2=252916&view=diff
> ==============================================================================
> --- llvm/trunk/include/llvm/ProfileData/InstrProf.h (original)
> +++ llvm/trunk/include/llvm/ProfileData/InstrProf.h Thu Nov 12 12:06:18 2015
> @@ -149,6 +149,7 @@ const std::error_category &instrprof_cat
> enum class instrprof_error {
> success = 0,
> eof,
> + unrecognized_format,
> bad_magic,
> bad_header,
> unsupported_version,
>
> Modified: llvm/trunk/include/llvm/ProfileData/InstrProfReader.h
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ProfileData/InstrProfReader.h?rev=252916&r1=252915&r2=252916&view=diff
> ==============================================================================
> --- llvm/trunk/include/llvm/ProfileData/InstrProfReader.h (original)
> +++ llvm/trunk/include/llvm/ProfileData/InstrProfReader.h Thu Nov 12 12:06:18 2015
> @@ -111,6 +111,9 @@ public:
> TextInstrProfReader(std::unique_ptr<MemoryBuffer> DataBuffer_)
> : DataBuffer(std::move(DataBuffer_)), Line(*DataBuffer, true, '#') {}
>
> + /// Return true if the given buffer is in text instrprof format.
> + static bool hasFormat(const MemoryBuffer &Buffer);
> +
> /// Read the header.
> std::error_code readHeader() override { return success(); }
> /// Read a single record.
>
> Modified: llvm/trunk/include/llvm/ProfileData/SampleProfReader.h
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ProfileData/SampleProfReader.h?rev=252916&r1=252915&r2=252916&view=diff
> ==============================================================================
> --- llvm/trunk/include/llvm/ProfileData/SampleProfReader.h (original)
> +++ llvm/trunk/include/llvm/ProfileData/SampleProfReader.h Thu Nov 12 12:06:18 2015
> @@ -292,6 +292,9 @@ public:
>
> /// \brief Read sample profiles from the associated file.
> std::error_code read() override;
> +
> + /// \brief Return true if \p Buffer is in the format supported by this class.
> + static bool hasFormat(const MemoryBuffer &Buffer);
> };
>
> class SampleProfileReaderBinary : public SampleProfileReader {
>
> Modified: llvm/trunk/lib/ProfileData/InstrProf.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ProfileData/InstrProf.cpp?rev=252916&r1=252915&r2=252916&view=diff
> ==============================================================================
> --- llvm/trunk/lib/ProfileData/InstrProf.cpp (original)
> +++ llvm/trunk/lib/ProfileData/InstrProf.cpp Thu Nov 12 12:06:18 2015
> @@ -32,20 +32,22 @@ class InstrProfErrorCategoryType : publi
> return "Success";
> case instrprof_error::eof:
> return "End of File";
> + case instrprof_error::unrecognized_format:
> + return "Unrecognized instrumentation profile encoding format";
> case instrprof_error::bad_magic:
> - return "Invalid profile data (bad magic)";
> + return "Invalid instrumentation profile data (bad magic)";
> case instrprof_error::bad_header:
> - return "Invalid profile data (file header is corrupt)";
> + return "Invalid instrumentation profile data (file header is corrupt)";
> case instrprof_error::unsupported_version:
> - return "Unsupported profiling format version";
> + return "Unsupported instrumentation profile format version";
> case instrprof_error::unsupported_hash_type:
> - return "Unsupported profiling hash";
> + return "Unsupported instrumentation profile hash type";
> case instrprof_error::too_large:
> return "Too much profile data";
> case instrprof_error::truncated:
> return "Truncated profile data";
> case instrprof_error::malformed:
> - return "Malformed profile data";
> + return "Malformed instrumentation profile data";
> case instrprof_error::unknown_function:
> return "No profile data available for function";
> case instrprof_error::hash_mismatch:
>
> Modified: llvm/trunk/lib/ProfileData/InstrProfReader.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ProfileData/InstrProfReader.cpp?rev=252916&r1=252915&r2=252916&view=diff
> ==============================================================================
> --- llvm/trunk/lib/ProfileData/InstrProfReader.cpp (original)
> +++ llvm/trunk/lib/ProfileData/InstrProfReader.cpp Thu Nov 12 12:06:18 2015
> @@ -54,8 +54,10 @@ InstrProfReader::create(std::unique_ptr<
> Result.reset(new RawInstrProfReader64(std::move(Buffer)));
> else if (RawInstrProfReader32::hasFormat(*Buffer))
> Result.reset(new RawInstrProfReader32(std::move(Buffer)));
> - else
> + else if (TextInstrProfReader::hasFormat(*Buffer))
> Result.reset(new TextInstrProfReader(std::move(Buffer)));
> + else
> + return instrprof_error::unrecognized_format;
>
> // Initialize the reader and return the result.
> if (std::error_code EC = initializeReader(*Result))
> @@ -97,6 +99,15 @@ void InstrProfIterator::Increment() {
> *this = InstrProfIterator();
> }
>
> +bool TextInstrProfReader::hasFormat(const MemoryBuffer &Buffer) {
> + // Verify that this really looks like plain ASCII text by checking a
> + // 'reasonable' number of characters (up to profile magic size).
> + size_t count = std::min(Buffer.getBufferSize(), sizeof(uint64_t));
> + StringRef buffer = Buffer.getBufferStart();
> + return count == 0 || std::all_of(buffer.begin(), buffer.begin() + count,
> + [](char c) { return ::isprint(c) || ::isspace(c); });
> +}
> +
> std::error_code TextInstrProfReader::readNextRecord(InstrProfRecord &Record) {
> // Skip empty lines and comments.
> while (!Line.is_at_end() && (Line->empty() || Line->startswith("#")))
>
> Modified: llvm/trunk/lib/ProfileData/SampleProf.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ProfileData/SampleProf.cpp?rev=252916&r1=252915&r2=252916&view=diff
> ==============================================================================
> --- llvm/trunk/lib/ProfileData/SampleProf.cpp (original)
> +++ llvm/trunk/lib/ProfileData/SampleProf.cpp Thu Nov 12 12:06:18 2015
> @@ -28,17 +28,17 @@ class SampleProfErrorCategoryType : publ
> case sampleprof_error::success:
> return "Success";
> case sampleprof_error::bad_magic:
> - return "Invalid file format (bad magic)";
> + return "Invalid sample profile data (bad magic)";
> case sampleprof_error::unsupported_version:
> - return "Unsupported format version";
> + return "Unsupported sample profile format version";
> case sampleprof_error::too_large:
> return "Too much profile data";
> case sampleprof_error::truncated:
> return "Truncated profile data";
> case sampleprof_error::malformed:
> - return "Malformed profile data";
> + return "Malformed sample profile data";
> case sampleprof_error::unrecognized_format:
> - return "Unrecognized profile encoding format";
> + return "Unrecognized sample profile encoding format";
> case sampleprof_error::unsupported_writing_format:
> return "Profile encoding format unsupported for writing operations";
> case sampleprof_error::truncated_name_table:
>
> Modified: llvm/trunk/lib/ProfileData/SampleProfReader.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ProfileData/SampleProfReader.cpp?rev=252916&r1=252915&r2=252916&view=diff
> ==============================================================================
> --- llvm/trunk/lib/ProfileData/SampleProfReader.cpp (original)
> +++ llvm/trunk/lib/ProfileData/SampleProfReader.cpp Thu Nov 12 12:06:18 2015
> @@ -222,6 +222,22 @@ std::error_code SampleProfileReaderText:
> return sampleprof_error::success;
> }
>
> +bool SampleProfileReaderText::hasFormat(const MemoryBuffer &Buffer) {
> + bool result = false;
> +
> + // Check that the first non-comment line is a valid function header.
> + line_iterator LineIt(Buffer, /*SkipBlanks=*/true, '#');
> + if (!LineIt.is_at_eof()) {
> + if ((*LineIt)[0] != ' ') {
> + uint64_t NumSamples, NumHeadSamples;
> + StringRef FName;
> + result = ParseHead(*LineIt, FName, NumSamples, NumHeadSamples);
> + }
> + }
> +
> + return result;
> +}
> +
> template <typename T> ErrorOr<T> SampleProfileReaderBinary::readNumber() {
> unsigned NumBytesRead = 0;
> std::error_code EC;
> @@ -685,8 +701,10 @@ SampleProfileReader::create(StringRef Fi
> Reader.reset(new SampleProfileReaderBinary(std::move(Buffer), C));
> else if (SampleProfileReaderGCC::hasFormat(*Buffer))
> Reader.reset(new SampleProfileReaderGCC(std::move(Buffer), C));
> - else
> + else if (SampleProfileReaderText::hasFormat(*Buffer))
> Reader.reset(new SampleProfileReaderText(std::move(Buffer), C));
> + else
> + return sampleprof_error::unrecognized_format;
>
> if (std::error_code EC = Reader->readHeader())
> return EC;
>
> Modified: llvm/trunk/test/tools/llvm-profdata/raw-magic-but-no-header.test
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-profdata/raw-magic-but-no-header.test?rev=252916&r1=252915&r2=252916&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/llvm-profdata/raw-magic-but-no-header.test (original)
> +++ llvm/trunk/test/tools/llvm-profdata/raw-magic-but-no-header.test Thu Nov 12 12:06:18 2015
> @@ -3,4 +3,4 @@ RUN: not llvm-profdata show %t 2>&1 | Fi
> RUN: printf '\377lprofr\201' > %t
> RUN: not llvm-profdata show %t 2>&1 | FileCheck %s
>
> -CHECK: error: {{.+}}: Invalid profile data (file header is corrupt)
> +CHECK: error: {{.+}}: Invalid instrumentation profile data (file header is corrupt)
>
> Modified: llvm/trunk/test/tools/llvm-profdata/sample-profile-basic.test
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-profdata/sample-profile-basic.test?rev=252916&r1=252915&r2=252916&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/llvm-profdata/sample-profile-basic.test (original)
> +++ llvm/trunk/test/tools/llvm-profdata/sample-profile-basic.test Thu Nov 12 12:06:18 2015
> @@ -28,3 +28,7 @@ RUN: llvm-profdata merge --sample --text
> MERGE1: main:368038:0
> MERGE1: 9: 4128 _Z3fooi:1262 _Z3bari:2942
> MERGE1: _Z3fooi:15422:1220
> +
> +5- Detect invalid text encoding (e.g. instrumentation profile text format).
> +RUN: not llvm-profdata show --sample %p/Inputs/foo3bar3-1.proftext 2>&1 | FileCheck %s --check-prefix=BADTEXT
> +BADTEXT: error: {{.+}}: Unrecognized sample profile encoding format
>
> Modified: llvm/trunk/test/tools/llvm-profdata/text-format-errors.test
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-profdata/text-format-errors.test?rev=252916&r1=252915&r2=252916&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/llvm-profdata/text-format-errors.test (original)
> +++ llvm/trunk/test/tools/llvm-profdata/text-format-errors.test Thu Nov 12 12:06:18 2015
> @@ -1,10 +1,21 @@
> +Tests for instrumentation profile bad encoding.
> +
> +1- Detect invalid count
> RUN: not llvm-profdata show %p/Inputs/invalid-count-later.proftext 2>&1 | FileCheck %s --check-prefix=INVALID-COUNT-LATER
> RUN: not llvm-profdata merge %p/Inputs/invalid-count-later.proftext %p/Inputs/invalid-count-later.profdata -o %t.out 2>&1 | FileCheck %s --check-prefix=INVALID-COUNT-LATER
> -INVALID-COUNT-LATER: error: {{.*}}invalid-count-later.proftext: Malformed profile data
> +INVALID-COUNT-LATER: error: {{.*}}invalid-count-later.proftext: Malformed instrumentation profile data
>
> +2- Detect bad hash
> RUN: not llvm-profdata show %p/Inputs/bad-hash.proftext 2>&1 | FileCheck %s --check-prefix=BAD-HASH
> RUN: not llvm-profdata merge %p/Inputs/bad-hash.proftext %p/Inputs/bad-hash.proftext -o %t.out 2>&1 | FileCheck %s --check-prefix=BAD-HASH
> -BAD-HASH: error: {{.*}}bad-hash.proftext: Malformed profile data
> +BAD-HASH: error: {{.*}}bad-hash.proftext: Malformed instrumentation profile data
>
> +3- Detect no counts
> RUN: not llvm-profdata show %p/Inputs/no-counts.proftext 2>&1 | FileCheck %s --check-prefix=NO-COUNTS
> -NO-COUNTS: error: {{.*}}no-counts.proftext: Malformed profile data
> +NO-COUNTS: error: {{.*}}no-counts.proftext: Malformed instrumentation profile data
> +
> +4- Detect binary input
> +RUN: echo -n $'\xff\xe5\xd0\xb1\xf4\c9\x94\xa8' > %t.bin
> +RUN: not llvm-profdata show %t.bin 2>&1 | FileCheck %s --check-prefix=BINARY
> +BINARY: error: {{.+}}: Unrecognized instrumentation profile encoding format
> +BINARY: Perhaps you forgot to use the -sample option?
>
> Modified: llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp?rev=252916&r1=252915&r2=252916&view=diff
> ==============================================================================
> --- llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp (original)
> +++ llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp Thu Nov 12 12:06:18 2015
> @@ -29,16 +29,32 @@
>
> using namespace llvm;
>
> -static void exitWithError(const Twine &Message, StringRef Whence = "") {
> +static void exitWithError(const Twine &Message,
> + StringRef Whence = "",
> + StringRef Hint = "") {
> errs() << "error: ";
> if (!Whence.empty())
> errs() << Whence << ": ";
> errs() << Message << "\n";
> + if (!Hint.empty())
> + errs() << Hint << "\n";
> ::exit(1);
> }
>
> +static void exitWithErrorCode(const std::error_code &Error, StringRef Whence = "") {
> + if (Error.category() == instrprof_category()) {
> + instrprof_error instrError = static_cast<instrprof_error>(Error.value());
> + if (instrError == instrprof_error::unrecognized_format) {
> + // Hint for common error of forgetting -sample for sample profiles.
> + exitWithError(Error.message(), Whence,
> + "Perhaps you forgot to use the -sample option?");
> + }
> + }
> + exitWithError(Error.message(), Whence);
> +}
> +
> namespace {
> -enum ProfileKinds { instr, sample };
> + enum ProfileKinds { instr, sample };
> }
>
> static void mergeInstrProfile(const cl::list<std::string> &Inputs,
> @@ -49,20 +65,20 @@ static void mergeInstrProfile(const cl::
> std::error_code EC;
> raw_fd_ostream Output(OutputFilename.data(), EC, sys::fs::F_None);
> if (EC)
> - exitWithError(EC.message(), OutputFilename);
> + exitWithErrorCode(EC, OutputFilename);
>
> InstrProfWriter Writer;
> for (const auto &Filename : Inputs) {
> auto ReaderOrErr = InstrProfReader::create(Filename);
> if (std::error_code ec = ReaderOrErr.getError())
> - exitWithError(ec.message(), Filename);
> + exitWithErrorCode(ec, Filename);
>
> auto Reader = std::move(ReaderOrErr.get());
> for (auto &I : *Reader)
> if (std::error_code EC = Writer.addRecord(std::move(I)))
> errs() << Filename << ": " << I.Name << ": " << EC.message() << "\n";
> if (Reader->hasError())
> - exitWithError(Reader->getError().message(), Filename);
> + exitWithErrorCode(Reader->getError(), Filename);
> }
> Writer.write(Output);
> }
> @@ -73,7 +89,7 @@ static void mergeSampleProfile(const cl:
> using namespace sampleprof;
> auto WriterOrErr = SampleProfileWriter::create(OutputFilename, OutputFormat);
> if (std::error_code EC = WriterOrErr.getError())
> - exitWithError(EC.message(), OutputFilename);
> + exitWithErrorCode(EC, OutputFilename);
>
> auto Writer = std::move(WriterOrErr.get());
> StringMap<FunctionSamples> ProfileMap;
> @@ -82,7 +98,7 @@ static void mergeSampleProfile(const cl:
> auto ReaderOrErr =
> SampleProfileReader::create(Filename, getGlobalContext());
> if (std::error_code EC = ReaderOrErr.getError())
> - exitWithError(EC.message(), Filename);
> + exitWithErrorCode(EC, Filename);
>
> // We need to keep the readers around until after all the files are
> // read so that we do not lose the function names stored in each
> @@ -91,7 +107,7 @@ static void mergeSampleProfile(const cl:
> Readers.push_back(std::move(ReaderOrErr.get()));
> const auto Reader = Readers.back().get();
> if (std::error_code EC = Reader->read())
> - exitWithError(EC.message(), Filename);
> + exitWithErrorCode(EC, Filename);
>
> StringMap<FunctionSamples> &Profiles = Reader->getProfiles();
> for (StringMap<FunctionSamples>::iterator I = Profiles.begin(),
> @@ -143,7 +159,7 @@ static int showInstrProfile(std::string
> std::string ShowFunction, raw_fd_ostream &OS) {
> auto ReaderOrErr = InstrProfReader::create(Filename);
> if (std::error_code EC = ReaderOrErr.getError())
> - exitWithError(EC.message(), Filename);
> + exitWithErrorCode(EC, Filename);
>
> auto Reader = std::move(ReaderOrErr.get());
> uint64_t MaxFunctionCount = 0, MaxBlockCount = 0;
> @@ -198,7 +214,7 @@ static int showInstrProfile(std::string
> }
> }
> if (Reader->hasError())
> - exitWithError(Reader->getError().message(), Filename);
> + exitWithErrorCode(Reader->getError(), Filename);
>
> if (ShowAllFunctions || !ShowFunction.empty())
> OS << "Functions shown: " << ShownFunctions << "\n";
> @@ -214,11 +230,11 @@ static int showSampleProfile(std::string
> using namespace sampleprof;
> auto ReaderOrErr = SampleProfileReader::create(Filename, getGlobalContext());
> if (std::error_code EC = ReaderOrErr.getError())
> - exitWithError(EC.message(), Filename);
> + exitWithErrorCode(EC, Filename);
>
> auto Reader = std::move(ReaderOrErr.get());
> if (std::error_code EC = Reader->read())
> - exitWithError(EC.message(), Filename);
> + exitWithErrorCode(EC, Filename);
>
> if (ShowAllFunctions || ShowFunction.empty())
> Reader->dump(OS);
> @@ -259,7 +275,7 @@ static int show_main(int argc, const cha
> std::error_code EC;
> raw_fd_ostream OS(OutputFilename.data(), EC, sys::fs::F_Text);
> if (EC)
> - exitWithError(EC.message(), OutputFilename);
> + exitWithErrorCode(EC, OutputFilename);
>
> if (ShowAllFunctions && !ShowFunction.empty())
> errs() << "warning: -function argument ignored: showing all functions\n";
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
--
Bruno Cardoso Lopes
http://www.brunocardoso.cc
More information about the llvm-commits
mailing list