[llvm] r271709 - [llvm-profdata] Add option to ingest filepaths from a file
Chandler Carruth via llvm-commits
llvm-commits at lists.llvm.org
Fri Jun 3 20:02:42 PDT 2016
Err... I see twhere were still further edits here. Sorry for perhaps
replying to the wrong commit. But the build bot I've cited is in fact still
failing.
On Fri, Jun 3, 2016 at 8:00 PM Chandler Carruth <chandlerc at gmail.com> wrote:
> On Fri, Jun 3, 2016 at 12:11 PM Vedant Kumar via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
>
>> Author: vedantk
>> Date: Fri Jun 3 14:05:20 2016
>> New Revision: 271709
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=271709&view=rev
>> Log:
>> [llvm-profdata] Add option to ingest filepaths from a file
>>
>
> This has been in the tree for 7 hours with failing tests on all the
> Windows bots. Please try to watch the bots more carefully in the future.
>
> I'm reverting for now. You'll need to do something to properly handle
> quoting and escaping with path names here.
>
> Here is one of the failures for your reference:
> http://lab.llvm.org:8011/builders/clang-x86-win2008-selfhost/builds/8317
>
>
>>
>> Differential Revision: http://reviews.llvm.org/D20980
>>
>> Added:
>> llvm/trunk/test/tools/llvm-profdata/input-filenames.test
>> Modified:
>> llvm/trunk/docs/CommandGuide/llvm-profdata.rst
>> llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp
>>
>> Modified: llvm/trunk/docs/CommandGuide/llvm-profdata.rst
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/CommandGuide/llvm-profdata.rst?rev=271709&r1=271708&r2=271709&view=diff
>>
>> ==============================================================================
>> --- llvm/trunk/docs/CommandGuide/llvm-profdata.rst (original)
>> +++ llvm/trunk/docs/CommandGuide/llvm-profdata.rst Fri Jun 3 14:05:20
>> 2016
>> @@ -44,6 +44,9 @@ interpreted as relatively more important
>> nature of the training runs it may be useful to adjust the weight given
>> to each
>> input file by using the ``-weighted-input`` option.
>>
>> +Profiles passed in via ``-weighted-input``, ``-input-files``, or via
>> positional
>> +arguments are processed once for each time they are seen.
>> +
>>
>> OPTIONS
>> ^^^^^^^
>> @@ -65,6 +68,12 @@ OPTIONS
>> Input files specified without using this option are assigned a default
>> weight of 1. Examples are shown below.
>>
>> +.. option:: -input-files=path, -f=path
>> +
>> + Specify a file which contains a list of files to merge. The entries in
>> this
>> + file are newline-separated. Lines starting with '#' are skipped.
>> Entries may
>> + be of the form <filename> or <weight>,<filename>.
>> +
>> .. option:: -instr (default)
>>
>> Specify that the input profile is an instrumentation-based profile.
>>
>> Added: llvm/trunk/test/tools/llvm-profdata/input-filenames.test
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-profdata/input-filenames.test?rev=271709&view=auto
>>
>> ==============================================================================
>> --- llvm/trunk/test/tools/llvm-profdata/input-filenames.test (added)
>> +++ llvm/trunk/test/tools/llvm-profdata/input-filenames.test Fri Jun 3
>> 14:05:20 2016
>> @@ -0,0 +1,16 @@
>> +# Create an input file.
>> +RUN: printf '# comment 1\n' > %t.input
>> +RUN: printf ' # comment 2\n' >> %t.input
>> +RUN: printf 'foo\n' >> %t.input
>> +RUN: printf ' bar\n' >> %t.input
>> +RUN: printf "2,%t.weighted\n" >> %t.input
>> +
>> +# Create the weighted file, since these actually need to exist.
>> +RUN: printf ' ' > %t.weighted
>> +
>> +# RUN: llvm-profdata merge -f %t.input -dump-input-file-list -o
>> /dev/null | FileCheck %s
>> +# RUN: llvm-profdata merge -input-files %t.input -dump-input-file-list
>> -o /dev/null | FileCheck %s
>> +
>> +# CHECK: 1,foo
>> +# CHECK-NEXT: 1,bar
>> +# CHECK-NEXT: 2,{{.*}}.weighted
>>
>> Modified: llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp?rev=271709&r1=271708&r2=271709&view=diff
>>
>> ==============================================================================
>> --- llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp (original)
>> +++ llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp Fri Jun 3 14:05:20
>> 2016
>> @@ -223,11 +223,45 @@ static WeightedFile parseWeightedFile(co
>> return WeightedFile(FileName, Weight);
>> }
>>
>> +static void parseInputFilenamesFile(const StringRef &InputFilenamesFile,
>> + WeightedFileVector &WFV) {
>> + if (InputFilenamesFile == "")
>> + return;
>> +
>> + auto Buf = MemoryBuffer::getFileOrSTDIN(InputFilenamesFile);
>> + if (!Buf)
>> + exitWithErrorCode(Buf.getError(), InputFilenamesFile);
>> +
>> + StringRef Data = Buf.get()->getBuffer();
>> + SmallVector<StringRef, 8> Entries;
>> + Data.split(Entries, '\n', /*MaxSplit=*/-1, /*KeepEmpty=*/false);
>> + for (const StringRef &FileWeightEntry : Entries) {
>> + StringRef SanitizedEntry = FileWeightEntry.trim(" \t\v\f\r");
>> + // Skip comments.
>> + if (SanitizedEntry.startswith("#"))
>> + continue;
>> + // If there's no comma, it's an unweighted profile.
>> + else if (SanitizedEntry.rfind(',') == StringRef::npos)
>> + WFV.emplace_back(SanitizedEntry, 1);
>> + else
>> + WFV.emplace_back(parseWeightedFile(SanitizedEntry));
>> + }
>> +}
>> +
>> static int merge_main(int argc, const char *argv[]) {
>> cl::list<std::string> InputFilenames(cl::Positional,
>> cl::desc("<filename...>"));
>> cl::list<std::string> WeightedInputFilenames("weighted-input",
>>
>> cl::desc("<weight>,<filename>"));
>> + cl::opt<std::string> InputFilenamesFile(
>> + "input-files", cl::init(""),
>> + cl::desc("Path to file containing newline-separated "
>> + "<filename>[,<weight>] entries"));
>> + cl::alias InputFilenamesFileA("f", cl::desc("Alias for --input-files"),
>> + cl::aliasopt(InputFilenamesFile));
>> + cl::opt<bool> DumpInputFileList(
>> + "dump-input-file-list", cl::init(false), cl::Hidden,
>> + cl::desc("Dump the list of input files and their weights, then
>> exit"));
>> cl::opt<std::string> OutputFilename("output", cl::value_desc("output"),
>> cl::init("-"), cl::Required,
>> cl::desc("Output file"));
>> @@ -249,15 +283,22 @@ static int merge_main(int argc, const ch
>>
>> cl::ParseCommandLineOptions(argc, argv, "LLVM profile data merger\n");
>>
>> - if (InputFilenames.empty() && WeightedInputFilenames.empty())
>> - exitWithError("No input files specified. See " +
>> - sys::path::filename(argv[0]) + " -help");
>> -
>> WeightedFileVector WeightedInputs;
>> for (StringRef Filename : InputFilenames)
>> WeightedInputs.push_back(WeightedFile(Filename, 1));
>> for (StringRef WeightedFilename : WeightedInputFilenames)
>> WeightedInputs.push_back(parseWeightedFile(WeightedFilename));
>> + parseInputFilenamesFile(InputFilenamesFile, WeightedInputs);
>> +
>> + if (WeightedInputs.empty())
>> + exitWithError("No input files specified. See " +
>> + sys::path::filename(argv[0]) + " -help");
>> +
>> + if (DumpInputFileList) {
>> + for (auto &WF : WeightedInputs)
>> + outs() << WF.Weight << "," << WF.Filename << "\n";
>> + return 0;
>> + }
>>
>> if (ProfileKind == instr)
>> mergeInstrProfile(WeightedInputs, OutputFilename, OutputFormat,
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160604/92f8018d/attachment.html>
More information about the llvm-commits
mailing list