<div dir="ltr">Err... I see twhere were still further edits here. Sorry for perhaps replying to the wrong commit. But the build bot I've cited is in fact still failing.</div><br><div class="gmail_quote"><div dir="ltr">On Fri, Jun 3, 2016 at 8:00 PM Chandler Carruth <<a href="mailto:chandlerc@gmail.com">chandlerc@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Fri, Jun 3, 2016 at 12:11 PM Vedant Kumar via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: vedantk<br>
Date: Fri Jun 3 14:05:20 2016<br>
New Revision: 271709<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=271709&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=271709&view=rev</a><br>
Log:<br>
[llvm-profdata] Add option to ingest filepaths from a file<br></blockquote><div><br></div></div></div><div dir="ltr"><div class="gmail_quote"><div>This has been in the tree for 7 hours with failing tests on all the Windows bots. Please try to watch the bots more carefully in the future.</div><div><br></div><div>I'm reverting for now. You'll need to do something to properly handle quoting and escaping with path names here.</div><div><br></div><div>Here is one of the failures for your reference:</div><div><a href="http://lab.llvm.org:8011/builders/clang-x86-win2008-selfhost/builds/8317" target="_blank">http://lab.llvm.org:8011/builders/clang-x86-win2008-selfhost/builds/8317</a><br></div></div></div><div dir="ltr"><div class="gmail_quote"><div><span style="line-height:1.5"> </span><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Differential Revision: <a href="http://reviews.llvm.org/D20980" rel="noreferrer" target="_blank">http://reviews.llvm.org/D20980</a><br>
<br>
Added:<br>
llvm/trunk/test/tools/llvm-profdata/input-filenames.test<br>
Modified:<br>
llvm/trunk/docs/CommandGuide/llvm-profdata.rst<br>
llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp<br>
<br>
Modified: llvm/trunk/docs/CommandGuide/llvm-profdata.rst<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/CommandGuide/llvm-profdata.rst?rev=271709&r1=271708&r2=271709&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/CommandGuide/llvm-profdata.rst?rev=271709&r1=271708&r2=271709&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/docs/CommandGuide/llvm-profdata.rst (original)<br>
+++ llvm/trunk/docs/CommandGuide/llvm-profdata.rst Fri Jun 3 14:05:20 2016<br>
@@ -44,6 +44,9 @@ interpreted as relatively more important<br>
nature of the training runs it may be useful to adjust the weight given to each<br>
input file by using the ``-weighted-input`` option.<br>
<br>
+Profiles passed in via ``-weighted-input``, ``-input-files``, or via positional<br>
+arguments are processed once for each time they are seen.<br>
+<br>
<br>
OPTIONS<br>
^^^^^^^<br>
@@ -65,6 +68,12 @@ OPTIONS<br>
Input files specified without using this option are assigned a default<br>
weight of 1. Examples are shown below.<br>
<br>
+.. option:: -input-files=path, -f=path<br>
+<br>
+ Specify a file which contains a list of files to merge. The entries in this<br>
+ file are newline-separated. Lines starting with '#' are skipped. Entries may<br>
+ be of the form <filename> or <weight>,<filename>.<br>
+<br>
.. option:: -instr (default)<br>
<br>
Specify that the input profile is an instrumentation-based profile.<br>
<br>
Added: llvm/trunk/test/tools/llvm-profdata/input-filenames.test<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-profdata/input-filenames.test?rev=271709&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-profdata/input-filenames.test?rev=271709&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/test/tools/llvm-profdata/input-filenames.test (added)<br>
+++ llvm/trunk/test/tools/llvm-profdata/input-filenames.test Fri Jun 3 14:05:20 2016<br>
@@ -0,0 +1,16 @@<br>
+# Create an input file.<br>
+RUN: printf '# comment 1\n' > %t.input<br>
+RUN: printf ' # comment 2\n' >> %t.input<br>
+RUN: printf 'foo\n' >> %t.input<br>
+RUN: printf ' bar\n' >> %t.input<br>
+RUN: printf "2,%t.weighted\n" >> %t.input<br>
+<br>
+# Create the weighted file, since these actually need to exist.<br>
+RUN: printf ' ' > %t.weighted<br>
+<br>
+# RUN: llvm-profdata merge -f %t.input -dump-input-file-list -o /dev/null | FileCheck %s<br>
+# RUN: llvm-profdata merge -input-files %t.input -dump-input-file-list -o /dev/null | FileCheck %s<br>
+<br>
+# CHECK: 1,foo<br>
+# CHECK-NEXT: 1,bar<br>
+# CHECK-NEXT: 2,{{.*}}.weighted<br>
<br>
Modified: llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp?rev=271709&r1=271708&r2=271709&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp?rev=271709&r1=271708&r2=271709&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp (original)<br>
+++ llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp Fri Jun 3 14:05:20 2016<br>
@@ -223,11 +223,45 @@ static WeightedFile parseWeightedFile(co<br>
return WeightedFile(FileName, Weight);<br>
}<br>
<br>
+static void parseInputFilenamesFile(const StringRef &InputFilenamesFile,<br>
+ WeightedFileVector &WFV) {<br>
+ if (InputFilenamesFile == "")<br>
+ return;<br>
+<br>
+ auto Buf = MemoryBuffer::getFileOrSTDIN(InputFilenamesFile);<br>
+ if (!Buf)<br>
+ exitWithErrorCode(Buf.getError(), InputFilenamesFile);<br>
+<br>
+ StringRef Data = Buf.get()->getBuffer();<br>
+ SmallVector<StringRef, 8> Entries;<br>
+ Data.split(Entries, '\n', /*MaxSplit=*/-1, /*KeepEmpty=*/false);<br>
+ for (const StringRef &FileWeightEntry : Entries) {<br>
+ StringRef SanitizedEntry = FileWeightEntry.trim(" \t\v\f\r");<br>
+ // Skip comments.<br>
+ if (SanitizedEntry.startswith("#"))<br>
+ continue;<br>
+ // If there's no comma, it's an unweighted profile.<br>
+ else if (SanitizedEntry.rfind(',') == StringRef::npos)<br>
+ WFV.emplace_back(SanitizedEntry, 1);<br>
+ else<br>
+ WFV.emplace_back(parseWeightedFile(SanitizedEntry));<br>
+ }<br>
+}<br>
+<br>
static int merge_main(int argc, const char *argv[]) {<br>
cl::list<std::string> InputFilenames(cl::Positional,<br>
cl::desc("<filename...>"));<br>
cl::list<std::string> WeightedInputFilenames("weighted-input",<br>
cl::desc("<weight>,<filename>"));<br>
+ cl::opt<std::string> InputFilenamesFile(<br>
+ "input-files", cl::init(""),<br>
+ cl::desc("Path to file containing newline-separated "<br>
+ "<filename>[,<weight>] entries"));<br>
+ cl::alias InputFilenamesFileA("f", cl::desc("Alias for --input-files"),<br>
+ cl::aliasopt(InputFilenamesFile));<br>
+ cl::opt<bool> DumpInputFileList(<br>
+ "dump-input-file-list", cl::init(false), cl::Hidden,<br>
+ cl::desc("Dump the list of input files and their weights, then exit"));<br>
cl::opt<std::string> OutputFilename("output", cl::value_desc("output"),<br>
cl::init("-"), cl::Required,<br>
cl::desc("Output file"));<br>
@@ -249,15 +283,22 @@ static int merge_main(int argc, const ch<br>
<br>
cl::ParseCommandLineOptions(argc, argv, "LLVM profile data merger\n");<br>
<br>
- if (InputFilenames.empty() && WeightedInputFilenames.empty())<br>
- exitWithError("No input files specified. See " +<br>
- sys::path::filename(argv[0]) + " -help");<br>
-<br>
WeightedFileVector WeightedInputs;<br>
for (StringRef Filename : InputFilenames)<br>
WeightedInputs.push_back(WeightedFile(Filename, 1));<br>
for (StringRef WeightedFilename : WeightedInputFilenames)<br>
WeightedInputs.push_back(parseWeightedFile(WeightedFilename));<br>
+ parseInputFilenamesFile(InputFilenamesFile, WeightedInputs);<br>
+<br>
+ if (WeightedInputs.empty())<br>
+ exitWithError("No input files specified. See " +<br>
+ sys::path::filename(argv[0]) + " -help");<br>
+<br>
+ if (DumpInputFileList) {<br>
+ for (auto &WF : WeightedInputs)<br>
+ outs() << WF.Weight << "," << WF.Filename << "\n";<br>
+ return 0;<br>
+ }<br>
<br>
if (ProfileKind == instr)<br>
mergeInstrProfile(WeightedInputs, OutputFilename, OutputFormat,<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
</blockquote></div></div></blockquote></div>