[llvm] r271709 - [llvm-profdata] Add option to ingest filepaths from a file
Chandler Carruth via llvm-commits
llvm-commits at lists.llvm.org
Mon Jun 6 11:29:06 PDT 2016
I'm very certain that wasn't the problem. Look at the build bot issues --
every character following a '\' in the path is missing because it was
escaped.
On Mon, Jun 6, 2016 at 11:27 AM Vedant Kumar via llvm-commits <
llvm-commits at lists.llvm.org> wrote:
> I'm sorry for the breakage. I'll be more attentive in the future.
>
> I think having extra hash ("#") characters preceding some RUN lines
> confused lit. I'll try again with these lines fixed and watch the bots.
>
> vedant
>
>
> > On Jun 3, 2016, at 8:14 PM, Chandler Carruth <chandlerc at gmail.com>
> wrote:
> >
> > Ok, looks like all the other patches were fixing other things, the
> Windows problem was in the original commit and persists. I've unwound the
> entire stack here so we get our test runs green again in r271760. Sorry for
> the trouble.
> >
> > On Fri, Jun 3, 2016 at 8:02 PM Chandler Carruth <chandlerc at gmail.com>
> wrote:
> > Err... I see twhere were still further edits here. Sorry for perhaps
> replying to the wrong commit. But the build bot I've cited is in fact still
> failing.
> >
> > On Fri, Jun 3, 2016 at 8:00 PM Chandler Carruth <chandlerc at gmail.com>
> wrote:
> > On Fri, Jun 3, 2016 at 12:11 PM Vedant Kumar via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
> > Author: vedantk
> > Date: Fri Jun 3 14:05:20 2016
> > New Revision: 271709
> >
> > URL: http://llvm.org/viewvc/llvm-project?rev=271709&view=rev
> > Log:
> > [llvm-profdata] Add option to ingest filepaths from a file
> >
> > This has been in the tree for 7 hours with failing tests on all the
> Windows bots. Please try to watch the bots more carefully in the future.
> >
> > I'm reverting for now. You'll need to do something to properly handle
> quoting and escaping with path names here.
> >
> > Here is one of the failures for your reference:
> > http://lab.llvm.org:8011/builders/clang-x86-win2008-selfhost/builds/8317
> >
> >
> > Differential Revision: http://reviews.llvm.org/D20980
> >
> > Added:
> > llvm/trunk/test/tools/llvm-profdata/input-filenames.test
> > Modified:
> > llvm/trunk/docs/CommandGuide/llvm-profdata.rst
> > llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp
> >
> > Modified: llvm/trunk/docs/CommandGuide/llvm-profdata.rst
> > URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/CommandGuide/llvm-profdata.rst?rev=271709&r1=271708&r2=271709&view=diff
> >
> ==============================================================================
> > --- llvm/trunk/docs/CommandGuide/llvm-profdata.rst (original)
> > +++ llvm/trunk/docs/CommandGuide/llvm-profdata.rst Fri Jun 3 14:05:20
> 2016
> > @@ -44,6 +44,9 @@ interpreted as relatively more important
> > nature of the training runs it may be useful to adjust the weight given
> to each
> > input file by using the ``-weighted-input`` option.
> >
> > +Profiles passed in via ``-weighted-input``, ``-input-files``, or via
> positional
> > +arguments are processed once for each time they are seen.
> > +
> >
> > OPTIONS
> > ^^^^^^^
> > @@ -65,6 +68,12 @@ OPTIONS
> > Input files specified without using this option are assigned a default
> > weight of 1. Examples are shown below.
> >
> > +.. option:: -input-files=path, -f=path
> > +
> > + Specify a file which contains a list of files to merge. The entries
> in this
> > + file are newline-separated. Lines starting with '#' are skipped.
> Entries may
> > + be of the form <filename> or <weight>,<filename>.
> > +
> > .. option:: -instr (default)
> >
> > Specify that the input profile is an instrumentation-based profile.
> >
> > Added: llvm/trunk/test/tools/llvm-profdata/input-filenames.test
> > URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-profdata/input-filenames.test?rev=271709&view=auto
> >
> ==============================================================================
> > --- llvm/trunk/test/tools/llvm-profdata/input-filenames.test (added)
> > +++ llvm/trunk/test/tools/llvm-profdata/input-filenames.test Fri Jun 3
> 14:05:20 2016
> > @@ -0,0 +1,16 @@
> > +# Create an input file.
> > +RUN: printf '# comment 1\n' > %t.input
> > +RUN: printf ' # comment 2\n' >> %t.input
> > +RUN: printf 'foo\n' >> %t.input
> > +RUN: printf ' bar\n' >> %t.input
> > +RUN: printf "2,%t.weighted\n" >> %t.input
> > +
> > +# Create the weighted file, since these actually need to exist.
> > +RUN: printf ' ' > %t.weighted
> > +
> > +# RUN: llvm-profdata merge -f %t.input -dump-input-file-list -o
> /dev/null | FileCheck %s
> > +# RUN: llvm-profdata merge -input-files %t.input -dump-input-file-list
> -o /dev/null | FileCheck %s
> > +
> > +# CHECK: 1,foo
> > +# CHECK-NEXT: 1,bar
> > +# CHECK-NEXT: 2,{{.*}}.weighted
> >
> > Modified: llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp
> > URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp?rev=271709&r1=271708&r2=271709&view=diff
> >
> ==============================================================================
> > --- llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp (original)
> > +++ llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp Fri Jun 3 14:05:20
> 2016
> > @@ -223,11 +223,45 @@ static WeightedFile parseWeightedFile(co
> > return WeightedFile(FileName, Weight);
> > }
> >
> > +static void parseInputFilenamesFile(const StringRef &InputFilenamesFile,
> > + WeightedFileVector &WFV) {
> > + if (InputFilenamesFile == "")
> > + return;
> > +
> > + auto Buf = MemoryBuffer::getFileOrSTDIN(InputFilenamesFile);
> > + if (!Buf)
> > + exitWithErrorCode(Buf.getError(), InputFilenamesFile);
> > +
> > + StringRef Data = Buf.get()->getBuffer();
> > + SmallVector<StringRef, 8> Entries;
> > + Data.split(Entries, '\n', /*MaxSplit=*/-1, /*KeepEmpty=*/false);
> > + for (const StringRef &FileWeightEntry : Entries) {
> > + StringRef SanitizedEntry = FileWeightEntry.trim(" \t\v\f\r");
> > + // Skip comments.
> > + if (SanitizedEntry.startswith("#"))
> > + continue;
> > + // If there's no comma, it's an unweighted profile.
> > + else if (SanitizedEntry.rfind(',') == StringRef::npos)
> > + WFV.emplace_back(SanitizedEntry, 1);
> > + else
> > + WFV.emplace_back(parseWeightedFile(SanitizedEntry));
> > + }
> > +}
> > +
> > static int merge_main(int argc, const char *argv[]) {
> > cl::list<std::string> InputFilenames(cl::Positional,
> > cl::desc("<filename...>"));
> > cl::list<std::string> WeightedInputFilenames("weighted-input",
> >
> cl::desc("<weight>,<filename>"));
> > + cl::opt<std::string> InputFilenamesFile(
> > + "input-files", cl::init(""),
> > + cl::desc("Path to file containing newline-separated "
> > + "<filename>[,<weight>] entries"));
> > + cl::alias InputFilenamesFileA("f", cl::desc("Alias for
> --input-files"),
> > + cl::aliasopt(InputFilenamesFile));
> > + cl::opt<bool> DumpInputFileList(
> > + "dump-input-file-list", cl::init(false), cl::Hidden,
> > + cl::desc("Dump the list of input files and their weights, then
> exit"));
> > cl::opt<std::string> OutputFilename("output",
> cl::value_desc("output"),
> > cl::init("-"), cl::Required,
> > cl::desc("Output file"));
> > @@ -249,15 +283,22 @@ static int merge_main(int argc, const ch
> >
> > cl::ParseCommandLineOptions(argc, argv, "LLVM profile data merger\n");
> >
> > - if (InputFilenames.empty() && WeightedInputFilenames.empty())
> > - exitWithError("No input files specified. See " +
> > - sys::path::filename(argv[0]) + " -help");
> > -
> > WeightedFileVector WeightedInputs;
> > for (StringRef Filename : InputFilenames)
> > WeightedInputs.push_back(WeightedFile(Filename, 1));
> > for (StringRef WeightedFilename : WeightedInputFilenames)
> > WeightedInputs.push_back(parseWeightedFile(WeightedFilename));
> > + parseInputFilenamesFile(InputFilenamesFile, WeightedInputs);
> > +
> > + if (WeightedInputs.empty())
> > + exitWithError("No input files specified. See " +
> > + sys::path::filename(argv[0]) + " -help");
> > +
> > + if (DumpInputFileList) {
> > + for (auto &WF : WeightedInputs)
> > + outs() << WF.Weight << "," << WF.Filename << "\n";
> > + return 0;
> > + }
> >
> > if (ProfileKind == instr)
> > mergeInstrProfile(WeightedInputs, OutputFilename, OutputFormat,
> >
> >
> > _______________________________________________
> > llvm-commits mailing list
> > llvm-commits at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160606/7bfd291b/attachment.html>
More information about the llvm-commits
mailing list