[llvm] r271709 - [llvm-profdata] Add option to ingest filepaths from a file
Vedant Kumar via llvm-commits
llvm-commits at lists.llvm.org
Mon Jun 6 11:27:50 PDT 2016
I'm sorry for the breakage. I'll be more attentive in the future.
I think having extra hash ("#") characters preceding some RUN lines confused lit. I'll try again with these lines fixed and watch the bots.
vedant
> On Jun 3, 2016, at 8:14 PM, Chandler Carruth <chandlerc at gmail.com> wrote:
>
> Ok, looks like all the other patches were fixing other things, the Windows problem was in the original commit and persists. I've unwound the entire stack here so we get our test runs green again in r271760. Sorry for the trouble.
>
> On Fri, Jun 3, 2016 at 8:02 PM Chandler Carruth <chandlerc at gmail.com> wrote:
> Err... I see twhere were still further edits here. Sorry for perhaps replying to the wrong commit. But the build bot I've cited is in fact still failing.
>
> On Fri, Jun 3, 2016 at 8:00 PM Chandler Carruth <chandlerc at gmail.com> wrote:
> On Fri, Jun 3, 2016 at 12:11 PM Vedant Kumar via llvm-commits <llvm-commits at lists.llvm.org> wrote:
> Author: vedantk
> Date: Fri Jun 3 14:05:20 2016
> New Revision: 271709
>
> URL: http://llvm.org/viewvc/llvm-project?rev=271709&view=rev
> Log:
> [llvm-profdata] Add option to ingest filepaths from a file
>
> This has been in the tree for 7 hours with failing tests on all the Windows bots. Please try to watch the bots more carefully in the future.
>
> I'm reverting for now. You'll need to do something to properly handle quoting and escaping with path names here.
>
> Here is one of the failures for your reference:
> http://lab.llvm.org:8011/builders/clang-x86-win2008-selfhost/builds/8317
>
>
> Differential Revision: http://reviews.llvm.org/D20980
>
> Added:
> llvm/trunk/test/tools/llvm-profdata/input-filenames.test
> Modified:
> llvm/trunk/docs/CommandGuide/llvm-profdata.rst
> llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp
>
> Modified: llvm/trunk/docs/CommandGuide/llvm-profdata.rst
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/CommandGuide/llvm-profdata.rst?rev=271709&r1=271708&r2=271709&view=diff
> ==============================================================================
> --- llvm/trunk/docs/CommandGuide/llvm-profdata.rst (original)
> +++ llvm/trunk/docs/CommandGuide/llvm-profdata.rst Fri Jun 3 14:05:20 2016
> @@ -44,6 +44,9 @@ interpreted as relatively more important
> nature of the training runs it may be useful to adjust the weight given to each
> input file by using the ``-weighted-input`` option.
>
> +Profiles passed in via ``-weighted-input``, ``-input-files``, or via positional
> +arguments are processed once for each time they are seen.
> +
>
> OPTIONS
> ^^^^^^^
> @@ -65,6 +68,12 @@ OPTIONS
> Input files specified without using this option are assigned a default
> weight of 1. Examples are shown below.
>
> +.. option:: -input-files=path, -f=path
> +
> + Specify a file which contains a list of files to merge. The entries in this
> + file are newline-separated. Lines starting with '#' are skipped. Entries may
> + be of the form <filename> or <weight>,<filename>.
> +
> .. option:: -instr (default)
>
> Specify that the input profile is an instrumentation-based profile.
>
> Added: llvm/trunk/test/tools/llvm-profdata/input-filenames.test
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-profdata/input-filenames.test?rev=271709&view=auto
> ==============================================================================
> --- llvm/trunk/test/tools/llvm-profdata/input-filenames.test (added)
> +++ llvm/trunk/test/tools/llvm-profdata/input-filenames.test Fri Jun 3 14:05:20 2016
> @@ -0,0 +1,16 @@
> +# Create an input file.
> +RUN: printf '# comment 1\n' > %t.input
> +RUN: printf ' # comment 2\n' >> %t.input
> +RUN: printf 'foo\n' >> %t.input
> +RUN: printf ' bar\n' >> %t.input
> +RUN: printf "2,%t.weighted\n" >> %t.input
> +
> +# Create the weighted file, since these actually need to exist.
> +RUN: printf ' ' > %t.weighted
> +
> +# RUN: llvm-profdata merge -f %t.input -dump-input-file-list -o /dev/null | FileCheck %s
> +# RUN: llvm-profdata merge -input-files %t.input -dump-input-file-list -o /dev/null | FileCheck %s
> +
> +# CHECK: 1,foo
> +# CHECK-NEXT: 1,bar
> +# CHECK-NEXT: 2,{{.*}}.weighted
>
> Modified: llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp?rev=271709&r1=271708&r2=271709&view=diff
> ==============================================================================
> --- llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp (original)
> +++ llvm/trunk/tools/llvm-profdata/llvm-profdata.cpp Fri Jun 3 14:05:20 2016
> @@ -223,11 +223,45 @@ static WeightedFile parseWeightedFile(co
> return WeightedFile(FileName, Weight);
> }
>
> +static void parseInputFilenamesFile(const StringRef &InputFilenamesFile,
> + WeightedFileVector &WFV) {
> + if (InputFilenamesFile == "")
> + return;
> +
> + auto Buf = MemoryBuffer::getFileOrSTDIN(InputFilenamesFile);
> + if (!Buf)
> + exitWithErrorCode(Buf.getError(), InputFilenamesFile);
> +
> + StringRef Data = Buf.get()->getBuffer();
> + SmallVector<StringRef, 8> Entries;
> + Data.split(Entries, '\n', /*MaxSplit=*/-1, /*KeepEmpty=*/false);
> + for (const StringRef &FileWeightEntry : Entries) {
> + StringRef SanitizedEntry = FileWeightEntry.trim(" \t\v\f\r");
> + // Skip comments.
> + if (SanitizedEntry.startswith("#"))
> + continue;
> + // If there's no comma, it's an unweighted profile.
> + else if (SanitizedEntry.rfind(',') == StringRef::npos)
> + WFV.emplace_back(SanitizedEntry, 1);
> + else
> + WFV.emplace_back(parseWeightedFile(SanitizedEntry));
> + }
> +}
> +
> static int merge_main(int argc, const char *argv[]) {
> cl::list<std::string> InputFilenames(cl::Positional,
> cl::desc("<filename...>"));
> cl::list<std::string> WeightedInputFilenames("weighted-input",
> cl::desc("<weight>,<filename>"));
> + cl::opt<std::string> InputFilenamesFile(
> + "input-files", cl::init(""),
> + cl::desc("Path to file containing newline-separated "
> + "<filename>[,<weight>] entries"));
> + cl::alias InputFilenamesFileA("f", cl::desc("Alias for --input-files"),
> + cl::aliasopt(InputFilenamesFile));
> + cl::opt<bool> DumpInputFileList(
> + "dump-input-file-list", cl::init(false), cl::Hidden,
> + cl::desc("Dump the list of input files and their weights, then exit"));
> cl::opt<std::string> OutputFilename("output", cl::value_desc("output"),
> cl::init("-"), cl::Required,
> cl::desc("Output file"));
> @@ -249,15 +283,22 @@ static int merge_main(int argc, const ch
>
> cl::ParseCommandLineOptions(argc, argv, "LLVM profile data merger\n");
>
> - if (InputFilenames.empty() && WeightedInputFilenames.empty())
> - exitWithError("No input files specified. See " +
> - sys::path::filename(argv[0]) + " -help");
> -
> WeightedFileVector WeightedInputs;
> for (StringRef Filename : InputFilenames)
> WeightedInputs.push_back(WeightedFile(Filename, 1));
> for (StringRef WeightedFilename : WeightedInputFilenames)
> WeightedInputs.push_back(parseWeightedFile(WeightedFilename));
> + parseInputFilenamesFile(InputFilenamesFile, WeightedInputs);
> +
> + if (WeightedInputs.empty())
> + exitWithError("No input files specified. See " +
> + sys::path::filename(argv[0]) + " -help");
> +
> + if (DumpInputFileList) {
> + for (auto &WF : WeightedInputs)
> + outs() << WF.Weight << "," << WF.Filename << "\n";
> + return 0;
> + }
>
> if (ProfileKind == instr)
> mergeInstrProfile(WeightedInputs, OutputFilename, OutputFormat,
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
More information about the llvm-commits
mailing list