r245036 - Add structed way to express command line options in the compilation database.

[Hans Wennborg via cfe-commits]

Yes, the run-time is now back to where it was before the regression. Thanks!

On Tue, Sep 8, 2015 at 8:16 AM, Manuel Klimek <klimek at google.com> wrote:
> In r247018 I get a ~8x speedup on a synthetic benchmark I tried. Can you
> validate this fixes the regression?
>
> On Sat, Sep 5, 2015 at 12:56 AM Hans Wennborg <hans at chromium.org> wrote:
>>
>> On Fri, Aug 14, 2015 at 2:55 AM, Manuel Klimek via cfe-commits
>> <cfe-commits at lists.llvm.org> wrote:
>> > Author: klimek
>> > Date: Fri Aug 14 04:55:36 2015
>> > New Revision: 245036
>> >
>> > URL: http://llvm.org/viewvc/llvm-project?rev=245036&view=rev
>> > Log:
>> > Add structed way to express command line options in the compilation
>> > database.
>> >
>> > Currently, arguments are passed via the string attribute 'command',
>> > assuming a shell-escaped / quoted command line to extract the original
>> > arguments. This works well enough on Unix systems, but turns out to be
>> > problematic for Windows tools to generate.
>> >
>> > This CL adds a new attribute 'arguments', an array of strings, which
>> > specifies the exact command line arguments. If 'arguments' is available
>> > in the compilation database, it is preferred to 'commands'.
>> >
>> > Currently there is no plan to retire 'commands': there are enough
>> > different use cases where users want to create their own mechanism for
>> > creating compilation databases, that it doesn't make sense to force them
>> > all to implement shell command line parsing.
>> >
>> > Patch by Daniel Dilts.
>>
>> This seems to have caused a bad performance regression for a tool we
>> use in Chromium. On the file I tried, run-time went from 0.4 s to 3.0
>> s (7.5x slow-down), and peak memory usage from 42 MB to 366 MB (8.7x).
>>
>> I suspect what's happened is that JSONCompilationDatabase::parse()
>> became significantly slower because it's now calling
>> unescapeCommandLine() and allocating a std::vector on each "command"
>> field during parsing, whereas previously the code wouldn't do that
>> until JSONCompilationDatabase::getCommands() was called on a specific
>> file or set of files.
>>
>> One idea for fixing this would be to make the second part of
>> CompileCommandRef be a yaml::Node pointer, and look at the node type:
>> if it's a ScalarNode, it contains "commands" and needs to be
>> unescaped; if it's a SequenceNode, it's "arguments".