r245036 - Add structed way to express command line options in the compilation database.

[Hans Wennborg via cfe-commits]

On Fri, Aug 14, 2015 at 2:55 AM, Manuel Klimek via cfe-commits
<cfe-commits at lists.llvm.org> wrote:
> Author: klimek
> Date: Fri Aug 14 04:55:36 2015
> New Revision: 245036
>
> URL: http://llvm.org/viewvc/llvm-project?rev=245036&view=rev
> Log:
> Add structed way to express command line options in the compilation database.
>
> Currently, arguments are passed via the string attribute 'command',
> assuming a shell-escaped / quoted command line to extract the original
> arguments. This works well enough on Unix systems, but turns out to be
> problematic for Windows tools to generate.
>
> This CL adds a new attribute 'arguments', an array of strings, which
> specifies the exact command line arguments. If 'arguments' is available
> in the compilation database, it is preferred to 'commands'.
>
> Currently there is no plan to retire 'commands': there are enough
> different use cases where users want to create their own mechanism for
> creating compilation databases, that it doesn't make sense to force them
> all to implement shell command line parsing.
>
> Patch by Daniel Dilts.

This seems to have caused a bad performance regression for a tool we
use in Chromium. On the file I tried, run-time went from 0.4 s to 3.0
s (7.5x slow-down), and peak memory usage from 42 MB to 366 MB (8.7x).

I suspect what's happened is that JSONCompilationDatabase::parse()
became significantly slower because it's now calling
unescapeCommandLine() and allocating a std::vector on each "command"
field during parsing, whereas previously the code wouldn't do that
until JSONCompilationDatabase::getCommands() was called on a specific
file or set of files.

One idea for fixing this would be to make the second part of
CompileCommandRef be a yaml::Node pointer, and look at the node type:
if it's a ScalarNode, it contains "commands" and needs to be
unescaped; if it's a SequenceNode, it's "arguments".