[PATCH] D30130: ArgList: cache index ranges containing arguments with each ID

[Richard Smith via llvm-commits]

On 22 February 2017 at 20:16, David Blaikie via llvm-commits <
llvm-commits at lists.llvm.org> wrote:

> Any sense of whether it's worth/possible to go the whole way and keep a
> list, rather than a range, for each one? (so that even if they're
> interleaved, it doesn't degenerate?) I may not be fully understanding the
> code to be clear on whether that's an option.
>

That would be possible, but somewhat awkward. We have a tree of option
groups, where the flags for each node are also considered to be part of the
parent node, so we'd end up duplicating a fair amount of information, and
when iterating over the options in multiple groups, we need to provide the
options in command-line order without duplicates, which means merging the
lists back together as we go.

I don't have any evidence that keeping a list rather than a range would
give any further performance improvements on realistic inputs. This was
enough to get the ArgList traversals to disappear from the top of the
profile in my test cases, and in practice it seems that options of the same
type are generally close together on the command line, so this
heuristically seems like it should work pretty well. (A pathological input
would now require a large number of pairs of similar options, with one near
the start of the arg list and one near the end.)

(this also assumes these args can be cleaned up for the real clang work
> occurs - so they don't increase the memory high water mark too much, though
> even with response-file.c, I would hope a list for each in-use flag group
> wouldn't be /too/ costly in the grand scheme of things)
>
> On Fri, Feb 17, 2017 at 5:34 PM Richard Smith via Phabricator <
> reviews at reviews.llvm.org> wrote:
>
>> rsmith created this revision.
>>
>> Improve performance of argument list parsing with large numbers of IDs
>> and large numbers of arguments, by tracking a conservative range of indexes
>> within the argument list that might contain an argument with each ID. In
>> the worst case (when the first and last argument with a given ID are at the
>> opposite ends of the argument list), this still results in a linear-time
>> walk of the list, but it helps substantially in the common case where each
>> ID occurs only once, or a few times close together in the list.
>>
>> This gives a ~10x speedup to clang's `test/Driver/response-file.c`, which
>> constructs a very large set of command line arguments and feeds them to the
>> clang driver.
>>
>> In passing I also converted the interface to use variadic templates. I
>> can split those changes into a separate patch if you'd prefer.
>>
>>
>> Repository:
>>   rL LLVM
>>
>> https://reviews.llvm.org/D30130
>>
>> Files:
>>   include/llvm/Option/ArgList.h
>>   lib/Option/ArgList.cpp
>>
>>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170407/9944b977/attachment.html>