[PATCH] D27152: Merge strings using sharded hash tables.

[Rafael Avila de Espindola via llvm-commits]

Rui Ueyama <ruiu at google.com> writes:

> On Fri, Dec 9, 2016 at 8:30 PM, Rafael Avila de Espindola <
> rafael.espindola at gmail.com> wrote:
>
>>
>> So, my only comment is that this seems to be a bit too much effort to
>> optimize string merging in a multi threaded environment. We should
>> really look into what .dwo gets us and then see if there are still so
>> many strings left to merge.
>>
>
> Do you mean you want this patch to not be submitted? Split DWARF is one
> good thing, but I think this is also useful for a common use case.
>
> Overall, this patch adds 170 lines and deletes 57 lines. If you subtract
> comment lines, this patch adds less than 100 lines. I don't think that's
> too complicated.

I am not completely opposed, but we have to make sure that

* Single thread performance is OK (bad for the concurrent one).
* We produce identical results without threads (bad for this one).

My main suggestion is still to try to get debug fission (and single file
debug fission) working and see if we still have a problem. All the work
you did here would still be relevant, but for dwp.

As for using multiple threads: Given that the single threaded algorithm
for the non tail merging case is probably as good as it gets, we have to
find a way to produce the same result with multiple threads.

What the single threaded case does is keep the first of each string it
sees. So maybe in the sharded hashes case what you need to do is
remember the string number and do a final parallel sort on that? It
should be fairly fast since the sort would be just on an integer.

Cheers,
Rafael