[PATCH] D17529: ELF: Implement ICF.

[Rui Ueyama via llvm-commits]

On Tue, Feb 23, 2016 at 4:30 PM, Sean Silva <chisophugis at gmail.com> wrote:

>
>
> On Tue, Feb 23, 2016 at 3:25 PM, Rui Ueyama <ruiu at google.com> wrote:
>
>> By the way, for ICF, I think we eventually want to have the compiler emit
>> a (cryptographically-safe) hash value for each section to eliminate
>> byte-by-byte comparison.
>>
>
> What fraction of time in ICF is spent doing section content comparisons?
> How many times is each section's content actually looked at? Currently we
> have to scan the entire section every time we get to the comparison end of
> equalsConstant. If we cache a hash value computed during that scan (even if
> it is not a super great hash) we could probably avoid a number of the scans.
>

We have data for COFF.
http://llvm.org/viewvc/llvm-project?view=revision&revision=247961

"LLD took 1198 ms to link itself to produce a 27.11 MB executable. Of
which, ICF accounted for 536 ms. This patch cuts it by 90 ms, which is 17%
speedup of ICF and 7.5% speedup overall. All numbers are median of ten
runs."

Note that checksums in COFF is not cryptographically strong but just CRC32,
so we do compare section contents at last. This could have been eliminated
if they chose to use MD5 or SHA1. That means the above number is more like
a lower-bound.

-- Sean Silva
>
>
>>
>> That would not only greatly reduce time we spent on ICF but also on
>> build-id computation. "--build-id" option is casually added to the linker
>> command line, but that is actually pretty expensive operation. For example,
>> GNU gold spends about 10% link time just for computing a hash value.
>>
>> On Tue, Feb 23, 2016 at 3:12 PM, Sean Silva via llvm-commits <
>> llvm-commits at lists.llvm.org> wrote:
>>
>>>
>>>
>>> On Tue, Feb 23, 2016 at 2:32 PM, Rafael Espíndola <
>>> llvm-commits at lists.llvm.org> wrote:
>>>
>>>> > I think the "take the address" part is the crucial hint. Can't you
>>>> tell
>>>> > from the combination of visibility and relocations what functions are
>>>> > safe to merge? The only tricky part I can imagine is identifying
>>>> > vtables, as they do not classify as pointer leak.
>>>>
>>>> Somewhat. Gold has that on its list of heuristics, but it is pretty
>>>> nasty. In particular, using files compiled with -fPIC to produce
>>>> regular executables breaks the heuristic.
>>>>
>>>> Lets focus on the non-safe version for now. When time comes I think
>>>> the correct thing to do is to implement the nop padding as it is
>>>> always safe and then propose adding a flag to ELF to say that a
>>>> particular section/symbol doesn't need an unique address.
>>>>
>>>
>>> Yeah. The compiler already knows the this information. No need for the
>>> linker to bend over backward to try to recover it.
>>>
>>> -- Sean Silva
>>>
>>>
>>>>
>>>> Cheers,
>>>> Rafael
>>>> _______________________________________________
>>>> llvm-commits mailing list
>>>> llvm-commits at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>>
>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160223/4ace8aa4/attachment.html>