<div dir="ltr">Ah, sorry, I missed that. I'll add that shortly.</div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Nov 30, 2017 at 4:19 PM, Rafael Avila de Espindola <span dir="ltr"><<a href="mailto:rafael.espindola@gmail.com" target="_blank">rafael.espindola@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I think you missed the request for having a test that checks the number<br>
of buckets that is created for a .so.<br>
<br>
Thanks,<br>
Rafael<br>
<div class="HOEnZb"><div class="h5"><br>
Rui Ueyama via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>> writes:<br>
<br>
> Author: ruiu<br>
> Date: Thu Nov 30 15:59:40 2017<br>
> New Revision: 319503<br>
><br>
> URL: <a href="http://llvm.org/viewvc/llvm-project?rev=319503&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project?rev=319503&view=rev</a><br>
> Log:<br>
> Make .gnu.hash section smaller.<br>
><br>
> Our on-disk hash table was unnecessarily large. The cost of collision is<br>
> not high in the .gnu.hash table because each symbol in the .gnu.hash<br>
> table has a hash value with it. So, for each collided symbol, the<br>
> dynamic linker just compares an integer, which is pretty cheap.<br>
><br>
> This patch increases the load factor by about 8. Here's a comparison.<br>
><br>
> $ readelf --histogram libclangSema.so.6.0.0svn-new-<wbr>lld<br>
> Histogram for `.gnu.hash' bucket list length (total of 582 buckets):<br>
> Length Number % of total Coverage<br>
> 0 11 ( 1.9%)<br>
> 1 35 ( 6.0%) 1.5%<br>
> 2 93 ( 16.0%) 9.5%<br>
> 3 108 ( 18.6%) 23.4%<br>
> 4 121 ( 20.8%) 44.1%<br>
> 5 86 ( 14.8%) 62.6%<br>
> 6 63 ( 10.8%) 78.8%<br>
> 7 38 ( 6.5%) 90.2%<br>
> 8 18 ( 3.1%) 96.4%<br>
> 9 6 ( 1.0%) 98.7%<br>
> 10 3 ( 0.5%) 100.0%<br>
><br>
> $ readelf --histogram libclangSema.so.6.0.0svn-old-<wbr>lld<br>
> Histogram for `.gnu.hash' bucket list length (total of 4093 buckets):<br>
> Length Number % of total Coverage<br>
> 0 1498 ( 36.6%)<br>
> 1 1545 ( 37.7%) 37.7%<br>
> 2 712 ( 17.4%) 72.5%<br>
> 3 251 ( 6.1%) 90.9%<br>
> 4 66 ( 1.6%) 97.3%<br>
> 5 16 ( 0.4%) 99.3%<br>
> 6 5 ( 0.1%) 100.0%<br>
><br>
> $ readelf --histogram libclangSema.so.6.0.0svn-bfd<br>
> Histogram for `.gnu.hash' bucket list length (total of 1004 buckets):<br>
> Length Number % of total Coverage<br>
> 0 92 ( 9.2%)<br>
> 1 227 ( 22.6%) 9.8%<br>
> 2 266 ( 26.5%) 32.6%<br>
> 3 222 ( 22.1%) 61.2%<br>
> 4 115 ( 11.5%) 81.0%<br>
> 5 55 ( 5.5%) 92.8%<br>
> 6 21 ( 2.1%) 98.2%<br>
> 7 6 ( 0.6%) 100.0%<br>
><br>
> $ readelf --histogram libclangSema.so.6.0.0svn-gold<br>
> Histogram for `.gnu.hash' bucket list length (total of 2053 buckets):<br>
> Length Number % of total Coverage<br>
> 0 671 ( 32.7%)<br>
> 1 709 ( 34.5%) 30.4%<br>
> 2 470 ( 22.9%) 70.7%<br>
> 3 141 ( 6.9%) 88.9%<br>
> 4 54 ( 2.6%) 98.2%<br>
> 5 5 ( 0.2%) 99.2%<br>
> 6 3 ( 0.1%) 100.0%<br>
><br>
> Differential Revision: <a href="https://reviews.llvm.org/D40683" rel="noreferrer" target="_blank">https://reviews.llvm.org/<wbr>D40683</a><br>
><br>
> Modified:<br>
> lld/trunk/ELF/<wbr>SyntheticSections.cpp<br>
> lld/trunk/test/ELF/gc-<wbr>sections-shared.s<br>
><br>
> Modified: lld/trunk/ELF/<wbr>SyntheticSections.cpp<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/SyntheticSections.cpp?rev=319503&r1=319502&r2=319503&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/lld/trunk/ELF/<wbr>SyntheticSections.cpp?rev=<wbr>319503&r1=319502&r2=319503&<wbr>view=diff</a><br>
> ==============================<wbr>==============================<wbr>==================<br>
> --- lld/trunk/ELF/<wbr>SyntheticSections.cpp (original)<br>
> +++ lld/trunk/ELF/<wbr>SyntheticSections.cpp Thu Nov 30 15:59:40 2017<br>
> @@ -1775,22 +1775,6 @@ static uint32_t hashGnu(StringRef Name)<br>
> return H;<br>
> }<br>
><br>
> -// Returns a number of hash buckets to accomodate given number of elements.<br>
> -// We want to choose a moderate number that is not too small (which<br>
> -// causes too many hash collisions) and not too large (which wastes<br>
> -// disk space.)<br>
> -//<br>
> -// We return a prime number because it (is believed to) achieve good<br>
> -// hash distribution.<br>
> -static size_t getBucketSize(size_t NumSymbols) {<br>
> - // List of largest prime numbers that are not greater than 2^n + 1.<br>
> - for (size_t N : {131071, 65521, 32749, 16381, 8191, 4093, 2039, 1021, 509,<br>
> - 251, 127, 61, 31, 13, 7, 3, 1})<br>
> - if (N <= NumSymbols)<br>
> - return N;<br>
> - return 0;<br>
> -}<br>
> -<br>
> // Add symbols to this symbol hash table. Note that this function<br>
> // destructively sort a given vector -- which is needed because<br>
> // GNU-style hash table places some sorting requirements.<br>
> @@ -1813,7 +1797,12 @@ void GnuHashTableSection::<wbr>addSymbols(std<br>
> Symbols.push_back({B, Ent.StrTabOffset, hashGnu(B->getName())});<br>
> }<br>
><br>
> - NBuckets = getBucketSize(Symbols.size());<br>
> + // We chose load factor 4 for the on-disk hash table. For each hash<br>
> + // collision, the dynamic linker will compare a uint32_t hash value.<br>
> + // Since the integer comparison is quite fast, we believe we can make<br>
> + // the load factor even larger. 4 is just a conservative choice.<br>
> + NBuckets = std::max<size_t>(Symbols.size(<wbr>) / 4, 1);<br>
> +<br>
> std::stable_sort(Symbols.<wbr>begin(), Symbols.end(),<br>
> [&](const Entry &L, const Entry &R) {<br>
> return L.Hash % NBuckets < R.Hash % NBuckets;<br>
><br>
> Modified: lld/trunk/test/ELF/gc-<wbr>sections-shared.s<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/lld/trunk/test/ELF/gc-sections-shared.s?rev=319503&r1=319502&r2=319503&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/lld/trunk/test/ELF/gc-<wbr>sections-shared.s?rev=319503&<wbr>r1=319502&r2=319503&view=diff</a><br>
> ==============================<wbr>==============================<wbr>==================<br>
> --- lld/trunk/test/ELF/gc-<wbr>sections-shared.s (original)<br>
> +++ lld/trunk/test/ELF/gc-<wbr>sections-shared.s Thu Nov 30 15:59:40 2017<br>
> @@ -34,28 +34,28 @@<br>
> # CHECK-NEXT: Section: .text<br>
> # CHECK-NEXT: }<br>
> # CHECK-NEXT: Symbol {<br>
> -# CHECK-NEXT: Name: foo<br>
> +# CHECK-NEXT: Name: baz<br>
> # CHECK-NEXT: Value:<br>
> # CHECK-NEXT: Size:<br>
> # CHECK-NEXT: Binding: Global<br>
> # CHECK-NEXT: Type:<br>
> # CHECK-NEXT: Other:<br>
> -# CHECK-NEXT: Section: .text<br>
> +# CHECK-NEXT: Section: Undefined<br>
> # CHECK-NEXT: }<br>
> # CHECK-NEXT: Symbol {<br>
> -# CHECK-NEXT: Name: qux<br>
> +# CHECK-NEXT: Name: foo<br>
> # CHECK-NEXT: Value:<br>
> # CHECK-NEXT: Size:<br>
> -# CHECK-NEXT: Binding: Weak<br>
> +# CHECK-NEXT: Binding: Global<br>
> # CHECK-NEXT: Type:<br>
> # CHECK-NEXT: Other:<br>
> -# CHECK-NEXT: Section: Undefined<br>
> +# CHECK-NEXT: Section: .text<br>
> # CHECK-NEXT: }<br>
> # CHECK-NEXT: Symbol {<br>
> -# CHECK-NEXT: Name: baz<br>
> +# CHECK-NEXT: Name: qux<br>
> # CHECK-NEXT: Value:<br>
> # CHECK-NEXT: Size:<br>
> -# CHECK-NEXT: Binding: Global<br>
> +# CHECK-NEXT: Binding: Weak<br>
> # CHECK-NEXT: Type:<br>
> # CHECK-NEXT: Other:<br>
> # CHECK-NEXT: Section: Undefined<br>
> @@ -90,31 +90,31 @@<br>
> # CHECK2-NEXT: Section: .text<br>
> # CHECK2-NEXT: }<br>
> # CHECK2-NEXT: Symbol {<br>
> -# CHECK2-NEXT: Name: qux<br>
> +# CHECK2-NEXT: Name: baz<br>
> # CHECK2-NEXT: Value:<br>
> # CHECK2-NEXT: Size:<br>
> -# CHECK2-NEXT: Binding: Weak<br>
> +# CHECK2-NEXT: Binding: Global<br>
> # CHECK2-NEXT: Type:<br>
> # CHECK2-NEXT: Other:<br>
> # CHECK2-NEXT: Section: Undefined<br>
> # CHECK2-NEXT: }<br>
> # CHECK2-NEXT: Symbol {<br>
> -# CHECK2-NEXT: Name: foo<br>
> +# CHECK2-NEXT: Name: qux<br>
> # CHECK2-NEXT: Value:<br>
> # CHECK2-NEXT: Size:<br>
> -# CHECK2-NEXT: Binding: Global<br>
> +# CHECK2-NEXT: Binding: Weak<br>
> # CHECK2-NEXT: Type:<br>
> # CHECK2-NEXT: Other:<br>
> -# CHECK2-NEXT: Section: .text<br>
> +# CHECK2-NEXT: Section: Undefined<br>
> # CHECK2-NEXT: }<br>
> # CHECK2-NEXT: Symbol {<br>
> -# CHECK2-NEXT: Name: baz<br>
> +# CHECK2-NEXT: Name: foo<br>
> # CHECK2-NEXT: Value:<br>
> # CHECK2-NEXT: Size:<br>
> # CHECK2-NEXT: Binding: Global<br>
> # CHECK2-NEXT: Type:<br>
> # CHECK2-NEXT: Other:<br>
> -# CHECK2-NEXT: Section: Undefined<br>
> +# CHECK2-NEXT: Section: .text<br>
> # CHECK2-NEXT: }<br>
> # CHECK2-NEXT: ]<br>
><br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> llvm-commits mailing list<br>
> <a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-commits</a><br>
</div></div></blockquote></div><br></div>