<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none"><!-- p { margin-top: 0px; margin-bottom: 0px; }--></style>
</head>
<body dir="ltr" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p><span style="color: rgb(33, 33, 33); font-size: 12pt;">>That said, I don't think this patch makes much sense. There's no theory behind it to explain why this could make things faster (it shouldn't as long as the hash >f</span><span style="color: rgb(33, 33, 33); font-size: 12pt;">unction
generates evenly distributed hash value). I think what we should do is</span><br>
</p>
<p><span style="color: rgb(33, 33, 33); font-size: 12pt;"><br>
</span></p>
<p><span style="color: rgb(33, 33, 33); font-size: 12pt;">I'll explain my theory at the end.</span></p>
<p><span style="color: rgb(33, 33, 33); font-size: 12pt;"><br>
</span></p>
<div style="color: rgb(33, 33, 33);">
<div>
<div dir="ltr">
<div>> </div>
<div>>1. first, verify that our bloom filter is really correct by taking a look at the output binary carefully, because even if the bloom filter is wrong, that doesn't cause any >correctness issue, but instead it just makes things slower, and then<br>
</div>
<div><br>
</div>
<div>That is not entirely correct. If you put few excessive bits to bloom filter, nothing too bad will happen,</div>
<div>you'll increase the amount of false positives, in that case you're right, <span style="font-size: 12pt;">but if you forget to set proper bit, things will most likely just stop working.</span></div>
<div><span style="font-size: 12pt;">It can be checked simple - I changed the position of one of bits
</span><span style="font-size: 12pt;">in our</span><span style="font-size: 12pt;"> fi</span><span style="font-size: 12pt;">lter to break it, </span></div>
<div><span style="font-size: 12pt;"></span><span style="font-size: 12pt;">and instantly faced with "symbol lookup error" from loader.</span><span style="font-size: 12pt;"> </span></div>
<div><br>
</div>
<div>So I think our filter is correct. Not only basing on above, but also I debugged gold, I tweaked LLD settings to be equal with gold's for sample I used</div>
<div>and compared what we <span style="font-size: 12pt;">do. (sample test.s I used and LLD patch are in attachment, it is based on one of our tests + had to add few symbols that</span></div>
<div><span style="font-size: 12pt;">gold adds to .dynsym, for gold had to add </span><span style="font-size: 12pt;">--noinhibit-exec</span><span style="font-size: 12pt;"> to produce the output</span><span style="font-size: 12pt;">).</span></div>
<div><span style="font-size: 12pt;"><br>
</span></div>
<div><span style="font-size: 12pt;">Even with the fact that gold uses slightly different order of filling bloom filter (we use list of symbols that is already sorted by bucket ids, gold does not, </span></div>
<div><span style="font-size: 12pt;">what breaks the order of filling, though symbols in the final output are still sorted fine), </span><span style="font-size: 12pt;">at the end I had 2 equal bloom filters.</span></div>
<div><br>
</div>
<div>What was different in output is a most of hash table part of .gnu_hash. I had no chance to investigate it deeply yet. I guess it is relative to symbols order too,<br>
</div>
<div>but probably worth to check to be sure we are not broken here. Have plans to do that.<br>
</div>
<div><br>
</div>
<div>>2. understand what's the difference between our output and GNU linker's output.</div>
<div>> </div>
<div>>I believe just changing parameters and benchmaking them is not a good strategy. I want to understand it first before getting some optimized parameters based on >experiments.</div>
</div>
<div class="gmail_extra"><br>
I think worth to mention following thing. When we fill bloom filter words, we calculate index of word as<br>
</div>
<div class="gmail_extra">size_t I = (Sym.Hash / C) & (MaskWords - 1);<br>
</div>
<div class="gmail_extra">(<a href="https://github.com/llvm-mirror/lld/blob/master/ELF/SyntheticSections.cpp#L1746">https://github.com/llvm-mirror/lld/blob/master/ELF/SyntheticSections.cpp#L1746</a>)<br>
</div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">It looks correct (and FWIW gold do the same logic), but noticable thing here is that even when we have<br>
</div>
<div class="gmail_extra">symbols that all has different Hash, left part which is "<span style="color: rgb(33, 33, 33); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; background-color: rgb(255, 255, 255);">Sym.Hash / C</span>" breaks things
a bit, so that in a list below:<br>
</div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">
<div>Name: __bss_start, Hash: 475558360, Hash/C: 7430599</div>
<div>Name: _end, Hash: 209000339, Hash/C: 32656270</div>
<div>Name: sym1, Hash: 209074775, Hash/C: 32667840</div>
<div>Name: sym2, Hash: 209074776, Hash/C: 32667840</div>
<div>Name: _edata, Hash: 397399875, Hash/C: 62084373</div>
<div>Name: sym3, Hash: 209074777, Hash/C: 32667840</div>
<div>Name: sym4, Hash: 209074778, Hash/C: 32667840</div>
<div>Name: sym5, Hash: 209074779, Hash/C: 32667840</div>
<div>Name: sym10, Hash: 27500887, Hash/C: 4296904</div>
<div>Name: sym6, Hash: 209074780, Hash/C: 32667840</div>
<div>Name: sym11, Hash: 27500888, Hash/C: 4296904</div>
<div>Name: sym7, Hash: 209074781, Hash/C: 32667840</div>
<div>Name: sym12, Hash: 27500889, Hash/C: 4296904</div>
<div>Name: sym8, Hash: 209074782, Hash/C: 32667840</div>
<div>Name: sym13, Hash: 27500890, Hash/C: 4296904</div>
<div>Name: sym9, Hash: 209074783, Hash/C: 32667840</div>
<div>Name: sym14, Hash: 27500891, Hash/C: 4296904</div>
<div>Name: sym15, Hash: 27500892, Hash/C: 4296904</div>
<div>Name: sym16, Hash: 27500893, Hash/C: 4296904<br>
</div>
<br>
</div>
<div class="gmail_extra">only __bss_start and _edata has I == 1, all other symbols has I == 0. I think that shows that fill of bloom filter can be </div>
<div class="gmail_extra">very <span style="font-size: 12pt;">depenent on symbol names.</span></div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">My theory was based on tweaking Shift2 so that it should set the new second bit as often as possible.<br>
</div>
<div class="gmail_extra">It was invented basing on following idea. Imaging we have some symbols "A", "B", "C".<br>
</div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">1) Lets assume we placed A to bloom filter and it became 11000000.<br>
</div>
<div class="gmail_extra">2) Lets try to lookup B. We should fail. If B has first bit equal to 10000000, then ideal situation for us is that B should have<br>
</div>
<div class="gmail_extra">second bit different from 01000000, because we do not want to have false positive during lookup.<br>
</div>
<div class="gmail_extra">3) So we want to place B, basing on (2), we want to setup Shift2 so that it will place second bit on a free space,<br>
</div>
<div class="gmail_extra">like result can be 11100000.<br>
</div>
<div class="gmail_extra">4) Repeat 1-3 with C. We should have something like 11110000 now.<br>
</div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">It shows it can be reasonable to fill as much bits as possible.
<span style="font-size: 12pt;">It is probably not too strong heuristics, </span></div>
<div class="gmail_extra"><span style="font-size: 12pt;">but it shows some good results.</span></div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">George.<br>
</div>
</div>
</div>
</body>
</html>