<div dir="ltr">If you can get profiles, it may help.<div>You may discover this is caused by some non-determinism causing something to do or not do an expensive thing.</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Feb 12, 2018 at 11:56 AM, Brian Rzycki via Phabricator <span dir="ltr"><<a href="mailto:reviews@reviews.llvm.org" target="_blank">reviews@reviews.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">brzycki added a comment.<br>
<br>
I'm seeing considerable variance on the tramp3d-v4 compile time test before and after the preservation of DT on JumpThreading patch:<br>
<br>
brzycki@cc01 /work/brzycki/test-suite $ ~/git/me/llvm/litdiff build.pre_dt_1/results.json build.pre_dt_2/results.json compile_time<br>
# left column: build.pre_dt_1/results.json<br>
# right column: build.pre_dt_2/results.json<br>
# metric name: compile_time<br>
524.8640 -> 474.9400 [ 10.51%] CTMark/tramp3d-v4/tramp3d-v4.<wbr>test<br>
343.1280 <- 369.4960 [ 7.68%] CTMark/sqlite3/sqlite3.test<br>
379.4560 <- 386.0880 [ 1.75%] CTMark/kimwitu++/kc.test<br>
250.3400 -> 248.0680 [ 0.92%] CTMark/mafft/pairlocalalign.<wbr>test<br>
583.2080 <- 587.5400 [ 0.74%] CTMark/lencod/lencod.test<br>
918.8360 -> 912.4920 [ 0.70%] CTMark/Bullet/bullet.test<br>
456.7720 -> 454.7480 [ 0.45%] CTMark/SPASS/SPASS.test<br>
367.2440 -> 366.3920 [ 0.23%] CTMark/consumer-typeset/<wbr>consumer-typeset.test<br>
1254.5240 -> 1252.3720 [ 0.17%] CTMark/7zip/7zip-benchmark.<wbr>test<br>
490.1600 -> 489.4360 [ 0.15%] CTMark/ClamAV/clamscan.test<br>
brzycki@cc01 /work/brzycki/test-suite $ ~/git/me/llvm/litdiff build.pre_dt_1/results.json build.pre_dt_2/results.json compile_time<br>
# left column: build.pre_dt_1/results.json<br>
# right column: build.pre_dt_2/results.json<br>
# metric name: compile_time<br>
524.8640 -> 474.9400 [ 10.51%] CTMark/tramp3d-v4/tramp3d-v4.<wbr>test<br>
343.1280 <- 369.4960 [ 7.68%] CTMark/sqlite3/sqlite3.test<br>
379.4560 <- 386.0880 [ 1.75%] CTMark/kimwitu++/kc.test<br>
250.3400 -> 248.0680 [ 0.92%] CTMark/mafft/pairlocalalign.<wbr>test<br>
583.2080 <- 587.5400 [ 0.74%] CTMark/lencod/lencod.test<br>
918.8360 -> 912.4920 [ 0.70%] CTMark/Bullet/bullet.test<br>
456.7720 -> 454.7480 [ 0.45%] CTMark/SPASS/SPASS.test<br>
367.2440 -> 366.3920 [ 0.23%] CTMark/consumer-typeset/<wbr>consumer-typeset.test<br>
1254.5240 -> 1252.3720 [ 0.17%] CTMark/7zip/7zip-benchmark.<wbr>test<br>
490.1600 -> 489.4360 [ 0.15%] CTMark/ClamAV/clamscan.test<br>
brzycki@cc01 /work/brzycki/test-suite $ ~/git/me/llvm/litdiff build.0cb78d41fca7bac_1/<wbr>results.json build.0cb78d41fca7bac_2/<wbr>results.json compile_time<br>
# left column: build.0cb78d41fca7bac_1/<wbr>results.json<br>
# right column: build.0cb78d41fca7bac_2/<wbr>results.json<br>
# metric name: compile_time<br>
535.8520 -> 483.3720 [ 10.86%] CTMark/tramp3d-v4/tramp3d-v4.<wbr>test<br>
372.9000 <- 378.4200 [ 1.48%] CTMark/sqlite3/sqlite3.test<br>
256.3040 -> 253.7600 [ 1.00%] CTMark/mafft/pairlocalalign.<wbr>test<br>
387.8480 -> 385.0480 [ 0.73%] CTMark/kimwitu++/kc.test<br>
369.8760 <- 372.3440 [ 0.67%] CTMark/consumer-typeset/<wbr>consumer-typeset.test<br>
458.8480 <- 461.5720 [ 0.59%] CTMark/SPASS/SPASS.test<br>
590.1440 -> 588.8520 [ 0.22%] CTMark/lencod/lencod.test<br>
499.0920 <- 499.9840 [ 0.18%] CTMark/ClamAV/clamscan.test<br>
1270.1200 -> 1267.9480 [ 0.17%] CTMark/7zip/7zip-benchmark.<wbr>test<br>
928.7720 <- 928.8000 [ 0.00%] CTMark/Bullet/bullet.test<br>
brzycki@cc01 /work/brzycki/test-suite $ ~/git/me/llvm/litdiff build.D42717_1/results.json build.D42717_2/results.json compile_time<br>
# left column: build.D42717_1/results.json<br>
# right column: build.D42717_2/results.json<br>
# metric name: compile_time<br>
494.1120 -> 487.4120 [ 1.37%] CTMark/tramp3d-v4/tramp3d-v4.<wbr>test<br>
369.8320 <- 373.5960 [ 1.02%] CTMark/consumer-typeset/<wbr>consumer-typeset.test<br>
386.5280 <- 390.0800 [ 0.92%] CTMark/kimwitu++/kc.test<br>
501.9440 -> 497.7200 [ 0.85%] CTMark/ClamAV/clamscan.test<br>
255.9640 -> 254.5120 [ 0.57%] CTMark/mafft/pairlocalalign.<wbr>test<br>
347.9000 -> 346.5880 [ 0.38%] CTMark/sqlite3/sqlite3.test<br>
1267.7040 <- 1269.8640 [ 0.17%] CTMark/7zip/7zip-benchmark.<wbr>test<br>
462.5080 <- 462.8000 [ 0.06%] CTMark/SPASS/SPASS.test<br>
923.5880 -> 923.0320 [ 0.06%] CTMark/Bullet/bullet.test<br>
589.6800 <- 589.8840 [ 0.03%] CTMark/lencod/lencod.test<br>
<br>
All of these are the same compiler compared against itself on as close to a quiescent x86_64 machine as I could find. The <a href="https://reviews.llvm.org/D42717" rel="noreferrer" target="_blank">https://reviews.llvm.org/<wbr>D42717</a> version has been rebased on top of llvm-project Git sha `0cb78d41fca7bac`. I'm going to run more iterations of the `pre_dt` compiler (llvm-project sha `788d5233ad1f2` from January 12) to see if the variance persists.<br>
<br>
If it does then this current patch does not meaningfully alter the compile time of `tramp3d-v4`.<br>
<br>
<br>
<a href="https://reviews.llvm.org/D42717" rel="noreferrer" target="_blank">https://reviews.llvm.org/<wbr>D42717</a><br>
<br>
<br>
<br>
</blockquote></div><br></div>