I have run the john the ripper test.<br>
I have used the official archive (same version as phoronix) from <a href="http://www.openwall.com/john/g/john-1.7.3.1.tar.bz2">http://www.openwall.com/john/g/john-1.7.3.1.tar.bz2</a><br>
<br>
To build with llvm-gcc, replace the line CC = gcc with CC = llvm-gcc.<br>
I have used the following command to build : make clean linux-x86-sse2 (seems to be the best on x86-32)<br>
The makefile invocation is "gcc -c -Wall -O2 -fomit-frame-pointer
-funroll-loops src.c" and "llvm-gcc -c -Wall -O2 -fomit-frame-pointer
-funroll-loops src.c".<br>
<br>
I have runned 10 tests : "./john --test" in the run directory. It does 6 benchmarking on various algorithms :<br><br>Benchmarking: Traditional DES [128/128 BS SSE2]... DONE<br><br> LLVM GCC Difference Difference %<br>
<br> Run 1 2371 2358 13 0.55<br> Run 2 2499 2497 2 0.08<br> Run 3 2489 2487 2 0.08<br> Run 4 2305 2504 -199 -8.63<br> Run 5 2499 2445 54 2.16<br>
Run 6 2404 2503 -99 -4.12<br> Run 7 2482 2502 -20 -0.81<br> Run 8 2479 2475 4 0.16<br> Run 9 2463 2489 -26 -1.06<br> Run 10 2484 2483 1 0.04<br> <br>
Average 2447.5 2474.3 -26.8 -1.15<br> Std dev 65.69 44.5 71.81 3.07<br><br>==> Similar results<br><br><br>Benchmarking: BSDI DES (x725) [128/128 BS SSE2]... DONE<br><br> LLVM GCC Difference Difference %<br>
<br> Run 1 72584 81280 -8696 -11.98<br> Run 2 76620 79795 -3175 -4.14<br> Run 3 79820 75264 4556 5.71<br> Run 4 76339 81484 -5145 -6.74<br> Run 5 81484 76441 5043 6.19<br>
Run 6 80742 81433 -691 -0.86<br> Run 7 81510 79104 2406 2.95<br> Run 8 81049 79872 1177 1.45<br> Run 9 80409 81100 -691 -0.86<br> Run 10 80204 80921 -717 -0.89<br>
<br> Average 79076.1 79669.4 -593.3 -0.92<br> Std dev 2937.56 2181.15 4262.03 5.59<br><br>==> Similar results<br><br>Benchmarking: FreeBSD MD5 [32/32]... DONE<br><br> LLVM GCC Difference Difference %<br>
<br> Run 1 7552 8009 -457 -6.05<br> Run 2 7739 7724 15 0.19<br> Run 3 7997 7696 301 3.76<br> Run 4 8038 8041 -3 -0.04<br> Run 5 7474 7938 -464 -6.21<br>
Run 6 7871 8078 -207 -2.63<br> Run 7 7884 7980 -96 -1.22<br> Run 8 7870 8025 -155 -1.97<br> Run 9 7989 8046 -57 -0.71<br> Run 10 7986 7989 -3 -0.04<br>
<br> Average 7840 7952.6 -112.6 -1.49<br> Std dev 193.87 133.78 227.92 2.98<br><br>==> Similar results<br><br>Benchmarking: OpenBSD Blowfish (x32) [32/32]... DONE<br><br>
LLVM GCC Difference Difference %<br> <br> Run 1 494 495 -1 -0.2<br> Run 2 457 485 -28 -6.13<br> Run 3 492 474 18 3.66<br> Run 4 494 492 2 0.4<br>
Run 5 486 469 17 3.5<br> Run 6 491 495 -4 -0.81<br> Run 7 495 493 2 0.4<br> Run 8 490 490 0 0<br> Run 9 493 494 -1 -0.2<br> Run 10 493 492 1 0.2<br>
<br> Average 488.5 487.9 0.6 0.08<br> Std dev 11.37 9.19 12.56 2.67<br><br>==> Similar results<br><br>Benchmarking: Kerberos AFS DES [48/64 4K MMX]... DONE<br><br> LLVM GCC Difference Difference %<br>
<br> Run 1 399001 403712 -4711 -1.18<br> Run 2 396697 377292 19405 4.89<br> Run 3 395520 401971 -6451 -1.63<br> Run 4 392396 404172 -11776 -3<br>
Run 5 392294 376217 16077 4.1<br> Run 6 395571 404172 -8601 -2.17<br> Run 7 400128 402995 -2867 -0.72<br> Run 8 397516 395110 2406 0.61<br> Run 9 396748 403507 -6759 -1.7<br>
Run 10 396263 403712 -7449 -1.88<br> <br> Average 396213.4 397286 -1072.6 -0.27<br> Std dev 2497.59 11150.79 10620.52 2.69<br><br>==> Similar results<br>
<br><br>Benchmarking: LM DES [128/128 BS SSE2]... DONE<br><br> LLVM GCC Difference Difference %<br> <br> Run 1 6879 11433 -4554 -66.2<br> Run 2 8984 12252 -3268 -36.38<br>
Run 3 9142 12182 -3040 -33.25<br> Run 4 8802 12205 -3403 -38.66<br> Run 5 8756 11971 -3215 -36.72<br> Run 6 9227 12224 -2997 -32.48<br> Run 7 8667 12191 -3524 -40.66<br>
Run 8 9163 11942 -2779 -30.33<br> Run 9 9117 12254 -3137 -34.41<br> Run 10 9076 12166 -3090 -34.05<br> <br> Average 8781.3 12082 -3300.7 -38.31<br>
Std dev 695.06 252.95 487.97 10.26<br><br>==> This one is interesting as gcc is better with near 40%<br>I have no idea why but anyone interested could take a look at LM_fmt.c which seems to define test and source for this algorithm.<br>
<br>Olivier.<br><br><br><br><br>
<br><br><div class="gmail_quote">On Wed, Sep 16, 2009 at 2:40 PM, Renato Golin <span dir="ltr"><<a href="mailto:rengolin@systemcall.org">rengolin@systemcall.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
2009/9/16 Olivier Meurant <<a href="mailto:meurant.olivier@gmail.com">meurant.olivier@gmail.com</a>>:<br>
<div class="im">> Average 13836499.46 12447973.17 1388526.29 10.03<br>
> Standard deviation 53189.13 515638.56<br>
> 522400.98 3.77<br>
<br>
<br>
</div>That was pretty much what I was expecting from the article... numbers.<br>
It doesn't matter who's best, you can't be best in all areas, but<br>
profiling must be done right. The standard deviation is, at least, one<br>
order of magnitude lower than the difference between the two averages,<br>
which means the measurements have a meaning and express some reality.<br>
<br>
There is little value in the results on the article. People had to<br>
re-do it properly again, anyway...<br>
<br>
Shouldn't it be part of the standard release? I mean, add profiling as<br>
an automated task before every big release, comparing to previous<br>
versions of LLVM and other important compilers. Not to be waving the<br>
results about, but to know the weakness and work on them on the next<br>
release. It might take a while to build such infrastructure, but it's<br>
a good thing to do, I guess.<br>
<div><div></div><div class="h5"><br>
cheers,<br>
--renato<br>
<br>
Reclaim your digital rights, eliminate DRM, learn more at<br>
<a href="http://www.defectivebydesign.org/what_is_drm" target="_blank">http://www.defectivebydesign.org/what_is_drm</a><br>
</div></div></blockquote></div><br>