[cfe-dev] Parsing benchmark: LibTomMath

[Steve Naroff]

On Nov 16, 2007, at 9:05 PM, Neil Booth wrote:

> Chris Lattner wrote:-
>
>> It would be better to run the gcc test 3  times then the clang test 3
>> times and take the minimum of each.  That said, we should be
>> significantly faster... 3x is awesome!
>
> I don't want to spoil the party :), but the comparison is not really
> fair.

> cc1 should be being invoked to avoid an extra fork/exec that
> penalizes GCC.  GCC handles character sets to some extent, which
> is a non-trivial cost, which clang does not.  Clang doesn't implement
> all the language yet; the extra semantic checking will likely slow
> it down slightly.  I believe GCC does lowering of some kind in
> addition to semantic analysis with -fsyntax-only, but I guess that
> can be viewed as an imlpementation weakness.
>

While everything you say is accurate, I don't think these  
implementation level issues should be "over analyzed" (particularly at  
this point in the project). For example, most developers don't invoke  
cc1 directly (so doing so in a benchmark seems like cheating).

While you can pick apart any benchmark, front-end performance is  
something that needs to be tracked early and often. Compile-time  
performance wasn't a gcc goal for many years (which led to some of  
gcc's bloat and compile-time woes).

I'd like to emphasize something (my own form of over analysis:-)...

Unlike gcc, clang is being developed as a set of reusable components  
(with the goal of supporting a diverse set of needs). From my  
perspective, striking the right balance between abstraction and  
performance is an "art". It's hard to do, and hasn't been a part of  
the C compiler development culture over the years (making it difficult  
to find people that respect/understand this idiom).

That said, the fact that clang is very competitive with gcc on  
"traditional" batch mode processing is very encouraging.

snaroff

> Of course, once complete, clang will probably be further improved
> speed-wise, and having said all that I'm sure clang will end up much
> faster than GCC.
>
> As another data point, here are timings on my (slow) machine,
> fastest of three runs, for compiling the preprocessor files of my
> own C front end.  Like clang, cfe is stand-alone, and doesn't handle
> charsets, so to that extent it is a fairer comparison.  Like clang
> there hasn't really been any time spent optimizing for speed.
> However unlike clang it does semantically analyze and diagnose
> pretty much the whole language.  clang/llvm was compiled with gmake
> MAKE_OPTIMIZED=1 -- I believe that's how to get an optimized
> executable.
>
> $ time for file in cpp/*.c; do /usr/libexec/cc1 -quiet -fsyntax-only  
> -I.
> $file ;done
>
> real    0m1.098s
> user    0m0.843s
> sys     0m0.190s
>
> $ time for file in cpp/*.c; do ~/src/nobackup/llvm/Release/bin/clang
> -fsyntax-only -I. $file ;done
> cpp/macro.c:686:15: error: variable has incomplete type 'char []'
>  static char compile_date[] = "\"Mmm dd yyyy\"";
>              ^
> cpp/macro.c:687:15: error: variable has incomplete type 'char []'
>  static char compile_time[] = "\"hh:mm:ss\"";
>              ^
> 2 diagnostics generated.
>
> real    0m0.456s
> user    0m0.281s
> sys     0m0.147s
>
> $ time for file in cpp/*.c; do ~/src/cfe/cfe -I. $file ;done
>
> real    0m0.205s
> user    0m0.118s
> sys     0m0.083s
>
> So there is still room for improvement, curing the C++ bloat and all
> that. 8)
>
> Neil.
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev