[cfe-dev] Parsing benchmark: LibTomMath

[Chris Lattner]

On Nov 16, 2007, at 9:05 PM, Neil Booth wrote:

> Chris Lattner wrote:-
>
>> It would be better to run the gcc test 3  times then the clang test 3
>> times and take the minimum of each.  That said, we should be
>> significantly faster... 3x is awesome!
>
> I don't want to spoil the party :), but the comparison is not really
> fair.

Sure, but it's a good indicator.

> cc1 should be being invoked to avoid an extra fork/exec that
> penalizes GCC.

Why?  On darwin, "gcc" is actually a driver-driver that invokes  
another "gcc" which is a driver, which invokes "cc1".  It doesn't  
matter why the architecture is like this, the actual effect for the  
user is that all this is done for every compile.

> GCC handles character sets to some extent, which
> is a non-trivial cost, which clang does not.  Clang doesn't implement
> all the language yet; the extra semantic checking will likely slow
> it down slightly.

Yep, this is true.

> I believe GCC does lowering of some kind in
> addition to semantic analysis with -fsyntax-only, but I guess that
> can be viewed as an imlpementation weakness.

Yep.

> Of course, once complete, clang will probably be further improved
> speed-wise, and having said all that I'm sure clang will end up much
> faster than GCC.

Of course. :)

> As another data point, here are timings on my (slow) machine,
> fastest of three runs, for compiling the preprocessor files of my
> own C front end.  Like clang, cfe is stand-alone, and doesn't handle
> charsets, so to that extent it is a fairer comparison.  Like clang
> there hasn't really been any time spent optimizing for speed.
> However unlike clang it does semantically analyze and diagnose
> pretty much the whole language.  clang/llvm was compiled with gmake
> MAKE_OPTIMIZED=1 -- I believe that's how to get an optimized
> executable.

Cool.

> $ time for file in cpp/*.c; do /usr/libexec/cc1 -quiet -fsyntax-only  
> -I.
> $file ;done
>
> real    0m1.098s
> user    0m0.843s
> sys     0m0.190s
>
> $ time for file in cpp/*.c; do ~/src/nobackup/llvm/Release/bin/clang
> -fsyntax-only -I. $file ;done
> cpp/macro.c:686:15: error: variable has incomplete type 'char []'
>  static char compile_date[] = "\"Mmm dd yyyy\"";
>              ^
> cpp/macro.c:687:15: error: variable has incomplete type 'char []'
>  static char compile_time[] = "\"hh:mm:ss\"";
>              ^
> 2 diagnostics generated.
>
> real    0m0.456s
> user    0m0.281s
> sys     0m0.147s
>
> $ time for file in cpp/*.c; do ~/src/cfe/cfe -I. $file ;done
>
> real    0m0.205s
> user    0m0.118s
> sys     0m0.083s
>
> So there is still room for improvement, curing the C++ bloat and all
> that. 8)

Wow, you have significantly less system time as well as user time.   
Off hand, I am not sure why you would have significantly less system  
time (do you have a trick that you are using?) other than the fact  
that we build in several system -I search paths that are probably  
being queried but ignored.

For user time, it is hard to guess what the difference is.  Is this  
source code available to use for timings?  I'd be interested to see  
where clang is spending all its time: your cfe is almost 2x faster,  
which points to us doing something pretty braindead :)

-Chris