[cfe-dev] Clang Preprocessor Speed Up

[Andy Gibbs via cfe-dev]

On Mon, May 16, 2016 at 10:46 AM Андрей Серебро wrote:

> So, I have implemented a prototype. For anyone interested, I attach the
> patch (I used clang release 38 from github as startup code).
>
> The prototype consumes 40% less time on preprocessing all boost headers
> than original clang (780 seconds vs 1330). On real-world source code
> patched clang seems to be usually faster than original.
>
> The big problem with it is that it currently doesn't generate proper
> source locations for expanded tokens. It doesn't affect the final code,
> but of course it makes diagnostics hard in case of errors. On the other
> hand, this situation is similar with inline functions debugging: without
> special flag, debugging will not be that easy.
>
> I have also measured timings for patched clang and clean clang, but with
> removed information about expansion locations (for it could happen, that
> all the profit came from switching off this info). But it turned out,
> that patched is still 28% faster (1100 seconds vs 780 seconds).
>
> I think, it may be useful probably to have some flag in clang that allows
> fast preprocessing, for sometimes profit can reach up to x4 times!
>
> I'm pretty sure there are some bugs now I haven't yet recognized, so any
> feedback is highly appreciated.

Hi,

I came across your post rummaging through the mailing list and I am most 
interested in it, not least because with minimal effort and an albeit fairly 
pathological macro expansion, I can make clang crash with the message "Ran 
out of source locations!" (assert inside SourceLocations.cpp).  So anything 
that will help there will be great.

However, I am unable to get your patch to work at all.  It seems to totally 
garble the macro expansion, even on simple expansions.  And the test-suite 
produces over 200 unexpected fails.  Other problems I see are that it gives 
an error on the following define (claiming an unterminated function-like 
macro invocation):

#define M1(...) M2(__VA_ARGS__

and I see lots of errors relating to supposed redefinitions of macros, when 
they aren't.

Is it possible that you attached an incomplete patch?  Do you have a working 
test-case that I can start with?  I would be delighted to test your work 
further.

Cheers,
Andy