[cfe-dev] Clang Preprocessor Speed Up

[Yaron Keren via cfe-dev]

Hi,

After your patch, you can run all clang regresion tests with "ninja
check-clang".
Passing these would be a good start.

Yaron


2016-06-02 11:59 GMT+03:00 Андрей Серебро <andy.silv at yandex.ru>:

> Hi,
>
> Yep, I have noticed several problems with the patch after I have sent it,
> it's my bad, really: first was with variadic macos, second there was an
> erroneous ## processing, and also now I'm experiencing some problems with
> compiling Boost 1.61.
> Currently I put a gap for the first problem, so that variadics are just
> not cached and so expanded in a standard way.
> The second I guess was fixed, now I'm struggling with the third one.
>
>
> However, I am unable to get your patch to work at all. It seems to totally
> garble the macro expansion, even on simple expansions. And the test-suite
> produces over 200 unexpected fails.
>
> That's interesting. Simple samples should definitely work fine (in case
> there were no variadics). You can share pieces that cause bad behavior, I
> will try to understand whats wrong with them (probably, it's about that
> errors with ##).
> Unexpeccted failures I guess are fine since llvm uses Lit to check the
> test results, and this checking consists of pattern matching the output
> with the one that is expected. Since I have removed SourceLocation
> information, the output definitely varies from the expected output that
> uses them. But I will also look at it in case something not connected with
> locations also breaks the tests.
>
>
> and I see lots of errors relating to supposed redefinitions of macros,
> when
> they aren't.
>
> Can you please attach some code that can reproduce redefinition problems?
> I hoped I have found all the cases where it happens, however, I could miss
> something.
> Thanks a lot for the feedback.
>
> 02.06.2016, 10:45, "Andy Gibbs" <andyg1001 at hotmail.co.uk>:
>
> On Mon, May 16, 2016 at 10:46 AM Андрей Серебро wrote:
>
>  So, I have implemented a prototype. For anyone interested, I attach the
>  patch (I used clang release 38 from github as startup code).
>
>  The prototype consumes 40% less time on preprocessing all boost headers
>  than original clang (780 seconds vs 1330). On real-world source code
>  patched clang seems to be usually faster than original.
>
>  The big problem with it is that it currently doesn't generate proper
>  source locations for expanded tokens. It doesn't affect the final code,
>  but of course it makes diagnostics hard in case of errors. On the other
>  hand, this situation is similar with inline functions debugging: without
>  special flag, debugging will not be that easy.
>
>  I have also measured timings for patched clang and clean clang, but with
>  removed information about expansion locations (for it could happen, that
>  all the profit came from switching off this info). But it turned out,
>  that patched is still 28% faster (1100 seconds vs 780 seconds).
>
>  I think, it may be useful probably to have some flag in clang that allows
>  fast preprocessing, for sometimes profit can reach up to x4 times!
>
>  I'm pretty sure there are some bugs now I haven't yet recognized, so any
>  feedback is highly appreciated.
>
>
> Hi,
>
> I came across your post rummaging through the mailing list and I am most
> interested in it, not least because with minimal effort and an albeit
> fairly
> pathological macro expansion, I can make clang crash with the message "Ran
> out of source locations!" (assert inside SourceLocations.cpp). So anything
> that will help there will be great.
>
> However, I am unable to get your patch to work at all. It seems to totally
> garble the macro expansion, even on simple expansions. And the test-suite
> produces over 200 unexpected fails. Other problems I see are that it gives
> an error on the following define (claiming an unterminated function-like
> macro invocation):
>
> #define M1(...) M2(__VA_ARGS__
>
> and I see lots of errors relating to supposed redefinitions of macros,
> when
> they aren't.
>
> Is it possible that you attached an incomplete patch? Do you have a
> working
> test-case that I can start with? I would be delighted to test your work
> further.
>
> Cheers,
> Andy
>
>
>
> --
> Regards,
> Andrei Serebro
> tel. +79111758381
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160602/25881886/attachment.html>