[cfe-dev] [llvm-dev] Clang Preprocessor Speed Up
Андрей Серебро via cfe-dev
cfe-dev at lists.llvm.org
Mon May 16 10:45:51 PDT 2016
So, I have implemented a prototype. For anyone interested, I attach the
patch (I used clang release 38 from github as startup code).
The prototype consumes 40% less time on preprocessing all boost headers
than original clang (780 seconds vs 1330). On real-world source code
patched clang seems to be usually faster than original.
The big problem with it is that it currently doesn't generate proper
source locations for expanded tokens. It doesn't affect the final code,
but of course it makes diagnostics hard in case of errors. On the other
hand, this situation is similar with inline functions debugging: without
special flag, debugging will not be that easy.
I have also measured timings for patched clang and clean clang, but with
removed information about expansion locations (for it could happen, that
all the profit came from switching off this info). But it turned out,
that patched is still 28% faster (1100 seconds vs 780 seconds).
I think, it may be useful probably to have some flag in clang that
allows fast preprocessing, for sometimes profit can reach up to x4 times!
I'm pretty sure there are some bugs now I haven't yet recognized, so any
feedback is highly appreciated.
25.03.2016, 12:05, "mats petersson" <mats at planetcatfish.com>:
> But even then, how much of the total time is expanding macros, and how
> much is "reading and finding the actual files" (and writing the output)?
>
> I'm not saying this is not worth doing, I'm just trying to avoid
> someone spending time on something that doesn't provide benefit - I
> speak from experience, I've "optimized" code, and then found that it
> didn't make any improvement at all - I've also done work with gives
> 3-30x speedups by some simple steps... So, measure, make improvement,
> measure.
>
> Or, use `perf` on some typical usecase, and figure out where the time
> goes...
>
> --
> Mats
>
> On 25 March 2016 at 08:29, Yaron Keren <yaron.keren at gmail.com
> <mailto:yaron.keren at gmail.com>> wrote:
>
> Once measured times for one of the Boost libraries example,
> preprocessing (-E) was about 20% of total compilation time. This
> is not typical in general but quite common with Boost libraries as
> 100s-1000s files may be included with tons of macros and nested
> macros.
>
> 2016-03-25 1:29 GMT+02:00 Андрей Серебро <llvm-dev at lists.llvm.org
> <mailto:llvm-dev at lists.llvm.org>>:
>
> Hi Mats,
> Thanks for the reply. Yep, you are right, the time should be
> measured and I guess I can imagine the typical workflow
>
> * implement prototype
> * take bunch of big real projects
> * compare preprocessing time for initial and changed clang
> * make conclusion whether the idea is sane or not
>
> About usage - probably, some IDEs can act better for they need
> iteratively relex source for correct autocomplete.
> What I'm also curious about is if somebody already did
> something on this or had thought about it.
> If the idea was already thought (which I guess is rather
> possible), it's interesting, did somebody already prove it's
> useless?
> 25.03.2016, 01:59, "mats petersson" <mats at planetcatfish.com
> <mailto:mats at planetcatfish.com>>:
>> First, surely the right place for this discussion is the
>> cfe-dev mailing list?
>>
>> Second, have you determined that this is a noticeable amount
>> of time when compiling? I have no idea - in my Pascal
>> compiler, parsing the code is ~0.1%, codegen to IR ~1.9% and
>> LLVM 98%. But I'm sure Clang is more complex in many ways, so
>> the proportion is probably a bit different - a measurement of
>> the time spent expanding macros would probably help determine
>> if it's worth doing or not.
>>
>> --
>> Mats
>>
>> On 24 March 2016 at 22:17, Andy via llvm-dev
>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>
>> Hello, folks!
>>
>> Currently me with one other guy are trying to play with
>> clang. The proposal may seem stupid, excuse me, if it was
>> already discussed, we just want to try to implement
>> something useful which seems absent for now.
>>
>> Ok, the idea. It seems interesting to try to make lexer a
>> little bit more efficient in terms of macro expanding by
>> applying partial expansion of macros. the idea is that
>> some libraries have rather deeply nested macro
>> definitions, and each time lexer sees it in code, it
>> reexpands definition fully. This seems to be overkill
>> sometimes, for rather often macros are not redefined in
>> code, so expansion can be reused.
>>
>> Of course, the typical nesting is rather low, but for
>> example BOOST_PP_REPEAT can cause such situations.
>>
>> So, the question is, what do you think about possible
>> utility of such research and the reasons for you think so?
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
> --
> Regards,
> Andrei Serebro
> tel. +79111758381 <tel:%2B79111758381>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
--
Regards,
Andrei Serebro
tel. +79111758381
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160516/5ebfc2f9/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: macros_patch.diff
Type: text/x-patch
Size: 48567 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160516/5ebfc2f9/attachment.bin>
More information about the cfe-dev
mailing list