[cfe-dev] Clang indexing library performance

[Douglas Gregor]

Sent from my iPhone

On Oct 2, 2011, at 2:06 AM, Alexander Bolodurin <alexander.bolodurin at gmail.com> wrote:

> On 02/10/2011, at 5:44 AM, Douglas Gregor wrote:
> 
>> 
>> On Oct 1, 2011, at 2:34 AM, Alexander Bolodurin wrote:
>> 
>>> I was playing with libclang-based indexing and clang_complete plugin for Vim.
>>> Having found completion responsiveness a bit lacking, I decided to track down whether it's plugin's fault.
>>> 
>>> To put the indexer under stress, I've made up an artificial test with the biggest and baddest C++ headers:
>>> 
>>> ---------------8<---------------
>>> #include <boost/asio.hpp>
>>> #include <boost/regex.hpp>
>>> #include <boost/thread.hpp>
>>> #include <boost/spirit.hpp>
>>> #include <boost/signals2.hpp>
>>> boost::
>>> ---------------8<---------------
>>> 
>>> ...and the following minimal test program using Python bindings:
>>> 
>>> ---------------8<---------------
>>> from clang import cindex
>>> 
>>> def clock(f):
>>>   import time
>>>   def wrapf(*args,**kw):
>>>       start = time.time()
>>>       f(*args,**kw)
>>>       end = time.time()
>>>       print f.func_name, end - start
>>>   return wrapf
>>> 
>>> fname='...path to file...'
>>> idx=cindex.Index.create()
>>> 
>>> @clock
>>> def parse():
>>>   global tu
>>>   opts = ['-I/opt/local/include'] # MacPorts Boost path
>>>   tu=idx.parse(fname,opts)
>>> 
>>> @clock
>>> def complete():
>>>   c = tu.codeComplete(fname,6,8)
>>> 
>>> parse()
>>> for i in range(4):
>>>   complete()
>>> ---------------8<---------------
>>> 
>>> This is the timing I get on a Macbook Pro:
>>> 
>>> parse 3.96061992645
>>> complete 3.31106615067
>>> complete 3.17438578606
>>> complete 3.37997102737
>>> complete 3.16793084145
>>> 
>>> Each one individually isn't *too* bad, taking under 1s, though still not too responsive, with exception of signals2.hpp taking over 1.5s and, and less exotic thread.hpp taking about 1.2s.
>>> 
>>> So, the questions:
>>> 
>>> Am I misusing the library? Should I try passing some additional flags or anything else?
>> 
>> You want to use the "default editing options" when parsing the translation unit
>> 
>>    clang_defaultEditingTranslationUnitOptions()
>> 
>> and then reparse at least once. That will enable the various code-completion optimizations that should bring this time down significantly.
>> 
>>> Or it's just the way it is? What puzzles me is that completion takes time comparable to parsing.
>> 
>> That's because it *is* parsing :)
>> 
>>> I take it headers have to be re-parsed in case the order of inclusion changes, as it may potentially change header contents, but 3s still seems a bit excessive.
>> 
>> Given the volume of code you're parsing, I wouldn't call 3s excessive. On my system, GCC takes 2.7s, Clang normally takes 2.3s, and Clang when parsing for code completion takes 1.6s (it takes some shortcuts).
>> 
>> If you then turn on the optimizations I mentioned above, it goes down to 0.4s. Of course, it could always be better.
>> 
>>    - Doug
> 
> I have set the appropriate flags and tried re-parsing, but I get nowhere near 0.4s completion time (which would be fantastic, of course).
> To be completely clear, I'm using official 2.9 release.

Ah. Some of these optimizations were not available for C++ in that release.