[cfe-dev] Clang indexing library performance

[Alexander Bolodurin]

On 02/10/2011, at 5:44 AM, Douglas Gregor wrote:

> 
> On Oct 1, 2011, at 2:34 AM, Alexander Bolodurin wrote:
> 
>> I was playing with libclang-based indexing and clang_complete plugin for Vim.
>> Having found completion responsiveness a bit lacking, I decided to track down whether it's plugin's fault.
>> 
>> To put the indexer under stress, I've made up an artificial test with the biggest and baddest C++ headers:
>> 
>> ---------------8<---------------
>> #include <boost/asio.hpp>
>> #include <boost/regex.hpp>
>> #include <boost/thread.hpp>
>> #include <boost/spirit.hpp>
>> #include <boost/signals2.hpp>
>> boost::
>> ---------------8<---------------
>> 
>> ...and the following minimal test program using Python bindings:
>> 
>> ---------------8<---------------
>> from clang import cindex
>> 
>> def clock(f):
>>    import time
>>    def wrapf(*args,**kw):
>>        start = time.time()
>>        f(*args,**kw)
>>        end = time.time()
>>        print f.func_name, end - start
>>    return wrapf
>> 
>> fname='...path to file...'
>> idx=cindex.Index.create()
>> 
>> @clock
>> def parse():
>>    global tu
>>    opts = ['-I/opt/local/include'] # MacPorts Boost path
>>    tu=idx.parse(fname,opts)
>> 
>> @clock
>> def complete():
>>    c = tu.codeComplete(fname,6,8)
>> 
>> parse()
>> for i in range(4):
>>    complete()
>> ---------------8<---------------
>> 
>> This is the timing I get on a Macbook Pro:
>> 
>> parse 3.96061992645
>> complete 3.31106615067
>> complete 3.17438578606
>> complete 3.37997102737
>> complete 3.16793084145
>> 
>> Each one individually isn't *too* bad, taking under 1s, though still not too responsive, with exception of signals2.hpp taking over 1.5s and, and less exotic thread.hpp taking about 1.2s.
>> 
>> So, the questions:
>> 
>> Am I misusing the library? Should I try passing some additional flags or anything else?
> 
> You want to use the "default editing options" when parsing the translation unit
> 
> 	clang_defaultEditingTranslationUnitOptions()
> 
> and then reparse at least once. That will enable the various code-completion optimizations that should bring this time down significantly.
> 
>> Or it's just the way it is? What puzzles me is that completion takes time comparable to parsing.
> 
> That's because it *is* parsing :)
> 
>> I take it headers have to be re-parsed in case the order of inclusion changes, as it may potentially change header contents, but 3s still seems a bit excessive.
> 
> Given the volume of code you're parsing, I wouldn't call 3s excessive. On my system, GCC takes 2.7s, Clang normally takes 2.3s, and Clang when parsing for code completion takes 1.6s (it takes some shortcuts).
> 
> If you then turn on the optimizations I mentioned above, it goes down to 0.4s. Of course, it could always be better.
> 
> 	- Doug

I have set the appropriate flags and tried re-parsing, but I get nowhere near 0.4s completion time (which would be fantastic, of course).
To be completely clear, I'm using official 2.9 release.