[cfe-dev] intermittent libclang Sema::LookupSpecialMember crash

[Ben Hendrickson]

There is an infrequent crash in libclang.so when calling
clang_reparseTranslationUnit, which I encounter when using the
YouCompleteMe vim plugin (which does auto-completion using libclang.so).
 It looks to me to be a libclang bug, and have traced it back enough that
perhaps someone familiar with the Sema code could quickly find the error
(I've cc'ed the author of YouCompleteMe in case he could add something).

I encountered this on the prebuilt x64 ubuntu 3.2 binaries, and
investigated using the source code from
http://llvm.org/svn/llvm-project/cfe/branches/release_32 r170678.

By recompiling libclang.so and every time I see the crash adding
increasingly relevant logging statements, here is what I've found:

Sema::LookupSpecialMember has the line:
    llvm::tie(I, E) = RD->lookup(Name);
Which calls DeclContext::lookup which calls
StoredDeclsList::getLookupResult to get iterators to the beginning and end
of an vector to loop over.  This is iterating over NamedDecl pointers.
 When it crashes, this array of pointers has around 10 elements in it, and
one of them (towards the middle of the array) is invalid (the value of the
pointer is always less than or equal to 0x35 - so clearly invalid -
although is never actually NULL) and thus crashes when it does the first
thing in this loop, the "if (Cand->isInvalidDecl())" call as this=0x35 for
the call to isInvalidDecl which doesn't work out.

After the call to RD->lookup(Name) line, I can add a check to make sure the
pointers in the array are all above 0x35, and indeed they are.
 Nevertheless, at some point actually step through the pointers in the
array, one will be less than 0x35, and we'll crash.  So at some point in
the loop, a pointer later in the array is getting changed.  This means one
can hack around the bug by making a local copy of the array one is
iterating over immediately after the RD->lookup call.  Indeed, I've done
that, and have not seen a crash since.  For example:

llvm::tie(I, E) = RD->lookup(Name);
> int decl_count = E - I;
> NamedDecl* Cands[decl_count];
> for (int i = 0; i < decl_count; i++) {
>   Cands[i] = I[i];
> }
> I = Cands;
> E = Cands + decl_count;
assert((I != E) &&
         "lookup for a constructor or assignment operator was empty");

Of course, this hack couldn't be checked in, and really whatever is
modifying the later elements in the array should be fixed. .

I've also looked at the code for the YouCompleteMe vim plugin, and do not
believe the bug in from that project.  Although modifying that project to
always rebuild translation units from scratch instead of reparsing existing
ones also avoids the crash.

Files I see this crashes on generally are long and are making use of
template meta programming, but even on these files, the crashes are
infrequent.

I hope this is useful.  As a c++ programmer, I deeply appreciate the Clang
project.

Cheers,
Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20130225/d8a8f2e6/attachment.html>