[cfe-dev] libclang: failing assert

[Erik Verbruggen]

On Mar 19, 2013, at 0:07, Argyrios Kyrtzidis <kyrtzidis at apple.com> wrote:

> On Mar 17, 2013, at 8:35 AM, Erik Verbruggen <erik.verbruggen at me.com> wrote:
> 
>> 
>> On Mar 16, 2013, at 18:56, Argyrios Kyrtzidis <kyrtzidis at apple.com> wrote:
>> 
>>> 
>>> On Mar 16, 2013, at 10:48 AM, Argyrios Kyrtzidis <kyrtzidis at apple.com> wrote:
>>> 
>>>> On Mar 16, 2013, at 3:09 AM, Erik Verbruggen <erik.verbruggen at me.com> wrote:
>>>> 
>>>>> Hi,
>>>>> 
>>>>> When using libclang's clang_annotateTokens, I have a consistently failing assert in CIndex.cpp, line 5429. This is for both 3.2 and trunk. When I comment the assert out, everything looks to be fine. Is this assert valid, and if so, what is it checking? 
>>>> 
>>>> It makes sure that when lexing for preprocessor directives tokens, that any cursors produced will refer to the correct clang_tokenize'd token.
>>>> Are you able to provide a preprocessed file for testing ?
>>>> 
>>> 
>>> You can use "-E -frewrite-includes" to preserve preprocessor directives.
>> 
>> See attachment.
> 
> I can't reproduce using:
> 
> $ c-index-test -test-annotate-tokens=main.pp:1:1:100000:1 -x c++ main.pp > /dev/null
> 
> Does it occur with a particular source range ? What is the file whose tokens you are annotating ? Does using c-index-test on it reproduce the assertion hit ?
> If yes, could you separate the includes of that file, for example:
> 
> #include "a.h"
> #include "b.h"
> #include "c.h"
> 
> <code>
> 
> ====>
> 
> #include "preprocessed-headers.h"
> 
> <code>

I cannot reproduce it either with c-index-test, so I modified it to do it in the same way as I use the annotation. Patch is attached. With that patch, I get:

erik at Road-Warrior:untitled6$ ~/dev/builds/llvm-release-build/bin/c-index-test -test-annotate-tokens=main.pp:62024:1:1000000:1 -std=gnu++98 -x c++ main.pp
Assertion failed: (Tok.getLocation() == SourceLocation::getFromRawEncoding(Tokens[TokIdx].int_data[1])), function annotatePreprocessorTokens, file /Users/erik/dev/clang-llvm/llvm-git/tools/clang/tools/libclang/CIndex.cpp, line 5429.
libclang: crash detected while annotating tokens
Identifier: "std" [62025:17 - 62025:20]
Identifier: "main" [62027:5 - 62027:9]
Identifier: "cout" [62029:5 - 62029:9]
Identifier: "endl" [62029:31 - 62029:35]

When I move everything up to (and including) line 62022 into the a header file (like you mentioned above), and include that file, I get exactly the same crash.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20130319/a219ea45/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: my-c-indexing.patch
Type: application/octet-stream
Size: 1894 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20130319/a219ea45/attachment.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20130319/a219ea45/attachment-0001.html>