r323008 - [Lex] Fix crash on code completion in comment in included file.

[Hans Wennborg via cfe-commits]

On Wed, Jan 24, 2018 at 4:30 PM, Hans Wennborg <hans at chromium.org> wrote:
> Merged to 6.0 in r323333 as requested in PR36043. Richard, please
> complain if you don't agree.

Actually cc'ing Richard this time.

>
> On Sat, Jan 20, 2018 at 12:41 AM, Volodymyr Sapsai via cfe-commits
> <cfe-commits at lists.llvm.org> wrote:
>> Author: vsapsai
>> Date: Fri Jan 19 15:41:47 2018
>> New Revision: 323008
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=323008&view=rev
>> Log:
>> [Lex] Fix crash on code completion in comment in included file.
>>
>> This fixes PR32732 by updating CurLexerKind to reflect available lexers.
>> We were hitting null pointer in Preprocessor::Lex because CurLexerKind
>> was CLK_Lexer but CurLexer was null. And we set it to null in
>> Preprocessor::HandleEndOfFile when exiting a file with code completion
>> point.
>>
>> To reproduce the crash it is important for a comment to be inside a
>> class specifier. In this case in Parser::ParseClassSpecifier we improve
>> error recovery by pushing a semicolon token back into the preprocessor
>> and later on try to lex a token because we haven't reached the end of
>> file.
>>
>> Also clang crashes only on code completion in included file, i.e. when
>> IncludeMacroStack is not empty. Though we reset CurLexer even if include
>> stack is empty. The difference is that during pushing back a semicolon
>> token, preprocessor calls EnterCachingLexMode which decides it is
>> already in caching mode because various lexers are null and
>> IncludeMacroStack is not empty. As the result, CurLexerKind remains
>> CLK_Lexer instead of updating to CLK_CachingLexer.
>>
>> rdar://problem/34787685
>>
>> Reviewers: akyrtzi, doug.gregor, arphaman
>>
>> Reviewed By: arphaman
>>
>> Subscribers: cfe-commits, kfunk, arphaman, nemanjai, kbarton
>>
>> Differential Revision: https://reviews.llvm.org/D41688
>>
>> Added:
>>     cfe/trunk/test/CodeCompletion/Inputs/comments.h
>>     cfe/trunk/test/CodeCompletion/comments.cpp
>> Modified:
>>     cfe/trunk/lib/Lex/PPCaching.cpp
>>     cfe/trunk/lib/Lex/PPLexerChange.cpp
>>
>> Modified: cfe/trunk/lib/Lex/PPCaching.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPCaching.cpp?rev=323008&r1=323007&r2=323008&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/Lex/PPCaching.cpp (original)
>> +++ cfe/trunk/lib/Lex/PPCaching.cpp Fri Jan 19 15:41:47 2018
>> @@ -105,8 +105,10 @@ void Preprocessor::CachingLex(Token &Res
>>  }
>>
>>  void Preprocessor::EnterCachingLexMode() {
>> -  if (InCachingLexMode())
>> +  if (InCachingLexMode()) {
>> +    assert(CurLexerKind == CLK_CachingLexer && "Unexpected lexer kind");
>>      return;
>> +  }
>>
>>    PushIncludeMacroStack();
>>    CurLexerKind = CLK_CachingLexer;
>>
>> Modified: cfe/trunk/lib/Lex/PPLexerChange.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPLexerChange.cpp?rev=323008&r1=323007&r2=323008&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/Lex/PPLexerChange.cpp (original)
>> +++ cfe/trunk/lib/Lex/PPLexerChange.cpp Fri Jan 19 15:41:47 2018
>> @@ -444,6 +444,7 @@ bool Preprocessor::HandleEndOfFile(Token
>>        }
>>
>>        CurPPLexer = nullptr;
>> +      recomputeCurLexerKind();
>>        return true;
>>      }
>>
>>
>> Added: cfe/trunk/test/CodeCompletion/Inputs/comments.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeCompletion/Inputs/comments.h?rev=323008&view=auto
>> ==============================================================================
>> --- cfe/trunk/test/CodeCompletion/Inputs/comments.h (added)
>> +++ cfe/trunk/test/CodeCompletion/Inputs/comments.h Fri Jan 19 15:41:47 2018
>> @@ -0,0 +1,4 @@
>> +// PR32732
>> +struct B {
>> +  // <- code completion
>> +};
>>
>> Added: cfe/trunk/test/CodeCompletion/comments.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeCompletion/comments.cpp?rev=323008&view=auto
>> ==============================================================================
>> --- cfe/trunk/test/CodeCompletion/comments.cpp (added)
>> +++ cfe/trunk/test/CodeCompletion/comments.cpp Fri Jan 19 15:41:47 2018
>> @@ -0,0 +1,13 @@
>> +// Note: the run lines follow their respective tests, since line/column
>> +// matter in this test.
>> +
>> +#include "comments.h"
>> +
>> +struct A {
>> +  // <- code completion
>> +  /* <- code completion */
>> +};
>> +
>> +// RUN: %clang_cc1 -I %S/Inputs -fsyntax-only -code-completion-at=%s:7:6 %s
>> +// RUN: %clang_cc1 -I %S/Inputs -fsyntax-only -code-completion-at=%s:8:6 %s
>> +// RUN: %clang_cc1 -I %S/Inputs -fsyntax-only -code-completion-at=%S/Inputs/comments.h:3:6 %s
>>
>>
>> _______________________________________________
>> cfe-commits mailing list
>> cfe-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits