r323008 - [Lex] Fix crash on code completion in comment in included file.

[Volodymyr Sapsai via cfe-commits]

Author: vsapsai
Date: Fri Jan 19 15:41:47 2018
New Revision: 323008

URL: http://llvm.org/viewvc/llvm-project?rev=323008&view=rev
Log:
[Lex] Fix crash on code completion in comment in included file.

This fixes PR32732 by updating CurLexerKind to reflect available lexers.
We were hitting null pointer in Preprocessor::Lex because CurLexerKind
was CLK_Lexer but CurLexer was null. And we set it to null in
Preprocessor::HandleEndOfFile when exiting a file with code completion
point.

To reproduce the crash it is important for a comment to be inside a
class specifier. In this case in Parser::ParseClassSpecifier we improve
error recovery by pushing a semicolon token back into the preprocessor
and later on try to lex a token because we haven't reached the end of
file.

Also clang crashes only on code completion in included file, i.e. when
IncludeMacroStack is not empty. Though we reset CurLexer even if include
stack is empty. The difference is that during pushing back a semicolon
token, preprocessor calls EnterCachingLexMode which decides it is
already in caching mode because various lexers are null and
IncludeMacroStack is not empty. As the result, CurLexerKind remains
CLK_Lexer instead of updating to CLK_CachingLexer.

rdar://problem/34787685

Reviewers: akyrtzi, doug.gregor, arphaman

Reviewed By: arphaman

Subscribers: cfe-commits, kfunk, arphaman, nemanjai, kbarton

Differential Revision: https://reviews.llvm.org/D41688

Added:
    cfe/trunk/test/CodeCompletion/Inputs/comments.h
    cfe/trunk/test/CodeCompletion/comments.cpp
Modified:
    cfe/trunk/lib/Lex/PPCaching.cpp
    cfe/trunk/lib/Lex/PPLexerChange.cpp

Modified: cfe/trunk/lib/Lex/PPCaching.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPCaching.cpp?rev=323008&r1=323007&r2=323008&view=diff
==============================================================================
--- cfe/trunk/lib/Lex/PPCaching.cpp (original)
+++ cfe/trunk/lib/Lex/PPCaching.cpp Fri Jan 19 15:41:47 2018
@@ -105,8 +105,10 @@ void Preprocessor::CachingLex(Token &Res
 }
 
 void Preprocessor::EnterCachingLexMode() {
-  if (InCachingLexMode())
+  if (InCachingLexMode()) {
+    assert(CurLexerKind == CLK_CachingLexer && "Unexpected lexer kind");
     return;
+  }
 
   PushIncludeMacroStack();
   CurLexerKind = CLK_CachingLexer;

Modified: cfe/trunk/lib/Lex/PPLexerChange.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPLexerChange.cpp?rev=323008&r1=323007&r2=323008&view=diff
==============================================================================
--- cfe/trunk/lib/Lex/PPLexerChange.cpp (original)
+++ cfe/trunk/lib/Lex/PPLexerChange.cpp Fri Jan 19 15:41:47 2018
@@ -444,6 +444,7 @@ bool Preprocessor::HandleEndOfFile(Token
       }
 
       CurPPLexer = nullptr;
+      recomputeCurLexerKind();
       return true;
     }
 

Added: cfe/trunk/test/CodeCompletion/Inputs/comments.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeCompletion/Inputs/comments.h?rev=323008&view=auto
==============================================================================
--- cfe/trunk/test/CodeCompletion/Inputs/comments.h (added)
+++ cfe/trunk/test/CodeCompletion/Inputs/comments.h Fri Jan 19 15:41:47 2018
@@ -0,0 +1,4 @@
+// PR32732
+struct B {
+  // <- code completion
+};

Added: cfe/trunk/test/CodeCompletion/comments.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeCompletion/comments.cpp?rev=323008&view=auto
==============================================================================
--- cfe/trunk/test/CodeCompletion/comments.cpp (added)
+++ cfe/trunk/test/CodeCompletion/comments.cpp Fri Jan 19 15:41:47 2018
@@ -0,0 +1,13 @@
+// Note: the run lines follow their respective tests, since line/column
+// matter in this test.
+
+#include "comments.h"
+
+struct A {
+  // <- code completion
+  /* <- code completion */
+};
+
+// RUN: %clang_cc1 -I %S/Inputs -fsyntax-only -code-completion-at=%s:7:6 %s
+// RUN: %clang_cc1 -I %S/Inputs -fsyntax-only -code-completion-at=%s:8:6 %s
+// RUN: %clang_cc1 -I %S/Inputs -fsyntax-only -code-completion-at=%S/Inputs/comments.h:3:6 %s