[clang] [clang][deps] Respect `Lexer::cutOffLexing()` (PR #134404)

[Jan Svoboda via cfe-commits]

https://github.com/jansvoboda11 created https://github.com/llvm/llvm-project/pull/134404

This is crucial when recovering from fatal loader errors. Without it, the `Lexer` keeps yielding more tokens and the compiler may access invalid `ASTReader` state.

rdar://133388373

>From 62a5616b139c5bfcd9629603de4eeca98e326ad4 Mon Sep 17 00:00:00 2001
From: Jan Svoboda <jan_svoboda at apple.com>
Date: Fri, 4 Apr 2025 09:00:36 -0700
Subject: [PATCH] [clang][deps] Respect `Lexer::cutOffLexing()`

This is crucial when recovering from fatal loader errors. Without it, the `Lexer` keeps yielding more tokens and the compiler may access invalid `ASTReader` state.
---
 clang/lib/Lex/Lexer.cpp                       |  3 ++
 .../modules-relocated-mm-macro.c              | 42 +++++++++++++++++++
 2 files changed, 45 insertions(+)
 create mode 100644 clang/test/ClangScanDeps/modules-relocated-mm-macro.c

diff --git a/clang/lib/Lex/Lexer.cpp b/clang/lib/Lex/Lexer.cpp
index 3128627490e28..93200458f04b4 100644
--- a/clang/lib/Lex/Lexer.cpp
+++ b/clang/lib/Lex/Lexer.cpp
@@ -4543,6 +4543,9 @@ bool Lexer::LexDependencyDirectiveToken(Token &Result) {
 
   using namespace dependency_directives_scan;
 
+  if (BufferPtr == BufferEnd)
+    return LexEndOfFile(Result, BufferPtr);
+
   while (NextDepDirectiveTokenIndex == DepDirectives.front().Tokens.size()) {
     if (DepDirectives.front().Kind == pp_eof)
       return LexEndOfFile(Result, BufferEnd);
diff --git a/clang/test/ClangScanDeps/modules-relocated-mm-macro.c b/clang/test/ClangScanDeps/modules-relocated-mm-macro.c
new file mode 100644
index 0000000000000..17f479d9e0046
--- /dev/null
+++ b/clang/test/ClangScanDeps/modules-relocated-mm-macro.c
@@ -0,0 +1,42 @@
+// This test checks that we don't crash when we load two conflicting PCM files
+// and instead emit the appropriate diagnostics.
+
+// RUN: rm -rf %t
+// RUN: split-file %s %t
+
+// RUN: mkdir %t/frameworks1
+
+// RUN: clang-scan-deps -format experimental-full -o %t/deps1.json -- \
+// RUN:   %clang -fmodules -fmodules-cache-path=%t/cache \
+// RUN:   -F %t/frameworks1 -F %t/frameworks2 \
+// RUN:   -c %t/tu1.m -o %t/tu1.o
+
+// RUN: cp -r %t/frameworks2/A.framework %t/frameworks1
+
+// RUN: not clang-scan-deps -format experimental-full -o %t/deps2.json -- \
+// RUN:   %clang -fmodules -fmodules-cache-path=%t/cache \
+// RUN:   -F %t/frameworks1 -F %t/frameworks2 \
+// RUN:   -c %t/tu2.m -o %t/tu2.o \
+// RUN:     2>&1 | FileCheck %s
+
+// CHECK: fatal error: module 'A' is defined in both '{{.*}}.pcm' and '{{.*}}.pcm'
+
+//--- frameworks2/A.framework/Modules/module.modulemap
+framework module A { header "A.h" }
+//--- frameworks2/A.framework/Headers/A.h
+#define MACRO_A 1
+
+//--- frameworks2/B.framework/Modules/module.modulemap
+framework module B { header "B.h" }
+//--- frameworks2/B.framework/Headers/B.h
+#include <A/A.h>
+
+//--- tu1.m
+#include <B/B.h>
+
+//--- tu2.m
+#include <A/A.h>
+#include <B/B.h> // This results in a conflict and a fatal loader error.
+
+#if MACRO_A // This crashes with lexer that does not respect `cutOfLexing()`.
+#endif