[clang] [clang-tools-extra] Remove clang-pseudo (PR #109154)
via cfe-commits
cfe-commits at lists.llvm.org
Wed Sep 18 08:01:19 PDT 2024
llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-clang
Author: Aaron Ballman (AaronBallman)
<details>
<summary>Changes</summary>
The functionality is incomplete and the authors have since shifted gears to other work, so this is effectively unmaintained.
The original design document for clang-pseudo can be found at:
https://docs.google.com/document/d/1eGkTOsFja63wsv8v0vd5JdoTonj-NlN3ujGF0T7xDbM/edit
in case anyone wishes to pick this project back up again in the future.
Original RFC: https://discourse.llvm.org/t/removing-pseudo-parser/71131/
---
Patch is 392.50 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/109154.diff
102 Files Affected:
- (modified) clang-tools-extra/CMakeLists.txt (-1)
- (modified) clang-tools-extra/clangd/CMakeLists.txt (-1)
- (modified) clang-tools-extra/clangd/SemanticSelection.cpp (+16-16)
- (renamed) clang-tools-extra/clangd/support/Bracket.cpp (+4-4)
- (renamed) clang-tools-extra/clangd/support/Bracket.h (+6-6)
- (modified) clang-tools-extra/clangd/support/CMakeLists.txt (+4)
- (renamed) clang-tools-extra/clangd/support/DirectiveTree.cpp (+3-3)
- (renamed) clang-tools-extra/clangd/support/DirectiveTree.h (+6-6)
- (renamed) clang-tools-extra/clangd/support/Lex.cpp (+3-3)
- (renamed) clang-tools-extra/clangd/support/Token.cpp (+3-3)
- (renamed) clang-tools-extra/clangd/support/Token.h (+5-5)
- (removed) clang-tools-extra/pseudo/CMakeLists.txt (-12)
- (removed) clang-tools-extra/pseudo/DesignNotes.md (-123)
- (removed) clang-tools-extra/pseudo/Disambiguation.md (-367)
- (removed) clang-tools-extra/pseudo/README.md (-37)
- (removed) clang-tools-extra/pseudo/benchmarks/Benchmark.cpp (-156)
- (removed) clang-tools-extra/pseudo/benchmarks/CMakeLists.txt (-9)
- (removed) clang-tools-extra/pseudo/fuzzer/CMakeLists.txt (-16)
- (removed) clang-tools-extra/pseudo/fuzzer/Fuzzer.cpp (-82)
- (removed) clang-tools-extra/pseudo/fuzzer/Main.cpp (-16)
- (removed) clang-tools-extra/pseudo/gen/CMakeLists.txt (-11)
- (removed) clang-tools-extra/pseudo/gen/Main.cpp (-172)
- (removed) clang-tools-extra/pseudo/include/CMakeLists.txt (-32)
- (removed) clang-tools-extra/pseudo/include/clang-pseudo/Disambiguate.h (-64)
- (removed) clang-tools-extra/pseudo/include/clang-pseudo/Forest.h (-236)
- (removed) clang-tools-extra/pseudo/include/clang-pseudo/GLR.h (-170)
- (removed) clang-tools-extra/pseudo/include/clang-pseudo/Language.h (-64)
- (removed) clang-tools-extra/pseudo/include/clang-pseudo/cli/CLI.h (-35)
- (removed) clang-tools-extra/pseudo/include/clang-pseudo/cxx/CXX.h (-91)
- (removed) clang-tools-extra/pseudo/include/clang-pseudo/grammar/Grammar.h (-230)
- (removed) clang-tools-extra/pseudo/include/clang-pseudo/grammar/LRGraph.h (-196)
- (removed) clang-tools-extra/pseudo/include/clang-pseudo/grammar/LRTable.h (-278)
- (removed) clang-tools-extra/pseudo/lib/CMakeLists.txt (-31)
- (removed) clang-tools-extra/pseudo/lib/Disambiguate.cpp (-48)
- (removed) clang-tools-extra/pseudo/lib/Forest.cpp (-199)
- (removed) clang-tools-extra/pseudo/lib/GLR.cpp (-772)
- (removed) clang-tools-extra/pseudo/lib/cli/CLI.cpp (-54)
- (removed) clang-tools-extra/pseudo/lib/cli/CMakeLists.txt (-15)
- (removed) clang-tools-extra/pseudo/lib/cxx/CMakeLists.txt (-19)
- (removed) clang-tools-extra/pseudo/lib/cxx/CXX.cpp (-452)
- (removed) clang-tools-extra/pseudo/lib/cxx/cxx.bnf (-775)
- (removed) clang-tools-extra/pseudo/lib/grammar/CMakeLists.txt (-10)
- (removed) clang-tools-extra/pseudo/lib/grammar/Grammar.cpp (-190)
- (removed) clang-tools-extra/pseudo/lib/grammar/GrammarBNF.cpp (-362)
- (removed) clang-tools-extra/pseudo/lib/grammar/LRGraph.cpp (-265)
- (removed) clang-tools-extra/pseudo/lib/grammar/LRTable.cpp (-79)
- (removed) clang-tools-extra/pseudo/lib/grammar/LRTableBuild.cpp (-121)
- (removed) clang-tools-extra/pseudo/test/.clang-format (-1)
- (removed) clang-tools-extra/pseudo/test/CMakeLists.txt (-29)
- (removed) clang-tools-extra/pseudo/test/Unit/lit.cfg.py (-25)
- (removed) clang-tools-extra/pseudo/test/Unit/lit.site.cfg.py.in (-11)
- (removed) clang-tools-extra/pseudo/test/check-cxx-bnf.test (-2)
- (removed) clang-tools-extra/pseudo/test/crash/backslashes.c (-4)
- (removed) clang-tools-extra/pseudo/test/cxx/capture-list.cpp (-23)
- (removed) clang-tools-extra/pseudo/test/cxx/contextual-keywords.cpp (-9)
- (removed) clang-tools-extra/pseudo/test/cxx/dangling-else.cpp (-22)
- (removed) clang-tools-extra/pseudo/test/cxx/decl-specfier-seq.cpp (-27)
- (removed) clang-tools-extra/pseudo/test/cxx/declarator-function.cpp (-9)
- (removed) clang-tools-extra/pseudo/test/cxx/declarator-var.cpp (-9)
- (removed) clang-tools-extra/pseudo/test/cxx/declator-member-function.cpp (-9)
- (removed) clang-tools-extra/pseudo/test/cxx/empty-member-declaration.cpp (-7)
- (removed) clang-tools-extra/pseudo/test/cxx/empty-member-spec.cpp (-13)
- (removed) clang-tools-extra/pseudo/test/cxx/keyword.cpp (-12)
- (removed) clang-tools-extra/pseudo/test/cxx/literals.cpp (-43)
- (removed) clang-tools-extra/pseudo/test/cxx/mixed-designator.cpp (-27)
- (removed) clang-tools-extra/pseudo/test/cxx/nested-name-specifier.cpp (-28)
- (removed) clang-tools-extra/pseudo/test/cxx/parameter-decl-clause.cpp (-14)
- (removed) clang-tools-extra/pseudo/test/cxx/predefined-identifier.cpp (-5)
- (removed) clang-tools-extra/pseudo/test/cxx/recovery-func-parameters.cpp (-13)
- (removed) clang-tools-extra/pseudo/test/cxx/recovery-init-list.cpp (-13)
- (removed) clang-tools-extra/pseudo/test/cxx/structured-binding.cpp (-6)
- (removed) clang-tools-extra/pseudo/test/cxx/template-empty-type-parameter.cpp (-3)
- (removed) clang-tools-extra/pseudo/test/cxx/unsized-array.cpp (-7)
- (removed) clang-tools-extra/pseudo/test/fuzzer.cpp (-4)
- (removed) clang-tools-extra/pseudo/test/glr-variant-start.cpp (-9)
- (removed) clang-tools-extra/pseudo/test/glr.cpp (-30)
- (removed) clang-tools-extra/pseudo/test/html-forest.c (-8)
- (removed) clang-tools-extra/pseudo/test/lex.c (-42)
- (removed) clang-tools-extra/pseudo/test/lit.cfg.py (-20)
- (removed) clang-tools-extra/pseudo/test/lit.local.cfg (-2)
- (removed) clang-tools-extra/pseudo/test/lit.site.cfg.py.in (-14)
- (removed) clang-tools-extra/pseudo/test/lr-build-basic.test (-32)
- (removed) clang-tools-extra/pseudo/test/lr-build-conflicts.test (-49)
- (removed) clang-tools-extra/pseudo/test/strip-directives.c (-49)
- (removed) clang-tools-extra/pseudo/tool/CMakeLists.txt (-30)
- (removed) clang-tools-extra/pseudo/tool/ClangPseudo.cpp (-243)
- (removed) clang-tools-extra/pseudo/tool/HTMLForest.cpp (-192)
- (removed) clang-tools-extra/pseudo/tool/HTMLForest.css (-93)
- (removed) clang-tools-extra/pseudo/tool/HTMLForest.html (-15)
- (removed) clang-tools-extra/pseudo/tool/HTMLForest.js (-290)
- (removed) clang-tools-extra/pseudo/unittests/BracketTest.cpp (-117)
- (removed) clang-tools-extra/pseudo/unittests/CMakeLists.txt (-32)
- (removed) clang-tools-extra/pseudo/unittests/CXXTest.cpp (-30)
- (removed) clang-tools-extra/pseudo/unittests/DirectiveTreeTest.cpp (-357)
- (removed) clang-tools-extra/pseudo/unittests/DisambiguateTest.cpp (-111)
- (removed) clang-tools-extra/pseudo/unittests/ForestTest.cpp (-180)
- (removed) clang-tools-extra/pseudo/unittests/GLRTest.cpp (-789)
- (removed) clang-tools-extra/pseudo/unittests/GrammarTest.cpp (-213)
- (removed) clang-tools-extra/pseudo/unittests/LRTableTest.cpp (-76)
- (removed) clang-tools-extra/pseudo/unittests/TokenTest.cpp (-224)
- (modified) clang/docs/ClangFormattedStatus.rst (-5)
- (modified) clang/docs/tools/clang-formatted-files.txt (-1)
``````````diff
diff --git a/clang-tools-extra/CMakeLists.txt b/clang-tools-extra/CMakeLists.txt
index f6a6b57b5ef0bc..6b6f2b1ca22765 100644
--- a/clang-tools-extra/CMakeLists.txt
+++ b/clang-tools-extra/CMakeLists.txt
@@ -27,7 +27,6 @@ add_subdirectory(clang-move)
add_subdirectory(clang-query)
add_subdirectory(include-cleaner)
add_subdirectory(pp-trace)
-add_subdirectory(pseudo)
add_subdirectory(tool-template)
option(CLANG_TOOLS_EXTRA_INCLUDE_DOCS "Generate build targets for the Clang Extra Tools docs."
diff --git a/clang-tools-extra/clangd/CMakeLists.txt b/clang-tools-extra/clangd/CMakeLists.txt
index c21d277d2ffcbd..8dcbf5f47e056a 100644
--- a/clang-tools-extra/clangd/CMakeLists.txt
+++ b/clang-tools-extra/clangd/CMakeLists.txt
@@ -183,7 +183,6 @@ target_link_libraries(clangDaemon
${LLVM_PTHREAD_LIB}
clangIncludeCleaner
- clangPseudo
clangTidy
clangTidyUtils
diff --git a/clang-tools-extra/clangd/SemanticSelection.cpp b/clang-tools-extra/clangd/SemanticSelection.cpp
index 3d687173b2be99..dd7116e619e6d0 100644
--- a/clang-tools-extra/clangd/SemanticSelection.cpp
+++ b/clang-tools-extra/clangd/SemanticSelection.cpp
@@ -11,9 +11,6 @@
#include "Protocol.h"
#include "Selection.h"
#include "SourceCode.h"
-#include "clang-pseudo/Bracket.h"
-#include "clang-pseudo/DirectiveTree.h"
-#include "clang-pseudo/Token.h"
#include "clang/AST/DeclBase.h"
#include "clang/Basic/SourceLocation.h"
#include "clang/Basic/SourceManager.h"
@@ -25,6 +22,9 @@
#include "llvm/ADT/StringRef.h"
#include "llvm/Support/Casting.h"
#include "llvm/Support/Error.h"
+#include "support/Bracket.h"
+#include "support/DirectiveTree.h"
+#include "support/Token.h"
#include <optional>
#include <queue>
#include <vector>
@@ -181,16 +181,16 @@ llvm::Expected<std::vector<FoldingRange>> getFoldingRanges(ParsedAST &AST) {
// Related issue: https://github.com/clangd/clangd/issues/310
llvm::Expected<std::vector<FoldingRange>>
getFoldingRanges(const std::string &Code, bool LineFoldingOnly) {
- auto OrigStream = pseudo::lex(Code, clang::pseudo::genericLangOpts());
+ auto OrigStream = lex(Code, genericLangOpts());
- auto DirectiveStructure = pseudo::DirectiveTree::parse(OrigStream);
- pseudo::chooseConditionalBranches(DirectiveStructure, OrigStream);
+ auto DirectiveStructure = DirectiveTree::parse(OrigStream);
+ chooseConditionalBranches(DirectiveStructure, OrigStream);
// FIXME: Provide ranges in the disabled-PP regions as well.
auto Preprocessed = DirectiveStructure.stripDirectives(OrigStream);
- auto ParseableStream = cook(Preprocessed, clang::pseudo::genericLangOpts());
- pseudo::pairBrackets(ParseableStream);
+ auto ParseableStream = cook(Preprocessed, genericLangOpts());
+ pairBrackets(ParseableStream);
std::vector<FoldingRange> Result;
auto AddFoldingRange = [&](Position Start, Position End,
@@ -205,19 +205,19 @@ getFoldingRanges(const std::string &Code, bool LineFoldingOnly) {
FR.kind = Kind.str();
Result.push_back(FR);
};
- auto OriginalToken = [&](const pseudo::Token &T) {
+ auto OriginalToken = [&](const Token &T) {
return OrigStream.tokens()[T.OriginalIndex];
};
- auto StartOffset = [&](const pseudo::Token &T) {
+ auto StartOffset = [&](const Token &T) {
return OriginalToken(T).text().data() - Code.data();
};
- auto StartPosition = [&](const pseudo::Token &T) {
+ auto StartPosition = [&](const Token &T) {
return offsetToPosition(Code, StartOffset(T));
};
- auto EndOffset = [&](const pseudo::Token &T) {
+ auto EndOffset = [&](const Token &T) {
return StartOffset(T) + OriginalToken(T).Length;
};
- auto EndPosition = [&](const pseudo::Token &T) {
+ auto EndPosition = [&](const Token &T) {
return offsetToPosition(Code, EndOffset(T));
};
auto Tokens = ParseableStream.tokens();
@@ -235,7 +235,7 @@ getFoldingRanges(const std::string &Code, bool LineFoldingOnly) {
}
}
}
- auto IsBlockComment = [&](const pseudo::Token &T) {
+ auto IsBlockComment = [&](const Token &T) {
assert(T.Kind == tok::comment);
return OriginalToken(T).Length >= 2 &&
Code.substr(StartOffset(T), 2) == "/*";
@@ -246,10 +246,10 @@ getFoldingRanges(const std::string &Code, bool LineFoldingOnly) {
T++;
continue;
}
- pseudo::Token *FirstComment = T;
+ Token *FirstComment = T;
// Show starting sentinals (// and /*) of the comment.
Position Start = offsetToPosition(Code, 2 + StartOffset(*FirstComment));
- pseudo::Token *LastComment = T;
+ Token *LastComment = T;
Position End = EndPosition(*T);
while (T != Tokens.end() && T->Kind == tok::comment &&
StartPosition(*T).line <= End.line + 1) {
diff --git a/clang-tools-extra/pseudo/lib/Bracket.cpp b/clang-tools-extra/clangd/support/Bracket.cpp
similarity index 97%
rename from clang-tools-extra/pseudo/lib/Bracket.cpp
rename to clang-tools-extra/clangd/support/Bracket.cpp
index 07836146ad8a58..93d0f38015efbf 100644
--- a/clang-tools-extra/pseudo/lib/Bracket.cpp
+++ b/clang-tools-extra/clangd/support/Bracket.cpp
@@ -62,10 +62,10 @@
//
//===----------------------------------------------------------------------===//
-#include "clang-pseudo/Bracket.h"
+#include "Bracket.h"
namespace clang {
-namespace pseudo {
+namespace clangd {
namespace {
struct Bracket {
@@ -83,7 +83,7 @@ struct Bracket {
// Find brackets in the stream and convert to Bracket struct.
std::vector<Bracket> findBrackets(const TokenStream &Stream) {
std::vector<Bracket> Brackets;
- auto Add = [&](const pseudo::Token &Tok, Bracket::BracketKind K,
+ auto Add = [&](const Token &Tok, Bracket::BracketKind K,
Bracket::Direction D) {
Brackets.push_back(
{K, D, Tok.Line, Tok.Indent, Stream.index(Tok), Bracket::None});
@@ -151,5 +151,5 @@ void pairBrackets(TokenStream &Stream) {
applyPairings(Brackets, Stream);
}
-} // namespace pseudo
+} // namespace clangd
} // namespace clang
diff --git a/clang-tools-extra/pseudo/include/clang-pseudo/Bracket.h b/clang-tools-extra/clangd/support/Bracket.h
similarity index 87%
rename from clang-tools-extra/pseudo/include/clang-pseudo/Bracket.h
rename to clang-tools-extra/clangd/support/Bracket.h
index 268cfff1ab07ab..b43c22cea06d06 100644
--- a/clang-tools-extra/pseudo/include/clang-pseudo/Bracket.h
+++ b/clang-tools-extra/clangd/support/Bracket.h
@@ -23,19 +23,19 @@
//
//===----------------------------------------------------------------------===//
-#ifndef CLANG_PSEUDO_BRACKET_H
-#define CLANG_PSEUDO_BRACKET_H
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_BRACKET_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_BRACKET_H
-#include "clang-pseudo/Token.h"
+#include "Token.h"
namespace clang {
-namespace pseudo {
+namespace clangd {
/// Identifies bracket token in the stream which should be paired.
/// Sets Token::Pair accordingly.
void pairBrackets(TokenStream &);
-} // namespace pseudo
+} // namespace clangd
} // namespace clang
-#endif
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANGD_BRACKET_H
diff --git a/clang-tools-extra/clangd/support/CMakeLists.txt b/clang-tools-extra/clangd/support/CMakeLists.txt
index 0c8c199dd4a4c2..a1a2a7765fae95 100644
--- a/clang-tools-extra/clangd/support/CMakeLists.txt
+++ b/clang-tools-extra/clangd/support/CMakeLists.txt
@@ -16,9 +16,12 @@ if(NOT HAVE_CXX_ATOMICS_WITHOUT_LIB OR NOT HAVE_CXX_ATOMICS64_WITHOUT_LIB)
endif()
add_clang_library(clangdSupport
+ Bracket.cpp
Cancellation.cpp
Context.cpp
+ DirectiveTree.cpp
FileCache.cpp
+ Lex.cpp
Logger.cpp
Markup.cpp
MemoryTree.cpp
@@ -27,6 +30,7 @@ add_clang_library(clangdSupport
ThreadCrashReporter.cpp
Threading.cpp
ThreadsafeFS.cpp
+ Token.cpp
Trace.cpp
LINK_LIBS
diff --git a/clang-tools-extra/pseudo/lib/DirectiveTree.cpp b/clang-tools-extra/clangd/support/DirectiveTree.cpp
similarity index 99%
rename from clang-tools-extra/pseudo/lib/DirectiveTree.cpp
rename to clang-tools-extra/clangd/support/DirectiveTree.cpp
index 9e853e46edc232..d25da111681afc 100644
--- a/clang-tools-extra/pseudo/lib/DirectiveTree.cpp
+++ b/clang-tools-extra/clangd/support/DirectiveTree.cpp
@@ -6,7 +6,7 @@
//
//===----------------------------------------------------------------------===//
-#include "clang-pseudo/DirectiveTree.h"
+#include "DirectiveTree.h"
#include "clang/Basic/IdentifierTable.h"
#include "clang/Basic/TokenKinds.h"
#include "llvm/Support/FormatVariadic.h"
@@ -14,7 +14,7 @@
#include <variant>
namespace clang {
-namespace pseudo {
+namespace clangd {
namespace {
class DirectiveParser {
@@ -353,5 +353,5 @@ TokenStream DirectiveTree::stripDirectives(const TokenStream &In) const {
return Out;
}
-} // namespace pseudo
+} // namespace clangd
} // namespace clang
diff --git a/clang-tools-extra/pseudo/include/clang-pseudo/DirectiveTree.h b/clang-tools-extra/clangd/support/DirectiveTree.h
similarity index 95%
rename from clang-tools-extra/pseudo/include/clang-pseudo/DirectiveTree.h
rename to clang-tools-extra/clangd/support/DirectiveTree.h
index 2b6cb63297915d..34f5a888863f26 100644
--- a/clang-tools-extra/pseudo/include/clang-pseudo/DirectiveTree.h
+++ b/clang-tools-extra/clangd/support/DirectiveTree.h
@@ -25,17 +25,17 @@
//
//===----------------------------------------------------------------------===//
-#ifndef CLANG_PSEUDO_DIRECTIVETREE_H
-#define CLANG_PSEUDO_DIRECTIVETREE_H
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_DIRECTIVETREE_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_DIRECTIVETREE_H
-#include "clang-pseudo/Token.h"
+#include "Token.h"
#include "clang/Basic/TokenKinds.h"
#include <optional>
#include <variant>
#include <vector>
namespace clang {
-namespace pseudo {
+namespace clangd {
/// Describes the structure of a source file, as seen by the preprocessor.
///
@@ -124,7 +124,7 @@ llvm::raw_ostream &operator<<(llvm::raw_ostream &,
/// The choices are stored in Conditional::Taken nodes.
void chooseConditionalBranches(DirectiveTree &, const TokenStream &Code);
-} // namespace pseudo
+} // namespace clangd
} // namespace clang
-#endif // CLANG_PSEUDO_DIRECTIVETREE_H
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANGD_DIRECTIVETREE_H
diff --git a/clang-tools-extra/pseudo/lib/Lex.cpp b/clang-tools-extra/clangd/support/Lex.cpp
similarity index 98%
rename from clang-tools-extra/pseudo/lib/Lex.cpp
rename to clang-tools-extra/clangd/support/Lex.cpp
index 2111476f04dc5b..f043b551b6bc6c 100644
--- a/clang-tools-extra/pseudo/lib/Lex.cpp
+++ b/clang-tools-extra/clangd/support/Lex.cpp
@@ -6,7 +6,7 @@
//
//===----------------------------------------------------------------------===//
-#include "clang-pseudo/Token.h"
+#include "Token.h"
#include "clang/Basic/IdentifierTable.h"
#include "clang/Basic/SourceLocation.h"
#include "clang/Basic/TokenKinds.h"
@@ -14,7 +14,7 @@
#include "clang/Lex/LiteralSupport.h"
namespace clang {
-namespace pseudo {
+namespace clangd {
TokenStream lex(const std::string &Code, const clang::LangOptions &LangOpts) {
clang::SourceLocation Start;
@@ -135,5 +135,5 @@ TokenStream cook(const TokenStream &Code, const LangOptions &LangOpts) {
return Result;
}
-} // namespace pseudo
+} // namespace clangd
} // namespace clang
diff --git a/clang-tools-extra/pseudo/lib/Token.cpp b/clang-tools-extra/clangd/support/Token.cpp
similarity index 98%
rename from clang-tools-extra/pseudo/lib/Token.cpp
rename to clang-tools-extra/clangd/support/Token.cpp
index 5b07a62f37fb2f..13eee66606061d 100644
--- a/clang-tools-extra/pseudo/lib/Token.cpp
+++ b/clang-tools-extra/clangd/support/Token.cpp
@@ -6,14 +6,14 @@
//
//===----------------------------------------------------------------------===//
-#include "clang-pseudo/Token.h"
+#include "Token.h"
#include "clang/Basic/LangOptions.h"
#include "llvm/ADT/StringExtras.h"
#include "llvm/Support/Format.h"
#include "llvm/Support/FormatVariadic.h"
namespace clang {
-namespace pseudo {
+namespace clangd {
llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, const Token &T) {
OS << llvm::formatv("{0} {1}:{2} ", clang::tok::getTokenName(T.Kind), T.Line,
@@ -126,5 +126,5 @@ TokenStream stripComments(const TokenStream &Input) {
return Out;
}
-} // namespace pseudo
+} // namespace clangd
} // namespace clang
diff --git a/clang-tools-extra/pseudo/include/clang-pseudo/Token.h b/clang-tools-extra/clangd/support/Token.h
similarity index 98%
rename from clang-tools-extra/pseudo/include/clang-pseudo/Token.h
rename to clang-tools-extra/clangd/support/Token.h
index 859fd7d2b3dfe2..555b6b0e4ce570 100644
--- a/clang-tools-extra/pseudo/include/clang-pseudo/Token.h
+++ b/clang-tools-extra/clangd/support/Token.h
@@ -25,8 +25,8 @@
//
//===----------------------------------------------------------------------===//
-#ifndef CLANG_PSEUDO_TOKEN_H
-#define CLANG_PSEUDO_TOKEN_H
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_TOKEN_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_TOKEN_H
#include "clang/Basic/LLVM.h"
#include "clang/Basic/LangStandard.h"
@@ -41,7 +41,7 @@
namespace clang {
class LangOptions;
-namespace pseudo {
+namespace clangd {
/// A single C++ or preprocessor token.
///
@@ -249,7 +249,7 @@ TokenStream cook(const TokenStream &, const clang::LangOptions &);
/// Drops comment tokens.
TokenStream stripComments(const TokenStream &);
-} // namespace pseudo
+} // namespace clangd
} // namespace clang
-#endif // CLANG_PSEUDO_TOKEN_H
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANGD_TOKEN_H
diff --git a/clang-tools-extra/pseudo/CMakeLists.txt b/clang-tools-extra/pseudo/CMakeLists.txt
deleted file mode 100644
index 24bc1530bb7d6f..00000000000000
--- a/clang-tools-extra/pseudo/CMakeLists.txt
+++ /dev/null
@@ -1,12 +0,0 @@
-include_directories(include)
-include_directories(${CMAKE_CURRENT_BINARY_DIR}/include)
-add_subdirectory(include)
-add_subdirectory(gen)
-add_subdirectory(lib)
-add_subdirectory(tool)
-add_subdirectory(fuzzer)
-add_subdirectory(benchmarks)
-if(CLANG_INCLUDE_TESTS)
- add_subdirectory(unittests)
- add_subdirectory(test)
-endif()
diff --git a/clang-tools-extra/pseudo/DesignNotes.md b/clang-tools-extra/pseudo/DesignNotes.md
deleted file mode 100644
index 421cc02aef7576..00000000000000
--- a/clang-tools-extra/pseudo/DesignNotes.md
+++ /dev/null
@@ -1,123 +0,0 @@
-# Error recovery (2022-05-07)
-
-Problem: we have two fairly generic cases of recovery bounded within a range:
- - sequences: `int x; this is absolute garbage; x++;`
- - brackets: `void foo() { this is garbage too }`
-
-So far, we've thought of these as different, and had precise ideas about
-brackets ("lazy parsing") and vague ones about sequences.
-Con we unify them instead?
-
-In both cases we want to recognize the bounds of the garbage item based on
-basic token-level features of the surrounding code, and avoid any interference
-with the surrounding code.
-
-## Brackets
-
-Consider a rule like `compound-stmt := { stmt-seq }`.
-
-The desired recovery is:
-- if we fail at `{ . stmt-seq }`
-- ... and we can find for the matching `}`
-- then consume up to that token as an opaque broken `stmt-seq`
-- ... and advance state to `{ stmt-seq . }`
-
-We can annotate the rule to describe this: `{ stmt-seq [recovery] }`.
-We can generalize as `{ stmt-seq [recovery=rbrace] }`, allowing for different
-**strategies** to find the resume point.
-
-(It's OK to assume we're skipping over one RHS nonterminal, we can always
-introduce a nonterminal for the bracket contents if necessary).
-
-## Sequences
-
-Can we apply the same technique to sequences?
-Simplest case: delimited right-recursive sequence.
-
-```
-param-list := param
-param-list := param , param-list
-```
-
-We need recovery in **both** rules.
-`param` in the first rule matches the *last* param in a list,
-in the second rule it matches all others. We want to recover in any position.
-
-If we want to be able to recovery `int x int y` as two parameters, then we
-should extract a `param-and-comma` rule that recovery could operate on.
-
-### Last vs not-last elements
-
-Sequences will usually have two rules with recovery, we discussed:
- - how to pick the correct one to recover with
- - in a left-recursive rule they correspond to last & not-last elements
- - the recovery strategy knows whether we're recoverying last or not-last
- - we could have the strategy return (pos, symbol parsed), and artificially
- require distinct symbols (e.g. `stmt` vs `last_stmt`).
- - we can rewrite left-recursion in the grammar as right-recursion
-
-However, on reflection I think we can simply allow recovery according to both
-rules. The "wrong" recovery will produce a parse head that dies.
-
-## How recovery fits into GLR
-
-Recovery should kick in at the point where we would otherwise abandon all
-variants of an high-level parse.
-
-e.g. Suppose we're parsing `static_cast<foo bar baz>(1)` and are up to `bar`.
-Our GSS looks something like:
-
-```
- "the static cast's type starts at foo"
----> {expr := static_cast < . type > ( expr )}
- | "foo... is a class name"
- +---- {type := identifier .}
- | "foo... is a template ID"
- +---- {type := identifier . < template-arg-list >}
-```
-
-Since `foo bar baz` isn't a valid class name or template ID, both active heads
-will soon die, as will the parent GSS Node - the latter should trigger recovery.
-
-- we need a refcount in GSS nodes so we can recognize never-reduced node death
-- when a node dies, we look up its recovery actions (symbol, strategy).
- These are the union of the recovery actions for each item.
- They can be stored in the action table.
- Here: `actions[State, death] = Recovery(type, matching-angle-bracket)`
-- we try each strategy: feeding in the start position = token of the dying node
- (`foo`) getting out the end position (`>`).
-- We form an opaque forest node with the correct symbol (`type`) spanning
- [start, end)
-- We create a GSS node to represent the state after recovery.
- The new state is found in the Goto table in the usual way.
-
-```
- "the static cast's type starts at foo"
----> {expr := static_cast < . type > ( expr )}
- | "`foo bar baz` is an unparseable type"
- +---- {expr := static_cast < type . > (expr)}
-```
-
-## Which recovery heads to activate
-
-We probably shouldn't *always* create active recovery heads when a recoverable
-node dies (and thus keep it alive).
-By design GLR creates multiple speculative parse heads and lets incorrect heads
-disappear.
-
-Concretely, the expression `(int *)(x)` is a valid cast, we probably shouldn't
-also parse it as a call whose callee is a broken expr.
-
-The simplest solution is to only create recovery heads if there are no normal
-heads remaining, i.e. if parsing is completely stuck. This is vulnerable if the
-"wrong" parse makes slightly more progress than the "right" parse which has
-better error recovery.
-
-A sophisticated variant might record recovery opportunities and pick the one
-with the rightmost *endpoint* when the last parse head dies.
-
-We should consider whether including every recovery in the parse forest might
-work after all - this would let disambiguation choose "broken" but likely parses
-over "valid" but unlikely ones.
-
-
diff --git a/clang-tools-extra/pseudo/Disambiguation.md b/clang-tools-extra/pseudo/Disambiguation.md
deleted file mode 100644
index 39e246a523beb9..00000000000000
--- a/clang-tools-extra/pseudo/Disambiguation.md
+++ /dev/null
@@ -1,367 +0,0 @@
-# Disambiguation
-
-The C++ grammar is highly ambiguous, so the GLR parser produces a forest of
-parses, represented compactly by a DAG.
-A real C++ parser finds the correct parse through semantic analysis: mostly
-resolving names. But we can't do that, as we don't parse the headers.
-
-Our disambiguation phase should take the parse forest, and choose a single parse
-tree that is most likely.
-It might **optionally** use some other hints (e.g. coding style, or what
-specific names tend to mean in this codebase).
-
-There are some grammatical ambiguities that can be resolved without semantic
-analysis, e.g. whether `int <declarator>{}` is a function-definition.
-We eliminate these earlier e.g., with rule guards. By "disambiguation" we mean
-choosing between interpretations that we can't reject confidently and locally.
-
-## Types of evidence
-
-We have limited information to go on, and strive to use similar heuristics a
-human reader might.
-
-### Likely and unlikely structure
-
-In some cases, the shape of a pa...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/109154
More information about the cfe-commits
mailing list