r348266 - PTH-- Remove feature entirely-
Erich Keane via cfe-commits
cfe-commits at lists.llvm.org
Tue Dec 4 06:34:09 PST 2018
Author: erichkeane
Date: Tue Dec 4 06:34:09 2018
New Revision: 348266
URL: http://llvm.org/viewvc/llvm-project?rev=348266&view=rev
Log:
PTH-- Remove feature entirely-
When debugging a boost build with a modified
version of Clang, I discovered that the PTH implementation
stores TokenKind in 8 bits. However, we currently have 368
TokenKinds.
The result is that the value gets truncated and the wrong token
gets picked up when including PTH files. It seems that this will
go wrong every time someone uses a token that uses the 9th bit.
Upon asking on IRC, it was brought up that this was a highly
experimental features that was considered a failure. I discovered
via googling that BoostBuild (mostly Boost.Math) is the only user of
this
feature, using the CC1 flag directly. I believe that this can be
transferred over to normal PCH with minimal effort:
https://github.com/boostorg/build/issues/367
Based on advice on IRC and research showing that this is a nearly
completely unused feature, this patch removes it entirely.
Note: I considered leaving the build-flags in place and making them
emit an error/warning, however since I've basically identified and
warned the only user, it seemed better to just remove them.
Differential Revision: https://reviews.llvm.org/D54547
Change-Id: If32744275ef1f585357bd6c1c813d96973c4d8d9
Removed:
cfe/trunk/docs/PTHInternals.rst
cfe/trunk/include/clang/Lex/PTHLexer.h
cfe/trunk/include/clang/Lex/PTHManager.h
cfe/trunk/lib/Frontend/CacheTokens.cpp
cfe/trunk/lib/Lex/PTHLexer.cpp
cfe/trunk/test/Driver/pth.c
cfe/trunk/test/PCH/emit-pth.c
cfe/trunk/test/PCH/pth.c
cfe/trunk/test/Preprocessor/include-pth.c
Modified:
cfe/trunk/NOTES.txt
cfe/trunk/docs/InternalsManual.rst
cfe/trunk/docs/ReleaseNotes.rst
cfe/trunk/include/clang/Basic/DiagnosticFrontendKinds.td
cfe/trunk/include/clang/Basic/DiagnosticLexKinds.td
cfe/trunk/include/clang/Driver/CC1Options.td
cfe/trunk/include/clang/Driver/Driver.h
cfe/trunk/include/clang/Driver/Options.td
cfe/trunk/include/clang/Frontend/FrontendActions.h
cfe/trunk/include/clang/Frontend/FrontendOptions.h
cfe/trunk/include/clang/Frontend/Utils.h
cfe/trunk/include/clang/Lex/Preprocessor.h
cfe/trunk/include/clang/Lex/PreprocessorOptions.h
cfe/trunk/lib/ARCMigrate/ARCMT.cpp
cfe/trunk/lib/Driver/Driver.cpp
cfe/trunk/lib/Driver/ToolChains/Clang.cpp
cfe/trunk/lib/Frontend/CMakeLists.txt
cfe/trunk/lib/Frontend/ChainedIncludesSource.cpp
cfe/trunk/lib/Frontend/CompilerInstance.cpp
cfe/trunk/lib/Frontend/CompilerInvocation.cpp
cfe/trunk/lib/Frontend/FrontendActions.cpp
cfe/trunk/lib/Frontend/InitPreprocessor.cpp
cfe/trunk/lib/FrontendTool/ExecuteCompilerInvocation.cpp
cfe/trunk/lib/Lex/CMakeLists.txt
cfe/trunk/lib/Lex/PPDirectives.cpp
cfe/trunk/lib/Lex/PPLexerChange.cpp
cfe/trunk/lib/Lex/PPMacroExpansion.cpp
cfe/trunk/lib/Lex/Pragma.cpp
cfe/trunk/lib/Lex/Preprocessor.cpp
cfe/trunk/lib/Serialization/ASTReader.cpp
cfe/trunk/lib/Serialization/ASTWriter.cpp
Modified: cfe/trunk/NOTES.txt
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/NOTES.txt?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/NOTES.txt (original)
+++ cfe/trunk/NOTES.txt Tue Dec 4 06:34:09 2018
@@ -10,13 +10,6 @@ This is similar to -Eonly.
//===---------------------------------------------------------------------===//
-Creating and using a PTH file for performance measurement (use a release build).
-
-$ clang -ccc-pch-is-pth -x objective-c-header INPUTS/Cocoa_h.m -o /tmp/tokencache
-$ clang -cc1 -token-cache /tmp/tokencache INPUTS/Cocoa_h.m
-
-//===---------------------------------------------------------------------===//
-
C++ Template Instantiation benchmark:
http://users.rcn.com/abrahams/instantiation_speed/index.html
Modified: cfe/trunk/docs/InternalsManual.rst
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/InternalsManual.rst?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/docs/InternalsManual.rst (original)
+++ cfe/trunk/docs/InternalsManual.rst Tue Dec 4 06:34:09 2018
@@ -559,13 +559,9 @@ The clang Driver and library are documen
Precompiled Headers
===================
-Clang supports two implementations of precompiled headers. The default
-implementation, precompiled headers (:doc:`PCH <PCHInternals>`) uses a
+Clang supports precompiled headers (:doc:`PCH <PCHInternals>`), which uses a
serialized representation of Clang's internal data structures, encoded with the
`LLVM bitstream format <https://llvm.org/docs/BitCodeFormat.html>`_.
-Pretokenized headers (:doc:`PTH <PTHInternals>`), on the other hand, contain a
-serialized representation of the tokens encountered when preprocessing a header
-(and anything that header includes).
The Frontend Library
====================
Removed: cfe/trunk/docs/PTHInternals.rst
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/PTHInternals.rst?rev=348265&view=auto
==============================================================================
--- cfe/trunk/docs/PTHInternals.rst (original)
+++ cfe/trunk/docs/PTHInternals.rst (removed)
@@ -1,163 +0,0 @@
-==========================
-Pretokenized Headers (PTH)
-==========================
-
-This document first describes the low-level interface for using PTH and
-then briefly elaborates on its design and implementation. If you are
-interested in the end-user view, please see the :ref:`User's Manual
-<usersmanual-precompiled-headers>`.
-
-Using Pretokenized Headers with ``clang`` (Low-level Interface)
-===============================================================
-
-The Clang compiler frontend, ``clang -cc1``, supports three command line
-options for generating and using PTH files.
-
-To generate PTH files using ``clang -cc1``, use the option ``-emit-pth``:
-
-.. code-block:: console
-
- $ clang -cc1 test.h -emit-pth -o test.h.pth
-
-This option is transparently used by ``clang`` when generating PTH
-files. Similarly, PTH files can be used as prefix headers using the
-``-include-pth`` option:
-
-.. code-block:: console
-
- $ clang -cc1 -include-pth test.h.pth test.c -o test.s
-
-Alternatively, Clang's PTH files can be used as a raw "token-cache" (or
-"content" cache) of the source included by the original header file.
-This means that the contents of the PTH file are searched as substitutes
-for *any* source files that are used by ``clang -cc1`` to process a
-source file. This is done by specifying the ``-token-cache`` option:
-
-.. code-block:: console
-
- $ cat test.h
- #include <stdio.h>
- $ clang -cc1 -emit-pth test.h -o test.h.pth
- $ cat test.c
- #include "test.h"
- $ clang -cc1 test.c -o test -token-cache test.h.pth
-
-In this example the contents of ``stdio.h`` (and the files it includes)
-will be retrieved from ``test.h.pth``, as the PTH file is being used in
-this case as a raw cache of the contents of ``test.h``. This is a
-low-level interface used to both implement the high-level PTH interface
-as well as to provide alternative means to use PTH-style caching.
-
-PTH Design and Implementation
-=============================
-
-Unlike GCC's precompiled headers, which cache the full ASTs and
-preprocessor state of a header file, Clang's pretokenized header files
-mainly cache the raw lexer *tokens* that are needed to segment the
-stream of characters in a source file into keywords, identifiers, and
-operators. Consequently, PTH serves to mainly directly speed up the
-lexing and preprocessing of a source file, while parsing and
-type-checking must be completely redone every time a PTH file is used.
-
-Basic Design Tradeoffs
-----------------------
-
-In the long term there are plans to provide an alternate PCH
-implementation for Clang that also caches the work for parsing and type
-checking the contents of header files. The current implementation of PCH
-in Clang as pretokenized header files was motivated by the following
-factors:
-
-**Language independence**
- PTH files work with any language that
- Clang's lexer can handle, including C, Objective-C, and (in the early
- stages) C++. This means development on language features at the
- parsing level or above (which is basically almost all interesting
- pieces) does not require PTH to be modified.
-
-**Simple design**
- Relatively speaking, PTH has a simple design and
- implementation, making it easy to test. Further, because the
- machinery for PTH resides at the lower-levels of the Clang library
- stack it is fairly straightforward to profile and optimize.
-
-Further, compared to GCC's PCH implementation (which is the dominate
-precompiled header file implementation that Clang can be directly
-compared against) the PTH design in Clang yields several attractive
-features:
-
-**Architecture independence**
- In contrast to GCC's PCH files (and
- those of several other compilers), Clang's PTH files are architecture
- independent, requiring only a single PTH file when building a
- program for multiple architectures.
-
- For example, on Mac OS X one may wish to compile a "universal binary"
- that runs on PowerPC, 32-bit Intel (i386), and 64-bit Intel
- architectures. In contrast, GCC requires a PCH file for each
- architecture, as the definitions of types in the AST are
- architecture-specific. Since a Clang PTH file essentially represents
- a lexical cache of header files, a single PTH file can be safely used
- when compiling for multiple architectures. This can also reduce
- compile times because only a single PTH file needs to be generated
- during a build instead of several.
-
-**Reduced memory pressure**
- Similar to GCC, Clang reads PTH files
- via the use of memory mapping (i.e., ``mmap``). Clang, however,
- memory maps PTH files as read-only, meaning that multiple invocations
- of ``clang -cc1`` can share the same pages in memory from a
- memory-mapped PTH file. In comparison, GCC also memory maps its PCH
- files but also modifies those pages in memory, incurring the
- copy-on-write costs. The read-only nature of PTH can greatly reduce
- memory pressure for builds involving multiple cores, thus improving
- overall scalability.
-
-**Fast generation**
- PTH files can be generated in a small fraction
- of the time needed to generate GCC's PCH files. Since PTH/PCH
- generation is a serial operation that typically blocks progress
- during a build, faster generation time leads to improved processor
- utilization with parallel builds on multicore machines.
-
-Despite these strengths, PTH's simple design suffers some algorithmic
-handicaps compared to other PCH strategies such as those used by GCC.
-While PTH can greatly speed up the processing time of a header file, the
-amount of work required to process a header file is still roughly linear
-in the size of the header file. In contrast, the amount of work done by
-GCC to process a precompiled header is (theoretically) constant (the
-ASTs for the header are literally memory mapped into the compiler). This
-means that only the pieces of the header file that are referenced by the
-source file including the header are the only ones the compiler needs to
-process during actual compilation. While GCC's particular implementation
-of PCH mitigates some of these algorithmic strengths via the use of
-copy-on-write pages, the approach itself can fundamentally dominate at
-an algorithmic level, especially when one considers header files of
-arbitrary size.
-
-There is also a PCH implementation for Clang based on the lazy
-deserialization of ASTs. This approach theoretically has the same
-constant-time algorithmic advantages just mentioned but also retains some
-of the strengths of PTH such as reduced memory pressure (ideal for
-multi-core builds).
-
-Internal PTH Optimizations
---------------------------
-
-While the main optimization employed by PTH is to reduce lexing time of
-header files by caching pre-lexed tokens, PTH also employs several other
-optimizations to speed up the processing of header files:
-
-- ``stat`` caching: PTH files cache information obtained via calls to
- ``stat`` that ``clang -cc1`` uses to resolve which files are included
- by ``#include`` directives. This greatly reduces the overhead
- involved in context-switching to the kernel to resolve included
- files.
-
-- Fast skipping of ``#ifdef`` ... ``#endif`` chains: PTH files
- record the basic structure of nested preprocessor blocks. When the
- condition of the preprocessor block is false, all of its tokens are
- immediately skipped instead of requiring them to be handled by
- Clang's preprocessor.
-
-
Modified: cfe/trunk/docs/ReleaseNotes.rst
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/ReleaseNotes.rst?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/docs/ReleaseNotes.rst (original)
+++ cfe/trunk/docs/ReleaseNotes.rst Tue Dec 4 06:34:09 2018
@@ -116,7 +116,9 @@ Improvements to Clang's diagnostics
Non-comprehensive list of changes in this release
-------------------------------------------------
-- ...
+- The experimental feature Pretokenized Headers (PTH) was removed in its
+ entirely from Clang. The feature did not properly work with about 1/3 of the
+ possible tokens available and was unmaintained.
New Compiler Flags
------------------
Modified: cfe/trunk/include/clang/Basic/DiagnosticFrontendKinds.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticFrontendKinds.td?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/include/clang/Basic/DiagnosticFrontendKinds.td (original)
+++ cfe/trunk/include/clang/Basic/DiagnosticFrontendKinds.td Tue Dec 4 06:34:09 2018
@@ -99,8 +99,6 @@ def err_fe_unable_to_interface_with_targ
"unable to interface with target machine">;
def err_fe_unable_to_open_output : Error<
"unable to open output file '%0': '%1'">;
-def err_fe_pth_file_has_no_source_header : Error<
- "PTH file '%0' does not designate an original source header file for -include-pth">;
def warn_fe_macro_contains_embedded_newline : Warning<
"macro '%0' contains embedded newline; text after the newline is ignored">;
def warn_fe_cc_print_header_failure : Warning<
Modified: cfe/trunk/include/clang/Basic/DiagnosticLexKinds.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticLexKinds.td?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/include/clang/Basic/DiagnosticLexKinds.td (original)
+++ cfe/trunk/include/clang/Basic/DiagnosticLexKinds.td Tue Dec 4 06:34:09 2018
@@ -258,13 +258,6 @@ def warn_bad_character_encoding : ExtWar
def err_lexing_string : Error<"failure when lexing a string">;
def err_placeholder_in_source : Error<"editor placeholder in source file">;
-
-//===----------------------------------------------------------------------===//
-// PTH Diagnostics
-//===----------------------------------------------------------------------===//
-def err_invalid_pth_file : Error<
- "invalid or corrupt PTH file '%0'">;
-
//===----------------------------------------------------------------------===//
// Preprocessor Diagnostics
//===----------------------------------------------------------------------===//
@@ -570,8 +563,6 @@ def err_pp_module_end_without_module_beg
"'#pragma clang module end'">;
def note_pp_module_begin_here : Note<
"entering module '%0' due to this pragma">;
-def err_pp_module_build_pth : Error<
- "'#pragma clang module build' not supported in pretokenized header">;
def err_pp_module_build_missing_end : Error<
"no matching '#pragma clang module endbuild' for this '#pragma clang module build'">;
Modified: cfe/trunk/include/clang/Driver/CC1Options.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/CC1Options.td?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/include/clang/Driver/CC1Options.td (original)
+++ cfe/trunk/include/clang/Driver/CC1Options.td Tue Dec 4 06:34:09 2018
@@ -562,8 +562,6 @@ def emit_module_interface : Flag<["-"],
HelpText<"Generate pre-compiled module file from a C++ module interface">;
def emit_header_module : Flag<["-"], "emit-header-module">,
HelpText<"Generate pre-compiled module file from a set of header files">;
-def emit_pth : Flag<["-"], "emit-pth">,
- HelpText<"Generate pre-tokenized header file">;
def emit_pch : Flag<["-"], "emit-pch">,
HelpText<"Generate pre-compiled header file">;
def emit_llvm_bc : Flag<["-"], "emit-llvm-bc">,
@@ -794,15 +792,11 @@ def internal_externc_isystem : JoinedOrS
// Preprocessor Options
//===----------------------------------------------------------------------===//
-def include_pth : Separate<["-"], "include-pth">, MetaVarName<"<file>">,
- HelpText<"Include file before parsing">;
def chain_include : Separate<["-"], "chain-include">, MetaVarName<"<file>">,
HelpText<"Include and chain a header file after turning it into PCH">;
def preamble_bytes_EQ : Joined<["-"], "preamble-bytes=">,
HelpText<"Assume that the precompiled header is a precompiled preamble "
"covering the first N bytes of the main file">;
-def token_cache : Separate<["-"], "token-cache">, MetaVarName<"<path>">,
- HelpText<"Use specified token cache file">;
def detailed_preprocessing_record : Flag<["-"], "detailed-preprocessing-record">,
HelpText<"include a detailed record of preprocessing actions">;
Modified: cfe/trunk/include/clang/Driver/Driver.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/Driver.h?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/include/clang/Driver/Driver.h (original)
+++ cfe/trunk/include/clang/Driver/Driver.h Tue Dec 4 06:34:09 2018
@@ -227,9 +227,6 @@ private:
unsigned CheckInputsExist : 1;
public:
- /// Use lazy precompiled headers for PCH support.
- unsigned CCCUsePCH : 1;
-
/// Force clang to emit reproducer for driver invocation. This is enabled
/// indirectly by setting FORCE_CLANG_DIAGNOSTICS_CRASH environment variable
/// or when using the -gen-reproducer driver flag.
Modified: cfe/trunk/include/clang/Driver/Options.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/Options.td?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/include/clang/Driver/Options.td (original)
+++ cfe/trunk/include/clang/Driver/Options.td Tue Dec 4 06:34:09 2018
@@ -258,10 +258,6 @@ def rsp_quoting : Joined<["--"], "rsp-qu
def ccc_gcc_name : Separate<["-"], "ccc-gcc-name">, InternalDriverOpt,
HelpText<"Name for native GCC compiler">,
MetaVarName<"<gcc-path>">;
-def ccc_pch_is_pch : Flag<["-"], "ccc-pch-is-pch">, InternalDriverOpt,
- HelpText<"Use lazy PCH for precompiled headers">;
-def ccc_pch_is_pth : Flag<["-"], "ccc-pch-is-pth">, InternalDriverOpt,
- HelpText<"Use pretokenized headers for precompiled headers">;
class InternalDebugOpt : Group<internal_debug_Group>,
Flags<[DriverOption, HelpHidden, CoreOption]>;
Modified: cfe/trunk/include/clang/Frontend/FrontendActions.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/FrontendActions.h?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/include/clang/Frontend/FrontendActions.h (original)
+++ cfe/trunk/include/clang/Frontend/FrontendActions.h Tue Dec 4 06:34:09 2018
@@ -261,11 +261,6 @@ protected:
void ExecuteAction() override;
};
-class GeneratePTHAction : public PreprocessorFrontendAction {
-protected:
- void ExecuteAction() override;
-};
-
class PreprocessOnlyAction : public PreprocessorFrontendAction {
protected:
void ExecuteAction() override;
Modified: cfe/trunk/include/clang/Frontend/FrontendOptions.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/FrontendOptions.h?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/include/clang/Frontend/FrontendOptions.h (original)
+++ cfe/trunk/include/clang/Frontend/FrontendOptions.h Tue Dec 4 06:34:09 2018
@@ -88,9 +88,6 @@ enum ActionKind {
/// Generate pre-compiled header.
GeneratePCH,
- /// Generate pre-tokenized header.
- GeneratePTH,
-
/// Only execute frontend initialization.
InitOnly,
Modified: cfe/trunk/include/clang/Frontend/Utils.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/Utils.h?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/include/clang/Frontend/Utils.h (original)
+++ cfe/trunk/include/clang/Frontend/Utils.h Tue Dec 4 06:34:09 2018
@@ -183,9 +183,6 @@ void AttachHeaderIncludeGen(Preprocessor
StringRef OutputPath = {},
bool ShowDepth = true, bool MSStyle = false);
-/// Cache tokens for use with PCH. Note that this requires a seekable stream.
-void CacheTokens(Preprocessor &PP, raw_pwrite_stream *OS);
-
/// The ChainedIncludesSource class converts headers to chained PCHs in
/// memory, mainly for testing.
IntrusiveRefCntPtr<ExternalSemaSource>
Removed: cfe/trunk/include/clang/Lex/PTHLexer.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/PTHLexer.h?rev=348265&view=auto
==============================================================================
--- cfe/trunk/include/clang/Lex/PTHLexer.h (original)
+++ cfe/trunk/include/clang/Lex/PTHLexer.h (removed)
@@ -1,107 +0,0 @@
-//===- PTHLexer.h - Lexer based on Pre-tokenized input ----------*- C++ -*-===//
-//
-// The LLVM Compiler Infrastructure
-//
-// This file is distributed under the University of Illinois Open Source
-// License. See LICENSE.TXT for details.
-//
-//===----------------------------------------------------------------------===//
-//
-// This file defines the PTHLexer interface.
-//
-//===----------------------------------------------------------------------===//
-
-#ifndef LLVM_CLANG_LEX_PTHLEXER_H
-#define LLVM_CLANG_LEX_PTHLEXER_H
-
-#include "clang/Basic/SourceLocation.h"
-#include "clang/Basic/TokenKinds.h"
-#include "clang/Lex/PreprocessorLexer.h"
-#include "clang/Lex/Token.h"
-
-namespace clang {
-
-class Preprocessor;
-class PTHManager;
-
-class PTHLexer : public PreprocessorLexer {
- SourceLocation FileStartLoc;
-
- /// TokBuf - Buffer from PTH file containing raw token data.
- const unsigned char* TokBuf;
-
- /// CurPtr - Pointer into current offset of the token buffer where
- /// the next token will be read.
- const unsigned char* CurPtr;
-
- /// LastHashTokPtr - Pointer into TokBuf of the last processed '#'
- /// token that appears at the start of a line.
- const unsigned char* LastHashTokPtr = nullptr;
-
- /// PPCond - Pointer to a side table in the PTH file that provides a
- /// a concise summary of the preprocessor conditional block structure.
- /// This is used to perform quick skipping of conditional blocks.
- const unsigned char* PPCond;
-
- /// CurPPCondPtr - Pointer inside PPCond that refers to the next entry
- /// to process when doing quick skipping of preprocessor blocks.
- const unsigned char* CurPPCondPtr;
-
- /// ReadToken - Used by PTHLexer to read tokens TokBuf.
- void ReadToken(Token &T);
-
- bool LexEndOfFile(Token &Result);
-
- /// PTHMgr - The PTHManager object that created this PTHLexer.
- PTHManager& PTHMgr;
-
- Token EofToken;
-
-protected:
- friend class PTHManager;
-
- /// Create a PTHLexer for the specified token stream.
- PTHLexer(Preprocessor &pp, FileID FID, const unsigned char *D,
- const unsigned char* ppcond, PTHManager &PM);
-
-public:
- PTHLexer(const PTHLexer &) = delete;
- PTHLexer &operator=(const PTHLexer &) = delete;
- ~PTHLexer() override = default;
-
- /// Lex - Return the next token.
- bool Lex(Token &Tok);
-
- void getEOF(Token &Tok);
-
- /// DiscardToEndOfLine - Read the rest of the current preprocessor line as an
- /// uninterpreted string. This switches the lexer out of directive mode.
- void DiscardToEndOfLine();
-
- /// isNextPPTokenLParen - Return 1 if the next unexpanded token will return a
- /// tok::l_paren token, 0 if it is something else and 2 if there are no more
- /// tokens controlled by this lexer.
- unsigned isNextPPTokenLParen() {
- // isNextPPTokenLParen is not on the hot path, and all we care about is
- // whether or not we are at a token with kind tok::eof or tok::l_paren.
- // Just read the first byte from the current token pointer to determine
- // its kind.
- tok::TokenKind x = (tok::TokenKind)*CurPtr;
- return x == tok::eof ? 2 : x == tok::l_paren;
- }
-
- /// IndirectLex - An indirect call to 'Lex' that can be invoked via
- /// the PreprocessorLexer interface.
- void IndirectLex(Token &Result) override { Lex(Result); }
-
- /// getSourceLocation - Return a source location for the token in
- /// the current file.
- SourceLocation getSourceLocation() override;
-
- /// SkipBlock - Used by Preprocessor to skip the current conditional block.
- bool SkipBlock();
-};
-
-} // namespace clang
-
-#endif // LLVM_CLANG_LEX_PTHLEXER_H
Removed: cfe/trunk/include/clang/Lex/PTHManager.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/PTHManager.h?rev=348265&view=auto
==============================================================================
--- cfe/trunk/include/clang/Lex/PTHManager.h (original)
+++ cfe/trunk/include/clang/Lex/PTHManager.h (removed)
@@ -1,149 +0,0 @@
-//===- PTHManager.h - Manager object for PTH processing ---------*- C++ -*-===//
-//
-// The LLVM Compiler Infrastructure
-//
-// This file is distributed under the University of Illinois Open Source
-// License. See LICENSE.TXT for details.
-//
-//===----------------------------------------------------------------------===//
-//
-// This file defines the PTHManager interface.
-//
-//===----------------------------------------------------------------------===//
-
-#ifndef LLVM_CLANG_LEX_PTHMANAGER_H
-#define LLVM_CLANG_LEX_PTHMANAGER_H
-
-#include "clang/Basic/IdentifierTable.h"
-#include "clang/Basic/SourceLocation.h"
-#include "llvm/ADT/STLExtras.h"
-#include "llvm/ADT/StringRef.h"
-#include "llvm/Support/Allocator.h"
-#include "llvm/Support/OnDiskHashTable.h"
-#include <memory>
-
-namespace llvm {
-
-class MemoryBuffer;
-
-} // namespace llvm
-
-namespace clang {
-
-class DiagnosticsEngine;
-class FileSystemStatCache;
-class Preprocessor;
-class PTHLexer;
-
-class PTHManager : public IdentifierInfoLookup {
- friend class PTHLexer;
- friend class PTHStatCache;
-
- class PTHFileLookupTrait;
- class PTHStringLookupTrait;
-
- using PTHStringIdLookup = llvm::OnDiskChainedHashTable<PTHStringLookupTrait>;
- using PTHFileLookup = llvm::OnDiskChainedHashTable<PTHFileLookupTrait>;
-
- /// The memory mapped PTH file.
- std::unique_ptr<const llvm::MemoryBuffer> Buf;
-
- /// Alloc - Allocator used for IdentifierInfo objects.
- llvm::BumpPtrAllocator Alloc;
-
- /// IdMap - A lazily generated cache mapping from persistent identifiers to
- /// IdentifierInfo*.
- std::unique_ptr<IdentifierInfo *[], llvm::FreeDeleter> PerIDCache;
-
- /// FileLookup - Abstract data structure used for mapping between files
- /// and token data in the PTH file.
- std::unique_ptr<PTHFileLookup> FileLookup;
-
- /// IdDataTable - Array representing the mapping from persistent IDs to the
- /// data offset within the PTH file containing the information to
- /// reconsitute an IdentifierInfo.
- const unsigned char* const IdDataTable;
-
- /// SortedIdTable - Abstract data structure mapping from strings to
- /// persistent IDs. This is used by get().
- std::unique_ptr<PTHStringIdLookup> StringIdLookup;
-
- /// NumIds - The number of identifiers in the PTH file.
- const unsigned NumIds;
-
- /// PP - The Preprocessor object that will use this PTHManager to create
- /// PTHLexer objects.
- Preprocessor* PP = nullptr;
-
- /// SpellingBase - The base offset within the PTH memory buffer that
- /// contains the cached spellings for literals.
- const unsigned char* const SpellingBase;
-
- /// OriginalSourceFile - A null-terminated C-string that specifies the name
- /// if the file (if any) that was to used to generate the PTH cache.
- const char* OriginalSourceFile;
-
- /// This constructor is intended to only be called by the static 'Create'
- /// method.
- PTHManager(std::unique_ptr<const llvm::MemoryBuffer> buf,
- std::unique_ptr<PTHFileLookup> fileLookup,
- const unsigned char *idDataTable,
- std::unique_ptr<IdentifierInfo *[], llvm::FreeDeleter> perIDCache,
- std::unique_ptr<PTHStringIdLookup> stringIdLookup, unsigned numIds,
- const unsigned char *spellingBase, const char *originalSourceFile);
-
- /// getSpellingAtPTHOffset - Used by PTHLexer classes to get the cached
- /// spelling for a token.
- unsigned getSpellingAtPTHOffset(unsigned PTHOffset, const char*& Buffer);
-
- /// GetIdentifierInfo - Used to reconstruct IdentifierInfo objects from the
- /// PTH file.
- IdentifierInfo *GetIdentifierInfo(unsigned PersistentID) {
- // Check if the IdentifierInfo has already been resolved.
- if (IdentifierInfo* II = PerIDCache[PersistentID])
- return II;
- return LazilyCreateIdentifierInfo(PersistentID);
- }
- IdentifierInfo* LazilyCreateIdentifierInfo(unsigned PersistentID);
-
-public:
- // The current PTH version.
- enum { Version = 10 };
-
- PTHManager(const PTHManager &) = delete;
- PTHManager &operator=(const PTHManager &) = delete;
- ~PTHManager() override;
-
- /// getOriginalSourceFile - Return the full path to the original header
- /// file name that was used to generate the PTH cache.
- const char* getOriginalSourceFile() const {
- return OriginalSourceFile;
- }
-
- /// get - Return the identifier token info for the specified named identifier.
- /// Unlike the version in IdentifierTable, this returns a pointer instead
- /// of a reference. If the pointer is nullptr then the IdentifierInfo cannot
- /// be found.
- IdentifierInfo *get(StringRef Name) override;
-
- /// Create - This method creates PTHManager objects. The 'file' argument
- /// is the name of the PTH file. This method returns nullptr upon failure.
- static PTHManager *Create(StringRef file, DiagnosticsEngine &Diags);
-
- void setPreprocessor(Preprocessor *pp) { PP = pp; }
-
- /// CreateLexer - Return a PTHLexer that "lexes" the cached tokens for the
- /// specified file. This method returns nullptr if no cached tokens exist.
- /// It is the responsibility of the caller to 'delete' the returned object.
- PTHLexer *CreateLexer(FileID FID);
-
- /// createStatCache - Returns a FileSystemStatCache object for use with
- /// FileManager objects. These objects use the PTH data to speed up
- /// calls to stat by memoizing their results from when the PTH file
- /// was generated.
- std::unique_ptr<FileSystemStatCache> createStatCache();
-};
-
-} // namespace clang
-
-#endif // LLVM_CLANG_LEX_PTHMANAGER_H
Modified: cfe/trunk/include/clang/Lex/Preprocessor.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/Preprocessor.h?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/include/clang/Lex/Preprocessor.h (original)
+++ cfe/trunk/include/clang/Lex/Preprocessor.h Tue Dec 4 06:34:09 2018
@@ -29,7 +29,6 @@
#include "clang/Lex/ModuleLoader.h"
#include "clang/Lex/ModuleMap.h"
#include "clang/Lex/PPCallbacks.h"
-#include "clang/Lex/PTHLexer.h"
#include "clang/Lex/Token.h"
#include "clang/Lex/TokenLexer.h"
#include "llvm/ADT/ArrayRef.h"
@@ -79,7 +78,6 @@ class PragmaNamespace;
class PreprocessingRecord;
class PreprocessorLexer;
class PreprocessorOptions;
-class PTHManager;
class ScratchBuffer;
class TargetInfo;
@@ -143,10 +141,6 @@ class Preprocessor {
/// External source of macros.
ExternalPreprocessorSource *ExternalSource;
- /// An optional PTHManager object used for getting tokens from
- /// a token cache rather than lexing the original source file.
- std::unique_ptr<PTHManager> PTH;
-
/// A BumpPtrAllocator object used to quickly allocate and release
/// objects internal to the Preprocessor.
llvm::BumpPtrAllocator BP;
@@ -392,19 +386,13 @@ private:
/// The current top of the stack that we're lexing from if
/// not expanding a macro and we are lexing directly from source code.
///
- /// Only one of CurLexer, CurPTHLexer, or CurTokenLexer will be non-null.
+ /// Only one of CurLexer, or CurTokenLexer will be non-null.
std::unique_ptr<Lexer> CurLexer;
- /// The current top of stack that we're lexing from if
- /// not expanding from a macro and we are lexing from a PTH cache.
- ///
- /// Only one of CurLexer, CurPTHLexer, or CurTokenLexer will be non-null.
- std::unique_ptr<PTHLexer> CurPTHLexer;
-
/// The current top of the stack what we're lexing from
/// if not expanding a macro.
///
- /// This is an alias for either CurLexer or CurPTHLexer.
+ /// This is an alias for CurLexer.
PreprocessorLexer *CurPPLexer = nullptr;
/// Used to find the current FileEntry, if CurLexer is non-null
@@ -422,7 +410,6 @@ private:
/// The kind of lexer we're currently working with.
enum CurLexerKind {
CLK_Lexer,
- CLK_PTHLexer,
CLK_TokenLexer,
CLK_CachingLexer,
CLK_LexAfterModuleImport
@@ -439,7 +426,6 @@ private:
enum CurLexerKind CurLexerKind;
Module *TheSubmodule;
std::unique_ptr<Lexer> TheLexer;
- std::unique_ptr<PTHLexer> ThePTHLexer;
PreprocessorLexer *ThePPLexer;
std::unique_ptr<TokenLexer> TheTokenLexer;
const DirectoryLookup *TheDirLookup;
@@ -448,13 +434,11 @@ private:
// versions, only needed to pacify MSVC.
IncludeStackInfo(enum CurLexerKind CurLexerKind, Module *TheSubmodule,
std::unique_ptr<Lexer> &&TheLexer,
- std::unique_ptr<PTHLexer> &&ThePTHLexer,
PreprocessorLexer *ThePPLexer,
std::unique_ptr<TokenLexer> &&TheTokenLexer,
const DirectoryLookup *TheDirLookup)
: CurLexerKind(std::move(CurLexerKind)),
TheSubmodule(std::move(TheSubmodule)), TheLexer(std::move(TheLexer)),
- ThePTHLexer(std::move(ThePTHLexer)),
ThePPLexer(std::move(ThePPLexer)),
TheTokenLexer(std::move(TheTokenLexer)),
TheDirLookup(std::move(TheDirLookup)) {}
@@ -838,10 +822,6 @@ public:
Builtin::Context &getBuiltinInfo() { return BuiltinInfo; }
llvm::BumpPtrAllocator &getPreprocessorAllocator() { return BP; }
- void setPTHManager(PTHManager* pm);
-
- PTHManager *getPTHManager() { return PTH.get(); }
-
void setExternalSource(ExternalPreprocessorSource *Source) {
ExternalSource = Source;
}
@@ -1456,7 +1436,7 @@ public:
CachedTokens[CachedLexPos-1] = Tok;
}
- /// Recompute the current lexer kind based on the CurLexer/CurPTHLexer/
+ /// Recompute the current lexer kind based on the CurLexer/
/// CurTokenLexer pointers.
void recomputeCurLexerKind();
@@ -1915,15 +1895,13 @@ private:
void PushIncludeMacroStack() {
assert(CurLexerKind != CLK_CachingLexer && "cannot push a caching lexer");
IncludeMacroStack.emplace_back(CurLexerKind, CurLexerSubmodule,
- std::move(CurLexer), std::move(CurPTHLexer),
- CurPPLexer, std::move(CurTokenLexer),
- CurDirLookup);
+ std::move(CurLexer), CurPPLexer,
+ std::move(CurTokenLexer), CurDirLookup);
CurPPLexer = nullptr;
}
void PopIncludeMacroStack() {
CurLexer = std::move(IncludeMacroStack.back().TheLexer);
- CurPTHLexer = std::move(IncludeMacroStack.back().ThePTHLexer);
CurPPLexer = IncludeMacroStack.back().ThePPLexer;
CurTokenLexer = std::move(IncludeMacroStack.back().TheTokenLexer);
CurDirLookup = IncludeMacroStack.back().TheDirLookup;
@@ -1992,9 +1970,6 @@ private:
bool FoundNonSkipPortion, bool FoundElse,
SourceLocation ElseLoc = SourceLocation());
- /// A fast PTH version of SkipExcludedConditionalBlock.
- void PTHSkipExcludedConditionalBlock();
-
/// Information about the result for evaluating an expression for a
/// preprocessor directive.
struct DirectiveEvalResult {
@@ -2060,10 +2035,6 @@ private:
/// start lexing tokens from it instead of the current buffer.
void EnterSourceFileWithLexer(Lexer *TheLexer, const DirectoryLookup *Dir);
- /// Add a lexer to the top of the include stack and
- /// start getting tokens from it using the PTH cache.
- void EnterSourceFileWithPTH(PTHLexer *PL, const DirectoryLookup *Dir);
-
/// Set the FileID for the preprocessor predefines.
void setPredefinesFileID(FileID FID) {
assert(PredefinesFileID.isInvalid() && "PredefinesFileID already set!");
@@ -2094,8 +2065,7 @@ private:
bool InCachingLexMode() const {
// If the Lexer pointers are 0 and IncludeMacroStack is empty, it means
// that we are past EOF, not that we are in CachingLex mode.
- return !CurPPLexer && !CurTokenLexer && !CurPTHLexer &&
- !IncludeMacroStack.empty();
+ return !CurPPLexer && !CurTokenLexer && !IncludeMacroStack.empty();
}
void EnterCachingLexMode();
Modified: cfe/trunk/include/clang/Lex/PreprocessorOptions.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/PreprocessorOptions.h?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/include/clang/Lex/PreprocessorOptions.h (original)
+++ cfe/trunk/include/clang/Lex/PreprocessorOptions.h Tue Dec 4 06:34:09 2018
@@ -110,13 +110,6 @@ public:
/// clients don't use them.
bool WriteCommentListToPCH = true;
- /// The implicit PTH input included at the start of the translation unit, or
- /// empty.
- std::string ImplicitPTHInclude;
-
- /// If given, a PTH cache file to use for speeding up header parsing.
- std::string TokenCache;
-
/// When enabled, preprocessor is in a mode for parsing a single file only.
///
/// Disables #includes of other files and if there are unresolved identifiers
@@ -204,8 +197,6 @@ public:
ChainedIncludes.clear();
DumpDeserializedPCHDecls = false;
ImplicitPCHInclude.clear();
- ImplicitPTHInclude.clear();
- TokenCache.clear();
SingleFileParseMode = false;
LexEditorPlaceholders = true;
RetainRemappedFileBuffers = true;
Modified: cfe/trunk/lib/ARCMigrate/ARCMT.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/ARCMigrate/ARCMT.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/ARCMigrate/ARCMT.cpp (original)
+++ cfe/trunk/lib/ARCMigrate/ARCMT.cpp Tue Dec 4 06:34:09 2018
@@ -190,8 +190,6 @@ createInvocationForMigration(CompilerInv
PPOpts.Includes.insert(PPOpts.Includes.begin(), OriginalFile);
PPOpts.ImplicitPCHInclude.clear();
}
- // FIXME: Get the original header of a PTH as well.
- CInvok->getPreprocessorOpts().ImplicitPTHInclude.clear();
std::string define = getARCMTMacroName();
define += '=';
CInvok->getPreprocessorOpts().addMacroDef(define);
Modified: cfe/trunk/lib/Driver/Driver.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/Driver.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Driver/Driver.cpp (original)
+++ cfe/trunk/lib/Driver/Driver.cpp Tue Dec 4 06:34:09 2018
@@ -101,8 +101,7 @@ Driver::Driver(StringRef ClangExecutable
CCPrintOptions(false), CCPrintHeaders(false), CCLogDiagnostics(false),
CCGenDiagnostics(false), TargetTriple(TargetTriple),
CCCGenericGCCName(""), Saver(Alloc), CheckInputsExist(true),
- CCCUsePCH(true), GenReproducer(false),
- SuppressMissingInputWarning(false) {
+ GenReproducer(false), SuppressMissingInputWarning(false) {
// Provide a sane fallback if no VFS is specified.
if (!this->VFS)
@@ -1007,8 +1006,6 @@ Compilation *Driver::BuildCompilation(Ar
CCCPrintBindings = Args.hasArg(options::OPT_ccc_print_bindings);
if (const Arg *A = Args.getLastArg(options::OPT_ccc_gcc_name))
CCCGenericGCCName = A->getValue();
- CCCUsePCH =
- Args.hasFlag(options::OPT_ccc_pch_is_pch, options::OPT_ccc_pch_is_pth);
GenReproducer = Args.hasFlag(options::OPT_gen_reproducer,
options::OPT_fno_crash_diagnostics,
!!::getenv("FORCE_CLANG_DIAGNOSTICS_CRASH"));
Modified: cfe/trunk/lib/Driver/ToolChains/Clang.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/ToolChains/Clang.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Driver/ToolChains/Clang.cpp (original)
+++ cfe/trunk/lib/Driver/ToolChains/Clang.cpp Tue Dec 4 06:34:09 2018
@@ -1152,42 +1152,26 @@ void Clang::AddPreprocessingOptions(Comp
bool IsFirstImplicitInclude = !RenderedImplicitInclude;
RenderedImplicitInclude = true;
- // Use PCH if the user requested it.
- bool UsePCH = D.CCCUsePCH;
-
- bool FoundPTH = false;
bool FoundPCH = false;
SmallString<128> P(A->getValue());
// We want the files to have a name like foo.h.pch. Add a dummy extension
// so that replace_extension does the right thing.
P += ".dummy";
- if (UsePCH) {
- llvm::sys::path::replace_extension(P, "pch");
- if (llvm::sys::fs::exists(P))
- FoundPCH = true;
- }
+ llvm::sys::path::replace_extension(P, "pch");
+ if (llvm::sys::fs::exists(P))
+ FoundPCH = true;
if (!FoundPCH) {
- llvm::sys::path::replace_extension(P, "pth");
- if (llvm::sys::fs::exists(P))
- FoundPTH = true;
- }
-
- if (!FoundPCH && !FoundPTH) {
llvm::sys::path::replace_extension(P, "gch");
if (llvm::sys::fs::exists(P)) {
- FoundPCH = UsePCH;
- FoundPTH = !UsePCH;
+ FoundPCH = true;
}
}
- if (FoundPCH || FoundPTH) {
+ if (FoundPCH) {
if (IsFirstImplicitInclude) {
A->claim();
- if (UsePCH)
- CmdArgs.push_back("-include-pch");
- else
- CmdArgs.push_back("-include-pth");
+ CmdArgs.push_back("-include-pch");
CmdArgs.push_back(Args.MakeArgString(P));
continue;
} else {
@@ -3467,19 +3451,14 @@ void Clang::ConstructJob(Compilation &C,
// Also ignore explicit -force_cpusubtype_ALL option.
(void)Args.hasArg(options::OPT_force__cpusubtype__ALL);
} else if (isa<PrecompileJobAction>(JA)) {
- // Use PCH if the user requested it.
- bool UsePCH = D.CCCUsePCH;
-
if (JA.getType() == types::TY_Nothing)
CmdArgs.push_back("-fsyntax-only");
else if (JA.getType() == types::TY_ModuleFile)
CmdArgs.push_back(IsHeaderModulePrecompile
? "-emit-header-module"
: "-emit-module-interface");
- else if (UsePCH)
- CmdArgs.push_back("-emit-pch");
else
- CmdArgs.push_back("-emit-pth");
+ CmdArgs.push_back("-emit-pch");
} else if (isa<VerifyPCHJobAction>(JA)) {
CmdArgs.push_back("-verify-pch");
} else {
@@ -5247,8 +5226,7 @@ void Clang::ConstructJob(Compilation &C,
// Claim some arguments which clang supports automatically.
// -fpch-preprocess is used with gcc to add a special marker in the output to
- // include the PCH file. Clang's PTH solution is completely transparent, so we
- // do not need to deal with it at all.
+ // include the PCH file.
Args.ClaimAllArgs(options::OPT_fpch_preprocess);
// Claim some arguments which clang doesn't support, but we don't
Modified: cfe/trunk/lib/Frontend/CMakeLists.txt
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/CMakeLists.txt?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/CMakeLists.txt (original)
+++ cfe/trunk/lib/Frontend/CMakeLists.txt Tue Dec 4 06:34:09 2018
@@ -16,7 +16,6 @@ add_clang_library(clangFrontend
ASTConsumers.cpp
ASTMerge.cpp
ASTUnit.cpp
- CacheTokens.cpp
ChainedDiagnosticConsumer.cpp
ChainedIncludesSource.cpp
CodeGenOptions.cpp
Removed: cfe/trunk/lib/Frontend/CacheTokens.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/CacheTokens.cpp?rev=348265&view=auto
==============================================================================
--- cfe/trunk/lib/Frontend/CacheTokens.cpp (original)
+++ cfe/trunk/lib/Frontend/CacheTokens.cpp (removed)
@@ -1,700 +0,0 @@
-//===--- CacheTokens.cpp - Caching of lexer tokens for PTH support --------===//
-//
-// The LLVM Compiler Infrastructure
-//
-// This file is distributed under the University of Illinois Open Source
-// License. See LICENSE.TXT for details.
-//
-//===----------------------------------------------------------------------===//
-//
-// This provides a possible implementation of PTH support for Clang that is
-// based on caching lexed tokens and identifiers.
-//
-//===----------------------------------------------------------------------===//
-
-#include "clang/Basic/Diagnostic.h"
-#include "clang/Basic/FileManager.h"
-#include "clang/Basic/FileSystemStatCache.h"
-#include "clang/Basic/IdentifierTable.h"
-#include "clang/Basic/SourceManager.h"
-#include "clang/Frontend/Utils.h"
-#include "clang/Lex/Lexer.h"
-#include "clang/Lex/PTHManager.h"
-#include "clang/Lex/Preprocessor.h"
-#include "llvm/ADT/StringMap.h"
-#include "llvm/Support/DJB.h"
-#include "llvm/Support/EndianStream.h"
-#include "llvm/Support/FileSystem.h"
-#include "llvm/Support/MemoryBuffer.h"
-#include "llvm/Support/OnDiskHashTable.h"
-#include "llvm/Support/Path.h"
-
-// FIXME: put this somewhere else?
-#ifndef S_ISDIR
-#define S_ISDIR(x) (((x)&_S_IFDIR)!=0)
-#endif
-
-using namespace clang;
-
-//===----------------------------------------------------------------------===//
-// PTH-specific stuff.
-//===----------------------------------------------------------------------===//
-
-typedef uint32_t Offset;
-
-namespace {
-class PTHEntry {
- Offset TokenData, PPCondData;
-
-public:
- PTHEntry() {}
-
- PTHEntry(Offset td, Offset ppcd)
- : TokenData(td), PPCondData(ppcd) {}
-
- Offset getTokenOffset() const { return TokenData; }
- Offset getPPCondTableOffset() const { return PPCondData; }
-};
-
-
-class PTHEntryKeyVariant {
- union {
- const FileEntry *FE;
- // FIXME: Use "StringRef Path;" when MSVC 2013 is dropped.
- const char *PathPtr;
- };
- size_t PathSize;
- enum { IsFE = 0x1, IsDE = 0x2, IsNoExist = 0x0 } Kind;
- FileData *Data;
-
-public:
- PTHEntryKeyVariant(const FileEntry *fe) : FE(fe), Kind(IsFE), Data(nullptr) {}
-
- PTHEntryKeyVariant(FileData *Data, StringRef Path)
- : PathPtr(Path.data()), PathSize(Path.size()), Kind(IsDE),
- Data(new FileData(*Data)) {}
-
- explicit PTHEntryKeyVariant(StringRef Path)
- : PathPtr(Path.data()), PathSize(Path.size()), Kind(IsNoExist),
- Data(nullptr) {}
-
- bool isFile() const { return Kind == IsFE; }
-
- StringRef getString() const {
- return Kind == IsFE ? FE->getName() : StringRef(PathPtr, PathSize);
- }
-
- unsigned getKind() const { return (unsigned) Kind; }
-
- void EmitData(raw_ostream& Out) {
- using namespace llvm::support;
- endian::Writer LE(Out, little);
- switch (Kind) {
- case IsFE: {
- // Emit stat information.
- llvm::sys::fs::UniqueID UID = FE->getUniqueID();
- LE.write<uint64_t>(UID.getFile());
- LE.write<uint64_t>(UID.getDevice());
- LE.write<uint64_t>(FE->getModificationTime());
- LE.write<uint64_t>(FE->getSize());
- } break;
- case IsDE:
- // Emit stat information.
- LE.write<uint64_t>(Data->UniqueID.getFile());
- LE.write<uint64_t>(Data->UniqueID.getDevice());
- LE.write<uint64_t>(Data->ModTime);
- LE.write<uint64_t>(Data->Size);
- delete Data;
- break;
- default:
- break;
- }
- }
-
- unsigned getRepresentationLength() const {
- return Kind == IsNoExist ? 0 : 4 * 8;
- }
-};
-
-class FileEntryPTHEntryInfo {
-public:
- typedef PTHEntryKeyVariant key_type;
- typedef key_type key_type_ref;
-
- typedef PTHEntry data_type;
- typedef const PTHEntry& data_type_ref;
-
- typedef unsigned hash_value_type;
- typedef unsigned offset_type;
-
- static hash_value_type ComputeHash(PTHEntryKeyVariant V) {
- return llvm::djbHash(V.getString());
- }
-
- static std::pair<unsigned,unsigned>
- EmitKeyDataLength(raw_ostream& Out, PTHEntryKeyVariant V,
- const PTHEntry& E) {
- using namespace llvm::support;
- endian::Writer LE(Out, little);
-
- unsigned n = V.getString().size() + 1 + 1;
- LE.write<uint16_t>(n);
-
- unsigned m = V.getRepresentationLength() + (V.isFile() ? 4 + 4 : 0);
- LE.write<uint8_t>(m);
-
- return std::make_pair(n, m);
- }
-
- static void EmitKey(raw_ostream& Out, PTHEntryKeyVariant V, unsigned n){
- using namespace llvm::support;
- // Emit the entry kind.
- Out << char(V.getKind());
- // Emit the string.
- Out.write(V.getString().data(), n - 1);
- }
-
- static void EmitData(raw_ostream& Out, PTHEntryKeyVariant V,
- const PTHEntry& E, unsigned) {
- using namespace llvm::support;
- endian::Writer LE(Out, little);
-
- // For file entries emit the offsets into the PTH file for token data
- // and the preprocessor blocks table.
- if (V.isFile()) {
- LE.write<uint32_t>(E.getTokenOffset());
- LE.write<uint32_t>(E.getPPCondTableOffset());
- }
-
- // Emit any other data associated with the key (i.e., stat information).
- V.EmitData(Out);
- }
-};
-
-class OffsetOpt {
- bool valid;
- Offset off;
-public:
- OffsetOpt() : valid(false) {}
- bool hasOffset() const { return valid; }
- Offset getOffset() const { assert(valid); return off; }
- void setOffset(Offset o) { off = o; valid = true; }
-};
-} // end anonymous namespace
-
-typedef llvm::OnDiskChainedHashTableGenerator<FileEntryPTHEntryInfo> PTHMap;
-
-namespace {
-class PTHWriter {
- typedef llvm::DenseMap<const IdentifierInfo*,uint32_t> IDMap;
- typedef llvm::StringMap<OffsetOpt, llvm::BumpPtrAllocator> CachedStrsTy;
-
- raw_pwrite_stream &Out;
- Preprocessor& PP;
- IDMap IM;
- std::vector<llvm::StringMapEntry<OffsetOpt>*> StrEntries;
- PTHMap PM;
- CachedStrsTy CachedStrs;
- uint32_t idcount;
- Offset CurStrOffset;
-
- //// Get the persistent id for the given IdentifierInfo*.
- uint32_t ResolveID(const IdentifierInfo* II);
-
- /// Emit a token to the PTH file.
- void EmitToken(const Token& T);
-
- void Emit8(uint32_t V) {
- Out << char(V);
- }
-
- void Emit16(uint32_t V) {
- using namespace llvm::support;
- endian::write<uint16_t>(Out, V, little);
- }
-
- void Emit32(uint32_t V) {
- using namespace llvm::support;
- endian::write<uint32_t>(Out, V, little);
- }
-
- void EmitBuf(const char *Ptr, unsigned NumBytes) {
- Out.write(Ptr, NumBytes);
- }
-
- void EmitString(StringRef V) {
- using namespace llvm::support;
- endian::write<uint16_t>(Out, V.size(), little);
- EmitBuf(V.data(), V.size());
- }
-
- /// EmitIdentifierTable - Emits two tables to the PTH file. The first is
- /// a hashtable mapping from identifier strings to persistent IDs.
- /// The second is a straight table mapping from persistent IDs to string data
- /// (the keys of the first table).
- std::pair<Offset, Offset> EmitIdentifierTable();
-
- /// EmitFileTable - Emit a table mapping from file name strings to PTH
- /// token data.
- Offset EmitFileTable() { return PM.Emit(Out); }
-
- PTHEntry LexTokens(Lexer& L);
- Offset EmitCachedSpellings();
-
-public:
- PTHWriter(raw_pwrite_stream &out, Preprocessor &pp)
- : Out(out), PP(pp), idcount(0), CurStrOffset(0) {}
-
- PTHMap &getPM() { return PM; }
- void GeneratePTH(StringRef MainFile);
-};
-} // end anonymous namespace
-
-uint32_t PTHWriter::ResolveID(const IdentifierInfo* II) {
- // Null IdentifierInfo's map to the persistent ID 0.
- if (!II)
- return 0;
-
- IDMap::iterator I = IM.find(II);
- if (I != IM.end())
- return I->second; // We've already added 1.
-
- IM[II] = ++idcount; // Pre-increment since '0' is reserved for NULL.
- return idcount;
-}
-
-void PTHWriter::EmitToken(const Token& T) {
- // Emit the token kind, flags, and length.
- Emit32(((uint32_t) T.getKind()) | ((((uint32_t) T.getFlags())) << 8)|
- (((uint32_t) T.getLength()) << 16));
-
- if (!T.isLiteral()) {
- Emit32(ResolveID(T.getIdentifierInfo()));
- } else {
- // We cache *un-cleaned* spellings. This gives us 100% fidelity with the
- // source code.
- StringRef s(T.getLiteralData(), T.getLength());
-
- // Get the string entry.
- auto &E = *CachedStrs.insert(std::make_pair(s, OffsetOpt())).first;
-
- // If this is a new string entry, bump the PTH offset.
- if (!E.second.hasOffset()) {
- E.second.setOffset(CurStrOffset);
- StrEntries.push_back(&E);
- CurStrOffset += s.size() + 1;
- }
-
- // Emit the relative offset into the PTH file for the spelling string.
- Emit32(E.second.getOffset());
- }
-
- // Emit the offset into the original source file of this token so that we
- // can reconstruct its SourceLocation.
- Emit32(PP.getSourceManager().getFileOffset(T.getLocation()));
-}
-
-PTHEntry PTHWriter::LexTokens(Lexer& L) {
- // Pad 0's so that we emit tokens to a 4-byte alignment.
- // This speed up reading them back in.
- using namespace llvm::support;
- endian::Writer LE(Out, little);
- uint32_t TokenOff = Out.tell();
- for (uint64_t N = llvm::OffsetToAlignment(TokenOff, 4); N; --N, ++TokenOff)
- LE.write<uint8_t>(0);
-
- // Keep track of matching '#if' ... '#endif'.
- typedef std::vector<std::pair<Offset, unsigned> > PPCondTable;
- PPCondTable PPCond;
- std::vector<unsigned> PPStartCond;
- bool ParsingPreprocessorDirective = false;
- Token Tok;
-
- do {
- L.LexFromRawLexer(Tok);
- NextToken:
-
- if ((Tok.isAtStartOfLine() || Tok.is(tok::eof)) &&
- ParsingPreprocessorDirective) {
- // Insert an eod token into the token cache. It has the same
- // position as the next token that is not on the same line as the
- // preprocessor directive. Observe that we continue processing
- // 'Tok' when we exit this branch.
- Token Tmp = Tok;
- Tmp.setKind(tok::eod);
- Tmp.clearFlag(Token::StartOfLine);
- Tmp.setIdentifierInfo(nullptr);
- EmitToken(Tmp);
- ParsingPreprocessorDirective = false;
- }
-
- if (Tok.is(tok::raw_identifier)) {
- PP.LookUpIdentifierInfo(Tok);
- EmitToken(Tok);
- continue;
- }
-
- if (Tok.is(tok::hash) && Tok.isAtStartOfLine()) {
- // Special processing for #include. Store the '#' token and lex
- // the next token.
- assert(!ParsingPreprocessorDirective);
- Offset HashOff = (Offset) Out.tell();
-
- // Get the next token.
- Token NextTok;
- L.LexFromRawLexer(NextTok);
-
- // If we see the start of line, then we had a null directive "#". In
- // this case, discard both tokens.
- if (NextTok.isAtStartOfLine())
- goto NextToken;
-
- // The token is the start of a directive. Emit it.
- EmitToken(Tok);
- Tok = NextTok;
-
- // Did we see 'include'/'import'/'include_next'?
- if (Tok.isNot(tok::raw_identifier)) {
- EmitToken(Tok);
- continue;
- }
-
- IdentifierInfo* II = PP.LookUpIdentifierInfo(Tok);
- tok::PPKeywordKind K = II->getPPKeywordID();
-
- ParsingPreprocessorDirective = true;
-
- switch (K) {
- case tok::pp_not_keyword:
- // Invalid directives "#foo" can occur in #if 0 blocks etc, just pass
- // them through.
- default:
- break;
-
- case tok::pp_include:
- case tok::pp_import:
- case tok::pp_include_next: {
- // Save the 'include' token.
- EmitToken(Tok);
- // Lex the next token as an include string.
- L.setParsingPreprocessorDirective(true);
- L.LexIncludeFilename(Tok);
- L.setParsingPreprocessorDirective(false);
- assert(!Tok.isAtStartOfLine());
- if (Tok.is(tok::raw_identifier))
- PP.LookUpIdentifierInfo(Tok);
-
- break;
- }
- case tok::pp_if:
- case tok::pp_ifdef:
- case tok::pp_ifndef: {
- // Add an entry for '#if' and friends. We initially set the target
- // index to 0. This will get backpatched when we hit #endif.
- PPStartCond.push_back(PPCond.size());
- PPCond.push_back(std::make_pair(HashOff, 0U));
- break;
- }
- case tok::pp_endif: {
- // Add an entry for '#endif'. We set the target table index to itself.
- // This will later be set to zero when emitting to the PTH file. We
- // use 0 for uninitialized indices because that is easier to debug.
- unsigned index = PPCond.size();
- // Backpatch the opening '#if' entry.
- assert(!PPStartCond.empty());
- assert(PPCond.size() > PPStartCond.back());
- assert(PPCond[PPStartCond.back()].second == 0);
- PPCond[PPStartCond.back()].second = index;
- PPStartCond.pop_back();
- // Add the new entry to PPCond.
- PPCond.push_back(std::make_pair(HashOff, index));
- EmitToken(Tok);
-
- // Some files have gibberish on the same line as '#endif'.
- // Discard these tokens.
- do
- L.LexFromRawLexer(Tok);
- while (Tok.isNot(tok::eof) && !Tok.isAtStartOfLine());
- // We have the next token in hand.
- // Don't immediately lex the next one.
- goto NextToken;
- }
- case tok::pp_elif:
- case tok::pp_else: {
- // Add an entry for #elif or #else.
- // This serves as both a closing and opening of a conditional block.
- // This means that its entry will get backpatched later.
- unsigned index = PPCond.size();
- // Backpatch the previous '#if' entry.
- assert(!PPStartCond.empty());
- assert(PPCond.size() > PPStartCond.back());
- assert(PPCond[PPStartCond.back()].second == 0);
- PPCond[PPStartCond.back()].second = index;
- PPStartCond.pop_back();
- // Now add '#elif' as a new block opening.
- PPCond.push_back(std::make_pair(HashOff, 0U));
- PPStartCond.push_back(index);
- break;
- }
- }
- }
-
- EmitToken(Tok);
- }
- while (Tok.isNot(tok::eof));
-
- assert(PPStartCond.empty() && "Error: imblanced preprocessor conditionals.");
-
- // Next write out PPCond.
- Offset PPCondOff = (Offset) Out.tell();
-
- // Write out the size of PPCond so that clients can identifer empty tables.
- Emit32(PPCond.size());
-
- for (unsigned i = 0, e = PPCond.size(); i!=e; ++i) {
- Emit32(PPCond[i].first - TokenOff);
- uint32_t x = PPCond[i].second;
- assert(x != 0 && "PPCond entry not backpatched.");
- // Emit zero for #endifs. This allows us to do checking when
- // we read the PTH file back in.
- Emit32(x == i ? 0 : x);
- }
-
- return PTHEntry(TokenOff, PPCondOff);
-}
-
-Offset PTHWriter::EmitCachedSpellings() {
- // Write each cached strings to the PTH file.
- Offset SpellingsOff = Out.tell();
-
- for (std::vector<llvm::StringMapEntry<OffsetOpt>*>::iterator
- I = StrEntries.begin(), E = StrEntries.end(); I!=E; ++I)
- EmitBuf((*I)->getKeyData(), (*I)->getKeyLength()+1 /*nul included*/);
-
- return SpellingsOff;
-}
-
-static uint32_t swap32le(uint32_t X) {
- return llvm::support::endian::byte_swap<uint32_t, llvm::support::little>(X);
-}
-
-static void pwrite32le(raw_pwrite_stream &OS, uint32_t Val, uint64_t &Off) {
- uint32_t LEVal = swap32le(Val);
- OS.pwrite(reinterpret_cast<const char *>(&LEVal), 4, Off);
- Off += 4;
-}
-
-void PTHWriter::GeneratePTH(StringRef MainFile) {
- // Generate the prologue.
- Out << "cfe-pth" << '\0';
- Emit32(PTHManager::Version);
-
- // Leave 4 words for the prologue.
- Offset PrologueOffset = Out.tell();
- for (unsigned i = 0; i < 4; ++i)
- Emit32(0);
-
- // Write the name of the MainFile.
- if (!MainFile.empty()) {
- EmitString(MainFile);
- } else {
- // String with 0 bytes.
- Emit16(0);
- }
- Emit8(0);
-
- // Iterate over all the files in SourceManager. Create a lexer
- // for each file and cache the tokens.
- SourceManager &SM = PP.getSourceManager();
- const LangOptions &LOpts = PP.getLangOpts();
-
- for (SourceManager::fileinfo_iterator I = SM.fileinfo_begin(),
- E = SM.fileinfo_end(); I != E; ++I) {
- const SrcMgr::ContentCache &C = *I->second;
- const FileEntry *FE = C.OrigEntry;
-
- // FIXME: Handle files with non-absolute paths.
- if (llvm::sys::path::is_relative(FE->getName()))
- continue;
-
- const llvm::MemoryBuffer *B = C.getBuffer(PP.getDiagnostics(), SM);
- if (!B) continue;
-
- FileID FID = SM.createFileID(FE, SourceLocation(), SrcMgr::C_User);
- const llvm::MemoryBuffer *FromFile = SM.getBuffer(FID);
- Lexer L(FID, FromFile, SM, LOpts);
- PM.insert(FE, LexTokens(L));
- }
-
- // Write out the identifier table.
- const std::pair<Offset,Offset> &IdTableOff = EmitIdentifierTable();
-
- // Write out the cached strings table.
- Offset SpellingOff = EmitCachedSpellings();
-
- // Write out the file table.
- Offset FileTableOff = EmitFileTable();
-
- // Finally, write the prologue.
- uint64_t Off = PrologueOffset;
- pwrite32le(Out, IdTableOff.first, Off);
- pwrite32le(Out, IdTableOff.second, Off);
- pwrite32le(Out, FileTableOff, Off);
- pwrite32le(Out, SpellingOff, Off);
-}
-
-namespace {
-/// StatListener - A simple "interpose" object used to monitor stat calls
-/// invoked by FileManager while processing the original sources used
-/// as input to PTH generation. StatListener populates the PTHWriter's
-/// file map with stat information for directories as well as negative stats.
-/// Stat information for files are populated elsewhere.
-class StatListener : public FileSystemStatCache {
- PTHMap &PM;
-public:
- StatListener(PTHMap &pm) : PM(pm) {}
- ~StatListener() override {}
-
- LookupResult getStat(StringRef Path, FileData &Data, bool isFile,
- std::unique_ptr<llvm::vfs::File> *F,
- llvm::vfs::FileSystem &FS) override {
- LookupResult Result = statChained(Path, Data, isFile, F, FS);
-
- if (Result == CacheMissing) // Failed 'stat'.
- PM.insert(PTHEntryKeyVariant(Path), PTHEntry());
- else if (Data.IsDirectory) {
- // Only cache directories with absolute paths.
- if (llvm::sys::path::is_relative(Path))
- return Result;
-
- PM.insert(PTHEntryKeyVariant(&Data, Path), PTHEntry());
- }
-
- return Result;
- }
-};
-} // end anonymous namespace
-
-void clang::CacheTokens(Preprocessor &PP, raw_pwrite_stream *OS) {
- // Get the name of the main file.
- const SourceManager &SrcMgr = PP.getSourceManager();
- const FileEntry *MainFile = SrcMgr.getFileEntryForID(SrcMgr.getMainFileID());
- SmallString<128> MainFilePath(MainFile->getName());
-
- llvm::sys::fs::make_absolute(MainFilePath);
-
- // Create the PTHWriter.
- PTHWriter PW(*OS, PP);
-
- // Install the 'stat' system call listener in the FileManager.
- auto StatCacheOwner = llvm::make_unique<StatListener>(PW.getPM());
- StatListener *StatCache = StatCacheOwner.get();
- PP.getFileManager().addStatCache(std::move(StatCacheOwner),
- /*AtBeginning=*/true);
-
- // Lex through the entire file. This will populate SourceManager with
- // all of the header information.
- Token Tok;
- PP.EnterMainSourceFile();
- do { PP.Lex(Tok); } while (Tok.isNot(tok::eof));
-
- // Generate the PTH file.
- PP.getFileManager().removeStatCache(StatCache);
- PW.GeneratePTH(MainFilePath.str());
-}
-
-//===----------------------------------------------------------------------===//
-
-namespace {
-class PTHIdKey {
-public:
- const IdentifierInfo* II;
- uint32_t FileOffset;
-};
-
-class PTHIdentifierTableTrait {
-public:
- typedef PTHIdKey* key_type;
- typedef key_type key_type_ref;
-
- typedef uint32_t data_type;
- typedef data_type data_type_ref;
-
- typedef unsigned hash_value_type;
- typedef unsigned offset_type;
-
- static hash_value_type ComputeHash(PTHIdKey* key) {
- return llvm::djbHash(key->II->getName());
- }
-
- static std::pair<unsigned,unsigned>
- EmitKeyDataLength(raw_ostream& Out, const PTHIdKey* key, uint32_t) {
- using namespace llvm::support;
- unsigned n = key->II->getLength() + 1;
- endian::write<uint16_t>(Out, n, little);
- return std::make_pair(n, sizeof(uint32_t));
- }
-
- static void EmitKey(raw_ostream& Out, PTHIdKey* key, unsigned n) {
- // Record the location of the key data. This is used when generating
- // the mapping from persistent IDs to strings.
- key->FileOffset = Out.tell();
- Out.write(key->II->getNameStart(), n);
- }
-
- static void EmitData(raw_ostream& Out, PTHIdKey*, uint32_t pID,
- unsigned) {
- using namespace llvm::support;
- endian::write<uint32_t>(Out, pID, little);
- }
-};
-} // end anonymous namespace
-
-/// EmitIdentifierTable - Emits two tables to the PTH file. The first is
-/// a hashtable mapping from identifier strings to persistent IDs. The second
-/// is a straight table mapping from persistent IDs to string data (the
-/// keys of the first table).
-///
-std::pair<Offset,Offset> PTHWriter::EmitIdentifierTable() {
- // Build two maps:
- // (1) an inverse map from persistent IDs -> (IdentifierInfo*,Offset)
- // (2) a map from (IdentifierInfo*, Offset)* -> persistent IDs
-
- // Note that we use 'calloc', so all the bytes are 0.
- PTHIdKey *IIDMap = static_cast<PTHIdKey*>(
- llvm::safe_calloc(idcount, sizeof(PTHIdKey)));
-
- // Create the hashtable.
- llvm::OnDiskChainedHashTableGenerator<PTHIdentifierTableTrait> IIOffMap;
-
- // Generate mapping from persistent IDs -> IdentifierInfo*.
- for (IDMap::iterator I = IM.begin(), E = IM.end(); I != E; ++I) {
- // Decrement by 1 because we are using a vector for the lookup and
- // 0 is reserved for NULL.
- assert(I->second > 0);
- assert(I->second-1 < idcount);
- unsigned idx = I->second-1;
-
- // Store the mapping from persistent ID to IdentifierInfo*
- IIDMap[idx].II = I->first;
-
- // Store the reverse mapping in a hashtable.
- IIOffMap.insert(&IIDMap[idx], I->second);
- }
-
- // Write out the inverse map first. This causes the PCIDKey entries to
- // record PTH file offsets for the string data. This is used to write
- // the second table.
- Offset StringTableOffset = IIOffMap.Emit(Out);
-
- // Now emit the table mapping from persistent IDs to PTH file offsets.
- Offset IDOff = Out.tell();
- Emit32(idcount); // Emit the number of identifiers.
- for (unsigned i = 0 ; i < idcount; ++i)
- Emit32(IIDMap[i].FileOffset);
-
- // Finally, release the inverse map.
- free(IIDMap);
-
- return std::make_pair(IDOff, StringTableOffset);
-}
Modified: cfe/trunk/lib/Frontend/ChainedIncludesSource.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/ChainedIncludesSource.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/ChainedIncludesSource.cpp (original)
+++ cfe/trunk/lib/Frontend/ChainedIncludesSource.cpp Tue Dec 4 06:34:09 2018
@@ -129,7 +129,6 @@ IntrusiveRefCntPtr<ExternalSemaSource> c
CInvok->getPreprocessorOpts().ChainedIncludes.clear();
CInvok->getPreprocessorOpts().ImplicitPCHInclude.clear();
- CInvok->getPreprocessorOpts().ImplicitPTHInclude.clear();
CInvok->getPreprocessorOpts().DisablePCHValidation = true;
CInvok->getPreprocessorOpts().Includes.clear();
CInvok->getPreprocessorOpts().MacroIncludes.clear();
Modified: cfe/trunk/lib/Frontend/CompilerInstance.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/CompilerInstance.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/CompilerInstance.cpp (original)
+++ cfe/trunk/lib/Frontend/CompilerInstance.cpp Tue Dec 4 06:34:09 2018
@@ -30,7 +30,6 @@
#include "clang/Frontend/Utils.h"
#include "clang/Frontend/VerifyDiagnosticConsumer.h"
#include "clang/Lex/HeaderSearch.h"
-#include "clang/Lex/PTHManager.h"
#include "clang/Lex/Preprocessor.h"
#include "clang/Lex/PreprocessorOptions.h"
#include "clang/Sema/CodeCompleteConsumer.h"
@@ -376,30 +375,18 @@ void CompilerInstance::createPreprocesso
// The module manager holds a reference to the old preprocessor (if any).
ModuleManager.reset();
- // Create a PTH manager if we are using some form of a token cache.
- PTHManager *PTHMgr = nullptr;
- if (!PPOpts.TokenCache.empty())
- PTHMgr = PTHManager::Create(PPOpts.TokenCache, getDiagnostics());
-
// Create the Preprocessor.
HeaderSearch *HeaderInfo =
new HeaderSearch(getHeaderSearchOptsPtr(), getSourceManager(),
getDiagnostics(), getLangOpts(), &getTarget());
PP = std::make_shared<Preprocessor>(
Invocation->getPreprocessorOptsPtr(), getDiagnostics(), getLangOpts(),
- getSourceManager(), getPCMCache(), *HeaderInfo, *this, PTHMgr,
+ getSourceManager(), getPCMCache(), *HeaderInfo, *this,
+ /*IdentifierInfoLookup=*/nullptr,
/*OwnsHeaderSearch=*/true, TUKind);
getTarget().adjust(getLangOpts());
PP->Initialize(getTarget(), getAuxTarget());
- // Note that this is different then passing PTHMgr to Preprocessor's ctor.
- // That argument is used as the IdentifierInfoLookup argument to
- // IdentifierTable's ctor.
- if (PTHMgr) {
- PTHMgr->setPreprocessor(&*PP);
- PP->setPTHManager(PTHMgr);
- }
-
if (PPOpts.DetailedRecord)
PP->createPreprocessingRecord();
Modified: cfe/trunk/lib/Frontend/CompilerInvocation.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/CompilerInvocation.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/CompilerInvocation.cpp (original)
+++ cfe/trunk/lib/Frontend/CompilerInvocation.cpp Tue Dec 4 06:34:09 2018
@@ -1619,8 +1619,6 @@ static InputKind ParseFrontendArgs(Front
Opts.ProgramAction = frontend::GenerateHeaderModule; break;
case OPT_emit_pch:
Opts.ProgramAction = frontend::GeneratePCH; break;
- case OPT_emit_pth:
- Opts.ProgramAction = frontend::GeneratePTH; break;
case OPT_init_only:
Opts.ProgramAction = frontend::InitOnly; break;
case OPT_fsyntax_only:
@@ -3020,7 +3018,6 @@ static bool isStrictlyPreprocessorAction
case frontend::GenerateModuleInterface:
case frontend::GenerateHeaderModule:
case frontend::GeneratePCH:
- case frontend::GeneratePTH:
case frontend::ParseSyntaxOnly:
case frontend::ModuleFileInfo:
case frontend::VerifyPCH:
@@ -3049,15 +3046,10 @@ static void ParsePreprocessorArgs(Prepro
DiagnosticsEngine &Diags,
frontend::ActionKind Action) {
Opts.ImplicitPCHInclude = Args.getLastArgValue(OPT_include_pch);
- Opts.ImplicitPTHInclude = Args.getLastArgValue(OPT_include_pth);
Opts.PCHWithHdrStop = Args.hasArg(OPT_pch_through_hdrstop_create) ||
Args.hasArg(OPT_pch_through_hdrstop_use);
Opts.PCHWithHdrStopCreate = Args.hasArg(OPT_pch_through_hdrstop_create);
Opts.PCHThroughHeader = Args.getLastArgValue(OPT_pch_through_header_EQ);
- if (const Arg *A = Args.getLastArg(OPT_token_cache))
- Opts.TokenCache = A->getValue();
- else
- Opts.TokenCache = Opts.ImplicitPTHInclude;
Opts.UsePredefines = !Args.hasArg(OPT_undef);
Opts.DetailedRecord = Args.hasArg(OPT_detailed_preprocessing_record);
Opts.DisablePCHValidation = Args.hasArg(OPT_fno_validate_pch);
Modified: cfe/trunk/lib/Frontend/FrontendActions.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/FrontendActions.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/FrontendActions.cpp (original)
+++ cfe/trunk/lib/Frontend/FrontendActions.cpp Tue Dec 4 06:34:09 2018
@@ -750,16 +750,6 @@ void DumpTokensAction::ExecuteAction() {
} while (Tok.isNot(tok::eof));
}
-void GeneratePTHAction::ExecuteAction() {
- CompilerInstance &CI = getCompilerInstance();
- std::unique_ptr<raw_pwrite_stream> OS =
- CI.createDefaultOutputFile(true, getCurrentFile());
- if (!OS)
- return;
-
- CacheTokens(CI.getPreprocessor(), OS.get());
-}
-
void PreprocessOnlyAction::ExecuteAction() {
Preprocessor &PP = getCompilerInstance().getPreprocessor();
Modified: cfe/trunk/lib/Frontend/InitPreprocessor.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/InitPreprocessor.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/InitPreprocessor.cpp (original)
+++ cfe/trunk/lib/Frontend/InitPreprocessor.cpp Tue Dec 4 06:34:09 2018
@@ -21,7 +21,6 @@
#include "clang/Frontend/FrontendOptions.h"
#include "clang/Frontend/Utils.h"
#include "clang/Lex/HeaderSearch.h"
-#include "clang/Lex/PTHManager.h"
#include "clang/Lex/Preprocessor.h"
#include "clang/Lex/PreprocessorOptions.h"
#include "clang/Serialization/ASTReader.h"
@@ -76,23 +75,6 @@ static void AddImplicitIncludeMacros(Mac
Builder.append("##"); // ##?
}
-/// AddImplicitIncludePTH - Add an implicit \#include using the original file
-/// used to generate a PTH cache.
-static void AddImplicitIncludePTH(MacroBuilder &Builder, Preprocessor &PP,
- StringRef ImplicitIncludePTH) {
- PTHManager *P = PP.getPTHManager();
- // Null check 'P' in the corner case where it couldn't be created.
- const char *OriginalFile = P ? P->getOriginalSourceFile() : nullptr;
-
- if (!OriginalFile) {
- PP.getDiagnostics().Report(diag::err_fe_pth_file_has_no_source_header)
- << ImplicitIncludePTH;
- return;
- }
-
- AddImplicitInclude(Builder, OriginalFile);
-}
-
/// Add an implicit \#include using the original file used to generate
/// a PCH file.
static void AddImplicitIncludePCH(MacroBuilder &Builder, Preprocessor &PP,
@@ -1177,8 +1159,6 @@ void clang::InitializePreprocessor(
if (!InitOpts.ImplicitPCHInclude.empty())
AddImplicitIncludePCH(Builder, PP, PCHContainerRdr,
InitOpts.ImplicitPCHInclude);
- if (!InitOpts.ImplicitPTHInclude.empty())
- AddImplicitIncludePTH(Builder, PP, InitOpts.ImplicitPTHInclude);
// Process -include directives.
for (unsigned i = 0, e = InitOpts.Includes.size(); i != e; ++i) {
Modified: cfe/trunk/lib/FrontendTool/ExecuteCompilerInvocation.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/FrontendTool/ExecuteCompilerInvocation.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/FrontendTool/ExecuteCompilerInvocation.cpp (original)
+++ cfe/trunk/lib/FrontendTool/ExecuteCompilerInvocation.cpp Tue Dec 4 06:34:09 2018
@@ -65,7 +65,6 @@ CreateFrontendBaseAction(CompilerInstanc
case GenerateHeaderModule:
return llvm::make_unique<GenerateHeaderModuleAction>();
case GeneratePCH: return llvm::make_unique<GeneratePCHAction>();
- case GeneratePTH: return llvm::make_unique<GeneratePTHAction>();
case InitOnly: return llvm::make_unique<InitOnlyAction>();
case ParseSyntaxOnly: return llvm::make_unique<SyntaxOnlyAction>();
case ModuleFileInfo: return llvm::make_unique<DumpModuleInfoAction>();
Modified: cfe/trunk/lib/Lex/CMakeLists.txt
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/CMakeLists.txt?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Lex/CMakeLists.txt (original)
+++ cfe/trunk/lib/Lex/CMakeLists.txt Tue Dec 4 06:34:09 2018
@@ -17,7 +17,6 @@ add_clang_library(clangLex
PPExpressions.cpp
PPLexerChange.cpp
PPMacroExpansion.cpp
- PTHLexer.cpp
Pragma.cpp
PreprocessingRecord.cpp
Preprocessor.cpp
Modified: cfe/trunk/lib/Lex/PPDirectives.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPDirectives.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Lex/PPDirectives.cpp (original)
+++ cfe/trunk/lib/Lex/PPDirectives.cpp Tue Dec 4 06:34:09 2018
@@ -31,7 +31,6 @@
#include "clang/Lex/Pragma.h"
#include "clang/Lex/Preprocessor.h"
#include "clang/Lex/PreprocessorOptions.h"
-#include "clang/Lex/PTHLexer.h"
#include "clang/Lex/Token.h"
#include "clang/Lex/VariadicMacroSupport.h"
#include "llvm/ADT/ArrayRef.h"
@@ -383,11 +382,6 @@ void Preprocessor::SkipExcludedCondition
CurPPLexer->pushConditionalLevel(IfTokenLoc, /*isSkipping*/ false,
FoundNonSkipPortion, FoundElse);
- if (CurPTHLexer) {
- PTHSkipExcludedConditionalBlock();
- return;
- }
-
// Enter raw mode to disable identifier lookup (and thus macro expansion),
// disabling warnings, etc.
CurPPLexer->LexingRawMode = true;
@@ -585,83 +579,6 @@ void Preprocessor::SkipExcludedCondition
Tok.getLocation());
}
-void Preprocessor::PTHSkipExcludedConditionalBlock() {
- while (true) {
- assert(CurPTHLexer);
- assert(CurPTHLexer->LexingRawMode == false);
-
- // Skip to the next '#else', '#elif', or #endif.
- if (CurPTHLexer->SkipBlock()) {
- // We have reached an #endif. Both the '#' and 'endif' tokens
- // have been consumed by the PTHLexer. Just pop off the condition level.
- PPConditionalInfo CondInfo;
- bool InCond = CurPTHLexer->popConditionalLevel(CondInfo);
- (void)InCond; // Silence warning in no-asserts mode.
- assert(!InCond && "Can't be skipping if not in a conditional!");
- break;
- }
-
- // We have reached a '#else' or '#elif'. Lex the next token to get
- // the directive flavor.
- Token Tok;
- LexUnexpandedToken(Tok);
-
- // We can actually look up the IdentifierInfo here since we aren't in
- // raw mode.
- tok::PPKeywordKind K = Tok.getIdentifierInfo()->getPPKeywordID();
-
- if (K == tok::pp_else) {
- // #else: Enter the else condition. We aren't in a nested condition
- // since we skip those. We're always in the one matching the last
- // blocked we skipped.
- PPConditionalInfo &CondInfo = CurPTHLexer->peekConditionalLevel();
- // Note that we've seen a #else in this conditional.
- CondInfo.FoundElse = true;
-
- // If the #if block wasn't entered then enter the #else block now.
- if (!CondInfo.FoundNonSkip) {
- CondInfo.FoundNonSkip = true;
-
- // Scan until the eod token.
- CurPTHLexer->ParsingPreprocessorDirective = true;
- DiscardUntilEndOfDirective();
- CurPTHLexer->ParsingPreprocessorDirective = false;
-
- break;
- }
-
- // Otherwise skip this block.
- continue;
- }
-
- assert(K == tok::pp_elif);
- PPConditionalInfo &CondInfo = CurPTHLexer->peekConditionalLevel();
-
- // If this is a #elif with a #else before it, report the error.
- if (CondInfo.FoundElse)
- Diag(Tok, diag::pp_err_elif_after_else);
-
- // If this is in a skipping block or if we're already handled this #if
- // block, don't bother parsing the condition. We just skip this block.
- if (CondInfo.FoundNonSkip)
- continue;
-
- // Evaluate the condition of the #elif.
- IdentifierInfo *IfNDefMacro = nullptr;
- CurPTHLexer->ParsingPreprocessorDirective = true;
- bool ShouldEnter = EvaluateDirectiveExpression(IfNDefMacro).Conditional;
- CurPTHLexer->ParsingPreprocessorDirective = false;
-
- // If this condition is true, enter it!
- if (ShouldEnter) {
- CondInfo.FoundNonSkip = true;
- break;
- }
-
- // Otherwise, skip this block and go to the next one.
- }
-}
-
Module *Preprocessor::getModuleForLocation(SourceLocation Loc) {
if (!SourceMgr.isInMainFile(Loc)) {
// Try to determine the module of the include directive.
@@ -1387,10 +1304,6 @@ void Preprocessor::HandleDigitDirective(
///
void Preprocessor::HandleUserDiagnosticDirective(Token &Tok,
bool isWarning) {
- // PTH doesn't emit #warning or #error directives.
- if (CurPTHLexer)
- return CurPTHLexer->DiscardToEndOfLine();
-
// Read the rest of the line raw. We do this because we don't want macros
// to be expanded and we don't require that the tokens be valid preprocessing
// tokens. For example, this is allowed: "#warning ` 'foo". GCC does
@@ -2007,14 +1920,10 @@ void Preprocessor::HandleIncludeDirectiv
if (hadModuleLoaderFatalFailure()) {
// With a fatal failure in the module loader, we abort parsing.
Token &Result = IncludeTok;
- if (CurLexer) {
- Result.startToken();
- CurLexer->FormTokenWithChars(Result, CurLexer->BufferEnd, tok::eof);
- CurLexer->cutOffLexing();
- } else {
- assert(CurPTHLexer && "#include but no current lexer set!");
- CurPTHLexer->getEOF(Result);
- }
+ assert(CurLexer && "#include but no current lexer set!");
+ Result.startToken();
+ CurLexer->FormTokenWithChars(Result, CurLexer->BufferEnd, tok::eof);
+ CurLexer->cutOffLexing();
}
return;
}
Modified: cfe/trunk/lib/Lex/PPLexerChange.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPLexerChange.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Lex/PPLexerChange.cpp (original)
+++ cfe/trunk/lib/Lex/PPLexerChange.cpp Tue Dec 4 06:34:09 2018
@@ -19,7 +19,6 @@
#include "clang/Lex/HeaderSearch.h"
#include "clang/Lex/LexDiagnostic.h"
#include "clang/Lex/MacroInfo.h"
-#include "clang/Lex/PTHManager.h"
#include "llvm/ADT/StringSwitch.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/MemoryBuffer.h"
@@ -76,13 +75,6 @@ bool Preprocessor::EnterSourceFile(FileI
if (MaxIncludeStackDepth < IncludeMacroStack.size())
MaxIncludeStackDepth = IncludeMacroStack.size();
- if (PTH) {
- if (PTHLexer *PL = PTH->CreateLexer(FID)) {
- EnterSourceFileWithPTH(PL, CurDir);
- return false;
- }
- }
-
// Get the MemoryBuffer for this FID, if it fails, we fail.
bool Invalid = false;
const llvm::MemoryBuffer *InputFile =
@@ -131,31 +123,6 @@ void Preprocessor::EnterSourceFileWithLe
}
}
-/// EnterSourceFileWithPTH - Add a source file to the top of the include stack
-/// and start getting tokens from it using the PTH cache.
-void Preprocessor::EnterSourceFileWithPTH(PTHLexer *PL,
- const DirectoryLookup *CurDir) {
-
- if (CurPPLexer || CurTokenLexer)
- PushIncludeMacroStack();
-
- CurDirLookup = CurDir;
- CurPTHLexer.reset(PL);
- CurPPLexer = CurPTHLexer.get();
- CurLexerSubmodule = nullptr;
- if (CurLexerKind != CLK_LexAfterModuleImport)
- CurLexerKind = CLK_PTHLexer;
-
- // Notify the client, if desired, that we are in a new source file.
- if (Callbacks) {
- FileID FID = CurPPLexer->getFileID();
- SourceLocation EnterLoc = SourceMgr.getLocForStartOfFile(FID);
- SrcMgr::CharacteristicKind FileType =
- SourceMgr.getFileCharacteristic(EnterLoc);
- Callbacks->FileChanged(EnterLoc, PPCallbacks::EnterFile, FileType);
- }
-}
-
/// EnterMacro - Add a Macro to the top of the include stack and start lexing
/// tokens from it instead of the current buffer.
void Preprocessor::EnterMacro(Token &Tok, SourceLocation ILEnd,
@@ -340,7 +307,6 @@ bool Preprocessor::HandleEndOfFile(Token
// If we have an unclosed module region from a pragma at the end of a
// module, complain and close it now.
- // FIXME: This is not correct if we are building a module from PTH.
const bool LeavingSubmodule = CurLexer && CurLexerSubmodule;
if ((LeavingSubmodule || IncludeMacroStack.empty()) &&
!BuildingSubmoduleStack.empty() &&
@@ -437,15 +403,10 @@ bool Preprocessor::HandleEndOfFile(Token
if (isCodeCompletionEnabled() && CurPPLexer &&
SourceMgr.getLocForStartOfFile(CurPPLexer->getFileID()) ==
CodeCompletionFileLoc) {
- if (CurLexer) {
- Result.startToken();
- CurLexer->FormTokenWithChars(Result, CurLexer->BufferEnd, tok::eof);
- CurLexer.reset();
- } else {
- assert(CurPTHLexer && "Got EOF but no current lexer set!");
- CurPTHLexer->getEOF(Result);
- CurPTHLexer.reset();
- }
+ assert(CurLexer && "Got EOF but no current lexer set!");
+ Result.startToken();
+ CurLexer->FormTokenWithChars(Result, CurLexer->BufferEnd, tok::eof);
+ CurLexer.reset();
CurPPLexer = nullptr;
recomputeCurLexerKind();
@@ -523,39 +484,34 @@ bool Preprocessor::HandleEndOfFile(Token
}
// If this is the end of the main file, form an EOF token.
- if (CurLexer) {
- const char *EndPos = getCurLexerEndPos();
- Result.startToken();
- CurLexer->BufferPtr = EndPos;
- CurLexer->FormTokenWithChars(Result, EndPos, tok::eof);
-
- if (isCodeCompletionEnabled()) {
- // Inserting the code-completion point increases the source buffer by 1,
- // but the main FileID was created before inserting the point.
- // Compensate by reducing the EOF location by 1, otherwise the location
- // will point to the next FileID.
- // FIXME: This is hacky, the code-completion point should probably be
- // inserted before the main FileID is created.
- if (CurLexer->getFileLoc() == CodeCompletionFileLoc)
- Result.setLocation(Result.getLocation().getLocWithOffset(-1));
- }
-
- if (creatingPCHWithThroughHeader() && !LeavingPCHThroughHeader) {
- // Reached the end of the compilation without finding the through header.
- Diag(CurLexer->getFileLoc(), diag::err_pp_through_header_not_seen)
- << PPOpts->PCHThroughHeader << 0;
- }
-
- if (!isIncrementalProcessingEnabled())
- // We're done with lexing.
- CurLexer.reset();
- } else {
- assert(CurPTHLexer && "Got EOF but no current lexer set!");
- CurPTHLexer->getEOF(Result);
- CurPTHLexer.reset();
+ assert(CurLexer && "Got EOF but no current lexer set!");
+ const char *EndPos = getCurLexerEndPos();
+ Result.startToken();
+ CurLexer->BufferPtr = EndPos;
+ CurLexer->FormTokenWithChars(Result, EndPos, tok::eof);
+
+ if (isCodeCompletionEnabled()) {
+ // Inserting the code-completion point increases the source buffer by 1,
+ // but the main FileID was created before inserting the point.
+ // Compensate by reducing the EOF location by 1, otherwise the location
+ // will point to the next FileID.
+ // FIXME: This is hacky, the code-completion point should probably be
+ // inserted before the main FileID is created.
+ if (CurLexer->getFileLoc() == CodeCompletionFileLoc)
+ Result.setLocation(Result.getLocation().getLocWithOffset(-1));
+ }
+
+ if (creatingPCHWithThroughHeader() && !LeavingPCHThroughHeader) {
+ // Reached the end of the compilation without finding the through header.
+ Diag(CurLexer->getFileLoc(), diag::err_pp_through_header_not_seen)
+ << PPOpts->PCHThroughHeader << 0;
}
if (!isIncrementalProcessingEnabled())
+ // We're done with lexing.
+ CurLexer.reset();
+
+ if (!isIncrementalProcessingEnabled())
CurPPLexer = nullptr;
if (TUKind == TU_Complete) {
Modified: cfe/trunk/lib/Lex/PPMacroExpansion.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPMacroExpansion.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Lex/PPMacroExpansion.cpp (original)
+++ cfe/trunk/lib/Lex/PPMacroExpansion.cpp Tue Dec 4 06:34:09 2018
@@ -29,7 +29,6 @@
#include "clang/Lex/MacroInfo.h"
#include "clang/Lex/Preprocessor.h"
#include "clang/Lex/PreprocessorLexer.h"
-#include "clang/Lex/PTHLexer.h"
#include "clang/Lex/Token.h"
#include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/DenseMap.h"
@@ -429,8 +428,6 @@ bool Preprocessor::isNextPPTokenLParen()
unsigned Val;
if (CurLexer)
Val = CurLexer->isNextPPTokenLParen();
- else if (CurPTHLexer)
- Val = CurPTHLexer->isNextPPTokenLParen();
else
Val = CurTokenLexer->isNextTokenLParen();
@@ -443,8 +440,6 @@ bool Preprocessor::isNextPPTokenLParen()
for (const IncludeStackInfo &Entry : llvm::reverse(IncludeMacroStack)) {
if (Entry.TheLexer)
Val = Entry.TheLexer->isNextPPTokenLParen();
- else if (Entry.ThePTHLexer)
- Val = Entry.ThePTHLexer->isNextPPTokenLParen();
else
Val = Entry.TheTokenLexer->isNextTokenLParen();
Removed: cfe/trunk/lib/Lex/PTHLexer.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PTHLexer.cpp?rev=348265&view=auto
==============================================================================
--- cfe/trunk/lib/Lex/PTHLexer.cpp (original)
+++ cfe/trunk/lib/Lex/PTHLexer.cpp (removed)
@@ -1,748 +0,0 @@
-//===- PTHLexer.cpp - Lex from a token stream -----------------------------===//
-//
-// The LLVM Compiler Infrastructure
-//
-// This file is distributed under the University of Illinois Open Source
-// License. See LICENSE.TXT for details.
-//
-//===----------------------------------------------------------------------===//
-//
-// This file implements the PTHLexer interface.
-//
-//===----------------------------------------------------------------------===//
-
-#include "clang/Lex/PTHLexer.h"
-#include "clang/Basic/Diagnostic.h"
-#include "clang/Basic/FileManager.h"
-#include "clang/Basic/FileSystemStatCache.h"
-#include "clang/Basic/IdentifierTable.h"
-#include "clang/Basic/SourceManager.h"
-#include "clang/Basic/TokenKinds.h"
-#include "clang/Lex/LexDiagnostic.h"
-#include "clang/Lex/PTHManager.h"
-#include "clang/Lex/Preprocessor.h"
-#include "clang/Lex/Token.h"
-#include "llvm/ADT/STLExtras.h"
-#include "llvm/ADT/StringRef.h"
-#include "llvm/Support/DJB.h"
-#include "llvm/Support/Endian.h"
-#include "llvm/Support/ErrorOr.h"
-#include "llvm/Support/FileSystem.h"
-#include "llvm/Support/MemoryBuffer.h"
-#include "llvm/Support/OnDiskHashTable.h"
-#include <cassert>
-#include <cstdint>
-#include <cstdlib>
-#include <cstring>
-#include <ctime>
-#include <memory>
-#include <utility>
-
-using namespace clang;
-
-static const unsigned StoredTokenSize = 1 + 1 + 2 + 4 + 4;
-
-//===----------------------------------------------------------------------===//
-// PTHLexer methods.
-//===----------------------------------------------------------------------===//
-
-PTHLexer::PTHLexer(Preprocessor &PP, FileID FID, const unsigned char *D,
- const unsigned char *ppcond, PTHManager &PM)
- : PreprocessorLexer(&PP, FID), TokBuf(D), CurPtr(D), PPCond(ppcond),
- CurPPCondPtr(ppcond), PTHMgr(PM) {
- FileStartLoc = PP.getSourceManager().getLocForStartOfFile(FID);
-}
-
-bool PTHLexer::Lex(Token& Tok) {
- //===--------------------------------------==//
- // Read the raw token data.
- //===--------------------------------------==//
- using namespace llvm::support;
-
- // Shadow CurPtr into an automatic variable.
- const unsigned char *CurPtrShadow = CurPtr;
-
- // Read in the data for the token.
- unsigned Word0 = endian::readNext<uint32_t, little, aligned>(CurPtrShadow);
- uint32_t IdentifierID =
- endian::readNext<uint32_t, little, aligned>(CurPtrShadow);
- uint32_t FileOffset =
- endian::readNext<uint32_t, little, aligned>(CurPtrShadow);
-
- tok::TokenKind TKind = (tok::TokenKind) (Word0 & 0xFF);
- Token::TokenFlags TFlags = (Token::TokenFlags) ((Word0 >> 8) & 0xFF);
- uint32_t Len = Word0 >> 16;
-
- CurPtr = CurPtrShadow;
-
- //===--------------------------------------==//
- // Construct the token itself.
- //===--------------------------------------==//
-
- Tok.startToken();
- Tok.setKind(TKind);
- Tok.setFlag(TFlags);
- assert(!LexingRawMode);
- Tok.setLocation(FileStartLoc.getLocWithOffset(FileOffset));
- Tok.setLength(Len);
-
- // Handle identifiers.
- if (Tok.isLiteral()) {
- Tok.setLiteralData((const char*) (PTHMgr.SpellingBase + IdentifierID));
- }
- else if (IdentifierID) {
- MIOpt.ReadToken();
- IdentifierInfo *II = PTHMgr.GetIdentifierInfo(IdentifierID-1);
-
- Tok.setIdentifierInfo(II);
-
- // Change the kind of this identifier to the appropriate token kind, e.g.
- // turning "for" into a keyword.
- Tok.setKind(II->getTokenID());
-
- if (II->isHandleIdentifierCase())
- return PP->HandleIdentifier(Tok);
-
- return true;
- }
-
- //===--------------------------------------==//
- // Process the token.
- //===--------------------------------------==//
- if (TKind == tok::eof) {
- // Save the end-of-file token.
- EofToken = Tok;
-
- assert(!ParsingPreprocessorDirective);
- assert(!LexingRawMode);
-
- return LexEndOfFile(Tok);
- }
-
- if (TKind == tok::hash && Tok.isAtStartOfLine()) {
- LastHashTokPtr = CurPtr - StoredTokenSize;
- assert(!LexingRawMode);
- PP->HandleDirective(Tok);
-
- return false;
- }
-
- if (TKind == tok::eod) {
- assert(ParsingPreprocessorDirective);
- ParsingPreprocessorDirective = false;
- return true;
- }
-
- MIOpt.ReadToken();
- return true;
-}
-
-bool PTHLexer::LexEndOfFile(Token &Result) {
- // If we hit the end of the file while parsing a preprocessor directive,
- // end the preprocessor directive first. The next token returned will
- // then be the end of file.
- if (ParsingPreprocessorDirective) {
- ParsingPreprocessorDirective = false; // Done parsing the "line".
- return true; // Have a token.
- }
-
- assert(!LexingRawMode);
-
- // If we are in a #if directive, emit an error.
- while (!ConditionalStack.empty()) {
- if (PP->getCodeCompletionFileLoc() != FileStartLoc)
- PP->Diag(ConditionalStack.back().IfLoc,
- diag::err_pp_unterminated_conditional);
- ConditionalStack.pop_back();
- }
-
- // Finally, let the preprocessor handle this.
- return PP->HandleEndOfFile(Result);
-}
-
-// FIXME: We can just grab the last token instead of storing a copy
-// into EofToken.
-void PTHLexer::getEOF(Token& Tok) {
- assert(EofToken.is(tok::eof));
- Tok = EofToken;
-}
-
-void PTHLexer::DiscardToEndOfLine() {
- assert(ParsingPreprocessorDirective && ParsingFilename == false &&
- "Must be in a preprocessing directive!");
-
- // We assume that if the preprocessor wishes to discard to the end of
- // the line that it also means to end the current preprocessor directive.
- ParsingPreprocessorDirective = false;
-
- // Skip tokens by only peeking at their token kind and the flags.
- // We don't need to actually reconstruct full tokens from the token buffer.
- // This saves some copies and it also reduces IdentifierInfo* lookup.
- const unsigned char* p = CurPtr;
- while (true) {
- // Read the token kind. Are we at the end of the file?
- tok::TokenKind x = (tok::TokenKind) (uint8_t) *p;
- if (x == tok::eof) break;
-
- // Read the token flags. Are we at the start of the next line?
- Token::TokenFlags y = (Token::TokenFlags) (uint8_t) p[1];
- if (y & Token::StartOfLine) break;
-
- // Skip to the next token.
- p += StoredTokenSize;
- }
-
- CurPtr = p;
-}
-
-/// SkipBlock - Used by Preprocessor to skip the current conditional block.
-bool PTHLexer::SkipBlock() {
- using namespace llvm::support;
-
- assert(CurPPCondPtr && "No cached PP conditional information.");
- assert(LastHashTokPtr && "No known '#' token.");
-
- const unsigned char *HashEntryI = nullptr;
- uint32_t TableIdx;
-
- do {
- // Read the token offset from the side-table.
- uint32_t Offset = endian::readNext<uint32_t, little, aligned>(CurPPCondPtr);
-
- // Read the target table index from the side-table.
- TableIdx = endian::readNext<uint32_t, little, aligned>(CurPPCondPtr);
-
- // Compute the actual memory address of the '#' token data for this entry.
- HashEntryI = TokBuf + Offset;
-
- // Optimization: "Sibling jumping". #if...#else...#endif blocks can
- // contain nested blocks. In the side-table we can jump over these
- // nested blocks instead of doing a linear search if the next "sibling"
- // entry is not at a location greater than LastHashTokPtr.
- if (HashEntryI < LastHashTokPtr && TableIdx) {
- // In the side-table we are still at an entry for a '#' token that
- // is earlier than the last one we saw. Check if the location we would
- // stride gets us closer.
- const unsigned char* NextPPCondPtr =
- PPCond + TableIdx*(sizeof(uint32_t)*2);
- assert(NextPPCondPtr >= CurPPCondPtr);
- // Read where we should jump to.
- const unsigned char *HashEntryJ =
- TokBuf + endian::readNext<uint32_t, little, aligned>(NextPPCondPtr);
-
- if (HashEntryJ <= LastHashTokPtr) {
- // Jump directly to the next entry in the side table.
- HashEntryI = HashEntryJ;
- TableIdx = endian::readNext<uint32_t, little, aligned>(NextPPCondPtr);
- CurPPCondPtr = NextPPCondPtr;
- }
- }
- }
- while (HashEntryI < LastHashTokPtr);
- assert(HashEntryI == LastHashTokPtr && "No PP-cond entry found for '#'");
- assert(TableIdx && "No jumping from #endifs.");
-
- // Update our side-table iterator.
- const unsigned char* NextPPCondPtr = PPCond + TableIdx*(sizeof(uint32_t)*2);
- assert(NextPPCondPtr >= CurPPCondPtr);
- CurPPCondPtr = NextPPCondPtr;
-
- // Read where we should jump to.
- HashEntryI =
- TokBuf + endian::readNext<uint32_t, little, aligned>(NextPPCondPtr);
- uint32_t NextIdx = endian::readNext<uint32_t, little, aligned>(NextPPCondPtr);
-
- // By construction NextIdx will be zero if this is a #endif. This is useful
- // to know to obviate lexing another token.
- bool isEndif = NextIdx == 0;
-
- // This case can occur when we see something like this:
- //
- // #if ...
- // /* a comment or nothing */
- // #elif
- //
- // If we are skipping the first #if block it will be the case that CurPtr
- // already points 'elif'. Just return.
-
- if (CurPtr > HashEntryI) {
- assert(CurPtr == HashEntryI + StoredTokenSize);
- // Did we reach a #endif? If so, go ahead and consume that token as well.
- if (isEndif)
- CurPtr += StoredTokenSize * 2;
- else
- LastHashTokPtr = HashEntryI;
-
- return isEndif;
- }
-
- // Otherwise, we need to advance. Update CurPtr to point to the '#' token.
- CurPtr = HashEntryI;
-
- // Update the location of the last observed '#'. This is useful if we
- // are skipping multiple blocks.
- LastHashTokPtr = CurPtr;
-
- // Skip the '#' token.
- assert(((tok::TokenKind)*CurPtr) == tok::hash);
- CurPtr += StoredTokenSize;
-
- // Did we reach a #endif? If so, go ahead and consume that token as well.
- if (isEndif) {
- CurPtr += StoredTokenSize * 2;
- }
-
- return isEndif;
-}
-
-SourceLocation PTHLexer::getSourceLocation() {
- // getSourceLocation is not on the hot path. It is used to get the location
- // of the next token when transitioning back to this lexer when done
- // handling a #included file. Just read the necessary data from the token
- // data buffer to construct the SourceLocation object.
- // NOTE: This is a virtual function; hence it is defined out-of-line.
- using namespace llvm::support;
-
- const unsigned char *OffsetPtr = CurPtr + (StoredTokenSize - 4);
- uint32_t Offset = endian::readNext<uint32_t, little, aligned>(OffsetPtr);
- return FileStartLoc.getLocWithOffset(Offset);
-}
-
-//===----------------------------------------------------------------------===//
-// PTH file lookup: map from strings to file data.
-//===----------------------------------------------------------------------===//
-
-/// PTHFileLookup - This internal data structure is used by the PTHManager
-/// to map from FileEntry objects managed by FileManager to offsets within
-/// the PTH file.
-namespace {
-
-class PTHFileData {
- const uint32_t TokenOff;
- const uint32_t PPCondOff;
-
-public:
- PTHFileData(uint32_t tokenOff, uint32_t ppCondOff)
- : TokenOff(tokenOff), PPCondOff(ppCondOff) {}
-
- uint32_t getTokenOffset() const { return TokenOff; }
- uint32_t getPPCondOffset() const { return PPCondOff; }
-};
-
-class PTHFileLookupCommonTrait {
-public:
- using internal_key_type = std::pair<unsigned char, StringRef>;
- using hash_value_type = unsigned;
- using offset_type = unsigned;
-
- static hash_value_type ComputeHash(internal_key_type x) {
- return llvm::djbHash(x.second);
- }
-
- static std::pair<unsigned, unsigned>
- ReadKeyDataLength(const unsigned char*& d) {
- using namespace llvm::support;
-
- unsigned keyLen =
- (unsigned)endian::readNext<uint16_t, little, unaligned>(d);
- unsigned dataLen = (unsigned) *(d++);
- return std::make_pair(keyLen, dataLen);
- }
-
- static internal_key_type ReadKey(const unsigned char* d, unsigned) {
- unsigned char k = *(d++); // Read the entry kind.
- return std::make_pair(k, (const char*) d);
- }
-};
-
-} // namespace
-
-class PTHManager::PTHFileLookupTrait : public PTHFileLookupCommonTrait {
-public:
- using external_key_type = const FileEntry *;
- using data_type = PTHFileData;
-
- static internal_key_type GetInternalKey(const FileEntry* FE) {
- return std::make_pair((unsigned char) 0x1, FE->getName());
- }
-
- static bool EqualKey(internal_key_type a, internal_key_type b) {
- return a.first == b.first && a.second == b.second;
- }
-
- static PTHFileData ReadData(const internal_key_type& k,
- const unsigned char* d, unsigned) {
- using namespace llvm::support;
-
- assert(k.first == 0x1 && "Only file lookups can match!");
- uint32_t x = endian::readNext<uint32_t, little, unaligned>(d);
- uint32_t y = endian::readNext<uint32_t, little, unaligned>(d);
- return PTHFileData(x, y);
- }
-};
-
-class PTHManager::PTHStringLookupTrait {
-public:
- using data_type = uint32_t;
- using external_key_type = const std::pair<const char *, unsigned>;
- using internal_key_type = external_key_type;
- using hash_value_type = uint32_t;
- using offset_type = unsigned;
-
- static bool EqualKey(const internal_key_type& a,
- const internal_key_type& b) {
- return (a.second == b.second) ? memcmp(a.first, b.first, a.second) == 0
- : false;
- }
-
- static hash_value_type ComputeHash(const internal_key_type& a) {
- return llvm::djbHash(StringRef(a.first, a.second));
- }
-
- // This hopefully will just get inlined and removed by the optimizer.
- static const internal_key_type&
- GetInternalKey(const external_key_type& x) { return x; }
-
- static std::pair<unsigned, unsigned>
- ReadKeyDataLength(const unsigned char*& d) {
- using namespace llvm::support;
-
- return std::make_pair(
- (unsigned)endian::readNext<uint16_t, little, unaligned>(d),
- sizeof(uint32_t));
- }
-
- static std::pair<const char*, unsigned>
- ReadKey(const unsigned char* d, unsigned n) {
- assert(n >= 2 && d[n-1] == '\0');
- return std::make_pair((const char*) d, n-1);
- }
-
- static uint32_t ReadData(const internal_key_type& k, const unsigned char* d,
- unsigned) {
- using namespace llvm::support;
-
- return endian::readNext<uint32_t, little, unaligned>(d);
- }
-};
-
-//===----------------------------------------------------------------------===//
-// PTHManager methods.
-//===----------------------------------------------------------------------===//
-
-PTHManager::PTHManager(
- std::unique_ptr<const llvm::MemoryBuffer> buf,
- std::unique_ptr<PTHFileLookup> fileLookup, const unsigned char *idDataTable,
- std::unique_ptr<IdentifierInfo *[], llvm::FreeDeleter> perIDCache,
- std::unique_ptr<PTHStringIdLookup> stringIdLookup, unsigned numIds,
- const unsigned char *spellingBase, const char *originalSourceFile)
- : Buf(std::move(buf)), PerIDCache(std::move(perIDCache)),
- FileLookup(std::move(fileLookup)), IdDataTable(idDataTable),
- StringIdLookup(std::move(stringIdLookup)), NumIds(numIds),
- SpellingBase(spellingBase), OriginalSourceFile(originalSourceFile) {}
-
-PTHManager::~PTHManager() = default;
-
-static void InvalidPTH(DiagnosticsEngine &Diags, const char *Msg) {
- Diags.Report(Diags.getCustomDiagID(DiagnosticsEngine::Error, "%0")) << Msg;
-}
-
-PTHManager *PTHManager::Create(StringRef file, DiagnosticsEngine &Diags) {
- // Memory map the PTH file.
- llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>> FileOrErr =
- llvm::MemoryBuffer::getFile(file);
-
- if (!FileOrErr) {
- // FIXME: Add ec.message() to this diag.
- Diags.Report(diag::err_invalid_pth_file) << file;
- return nullptr;
- }
- std::unique_ptr<llvm::MemoryBuffer> File = std::move(FileOrErr.get());
-
- using namespace llvm::support;
-
- // Get the buffer ranges and check if there are at least three 32-bit
- // words at the end of the file.
- const unsigned char *BufBeg = (const unsigned char*)File->getBufferStart();
- const unsigned char *BufEnd = (const unsigned char*)File->getBufferEnd();
-
- // Check the prologue of the file.
- if ((BufEnd - BufBeg) < (signed)(sizeof("cfe-pth") + 4 + 4) ||
- memcmp(BufBeg, "cfe-pth", sizeof("cfe-pth")) != 0) {
- Diags.Report(diag::err_invalid_pth_file) << file;
- return nullptr;
- }
-
- // Read the PTH version.
- const unsigned char *p = BufBeg + (sizeof("cfe-pth"));
- unsigned Version = endian::readNext<uint32_t, little, aligned>(p);
-
- if (Version < PTHManager::Version) {
- InvalidPTH(Diags,
- Version < PTHManager::Version
- ? "PTH file uses an older PTH format that is no longer supported"
- : "PTH file uses a newer PTH format that cannot be read");
- return nullptr;
- }
-
- // Compute the address of the index table at the end of the PTH file.
- const unsigned char *PrologueOffset = p;
-
- if (PrologueOffset >= BufEnd) {
- Diags.Report(diag::err_invalid_pth_file) << file;
- return nullptr;
- }
-
- // Construct the file lookup table. This will be used for mapping from
- // FileEntry*'s to cached tokens.
- const unsigned char* FileTableOffset = PrologueOffset + sizeof(uint32_t)*2;
- const unsigned char *FileTable =
- BufBeg + endian::readNext<uint32_t, little, aligned>(FileTableOffset);
-
- if (!(FileTable > BufBeg && FileTable < BufEnd)) {
- Diags.Report(diag::err_invalid_pth_file) << file;
- return nullptr; // FIXME: Proper error diagnostic?
- }
-
- std::unique_ptr<PTHFileLookup> FL(PTHFileLookup::Create(FileTable, BufBeg));
-
- // Warn if the PTH file is empty. We still want to create a PTHManager
- // as the PTH could be used with -include-pth.
- if (FL->isEmpty())
- InvalidPTH(Diags, "PTH file contains no cached source data");
-
- // Get the location of the table mapping from persistent ids to the
- // data needed to reconstruct identifiers.
- const unsigned char* IDTableOffset = PrologueOffset + sizeof(uint32_t)*0;
- const unsigned char *IData =
- BufBeg + endian::readNext<uint32_t, little, aligned>(IDTableOffset);
-
- if (!(IData >= BufBeg && IData < BufEnd)) {
- Diags.Report(diag::err_invalid_pth_file) << file;
- return nullptr;
- }
-
- // Get the location of the hashtable mapping between strings and
- // persistent IDs.
- const unsigned char* StringIdTableOffset = PrologueOffset + sizeof(uint32_t)*1;
- const unsigned char *StringIdTable =
- BufBeg + endian::readNext<uint32_t, little, aligned>(StringIdTableOffset);
- if (!(StringIdTable >= BufBeg && StringIdTable < BufEnd)) {
- Diags.Report(diag::err_invalid_pth_file) << file;
- return nullptr;
- }
-
- std::unique_ptr<PTHStringIdLookup> SL(
- PTHStringIdLookup::Create(StringIdTable, BufBeg));
-
- // Get the location of the spelling cache.
- const unsigned char* spellingBaseOffset = PrologueOffset + sizeof(uint32_t)*3;
- const unsigned char *spellingBase =
- BufBeg + endian::readNext<uint32_t, little, aligned>(spellingBaseOffset);
- if (!(spellingBase >= BufBeg && spellingBase < BufEnd)) {
- Diags.Report(diag::err_invalid_pth_file) << file;
- return nullptr;
- }
-
- // Get the number of IdentifierInfos and pre-allocate the identifier cache.
- uint32_t NumIds = endian::readNext<uint32_t, little, aligned>(IData);
-
- // Pre-allocate the persistent ID -> IdentifierInfo* cache. We use calloc()
- // so that we in the best case only zero out memory once when the OS returns
- // us new pages.
- std::unique_ptr<IdentifierInfo *[], llvm::FreeDeleter> PerIDCache;
-
- if (NumIds) {
- PerIDCache.reset((IdentifierInfo **)calloc(NumIds, sizeof(PerIDCache[0])));
- if (!PerIDCache) {
- InvalidPTH(Diags, "Could not allocate memory for processing PTH file");
- return nullptr;
- }
- }
-
- // Compute the address of the original source file.
- const unsigned char* originalSourceBase = PrologueOffset + sizeof(uint32_t)*4;
- unsigned len =
- endian::readNext<uint16_t, little, unaligned>(originalSourceBase);
- if (!len) originalSourceBase = nullptr;
-
- // Create the new PTHManager.
- return new PTHManager(std::move(File), std::move(FL), IData,
- std::move(PerIDCache), std::move(SL), NumIds,
- spellingBase, (const char *)originalSourceBase);
-}
-
-IdentifierInfo* PTHManager::LazilyCreateIdentifierInfo(unsigned PersistentID) {
- using namespace llvm::support;
-
- // Look in the PTH file for the string data for the IdentifierInfo object.
- const unsigned char* TableEntry = IdDataTable + sizeof(uint32_t)*PersistentID;
- const unsigned char *IDData =
- (const unsigned char *)Buf->getBufferStart() +
- endian::readNext<uint32_t, little, aligned>(TableEntry);
- assert(IDData < (const unsigned char*)Buf->getBufferEnd());
-
- // Allocate the object.
- std::pair<IdentifierInfo,const unsigned char*> *Mem =
- Alloc.Allocate<std::pair<IdentifierInfo, const unsigned char *>>();
-
- Mem->second = IDData;
- assert(IDData[0] != '\0');
- IdentifierInfo *II = new ((void*) Mem) IdentifierInfo();
-
- // Store the new IdentifierInfo in the cache.
- PerIDCache[PersistentID] = II;
- assert(II->getNameStart() && II->getNameStart()[0] != '\0');
- return II;
-}
-
-IdentifierInfo* PTHManager::get(StringRef Name) {
- // Double check our assumption that the last character isn't '\0'.
- assert(Name.empty() || Name.back() != '\0');
- PTHStringIdLookup::iterator I =
- StringIdLookup->find(std::make_pair(Name.data(), Name.size()));
- if (I == StringIdLookup->end()) // No identifier found?
- return nullptr;
-
- // Match found. Return the identifier!
- assert(*I > 0);
- return GetIdentifierInfo(*I-1);
-}
-
-PTHLexer *PTHManager::CreateLexer(FileID FID) {
- const FileEntry *FE = PP->getSourceManager().getFileEntryForID(FID);
- if (!FE)
- return nullptr;
-
- using namespace llvm::support;
-
- // Lookup the FileEntry object in our file lookup data structure. It will
- // return a variant that indicates whether or not there is an offset within
- // the PTH file that contains cached tokens.
- PTHFileLookup::iterator I = FileLookup->find(FE);
-
- if (I == FileLookup->end()) // No tokens available?
- return nullptr;
-
- const PTHFileData& FileData = *I;
-
- const unsigned char *BufStart = (const unsigned char *)Buf->getBufferStart();
- // Compute the offset of the token data within the buffer.
- const unsigned char* data = BufStart + FileData.getTokenOffset();
-
- // Get the location of pp-conditional table.
- const unsigned char* ppcond = BufStart + FileData.getPPCondOffset();
- uint32_t Len = endian::readNext<uint32_t, little, aligned>(ppcond);
- if (Len == 0) ppcond = nullptr;
-
- assert(PP && "No preprocessor set yet!");
- return new PTHLexer(*PP, FID, data, ppcond, *this);
-}
-
-//===----------------------------------------------------------------------===//
-// 'stat' caching.
-//===----------------------------------------------------------------------===//
-
-namespace {
-
-class PTHStatData {
-public:
- uint64_t Size;
- time_t ModTime;
- llvm::sys::fs::UniqueID UniqueID;
- const bool HasData = false;
- bool IsDirectory;
-
- PTHStatData() = default;
- PTHStatData(uint64_t Size, time_t ModTime, llvm::sys::fs::UniqueID UniqueID,
- bool IsDirectory)
- : Size(Size), ModTime(ModTime), UniqueID(UniqueID), HasData(true),
- IsDirectory(IsDirectory) {}
-};
-
-class PTHStatLookupTrait : public PTHFileLookupCommonTrait {
-public:
- using external_key_type = StringRef; // const char*
- using data_type = PTHStatData;
-
- static internal_key_type GetInternalKey(StringRef path) {
- // The key 'kind' doesn't matter here because it is ignored in EqualKey.
- return std::make_pair((unsigned char) 0x0, path);
- }
-
- static bool EqualKey(internal_key_type a, internal_key_type b) {
- // When doing 'stat' lookups we don't care about the kind of 'a' and 'b',
- // just the paths.
- return a.second == b.second;
- }
-
- static data_type ReadData(const internal_key_type& k, const unsigned char* d,
- unsigned) {
- if (k.first /* File or Directory */) {
- bool IsDirectory = true;
- if (k.first == 0x1 /* File */) {
- IsDirectory = false;
- d += 4 * 2; // Skip the first 2 words.
- }
-
- using namespace llvm::support;
-
- uint64_t File = endian::readNext<uint64_t, little, unaligned>(d);
- uint64_t Device = endian::readNext<uint64_t, little, unaligned>(d);
- llvm::sys::fs::UniqueID UniqueID(Device, File);
- time_t ModTime = endian::readNext<uint64_t, little, unaligned>(d);
- uint64_t Size = endian::readNext<uint64_t, little, unaligned>(d);
- return data_type(Size, ModTime, UniqueID, IsDirectory);
- }
-
- // Negative stat. Don't read anything.
- return data_type();
- }
-};
-
-} // namespace
-
-namespace clang {
-
-class PTHStatCache : public FileSystemStatCache {
- using CacheTy = llvm::OnDiskChainedHashTable<PTHStatLookupTrait>;
-
- CacheTy Cache;
-
-public:
- PTHStatCache(PTHManager::PTHFileLookup &FL)
- : Cache(FL.getNumBuckets(), FL.getNumEntries(), FL.getBuckets(),
- FL.getBase()) {}
-
- LookupResult getStat(StringRef Path, FileData &Data, bool isFile,
- std::unique_ptr<llvm::vfs::File> *F,
- llvm::vfs::FileSystem &FS) override {
- // Do the lookup for the file's data in the PTH file.
- CacheTy::iterator I = Cache.find(Path);
-
- // If we don't get a hit in the PTH file just forward to 'stat'.
- if (I == Cache.end())
- return statChained(Path, Data, isFile, F, FS);
-
- const PTHStatData &D = *I;
-
- if (!D.HasData)
- return CacheMissing;
-
- Data.Name = Path;
- Data.Size = D.Size;
- Data.ModTime = D.ModTime;
- Data.UniqueID = D.UniqueID;
- Data.IsDirectory = D.IsDirectory;
- Data.IsNamedPipe = false;
- Data.InPCH = true;
-
- return CacheExists;
- }
-};
-
-} // namespace clang
-
-std::unique_ptr<FileSystemStatCache> PTHManager::createStatCache() {
- return llvm::make_unique<PTHStatCache>(*FileLookup);
-}
Modified: cfe/trunk/lib/Lex/Pragma.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Pragma.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Lex/Pragma.cpp (original)
+++ cfe/trunk/lib/Lex/Pragma.cpp Tue Dec 4 06:34:09 2018
@@ -31,7 +31,6 @@
#include "clang/Lex/PPCallbacks.h"
#include "clang/Lex/Preprocessor.h"
#include "clang/Lex/PreprocessorLexer.h"
-#include "clang/Lex/PTHLexer.h"
#include "clang/Lex/Token.h"
#include "clang/Lex/TokenLexer.h"
#include "llvm/ADT/ArrayRef.h"
@@ -404,10 +403,7 @@ void Preprocessor::HandlePragmaOnce(Toke
void Preprocessor::HandlePragmaMark() {
assert(CurPPLexer && "No current lexer?");
- if (CurLexer)
- CurLexer->ReadToEndOfLine();
- else
- CurPTHLexer->DiscardToEndOfLine();
+ CurLexer->ReadToEndOfLine();
}
/// HandlePragmaPoison - Handle \#pragma GCC poison. PoisonTok is the 'poison'.
@@ -810,12 +806,6 @@ void Preprocessor::HandlePragmaModuleBui
DiscardUntilEndOfDirective();
}
- if (CurPTHLexer) {
- // FIXME: Support this somehow?
- Diag(Loc, diag::err_pp_module_build_pth);
- return;
- }
-
CurLexer->LexingRawMode = true;
auto TryConsumeIdentifier = [&](StringRef Ident) -> bool {
Modified: cfe/trunk/lib/Lex/Preprocessor.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Preprocessor.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Lex/Preprocessor.cpp (original)
+++ cfe/trunk/lib/Lex/Preprocessor.cpp Tue Dec 4 06:34:09 2018
@@ -44,8 +44,6 @@
#include "clang/Lex/MacroArgs.h"
#include "clang/Lex/MacroInfo.h"
#include "clang/Lex/ModuleLoader.h"
-#include "clang/Lex/PTHLexer.h"
-#include "clang/Lex/PTHManager.h"
#include "clang/Lex/Pragma.h"
#include "clang/Lex/PreprocessingRecord.h"
#include "clang/Lex/PreprocessorLexer.h"
@@ -224,11 +222,6 @@ void Preprocessor::FinalizeForModelFile(
PragmaHandlers = std::move(PragmaHandlersBackup);
}
-void Preprocessor::setPTHManager(PTHManager* pm) {
- PTH.reset(pm);
- FileMgr.addStatCache(PTH->createStatCache());
-}
-
void Preprocessor::DumpToken(const Token &Tok, bool DumpFlags) const {
llvm::errs() << tok::getTokenName(Tok.getKind()) << " '"
<< getSpelling(Tok) << "'";
@@ -379,8 +372,6 @@ StringRef Preprocessor::getLastMacroWith
void Preprocessor::recomputeCurLexerKind() {
if (CurLexer)
CurLexerKind = CLK_Lexer;
- else if (CurPTHLexer)
- CurLexerKind = CLK_PTHLexer;
else if (CurTokenLexer)
CurLexerKind = CLK_TokenLexer;
else
@@ -877,9 +868,6 @@ void Preprocessor::Lex(Token &Result) {
case CLK_Lexer:
ReturnedToken = CurLexer->Lex(Result);
break;
- case CLK_PTHLexer:
- ReturnedToken = CurPTHLexer->Lex(Result);
- break;
case CLK_TokenLexer:
ReturnedToken = CurTokenLexer->Lex(Result);
break;
Modified: cfe/trunk/lib/Serialization/ASTReader.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/ASTReader.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Serialization/ASTReader.cpp (original)
+++ cfe/trunk/lib/Serialization/ASTReader.cpp Tue Dec 4 06:34:09 2018
@@ -5388,7 +5388,6 @@ bool ASTReader::ParsePreprocessorOptions
PPOpts.UsePredefines = Record[Idx++];
PPOpts.DetailedRecord = Record[Idx++];
PPOpts.ImplicitPCHInclude = ReadString(Record, Idx);
- PPOpts.ImplicitPTHInclude = ReadString(Record, Idx);
PPOpts.ObjCXXARCStandardLibrary =
static_cast<ObjCXXARCStandardLibraryKind>(Record[Idx++]);
SuggestedPredefines.clear();
Modified: cfe/trunk/lib/Serialization/ASTWriter.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Serialization/ASTWriter.cpp?rev=348266&r1=348265&r2=348266&view=diff
==============================================================================
--- cfe/trunk/lib/Serialization/ASTWriter.cpp (original)
+++ cfe/trunk/lib/Serialization/ASTWriter.cpp Tue Dec 4 06:34:09 2018
@@ -1695,7 +1695,6 @@ void ASTWriter::WriteControlBlock(Prepro
// Detailed record is important since it is used for the module cache hash.
Record.push_back(PPOpts.DetailedRecord);
AddString(PPOpts.ImplicitPCHInclude, Record);
- AddString(PPOpts.ImplicitPTHInclude, Record);
Record.push_back(static_cast<unsigned>(PPOpts.ObjCXXARCStandardLibrary));
Stream.EmitRecord(PREPROCESSOR_OPTIONS, Record);
Removed: cfe/trunk/test/Driver/pth.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/pth.c?rev=348265&view=auto
==============================================================================
--- cfe/trunk/test/Driver/pth.c (original)
+++ cfe/trunk/test/Driver/pth.c (removed)
@@ -1,12 +0,0 @@
-// Test transparent PTH support.
-
-// RUN: %clang -no-canonical-prefixes -ccc-pch-is-pth -x c-header %s -o %t.h.pth -### 2> %t.log
-// RUN: FileCheck -check-prefix CHECK1 -input-file %t.log %s
-
-// CHECK1: "{{.*[/\\]}}clang{{.*}}" "-cc1" {{.*}} "-o" "{{.*}}.h.pth" "-x" "c-header" "{{.*}}pth.c"
-
-// RUN: touch %t.h.pth
-// RUN: %clang -no-canonical-prefixes -ccc-pch-is-pth -E -include %t.h %s -### 2> %t.log
-// RUN: FileCheck -check-prefix CHECK2 -input-file %t.log %s
-
-// CHECK2: "{{.*[/\\]}}clang{{.*}}" "-cc1" {{.*}}"-include-pth" "{{.*}}.h.pth" {{.*}}"-x" "c" "{{.*}}pth.c"
Removed: cfe/trunk/test/PCH/emit-pth.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/PCH/emit-pth.c?rev=348265&view=auto
==============================================================================
--- cfe/trunk/test/PCH/emit-pth.c (original)
+++ cfe/trunk/test/PCH/emit-pth.c (removed)
@@ -1,7 +0,0 @@
-// RUN: %clang_cc1 -triple i386-unknown-unknown -emit-pth -o %t1 %s
-// RUN: %clang_cc1 -triple i386-unknown-unknown -emit-pth -o - %s > %t2
-// RUN: cmp %t1 %t2
-// RUN: %clang_cc1 -triple i386-unknown-unknown -emit-pth -o - %s | \
-// RUN: FileCheck %s
-
-// CHECK: cfe-pth
Removed: cfe/trunk/test/PCH/pth.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/PCH/pth.c?rev=348265&view=auto
==============================================================================
--- cfe/trunk/test/PCH/pth.c (original)
+++ cfe/trunk/test/PCH/pth.c (removed)
@@ -1,7 +0,0 @@
-// RUN: %clang_cc1 -triple i386-unknown-unknown -emit-pth -o %t %S/pth.h
-// RUN: not %clang_cc1 -triple i386-unknown-unknown -include-pth %t -fsyntax-only %s 2>&1 | FileCheck %s
-
-#error This is the only diagnostic
-
-// CHECK: This is the only diagnostic
-// CHECK: 1 error generated.
Removed: cfe/trunk/test/Preprocessor/include-pth.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Preprocessor/include-pth.c?rev=348265&view=auto
==============================================================================
--- cfe/trunk/test/Preprocessor/include-pth.c (original)
+++ cfe/trunk/test/Preprocessor/include-pth.c (removed)
@@ -1,3 +0,0 @@
-// RUN: %clang_cc1 -emit-pth %s -o %t
-// RUN: %clang_cc1 -include-pth %t %s -E | grep 'file_to_include' | count 2
-#include "file_to_include.h"
More information about the cfe-commits
mailing list