[cfe-commits] r109445 - in /cfe/trunk: include/clang/Basic/ include/clang/Driver/ include/clang/Frontend/ include/clang/Lex/ lib/Basic/ lib/Frontend/ lib/Lex/ test/PCH/ test/PCH/Inputs/
Douglas Gregor
dgregor at apple.com
Mon Jul 26 14:36:20 PDT 2010
Author: dgregor
Date: Mon Jul 26 16:36:20 2010
New Revision: 109445
URL: http://llvm.org/viewvc/llvm-project?rev=109445&view=rev
Log:
Introduce basic support for loading a precompiled preamble while
reparsing an ASTUnit. When saving a preamble, create a buffer larger
than the actual file we're working with but fill everything from the
end of the preamble to the end of the file with spaces (so the lexer
will quickly skip them). When we load the file, create a buffer of the
same size, filling it with the file and then spaces. Then, instruct
the lexer to start lexing after the preamble, therefore continuing the
parse from the spot where the preamble left off.
It's now possible to perform a simple preamble build + parse (+
reparse) with ASTUnit. However, one has to disable a bunch of checking
in the PCH reader to do so. That part isn't committed; it will likely
be handled with some other kind of flag (e.g., -fno-validate-pch).
As part of this, fix some issues with null termination of the memory
buffers created for the preamble; we were trying to explicitly
NULL-terminate them, even though they were also getting implicitly
NULL terminated, leading to excess warnings about NULL characters in
source files.
Added:
cfe/trunk/test/PCH/Inputs/preamble.h
cfe/trunk/test/PCH/preamble.c
Modified:
cfe/trunk/include/clang/Basic/DiagnosticDriverKinds.td
cfe/trunk/include/clang/Basic/SourceManager.h
cfe/trunk/include/clang/Driver/CC1Options.td
cfe/trunk/include/clang/Frontend/ASTUnit.h
cfe/trunk/include/clang/Frontend/PreprocessorOptions.h
cfe/trunk/include/clang/Lex/Lexer.h
cfe/trunk/include/clang/Lex/Preprocessor.h
cfe/trunk/lib/Basic/SourceManager.cpp
cfe/trunk/lib/Frontend/ASTUnit.cpp
cfe/trunk/lib/Frontend/CompilerInvocation.cpp
cfe/trunk/lib/Frontend/FrontendActions.cpp
cfe/trunk/lib/Frontend/InitPreprocessor.cpp
cfe/trunk/lib/Lex/Lexer.cpp
cfe/trunk/lib/Lex/Preprocessor.cpp
Modified: cfe/trunk/include/clang/Basic/DiagnosticDriverKinds.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticDriverKinds.td?rev=109445&r1=109444&r2=109445&view=diff
==============================================================================
--- cfe/trunk/include/clang/Basic/DiagnosticDriverKinds.td (original)
+++ cfe/trunk/include/clang/Basic/DiagnosticDriverKinds.td Mon Jul 26 16:36:20 2010
@@ -68,6 +68,8 @@
"invalid output type '%0' for use with gcc tool">;
def err_drv_cc_print_options_failure : Error<
"unable to open CC_PRINT_OPTIONS file: %0">;
+def err_drv_preamble_format : Error<
+ "incorrect format for -preamble-bytes=N,END">;
def warn_drv_input_file_unused : Warning<
"%0: '%1' input unused when '%2' is present">;
Modified: cfe/trunk/include/clang/Basic/SourceManager.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/SourceManager.h?rev=109445&r1=109444&r2=109445&view=diff
==============================================================================
--- cfe/trunk/include/clang/Basic/SourceManager.h (original)
+++ cfe/trunk/include/clang/Basic/SourceManager.h Mon Jul 26 16:36:20 2010
@@ -53,10 +53,17 @@
/// ContentCache - Once instance of this struct is kept for every file
/// loaded or used. This object owns the MemoryBuffer object.
class ContentCache {
+ enum CCFlags {
+ /// \brief Whether the buffer is invalid.
+ InvalidFlag = 0x01,
+ /// \brief Whether the buffer should not be freed on destruction.
+ DoNotFreeFlag = 0x02
+ };
+
/// Buffer - The actual buffer containing the characters from the input
/// file. This is owned by the ContentCache object.
- /// The bit indicates whether the buffer is invalid.
- mutable llvm::PointerIntPair<const llvm::MemoryBuffer *, 1, bool> Buffer;
+ /// The bits indicate indicates whether the buffer is invalid.
+ mutable llvm::PointerIntPair<const llvm::MemoryBuffer *, 2> Buffer;
public:
/// Reference to the file entry. This reference does not own
@@ -106,8 +113,18 @@
/// \brief Replace the existing buffer (which will be deleted)
/// with the given buffer.
- void replaceBuffer(const llvm::MemoryBuffer *B);
+ void replaceBuffer(const llvm::MemoryBuffer *B, bool DoNotFree = false);
+ /// \brief Determine whether the buffer itself is invalid.
+ bool isBufferInvalid() const {
+ return Buffer.getInt() & InvalidFlag;
+ }
+
+ /// \brief Determine whether the buffer should be freed.
+ bool shouldFreeBuffer() const {
+ return (Buffer.getInt() & DoNotFreeFlag) == 0;
+ }
+
ContentCache(const FileEntry *Ent = 0)
: Buffer(0, false), Entry(Ent), SourceLineCache(0), NumLines(0) {}
@@ -490,9 +507,13 @@
/// \param Buffer the memory buffer whose contents will be used as the
/// data in the given source file.
///
+ /// \param DoNotFree If true, then the buffer will not be freed when the
+ /// source manager is destroyed.
+ ///
/// \returns true if an error occurred, false otherwise.
bool overrideFileContents(const FileEntry *SourceFile,
- const llvm::MemoryBuffer *Buffer);
+ const llvm::MemoryBuffer *Buffer,
+ bool DoNotFree = false);
//===--------------------------------------------------------------------===//
// FileID manipulation methods.
Modified: cfe/trunk/include/clang/Driver/CC1Options.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/CC1Options.td?rev=109445&r1=109444&r2=109445&view=diff
==============================================================================
--- cfe/trunk/include/clang/Driver/CC1Options.td (original)
+++ cfe/trunk/include/clang/Driver/CC1Options.td Mon Jul 26 16:36:20 2010
@@ -505,7 +505,7 @@
HelpText<"Include precompiled header file">;
def include_pth : Separate<"-include-pth">, MetaVarName<"<file>">,
HelpText<"Include file before parsing">;
-def use_preamble_EQ : Joined<"-use-preamble=">,
+def preamble_bytes_EQ : Joined<"-preamble-bytes=">,
HelpText<"Assume that the precompiled header is a precompiled preamble "
"covering the first N bytes of the main file">;
def token_cache : Separate<"-token-cache">, MetaVarName<"<path>">,
Modified: cfe/trunk/include/clang/Frontend/ASTUnit.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/ASTUnit.h?rev=109445&r1=109444&r2=109445&view=diff
==============================================================================
--- cfe/trunk/include/clang/Frontend/ASTUnit.h (original)
+++ cfe/trunk/include/clang/Frontend/ASTUnit.h Mon Jul 26 16:36:20 2010
@@ -125,6 +125,12 @@
/// \c PreambleFile.
std::vector<char> Preamble;
+ /// \brief Whether the preamble ends at the start of a new line.
+ ///
+ /// Used to inform the lexer as to whether it's starting at the beginning of
+ /// a line after skipping the preamble.
+ bool PreambleEndsAtStartOfLine;
+
/// \brief The size of the source buffer that we've reserved for the main
/// file within the precompiled preamble.
unsigned PreambleReservedSize;
@@ -137,9 +143,8 @@
void CleanTemporaryFiles();
bool Parse(llvm::MemoryBuffer *OverrideMainBuffer);
- std::pair<llvm::MemoryBuffer *, unsigned> ComputePreamble(
- CompilerInvocation &Invocation,
- bool &CreatedBuffer);
+ std::pair<llvm::MemoryBuffer *, std::pair<unsigned, bool> >
+ ComputePreamble(CompilerInvocation &Invocation, bool &CreatedBuffer);
llvm::MemoryBuffer *BuildPrecompiledPreamble();
Modified: cfe/trunk/include/clang/Frontend/PreprocessorOptions.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/PreprocessorOptions.h?rev=109445&r1=109444&r2=109445&view=diff
==============================================================================
--- cfe/trunk/include/clang/Frontend/PreprocessorOptions.h (original)
+++ cfe/trunk/include/clang/Frontend/PreprocessorOptions.h Mon Jul 26 16:36:20 2010
@@ -43,6 +43,13 @@
/// The implicit PCH included at the start of the translation unit, or empty.
std::string ImplicitPCHInclude;
+ /// \brief If non-zero, the implicit PCH include is actually a precompiled
+ /// preamble that covers this number of bytes in the main source file.
+ ///
+ /// The boolean indicates whether the preamble ends at the start of a new
+ /// line.
+ std::pair<unsigned, bool> PrecompiledPreambleBytes;
+
/// The implicit PTH input included at the start of the translation unit, or
/// empty.
std::string ImplicitPTHInclude;
@@ -62,6 +69,14 @@
std::vector<std::pair<std::string, const llvm::MemoryBuffer *> >
RemappedFileBuffers;
+ /// \brief Whether the compiler instance should retain (i.e., not free)
+ /// the buffers associated with remapped files.
+ ///
+ /// This flag defaults to false; it can be set true only through direct
+ /// manipulation of the compiler invocation object, in cases where the
+ /// compiler invocation and its buffers will be reused.
+ bool RetainRemappedFileBuffers;
+
typedef std::vector<std::pair<std::string, std::string> >::iterator
remapped_file_iterator;
typedef std::vector<std::pair<std::string, std::string> >::const_iterator
@@ -97,7 +112,9 @@
}
public:
- PreprocessorOptions() : UsePredefines(true), DetailedRecord(false) {}
+ PreprocessorOptions() : UsePredefines(true), DetailedRecord(false),
+ PrecompiledPreambleBytes(0, true),
+ RetainRemappedFileBuffers(false) { }
void addMacroDef(llvm::StringRef Name) {
Macros.push_back(std::make_pair(Name, false));
Modified: cfe/trunk/include/clang/Lex/Lexer.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/Lexer.h?rev=109445&r1=109444&r2=109445&view=diff
==============================================================================
--- cfe/trunk/include/clang/Lex/Lexer.h (original)
+++ cfe/trunk/include/clang/Lex/Lexer.h Mon Jul 26 16:36:20 2010
@@ -238,8 +238,10 @@
/// \param Buffer The memory buffer containing the file's contents.
///
/// \returns The offset into the file where the preamble ends and the rest
- /// of the file begins.
- static unsigned ComputePreamble(const llvm::MemoryBuffer *Buffer);
+ /// of the file begins along with a boolean value indicating whether
+ /// the preamble ends at the beginning of a new line.
+ static std::pair<unsigned, bool>
+ ComputePreamble(const llvm::MemoryBuffer *Buffer);
//===--------------------------------------------------------------------===//
// Internal implementation interfaces.
@@ -383,6 +385,8 @@
//===--------------------------------------------------------------------===//
// Other lexer functions.
+ void SkipBytes(unsigned Bytes, bool StartOfLine);
+
// Helper functions to lex the remainder of a token of the specific type.
void LexIdentifier (Token &Result, const char *CurPtr);
void LexNumericConstant (Token &Result, const char *CurPtr);
Modified: cfe/trunk/include/clang/Lex/Preprocessor.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/Preprocessor.h?rev=109445&r1=109444&r2=109445&view=diff
==============================================================================
--- cfe/trunk/include/clang/Lex/Preprocessor.h (original)
+++ cfe/trunk/include/clang/Lex/Preprocessor.h Mon Jul 26 16:36:20 2010
@@ -134,6 +134,12 @@
/// \brief The file that we're performing code-completion for, if any.
const FileEntry *CodeCompletionFile;
+ /// \brief The number of bytes that we will initially skip when entering the
+ /// main file, which is used when loading a precompiled preamble, along
+ /// with a flag that indicates whether skipping this number of bytes will
+ /// place the lexer at the start of a line.
+ std::pair<unsigned, bool> SkipMainFilePreamble;
+
/// CurLexer - This is the current top of the stack that we're lexing from if
/// not expanding a macro and we are lexing directly from source code.
/// Only one of CurLexer, CurPTHLexer, or CurTokenLexer will be non-null.
@@ -556,6 +562,18 @@
/// for which we are performing code completion.
bool isCodeCompletionFile(SourceLocation FileLoc) const;
+ /// \brief Instruct the preprocessor to skip part of the main
+ /// the main source file.
+ ///
+ /// \brief Bytes The number of bytes in the preamble to skip.
+ ///
+ /// \brief StartOfLine Whether skipping these bytes puts the lexer at the
+ /// start of a line.
+ void setSkipMainFilePreamble(unsigned Bytes, bool StartOfLine) {
+ SkipMainFilePreamble.first = Bytes;
+ SkipMainFilePreamble.second = StartOfLine;
+ }
+
/// Diag - Forwarding function for diagnostics. This emits a diagnostic at
/// the specified Token's location, translating the token's start
/// position in the current buffer into a SourcePosition object for rendering.
Modified: cfe/trunk/lib/Basic/SourceManager.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/SourceManager.cpp?rev=109445&r1=109444&r2=109445&view=diff
==============================================================================
--- cfe/trunk/lib/Basic/SourceManager.cpp (original)
+++ cfe/trunk/lib/Basic/SourceManager.cpp Mon Jul 26 16:36:20 2010
@@ -32,7 +32,8 @@
//===----------------------------------------------------------------------===//
ContentCache::~ContentCache() {
- delete Buffer.getPointer();
+ if (shouldFreeBuffer())
+ delete Buffer.getPointer();
}
/// getSizeBytesMapped - Returns the number of bytes actually mapped for
@@ -51,12 +52,14 @@
: (unsigned) Entry->getSize();
}
-void ContentCache::replaceBuffer(const llvm::MemoryBuffer *B) {
+void ContentCache::replaceBuffer(const llvm::MemoryBuffer *B,
+ bool DoNotFree) {
assert(B != Buffer.getPointer());
- delete Buffer.getPointer();
+ if (shouldFreeBuffer())
+ delete Buffer.getPointer();
Buffer.setPointer(B);
- Buffer.setInt(false);
+ Buffer.setInt(DoNotFree? DoNotFreeFlag : 0);
}
const llvm::MemoryBuffer *ContentCache::getBuffer(Diagnostic &Diag,
@@ -72,7 +75,6 @@
struct stat FileInfo;
Buffer.setPointer(MemoryBuffer::getFile(Entry->getName(), &ErrorStr,
Entry->getSize(), &FileInfo));
- Buffer.setInt(false);
// If we were unable to open the file, then we are in an inconsistent
// situation where the content cache referenced a file which no longer
@@ -99,7 +101,7 @@
Diag.Report(FullSourceLoc(Loc, SM), diag::err_cannot_open_file)
<< Entry->getName() << ErrorStr;
- Buffer.setInt(true);
+ Buffer.setInt(Buffer.getInt() | InvalidFlag);
// FIXME: This conditionalization is horrible, but we see spurious failures
// in the test suite due to this warning and no one has had time to hunt it
@@ -119,14 +121,14 @@
Diag.Report(FullSourceLoc(Loc, SM), diag::err_file_modified)
<< Entry->getName();
- Buffer.setInt(true);
+ Buffer.setInt(Buffer.getInt() | InvalidFlag);
#endif
}
// If the buffer is valid, check to see if it has a UTF Byte Order Mark
// (BOM). We only support UTF-8 without a BOM right now. See
// http://en.wikipedia.org/wiki/Byte_order_mark for more information.
- if (!Buffer.getInt()) {
+ if (!isBufferInvalid()) {
llvm::StringRef BufStr = Buffer.getPointer()->getBuffer();
const char *BOM = 0;
if (BufStr.startswith("\xFE\xBB\xBF"))
@@ -161,7 +163,7 @@
}
if (Invalid)
- *Invalid = Buffer.getInt();
+ *Invalid = isBufferInvalid();
return Buffer.getPointer();
}
@@ -521,12 +523,13 @@
}
bool SourceManager::overrideFileContents(const FileEntry *SourceFile,
- const llvm::MemoryBuffer *Buffer) {
+ const llvm::MemoryBuffer *Buffer,
+ bool DoNotFree) {
const SrcMgr::ContentCache *IR = getOrCreateContentCache(SourceFile);
if (IR == 0)
return true;
- const_cast<SrcMgr::ContentCache *>(IR)->replaceBuffer(Buffer);
+ const_cast<SrcMgr::ContentCache *>(IR)->replaceBuffer(Buffer, DoNotFree);
return false;
}
Modified: cfe/trunk/lib/Frontend/ASTUnit.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/ASTUnit.cpp?rev=109445&r1=109444&r2=109445&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/ASTUnit.cpp (original)
+++ cfe/trunk/lib/Frontend/ASTUnit.cpp Mon Jul 26 16:36:20 2010
@@ -46,6 +46,20 @@
CleanTemporaryFiles();
if (!PreambleFile.empty())
PreambleFile.eraseFromDisk();
+
+ // Free the buffers associated with remapped files. We are required to
+ // perform this operation here because we explicitly request that the
+ // compiler instance *not* free these buffers for each invocation of the
+ // parser.
+ if (Invocation.get()) {
+ PreprocessorOptions &PPOpts = Invocation->getPreprocessorOpts();
+ for (PreprocessorOptions::remapped_file_buffer_iterator
+ FB = PPOpts.remapped_file_buffer_begin(),
+ FBEnd = PPOpts.remapped_file_buffer_end();
+ FB != FBEnd;
+ ++FB)
+ delete FB->second;
+ }
}
void ASTUnit::CleanTemporaryFiles() {
@@ -371,6 +385,17 @@
// Create the source manager.
Clang.setSourceManager(&getSourceManager());
+ // If the main file has been overridden due to the use of a preamble,
+ // make that override happen and introduce the preamble.
+ PreprocessorOptions &PreprocessorOpts = Clang.getPreprocessorOpts();
+ if (OverrideMainBuffer) {
+ PreprocessorOpts.addRemappedFile(OriginalSourceFile, OverrideMainBuffer);
+ PreprocessorOpts.PrecompiledPreambleBytes.first = Preamble.size();
+ PreprocessorOpts.PrecompiledPreambleBytes.second
+ = PreambleEndsAtStartOfLine;
+ PreprocessorOpts.ImplicitPCHInclude = PreambleFile.str();
+ }
+
llvm::OwningPtr<TopLevelDeclTrackerAction> Act;
Act.reset(new TopLevelDeclTrackerAction(*this));
if (!Act->BeginSourceFile(Clang, Clang.getFrontendOpts().Inputs[0].second,
@@ -388,6 +413,11 @@
Target.reset(Clang.takeTarget());
Act->EndSourceFile();
+
+ // Remove the overridden buffer we used for the preamble.
+ if (OverrideMainBuffer)
+ PreprocessorOpts.eraseRemappedFile(
+ PreprocessorOpts.remapped_file_buffer_end() - 1);
Clang.takeDiagnosticClient();
@@ -395,6 +425,11 @@
return false;
error:
+ // Remove the overridden buffer we used for the preamble.
+ if (OverrideMainBuffer)
+ PreprocessorOpts.eraseRemappedFile(
+ PreprocessorOpts.remapped_file_buffer_end() - 1);
+
Clang.takeSourceManager();
Clang.takeFileManager();
Clang.takeDiagnosticClient();
@@ -424,8 +459,10 @@
return P.str();
}
-/// \brief Compute the preamble for the main file, providing
-std::pair<llvm::MemoryBuffer *, unsigned>
+/// \brief Compute the preamble for the main file, providing the source buffer
+/// that corresponds to the main file along with a pair (bytes, start-of-line)
+/// that describes the preamble.
+std::pair<llvm::MemoryBuffer *, std::pair<unsigned, bool> >
ASTUnit::ComputePreamble(CompilerInvocation &Invocation, bool &CreatedBuffer) {
FrontendOptions &FrontendOpts = Invocation.getFrontendOpts();
PreprocessorOptions &PreprocessorOpts
@@ -455,7 +492,8 @@
Buffer = llvm::MemoryBuffer::getFile(M->second);
if (!Buffer)
- return std::make_pair((llvm::MemoryBuffer*)0, 0);
+ return std::make_pair((llvm::MemoryBuffer*)0,
+ std::make_pair(0, true));
CreatedBuffer = true;
// Remove this remapping. We've captured the buffer already.
@@ -495,7 +533,7 @@
if (!Buffer) {
Buffer = llvm::MemoryBuffer::getFile(FrontendOpts.Inputs[0].second);
if (!Buffer)
- return std::make_pair((llvm::MemoryBuffer*)0, 0);
+ return std::make_pair((llvm::MemoryBuffer*)0, std::make_pair(0, true));
CreatedBuffer = true;
}
@@ -512,9 +550,8 @@
memcpy(const_cast<char*>(Result->getBufferStart()),
Old->getBufferStart(), Old->getBufferSize());
memset(const_cast<char*>(Result->getBufferStart()) + Old->getBufferSize(),
- ' ', NewSize - Old->getBufferSize() - 2);
- const_cast<char*>(Result->getBufferEnd())[-2] = '\n';
- const_cast<char*>(Result->getBufferEnd())[-1] = 0;
+ ' ', NewSize - Old->getBufferSize() - 1);
+ const_cast<char*>(Result->getBufferEnd())[-1] = '\n';
if (DeleteOld)
delete Old;
@@ -542,10 +579,10 @@
= PreambleInvocation.getPreprocessorOpts();
bool CreatedPreambleBuffer = false;
- std::pair<llvm::MemoryBuffer *, unsigned> NewPreamble
+ std::pair<llvm::MemoryBuffer *, std::pair<unsigned, bool> > NewPreamble
= ComputePreamble(PreambleInvocation, CreatedPreambleBuffer);
- if (!NewPreamble.second) {
+ if (!NewPreamble.second.first) {
// We couldn't find a preamble in the main source. Clear out the current
// preamble, if we have one. It's obviously no good any more.
Preamble.clear();
@@ -564,10 +601,11 @@
// preamble now that we did before, and that there's enough space in
// the main-file buffer within the precompiled preamble to fit the
// new main file.
- if (Preamble.size() == NewPreamble.second &&
+ if (Preamble.size() == NewPreamble.second.first &&
+ PreambleEndsAtStartOfLine == NewPreamble.second.second &&
NewPreamble.first->getBufferSize() < PreambleReservedSize-2 &&
memcmp(&Preamble[0], NewPreamble.first->getBufferStart(),
- NewPreamble.second) == 0) {
+ NewPreamble.second.first) == 0) {
// The preamble has not changed. We may be able to re-use the precompiled
// preamble.
// FIXME: Check that none of the files used by the preamble have changed.
@@ -593,7 +631,7 @@
// grows.
PreambleReservedSize = NewPreamble.first->getBufferSize();
if (PreambleReservedSize < 4096)
- PreambleReservedSize = 8192;
+ PreambleReservedSize = 8191;
else
PreambleReservedSize *= 2;
@@ -603,14 +641,15 @@
memcpy(const_cast<char*>(PreambleBuffer->getBufferStart()),
NewPreamble.first->getBufferStart(), Preamble.size());
memset(const_cast<char*>(PreambleBuffer->getBufferStart()) + Preamble.size(),
- ' ', PreambleReservedSize - Preamble.size() - 2);
- const_cast<char*>(PreambleBuffer->getBufferEnd())[-1] = 0;
- const_cast<char*>(PreambleBuffer->getBufferEnd())[-2] = '\n';
+ ' ', PreambleReservedSize - Preamble.size() - 1);
+ const_cast<char*>(PreambleBuffer->getBufferEnd())[-1] = '\n';
// Save the preamble text for later; we'll need to compare against it for
// subsequent reparses.
Preamble.assign(NewPreamble.first->getBufferStart(),
- NewPreamble.first->getBufferStart() + NewPreamble.second);
+ NewPreamble.first->getBufferStart()
+ + NewPreamble.second.first);
+ PreambleEndsAtStartOfLine = NewPreamble.second.second;
// Remap the main source file to the preamble buffer.
llvm::sys::PathWithStatus MainFilePath(FrontendOpts.Inputs[0].second);
@@ -734,6 +773,7 @@
AST->CaptureDiagnostics = CaptureDiagnostics;
AST->OnlyLocalDecls = OnlyLocalDecls;
AST->Invocation.reset(CI);
+ CI->getPreprocessorOpts().RetainRemappedFileBuffers = true;
llvm::MemoryBuffer *OverrideMainBuffer = 0;
if (PrecompilePreamble)
Modified: cfe/trunk/lib/Frontend/CompilerInvocation.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/CompilerInvocation.cpp?rev=109445&r1=109444&r2=109445&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/CompilerInvocation.cpp (original)
+++ cfe/trunk/lib/Frontend/CompilerInvocation.cpp Mon Jul 26 16:36:20 2010
@@ -1353,6 +1353,23 @@
Opts.TokenCache = Opts.ImplicitPTHInclude;
Opts.UsePredefines = !Args.hasArg(OPT_undef);
Opts.DetailedRecord = Args.hasArg(OPT_detailed_preprocessing_record);
+
+ if (const Arg *A = Args.getLastArg(OPT_preamble_bytes_EQ)) {
+ llvm::StringRef Value(A->getValue(Args));
+ size_t Comma = Value.find(',');
+ unsigned Bytes = 0;
+ unsigned EndOfLine = 0;
+
+ if (Comma == llvm::StringRef::npos ||
+ Value.substr(0, Comma).getAsInteger(10, Bytes) ||
+ Value.substr(Comma + 1).getAsInteger(10, EndOfLine))
+ Diags.Report(diag::err_drv_preamble_format);
+ else {
+ Opts.PrecompiledPreambleBytes.first = Bytes;
+ Opts.PrecompiledPreambleBytes.second = (EndOfLine != 0);
+ }
+ }
+
// Add macros from the command line.
for (arg_iterator it = Args.filtered_begin(OPT_D, OPT_U),
ie = Args.filtered_end(); it != ie; ++it) {
Modified: cfe/trunk/lib/Frontend/FrontendActions.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/FrontendActions.cpp?rev=109445&r1=109444&r2=109445&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/FrontendActions.cpp (original)
+++ cfe/trunk/lib/Frontend/FrontendActions.cpp Mon Jul 26 16:36:20 2010
@@ -195,7 +195,7 @@
llvm::MemoryBuffer *Buffer = llvm::MemoryBuffer::getFile(getCurrentFile());
if (Buffer) {
- unsigned Preamble = Lexer::ComputePreamble(Buffer);
+ unsigned Preamble = Lexer::ComputePreamble(Buffer).first;
llvm::outs().write(Buffer->getBufferStart(), Preamble);
delete Buffer;
}
Modified: cfe/trunk/lib/Frontend/InitPreprocessor.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/InitPreprocessor.cpp?rev=109445&r1=109444&r2=109445&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/InitPreprocessor.cpp (original)
+++ cfe/trunk/lib/Frontend/InitPreprocessor.cpp Mon Jul 26 16:36:20 2010
@@ -489,13 +489,15 @@
if (!FromFile) {
Diags.Report(diag::err_fe_remap_missing_from_file)
<< Remap->first;
- delete Remap->second;
+ if (!InitOpts.RetainRemappedFileBuffers)
+ delete Remap->second;
continue;
}
// Override the contents of the "from" file with the contents of
// the "to" file.
- SourceMgr.overrideFileContents(FromFile, Remap->second);
+ SourceMgr.overrideFileContents(FromFile, Remap->second,
+ InitOpts.RetainRemappedFileBuffers);
}
// Remap files in the source manager (with other files).
@@ -596,6 +598,10 @@
if (!PP.getLangOptions().AsmPreprocessor)
Builder.append("# 1 \"<built-in>\" 2");
+ // Instruct the preprocessor to skip the preamble.
+ PP.setSkipMainFilePreamble(InitOpts.PrecompiledPreambleBytes.first,
+ InitOpts.PrecompiledPreambleBytes.second);
+
// Copy PredefinedBuffer into the Preprocessor.
PP.setPredefines(Predefines.str());
Modified: cfe/trunk/lib/Lex/Lexer.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Lexer.cpp?rev=109445&r1=109444&r2=109445&view=diff
==============================================================================
--- cfe/trunk/lib/Lex/Lexer.cpp (original)
+++ cfe/trunk/lib/Lex/Lexer.cpp Mon Jul 26 16:36:20 2010
@@ -310,7 +310,8 @@
};
}
-unsigned Lexer::ComputePreamble(const llvm::MemoryBuffer *Buffer) {
+std::pair<unsigned, bool>
+Lexer::ComputePreamble(const llvm::MemoryBuffer *Buffer) {
// Create a lexer starting at the beginning of the file. Note that we use a
// "fake" file source location at offset 1 so that the lexer will track our
// position within the file.
@@ -422,7 +423,9 @@
} while (true);
SourceLocation End = IfCount? IfStartTok.getLocation() : TheTok.getLocation();
- return End.getRawEncoding() - StartLoc.getRawEncoding();
+ return std::make_pair(End.getRawEncoding() - StartLoc.getRawEncoding(),
+ IfCount? IfStartTok.isAtStartOfLine()
+ : TheTok.isAtStartOfLine());
}
//===----------------------------------------------------------------------===//
@@ -825,6 +828,14 @@
// Helper methods for lexing.
//===----------------------------------------------------------------------===//
+/// \brief Routine that indiscriminately skips bytes in the source file.
+void Lexer::SkipBytes(unsigned Bytes, bool StartOfLine) {
+ BufferPtr += Bytes;
+ if (BufferPtr > BufferEnd)
+ BufferPtr = BufferEnd;
+ IsAtStartOfLine = StartOfLine;
+}
+
void Lexer::LexIdentifier(Token &Result, const char *CurPtr) {
// Match [_A-Za-z0-9]*, we have already matched [_A-Za-z$]
unsigned Size;
Modified: cfe/trunk/lib/Lex/Preprocessor.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Preprocessor.cpp?rev=109445&r1=109444&r2=109445&view=diff
==============================================================================
--- cfe/trunk/lib/Lex/Preprocessor.cpp (original)
+++ cfe/trunk/lib/Lex/Preprocessor.cpp Mon Jul 26 16:36:20 2010
@@ -54,7 +54,8 @@
: Diags(&diags), Features(opts), Target(target),FileMgr(Headers.getFileMgr()),
SourceMgr(SM), HeaderInfo(Headers), ExternalSource(0),
Identifiers(opts, IILookup), BuiltinInfo(Target), CodeCompletionFile(0),
- CurPPLexer(0), CurDirLookup(0), Callbacks(0), MacroArgCache(0), Record(0) {
+ SkipMainFilePreamble(0, true), CurPPLexer(0), CurDirLookup(0), Callbacks(0),
+ MacroArgCache(0), Record(0) {
ScratchBuf = new ScratchBuffer(SourceMgr);
CounterValue = 0; // __COUNTER__ starts at 0.
OwnsHeaderSearch = OwnsHeaders;
@@ -508,6 +509,12 @@
// Enter the main file source buffer.
EnterSourceFile(MainFileID, 0, SourceLocation());
+ // If we've been asked to skip bytes in the main file (e.g., as part of a
+ // precompiled preamble), do so now.
+ if (SkipMainFilePreamble.first > 0)
+ CurLexer->SkipBytes(SkipMainFilePreamble.first,
+ SkipMainFilePreamble.second);
+
// Tell the header info that the main file was entered. If the file is later
// #imported, it won't be re-entered.
if (const FileEntry *FE = SourceMgr.getFileEntryForID(MainFileID))
Added: cfe/trunk/test/PCH/Inputs/preamble.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/PCH/Inputs/preamble.h?rev=109445&view=auto
==============================================================================
--- cfe/trunk/test/PCH/Inputs/preamble.h (added)
+++ cfe/trunk/test/PCH/Inputs/preamble.h Mon Jul 26 16:36:20 2010
@@ -0,0 +1 @@
+int f(int);
Added: cfe/trunk/test/PCH/preamble.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/PCH/preamble.c?rev=109445&view=auto
==============================================================================
--- cfe/trunk/test/PCH/preamble.c (added)
+++ cfe/trunk/test/PCH/preamble.c Mon Jul 26 16:36:20 2010
@@ -0,0 +1,21 @@
+// Check that using the preamble option actually skips the preamble.
+
+// RUN: %clang_cc1 -emit-pch -o %t %S/Inputs/preamble.h
+// RUN: %clang_cc1 -include-pch %t -preamble-bytes=278,1 -DFOO=f -verify %s
+
+float f(int); // Not an error, because we skip this via the preamble!
+
+
+
+
+
+
+
+
+
+
+
+
+int g(int x) {
+ return FOO(x);
+}
More information about the cfe-commits
mailing list