[cfe-commits] r163022 - in /cfe/trunk: include/clang/Lex/Token.h lib/Lex/PPMacroExpansion.cpp lib/Lex/TokenLexer.cpp test/Preprocessor/microsoft-ext.c
Nico Weber
thakis at chromium.org
Wed Sep 26 01:21:13 PDT 2012
Reverted in r164672. The test you added surprisingly passes even with
the code changes reverted, so I kept the test around.
Nico
On Wed, Sep 26, 2012 at 4:56 PM, Nico Weber <thakis at chromium.org> wrote:
> In case anyone was wondering, here's the actual diff to PPMacroExpansion.cpp:
>
> Nicos-MacBook-Pro:clang thakis$ diff -u ~/triton-before.txt ~/triton-after.txt
> --- /Users/thakis/triton-before.txt 2012-09-26 16:54:53.000000000 +0900
> +++ /Users/thakis/triton-after.txt 2012-09-26 16:55:04.000000000 +0900
> @@ -403,7 +403,11 @@
> }
> } else if (Tok.is(tok::l_paren)) {
> ++NumParens;
> - } else if (Tok.is(tok::comma) && NumParens == 0) {
> + // In Microsoft-compatibility mode, commas from nested macro expan-
> + // sions should not be considered as argument separators. We test
> + // for this with the IgnoredComma token flag.
> + } else if (Tok.is(tok::comma)
> + && !(Tok.getFlags() & Token::IgnoredComma) && NumParens == 0) {
> // Comma ends this argument if there are more fixed arguments expected.
> // However, if this is a variadic macro, and this is part of the
> // variadic part, then the comma is just an argument token.
> @@ -1171,3 +1175,4 @@
> WarnUnusedMacroLocs.erase(MI->getDefinitionLoc());
> MI->setIsUsed(true);
> }
>
>
>
> This change caused PR13924, I might roll it out.
>
> On Wed, Sep 5, 2012 at 2:10 AM, João Matos <ripzonetriton at gmail.com> wrote:
>> Will do. Yes, the this line endings problem was really annoying. I was not
>> aware VS didn't respect them. I've introduced some measures locally so
>> hopefully this won't happen in the future.
>>
>> Regarding your concerns, I didn't benchmark this, but I did had the
>> performance nature of the preprocessor when writing the code, and tried to
>> make minimal changes to not affect it.
>>
>>
>> On Tue, Sep 4, 2012 at 5:56 PM, Chandler Carruth <chandlerc at google.com>
>> wrote:
>>>
>>> On Fri, Aug 31, 2012 at 5:10 PM, Joao Matos <ripzonetriton at gmail.com>
>>> wrote:
>>>>
>>>> Author: triton
>>>> Date: Fri Aug 31 16:10:54 2012
>>>> New Revision: 163022
>>>>
>>>> URL: http://llvm.org/viewvc/llvm-project?rev=163022&view=rev
>>>> Log:
>>>> Emulate MSVC's preprocessor macro argument separator behavior by not
>>>> considering commas from nested macro expansions as argument separators.
>>>> Fixes parsing of VS 2012 headers.
>>>
>>>
>>> Arg, no one can review this patch now because the line endings got
>>> thrashed.
>>>
>>> In the future, if you have trouble with line endings, *please* revert the
>>> patch first, and then commit a new patch with only the intended edits. That
>>> way our post-commit review can proceed normally.
>>>
>>> That said, I have some significant concerns about this patch that didn't
>>> come up in the initial review, and I think might merit reverting the patch
>>> temporarily until we understand them.
>>>
>>> What is the performance impact of this patch? The performance of the
>>> preprocessor is *incredibly* sensitive. Have you benchmarked the patch in
>>> microsoft mode and non-microsoft mode, before and after, with some of the
>>> heavy users of preprocess macros? The single-source GCC version, or some of
>>> the Boost preprocessor libraries might make excellent benchmarks.
>>>
>>> Also, more test cases would seem to be in order. How does this interact
>>> with token pasting? How does it interact with variadic macros which also
>>> have strange comma behavior?
>>>
>>>>
>>>>
>>>> Added:
>>>> cfe/trunk/test/Preprocessor/microsoft-ext.c
>>>> Modified:
>>>> cfe/trunk/include/clang/Lex/Token.h
>>>> cfe/trunk/lib/Lex/PPMacroExpansion.cpp
>>>> cfe/trunk/lib/Lex/TokenLexer.cpp
>>>>
>>>> Modified: cfe/trunk/include/clang/Lex/Token.h
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/Token.h?rev=163022&r1=163021&r2=163022&view=diff
>>>>
>>>> ==============================================================================
>>>> --- cfe/trunk/include/clang/Lex/Token.h (original)
>>>> +++ cfe/trunk/include/clang/Lex/Token.h Fri Aug 31 16:10:54 2012
>>>> @@ -76,7 +76,8 @@
>>>> DisableExpand = 0x04, // This identifier may never be macro
>>>> expanded.
>>>> NeedsCleaning = 0x08, // Contained an escaped newline or trigraph.
>>>> LeadingEmptyMacro = 0x10, // Empty macro exists before this token.
>>>> - HasUDSuffix = 0x20 // This string or character literal has a
>>>> ud-suffix.
>>>> + HasUDSuffix = 0x20, // This string or character literal has a
>>>> ud-suffix.
>>>> + IgnoredComma = 0x40 // Flags ignored commas from nested macro
>>>> expansions.
>>>> };
>>>>
>>>> tok::TokenKind getKind() const { return (tok::TokenKind)Kind; }
>>>>
>>>> Modified: cfe/trunk/lib/Lex/PPMacroExpansion.cpp
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPMacroExpansion.cpp?rev=163022&r1=163021&r2=163022&view=diff
>>>>
>>>> ==============================================================================
>>>> --- cfe/trunk/lib/Lex/PPMacroExpansion.cpp (original)
>>>> +++ cfe/trunk/lib/Lex/PPMacroExpansion.cpp Fri Aug 31 16:10:54 2012
>>>> @@ -1,1173 +1,1177 @@
>>>> -//===--- MacroExpansion.cpp - Top level Macro Expansion
>>>> -------------------===//
>>>> -//
>>>> -// The LLVM Compiler Infrastructure
>>>> -//
>>>> -// This file is distributed under the University of Illinois Open Source
>>>> -// License. See LICENSE.TXT for details.
>>>> -//
>>>>
>>>> -//===----------------------------------------------------------------------===//
>>>> -//
>>>> -// This file implements the top level handling of macro expasion for the
>>>> -// preprocessor.
>>>> -//
>>>>
>>>> -//===----------------------------------------------------------------------===//
>>>> -
>>>> -#include "clang/Lex/Preprocessor.h"
>>>> -#include "MacroArgs.h"
>>>> -#include "clang/Lex/MacroInfo.h"
>>>> -#include "clang/Basic/SourceManager.h"
>>>> -#include "clang/Basic/FileManager.h"
>>>> -#include "clang/Basic/TargetInfo.h"
>>>> -#include "clang/Lex/LexDiagnostic.h"
>>>> -#include "clang/Lex/CodeCompletionHandler.h"
>>>> -#include "clang/Lex/ExternalPreprocessorSource.h"
>>>> -#include "clang/Lex/LiteralSupport.h"
>>>> -#include "llvm/ADT/StringSwitch.h"
>>>> -#include "llvm/ADT/STLExtras.h"
>>>> -#include "llvm/Config/llvm-config.h"
>>>> -#include "llvm/Support/raw_ostream.h"
>>>> -#include "llvm/Support/ErrorHandling.h"
>>>> -#include <cstdio>
>>>> -#include <ctime>
>>>> -using namespace clang;
>>>> -
>>>> -MacroInfo *Preprocessor::getInfoForMacro(IdentifierInfo *II) const {
>>>> - assert(II->hasMacroDefinition() && "Identifier is not a macro!");
>>>> -
>>>> - macro_iterator Pos = Macros.find(II);
>>>> - if (Pos == Macros.end()) {
>>>> - // Load this macro from the external source.
>>>> - getExternalSource()->LoadMacroDefinition(II);
>>>> - Pos = Macros.find(II);
>>>> - }
>>>> - assert(Pos != Macros.end() && "Identifier macro info is missing!");
>>>> - assert(Pos->second->getUndefLoc().isInvalid() && "Macro is
>>>> undefined!");
>>>> - return Pos->second;
>>>> -}
>>>> -
>>>> -/// setMacroInfo - Specify a macro for this identifier.
>>>> -///
>>>> -void Preprocessor::setMacroInfo(IdentifierInfo *II, MacroInfo *MI,
>>>> - bool LoadedFromAST) {
>>>> - assert(MI && "MacroInfo should be non-zero!");
>>>> - MI->setPreviousDefinition(Macros[II]);
>>>> - Macros[II] = MI;
>>>> - II->setHasMacroDefinition(true);
>>>> - if (II->isFromAST() && !LoadedFromAST)
>>>> - II->setChangedSinceDeserialization();
>>>> -}
>>>> -
>>>> -/// \brief Undefine a macro for this identifier.
>>>> -void Preprocessor::clearMacroInfo(IdentifierInfo *II) {
>>>> - assert(II->hasMacroDefinition() && "Macro is not defined!");
>>>> - assert(Macros[II]->getUndefLoc().isValid() && "Macro is still
>>>> defined!");
>>>> - II->setHasMacroDefinition(false);
>>>> - if (II->isFromAST())
>>>> - II->setChangedSinceDeserialization();
>>>> -}
>>>> -
>>>> -/// RegisterBuiltinMacro - Register the specified identifier in the
>>>> identifier
>>>> -/// table and mark it as a builtin macro to be expanded.
>>>> -static IdentifierInfo *RegisterBuiltinMacro(Preprocessor &PP, const char
>>>> *Name){
>>>> - // Get the identifier.
>>>> - IdentifierInfo *Id = PP.getIdentifierInfo(Name);
>>>> -
>>>> - // Mark it as being a macro that is builtin.
>>>> - MacroInfo *MI = PP.AllocateMacroInfo(SourceLocation());
>>>> - MI->setIsBuiltinMacro();
>>>> - PP.setMacroInfo(Id, MI);
>>>> - return Id;
>>>> -}
>>>> -
>>>> -
>>>> -/// RegisterBuiltinMacros - Register builtin macros, such as __LINE__
>>>> with the
>>>> -/// identifier table.
>>>> -void Preprocessor::RegisterBuiltinMacros() {
>>>> - Ident__LINE__ = RegisterBuiltinMacro(*this, "__LINE__");
>>>> - Ident__FILE__ = RegisterBuiltinMacro(*this, "__FILE__");
>>>> - Ident__DATE__ = RegisterBuiltinMacro(*this, "__DATE__");
>>>> - Ident__TIME__ = RegisterBuiltinMacro(*this, "__TIME__");
>>>> - Ident__COUNTER__ = RegisterBuiltinMacro(*this, "__COUNTER__");
>>>> - Ident_Pragma = RegisterBuiltinMacro(*this, "_Pragma");
>>>> -
>>>> - // GCC Extensions.
>>>> - Ident__BASE_FILE__ = RegisterBuiltinMacro(*this, "__BASE_FILE__");
>>>> - Ident__INCLUDE_LEVEL__ = RegisterBuiltinMacro(*this,
>>>> "__INCLUDE_LEVEL__");
>>>> - Ident__TIMESTAMP__ = RegisterBuiltinMacro(*this, "__TIMESTAMP__");
>>>> -
>>>> - // Clang Extensions.
>>>> - Ident__has_feature = RegisterBuiltinMacro(*this,
>>>> "__has_feature");
>>>> - Ident__has_extension = RegisterBuiltinMacro(*this,
>>>> "__has_extension");
>>>> - Ident__has_builtin = RegisterBuiltinMacro(*this,
>>>> "__has_builtin");
>>>> - Ident__has_attribute = RegisterBuiltinMacro(*this,
>>>> "__has_attribute");
>>>> - Ident__has_include = RegisterBuiltinMacro(*this,
>>>> "__has_include");
>>>> - Ident__has_include_next = RegisterBuiltinMacro(*this,
>>>> "__has_include_next");
>>>> - Ident__has_warning = RegisterBuiltinMacro(*this,
>>>> "__has_warning");
>>>> -
>>>> - // Microsoft Extensions.
>>>> - if (LangOpts.MicrosoftExt)
>>>> - Ident__pragma = RegisterBuiltinMacro(*this, "__pragma");
>>>> - else
>>>> - Ident__pragma = 0;
>>>> -}
>>>> -
>>>> -/// isTrivialSingleTokenExpansion - Return true if MI, which has a
>>>> single token
>>>> -/// in its expansion, currently expands to that token literally.
>>>> -static bool isTrivialSingleTokenExpansion(const MacroInfo *MI,
>>>> - const IdentifierInfo
>>>> *MacroIdent,
>>>> - Preprocessor &PP) {
>>>> - IdentifierInfo *II = MI->getReplacementToken(0).getIdentifierInfo();
>>>> -
>>>> - // If the token isn't an identifier, it's always literally expanded.
>>>> - if (II == 0) return true;
>>>> -
>>>> - // If the information about this identifier is out of date, update it
>>>> from
>>>> - // the external source.
>>>> - if (II->isOutOfDate())
>>>> - PP.getExternalSource()->updateOutOfDateIdentifier(*II);
>>>> -
>>>> - // If the identifier is a macro, and if that macro is enabled, it may
>>>> be
>>>> - // expanded so it's not a trivial expansion.
>>>> - if (II->hasMacroDefinition() && PP.getMacroInfo(II)->isEnabled() &&
>>>> - // Fast expanding "#define X X" is ok, because X would be
>>>> disabled.
>>>> - II != MacroIdent)
>>>> - return false;
>>>> -
>>>> - // If this is an object-like macro invocation, it is safe to trivially
>>>> expand
>>>> - // it.
>>>> - if (MI->isObjectLike()) return true;
>>>> -
>>>> - // If this is a function-like macro invocation, it's safe to trivially
>>>> expand
>>>> - // as long as the identifier is not a macro argument.
>>>> - for (MacroInfo::arg_iterator I = MI->arg_begin(), E = MI->arg_end();
>>>> - I != E; ++I)
>>>> - if (*I == II)
>>>> - return false; // Identifier is a macro argument.
>>>> -
>>>> - return true;
>>>> -}
>>>> -
>>>> -
>>>> -/// isNextPPTokenLParen - Determine whether the next preprocessor token
>>>> to be
>>>> -/// lexed is a '('. If so, consume the token and return true, if not,
>>>> this
>>>> -/// method should have no observable side-effect on the lexed tokens.
>>>> -bool Preprocessor::isNextPPTokenLParen() {
>>>> - // Do some quick tests for rejection cases.
>>>> - unsigned Val;
>>>> - if (CurLexer)
>>>> - Val = CurLexer->isNextPPTokenLParen();
>>>> - else if (CurPTHLexer)
>>>> - Val = CurPTHLexer->isNextPPTokenLParen();
>>>> - else
>>>> - Val = CurTokenLexer->isNextTokenLParen();
>>>> -
>>>> - if (Val == 2) {
>>>> - // We have run off the end. If it's a source file we don't
>>>> - // examine enclosing ones (C99 5.1.1.2p4). Otherwise walk up the
>>>> - // macro stack.
>>>> - if (CurPPLexer)
>>>> - return false;
>>>> - for (unsigned i = IncludeMacroStack.size(); i != 0; --i) {
>>>> - IncludeStackInfo &Entry = IncludeMacroStack[i-1];
>>>> - if (Entry.TheLexer)
>>>> - Val = Entry.TheLexer->isNextPPTokenLParen();
>>>> - else if (Entry.ThePTHLexer)
>>>> - Val = Entry.ThePTHLexer->isNextPPTokenLParen();
>>>> - else
>>>> - Val = Entry.TheTokenLexer->isNextTokenLParen();
>>>> -
>>>> - if (Val != 2)
>>>> - break;
>>>> -
>>>> - // Ran off the end of a source file?
>>>> - if (Entry.ThePPLexer)
>>>> - return false;
>>>> - }
>>>> - }
>>>> -
>>>> - // Okay, if we know that the token is a '(', lex it and return.
>>>> Otherwise we
>>>> - // have found something that isn't a '(' or we found the end of the
>>>> - // translation unit. In either case, return false.
>>>> - return Val == 1;
>>>> -}
>>>> -
>>>> -/// HandleMacroExpandedIdentifier - If an identifier token is read that
>>>> is to be
>>>> -/// expanded as a macro, handle it and return the next token as
>>>> 'Identifier'.
>>>> -bool Preprocessor::HandleMacroExpandedIdentifier(Token &Identifier,
>>>> - MacroInfo *MI) {
>>>> - // If this is a macro expansion in the "#if !defined(x)" line for the
>>>> file,
>>>> - // then the macro could expand to different things in other contexts,
>>>> we need
>>>> - // to disable the optimization in this case.
>>>> - if (CurPPLexer) CurPPLexer->MIOpt.ExpandedMacro();
>>>> -
>>>> - // If this is a builtin macro, like __LINE__ or _Pragma, handle it
>>>> specially.
>>>> - if (MI->isBuiltinMacro()) {
>>>> - if (Callbacks) Callbacks->MacroExpands(Identifier, MI,
>>>> - Identifier.getLocation());
>>>> - ExpandBuiltinMacro(Identifier);
>>>> - return false;
>>>> - }
>>>> -
>>>> - /// Args - If this is a function-like macro expansion, this contains,
>>>> - /// for each macro argument, the list of tokens that were provided to
>>>> the
>>>> - /// invocation.
>>>> - MacroArgs *Args = 0;
>>>> -
>>>> - // Remember where the end of the expansion occurred. For an
>>>> object-like
>>>> - // macro, this is the identifier. For a function-like macro, this is
>>>> the ')'.
>>>> - SourceLocation ExpansionEnd = Identifier.getLocation();
>>>> -
>>>> - // If this is a function-like macro, read the arguments.
>>>> - if (MI->isFunctionLike()) {
>>>> - // C99 6.10.3p10: If the preprocessing token immediately after the
>>>> macro
>>>> - // name isn't a '(', this macro should not be expanded.
>>>> - if (!isNextPPTokenLParen())
>>>> - return true;
>>>> -
>>>> - // Remember that we are now parsing the arguments to a macro
>>>> invocation.
>>>> - // Preprocessor directives used inside macro arguments are not
>>>> portable, and
>>>> - // this enables the warning.
>>>> - InMacroArgs = true;
>>>> - Args = ReadFunctionLikeMacroArgs(Identifier, MI, ExpansionEnd);
>>>> -
>>>> - // Finished parsing args.
>>>> - InMacroArgs = false;
>>>> -
>>>> - // If there was an error parsing the arguments, bail out.
>>>> - if (Args == 0) return false;
>>>> -
>>>> - ++NumFnMacroExpanded;
>>>> - } else {
>>>> - ++NumMacroExpanded;
>>>> - }
>>>> -
>>>> - // Notice that this macro has been used.
>>>> - markMacroAsUsed(MI);
>>>> -
>>>> - // Remember where the token is expanded.
>>>> - SourceLocation ExpandLoc = Identifier.getLocation();
>>>> - SourceRange ExpansionRange(ExpandLoc, ExpansionEnd);
>>>> -
>>>> - if (Callbacks) {
>>>> - if (InMacroArgs) {
>>>> - // We can have macro expansion inside a conditional directive
>>>> while
>>>> - // reading the function macro arguments. To ensure, in that case,
>>>> that
>>>> - // MacroExpands callbacks still happen in source order, queue this
>>>> - // callback to have it happen after the function macro callback.
>>>> - DelayedMacroExpandsCallbacks.push_back(
>>>> - MacroExpandsInfo(Identifier, MI,
>>>> ExpansionRange));
>>>> - } else {
>>>> - Callbacks->MacroExpands(Identifier, MI, ExpansionRange);
>>>> - if (!DelayedMacroExpandsCallbacks.empty()) {
>>>> - for (unsigned i=0, e = DelayedMacroExpandsCallbacks.size();
>>>> i!=e; ++i) {
>>>> - MacroExpandsInfo &Info = DelayedMacroExpandsCallbacks[i];
>>>> - Callbacks->MacroExpands(Info.Tok, Info.MI, Info.Range);
>>>> - }
>>>> - DelayedMacroExpandsCallbacks.clear();
>>>> - }
>>>> - }
>>>> - }
>>>> -
>>>> - // If we started lexing a macro, enter the macro expansion body.
>>>> -
>>>> - // If this macro expands to no tokens, don't bother to push it onto
>>>> the
>>>> - // expansion stack, only to take it right back off.
>>>> - if (MI->getNumTokens() == 0) {
>>>> - // No need for arg info.
>>>> - if (Args) Args->destroy(*this);
>>>> -
>>>> - // Ignore this macro use, just return the next token in the current
>>>> - // buffer.
>>>> - bool HadLeadingSpace = Identifier.hasLeadingSpace();
>>>> - bool IsAtStartOfLine = Identifier.isAtStartOfLine();
>>>> -
>>>> - Lex(Identifier);
>>>> -
>>>> - // If the identifier isn't on some OTHER line, inherit the leading
>>>> - // whitespace/first-on-a-line property of this token. This handles
>>>> - // stuff like "! XX," -> "! ," and " XX," -> " ,", when XX is
>>>> - // empty.
>>>> - if (!Identifier.isAtStartOfLine()) {
>>>> - if (IsAtStartOfLine) Identifier.setFlag(Token::StartOfLine);
>>>> - if (HadLeadingSpace) Identifier.setFlag(Token::LeadingSpace);
>>>> - }
>>>> - Identifier.setFlag(Token::LeadingEmptyMacro);
>>>> - ++NumFastMacroExpanded;
>>>> - return false;
>>>> -
>>>> - } else if (MI->getNumTokens() == 1 &&
>>>> - isTrivialSingleTokenExpansion(MI,
>>>> Identifier.getIdentifierInfo(),
>>>> - *this)) {
>>>> - // Otherwise, if this macro expands into a single trivially-expanded
>>>> - // token: expand it now. This handles common cases like
>>>> - // "#define VAL 42".
>>>> -
>>>> - // No need for arg info.
>>>> - if (Args) Args->destroy(*this);
>>>> -
>>>> - // Propagate the isAtStartOfLine/hasLeadingSpace markers of the
>>>> macro
>>>> - // identifier to the expanded token.
>>>> - bool isAtStartOfLine = Identifier.isAtStartOfLine();
>>>> - bool hasLeadingSpace = Identifier.hasLeadingSpace();
>>>> -
>>>> - // Replace the result token.
>>>> - Identifier = MI->getReplacementToken(0);
>>>> -
>>>> - // Restore the StartOfLine/LeadingSpace markers.
>>>> - Identifier.setFlagValue(Token::StartOfLine , isAtStartOfLine);
>>>> - Identifier.setFlagValue(Token::LeadingSpace, hasLeadingSpace);
>>>> -
>>>> - // Update the tokens location to include both its expansion and
>>>> physical
>>>> - // locations.
>>>> - SourceLocation Loc =
>>>> - SourceMgr.createExpansionLoc(Identifier.getLocation(), ExpandLoc,
>>>> - ExpansionEnd,Identifier.getLength());
>>>> - Identifier.setLocation(Loc);
>>>> -
>>>> - // If this is a disabled macro or #define X X, we must mark the
>>>> result as
>>>> - // unexpandable.
>>>> - if (IdentifierInfo *NewII = Identifier.getIdentifierInfo()) {
>>>> - if (MacroInfo *NewMI = getMacroInfo(NewII))
>>>> - if (!NewMI->isEnabled() || NewMI == MI) {
>>>> - Identifier.setFlag(Token::DisableExpand);
>>>> - Diag(Identifier, diag::pp_disabled_macro_expansion);
>>>> - }
>>>> - }
>>>> -
>>>> - // Since this is not an identifier token, it can't be macro
>>>> expanded, so
>>>> - // we're done.
>>>> - ++NumFastMacroExpanded;
>>>> - return false;
>>>> - }
>>>> -
>>>> - // Start expanding the macro.
>>>> - EnterMacro(Identifier, ExpansionEnd, MI, Args);
>>>> -
>>>> - // Now that the macro is at the top of the include stack, ask the
>>>> - // preprocessor to read the next token from it.
>>>> - Lex(Identifier);
>>>> - return false;
>>>> -}
>>>> -
>>>> -/// ReadFunctionLikeMacroArgs - After reading "MACRO" and knowing that
>>>> the next
>>>> -/// token is the '(' of the macro, this method is invoked to read all of
>>>> the
>>>> -/// actual arguments specified for the macro invocation. This returns
>>>> null on
>>>> -/// error.
>>>> -MacroArgs *Preprocessor::ReadFunctionLikeMacroArgs(Token &MacroName,
>>>> - MacroInfo *MI,
>>>> - SourceLocation
>>>> &MacroEnd) {
>>>> - // The number of fixed arguments to parse.
>>>> - unsigned NumFixedArgsLeft = MI->getNumArgs();
>>>> - bool isVariadic = MI->isVariadic();
>>>> -
>>>> - // Outer loop, while there are more arguments, keep reading them.
>>>> - Token Tok;
>>>> -
>>>> - // Read arguments as unexpanded tokens. This avoids issues, e.g.,
>>>> where
>>>> - // an argument value in a macro could expand to ',' or '(' or ')'.
>>>> - LexUnexpandedToken(Tok);
>>>> - assert(Tok.is(tok::l_paren) && "Error computing l-paren-ness?");
>>>> -
>>>> - // ArgTokens - Build up a list of tokens that make up each argument.
>>>> Each
>>>> - // argument is separated by an EOF token. Use a SmallVector so we can
>>>> avoid
>>>> - // heap allocations in the common case.
>>>> - SmallVector<Token, 64> ArgTokens;
>>>> -
>>>> - unsigned NumActuals = 0;
>>>> - while (Tok.isNot(tok::r_paren)) {
>>>> - assert((Tok.is(tok::l_paren) || Tok.is(tok::comma)) &&
>>>> - "only expect argument separators here");
>>>> -
>>>> - unsigned ArgTokenStart = ArgTokens.size();
>>>> - SourceLocation ArgStartLoc = Tok.getLocation();
>>>> -
>>>> - // C99 6.10.3p11: Keep track of the number of l_parens we have seen.
>>>> Note
>>>> - // that we already consumed the first one.
>>>> - unsigned NumParens = 0;
>>>> -
>>>> - while (1) {
>>>> - // Read arguments as unexpanded tokens. This avoids issues, e.g.,
>>>> where
>>>> - // an argument value in a macro could expand to ',' or '(' or ')'.
>>>> - LexUnexpandedToken(Tok);
>>>> -
>>>> - if (Tok.is(tok::eof) || Tok.is(tok::eod)) { // "#if f(<eof>" &
>>>> "#if f(\n"
>>>> - Diag(MacroName, diag::err_unterm_macro_invoc);
>>>> - // Do not lose the EOF/EOD. Return it to the client.
>>>> - MacroName = Tok;
>>>> - return 0;
>>>> - } else if (Tok.is(tok::r_paren)) {
>>>> - // If we found the ) token, the macro arg list is done.
>>>> - if (NumParens-- == 0) {
>>>> - MacroEnd = Tok.getLocation();
>>>> - break;
>>>> - }
>>>> - } else if (Tok.is(tok::l_paren)) {
>>>> - ++NumParens;
>>>> - } else if (Tok.is(tok::comma) && NumParens == 0) {
>>>> - // Comma ends this argument if there are more fixed arguments
>>>> expected.
>>>> - // However, if this is a variadic macro, and this is part of the
>>>> - // variadic part, then the comma is just an argument token.
>>>> - if (!isVariadic) break;
>>>> - if (NumFixedArgsLeft > 1)
>>>> - break;
>>>> - } else if (Tok.is(tok::comment) && !KeepMacroComments) {
>>>> - // If this is a comment token in the argument list and we're
>>>> just in
>>>> - // -C mode (not -CC mode), discard the comment.
>>>> - continue;
>>>> - } else if (Tok.getIdentifierInfo() != 0) {
>>>> - // Reading macro arguments can cause macros that we are
>>>> currently
>>>> - // expanding from to be popped off the expansion stack. Doing
>>>> so causes
>>>> - // them to be reenabled for expansion. Here we record whether
>>>> any
>>>> - // identifiers we lex as macro arguments correspond to disabled
>>>> macros.
>>>> - // If so, we mark the token as noexpand. This is a subtle
>>>> aspect of
>>>> - // C99 6.10.3.4p2.
>>>> - if (MacroInfo *MI = getMacroInfo(Tok.getIdentifierInfo()))
>>>> - if (!MI->isEnabled())
>>>> - Tok.setFlag(Token::DisableExpand);
>>>> - } else if (Tok.is(tok::code_completion)) {
>>>> - if (CodeComplete)
>>>> -
>>>> CodeComplete->CodeCompleteMacroArgument(MacroName.getIdentifierInfo(),
>>>> - MI, NumActuals);
>>>> - // Don't mark that we reached the code-completion point because
>>>> the
>>>> - // parser is going to handle the token and there will be another
>>>> - // code-completion callback.
>>>> - }
>>>> -
>>>> - ArgTokens.push_back(Tok);
>>>> - }
>>>> -
>>>> - // If this was an empty argument list foo(), don't add this as an
>>>> empty
>>>> - // argument.
>>>> - if (ArgTokens.empty() && Tok.getKind() == tok::r_paren)
>>>> - break;
>>>> -
>>>> - // If this is not a variadic macro, and too many args were
>>>> specified, emit
>>>> - // an error.
>>>> - if (!isVariadic && NumFixedArgsLeft == 0) {
>>>> - if (ArgTokens.size() != ArgTokenStart)
>>>> - ArgStartLoc = ArgTokens[ArgTokenStart].getLocation();
>>>> -
>>>> - // Emit the diagnostic at the macro name in case there is a
>>>> missing ).
>>>> - // Emitting it at the , could be far away from the macro name.
>>>> - Diag(ArgStartLoc, diag::err_too_many_args_in_macro_invoc);
>>>> - return 0;
>>>> - }
>>>> -
>>>> - // Empty arguments are standard in C99 and C++0x, and are supported
>>>> as an extension in
>>>> - // other modes.
>>>> - if (ArgTokens.size() == ArgTokenStart && !LangOpts.C99)
>>>> - Diag(Tok, LangOpts.CPlusPlus0x ?
>>>> - diag::warn_cxx98_compat_empty_fnmacro_arg :
>>>> - diag::ext_empty_fnmacro_arg);
>>>> -
>>>> - // Add a marker EOF token to the end of the token list for this
>>>> argument.
>>>> - Token EOFTok;
>>>> - EOFTok.startToken();
>>>> - EOFTok.setKind(tok::eof);
>>>> - EOFTok.setLocation(Tok.getLocation());
>>>> - EOFTok.setLength(0);
>>>> - ArgTokens.push_back(EOFTok);
>>>> - ++NumActuals;
>>>> - assert(NumFixedArgsLeft != 0 && "Too many arguments parsed");
>>>> - --NumFixedArgsLeft;
>>>> - }
>>>> -
>>>> - // Okay, we either found the r_paren. Check to see if we parsed too
>>>> few
>>>> - // arguments.
>>>> - unsigned MinArgsExpected = MI->getNumArgs();
>>>> -
>>>> - // See MacroArgs instance var for description of this.
>>>> - bool isVarargsElided = false;
>>>> -
>>>> - if (NumActuals < MinArgsExpected) {
>>>> - // There are several cases where too few arguments is ok, handle
>>>> them now.
>>>> - if (NumActuals == 0 && MinArgsExpected == 1) {
>>>> - // #define A(X) or #define A(...) ---> A()
>>>> -
>>>> - // If there is exactly one argument, and that argument is missing,
>>>> - // then we have an empty "()" argument empty list. This is fine,
>>>> even if
>>>> - // the macro expects one argument (the argument is just empty).
>>>> - isVarargsElided = MI->isVariadic();
>>>> - } else if (MI->isVariadic() &&
>>>> - (NumActuals+1 == MinArgsExpected || // A(x, ...) -> A(X)
>>>> - (NumActuals == 0 && MinArgsExpected == 2))) {// A(x,...)
>>>> -> A()
>>>> - // Varargs where the named vararg parameter is missing: OK as
>>>> extension.
>>>> - // #define A(x, ...)
>>>> - // A("blah")
>>>> - Diag(Tok, diag::ext_missing_varargs_arg);
>>>> - Diag(MI->getDefinitionLoc(), diag::note_macro_here)
>>>> - << MacroName.getIdentifierInfo();
>>>> -
>>>> - // Remember this occurred, allowing us to elide the comma when
>>>> used for
>>>> - // cases like:
>>>> - // #define A(x, foo...) blah(a, ## foo)
>>>> - // #define B(x, ...) blah(a, ## __VA_ARGS__)
>>>> - // #define C(...) blah(a, ## __VA_ARGS__)
>>>> - // A(x) B(x) C()
>>>> - isVarargsElided = true;
>>>> - } else {
>>>> - // Otherwise, emit the error.
>>>> - Diag(Tok, diag::err_too_few_args_in_macro_invoc);
>>>> - return 0;
>>>> - }
>>>> -
>>>> - // Add a marker EOF token to the end of the token list for this
>>>> argument.
>>>> - SourceLocation EndLoc = Tok.getLocation();
>>>> - Tok.startToken();
>>>> - Tok.setKind(tok::eof);
>>>> - Tok.setLocation(EndLoc);
>>>> - Tok.setLength(0);
>>>> - ArgTokens.push_back(Tok);
>>>> -
>>>> - // If we expect two arguments, add both as empty.
>>>> - if (NumActuals == 0 && MinArgsExpected == 2)
>>>> - ArgTokens.push_back(Tok);
>>>> -
>>>> - } else if (NumActuals > MinArgsExpected && !MI->isVariadic()) {
>>>> - // Emit the diagnostic at the macro name in case there is a missing
>>>> ).
>>>> - // Emitting it at the , could be far away from the macro name.
>>>> - Diag(MacroName, diag::err_too_many_args_in_macro_invoc);
>>>> - return 0;
>>>> - }
>>>> -
>>>> - return MacroArgs::create(MI, ArgTokens, isVarargsElided, *this);
>>>> -}
>>>> -
>>>> -/// \brief Keeps macro expanded tokens for TokenLexers.
>>>> -//
>>>> -/// Works like a stack; a TokenLexer adds the macro expanded tokens that
>>>> is
>>>> -/// going to lex in the cache and when it finishes the tokens are
>>>> removed
>>>> -/// from the end of the cache.
>>>> -Token *Preprocessor::cacheMacroExpandedTokens(TokenLexer *tokLexer,
>>>> - ArrayRef<Token> tokens) {
>>>> - assert(tokLexer);
>>>> - if (tokens.empty())
>>>> - return 0;
>>>> -
>>>> - size_t newIndex = MacroExpandedTokens.size();
>>>> - bool cacheNeedsToGrow = tokens.size() >
>>>> -
>>>> MacroExpandedTokens.capacity()-MacroExpandedTokens.size();
>>>> - MacroExpandedTokens.append(tokens.begin(), tokens.end());
>>>> -
>>>> - if (cacheNeedsToGrow) {
>>>> - // Go through all the TokenLexers whose 'Tokens' pointer points in
>>>> the
>>>> - // buffer and update the pointers to the (potential) new buffer
>>>> array.
>>>> - for (unsigned i = 0, e = MacroExpandingLexersStack.size(); i != e;
>>>> ++i) {
>>>> - TokenLexer *prevLexer;
>>>> - size_t tokIndex;
>>>> - llvm::tie(prevLexer, tokIndex) = MacroExpandingLexersStack[i];
>>>> - prevLexer->Tokens = MacroExpandedTokens.data() + tokIndex;
>>>> - }
>>>> - }
>>>> -
>>>> - MacroExpandingLexersStack.push_back(std::make_pair(tokLexer,
>>>> newIndex));
>>>> - return MacroExpandedTokens.data() + newIndex;
>>>> -}
>>>> -
>>>> -void Preprocessor::removeCachedMacroExpandedTokensOfLastLexer() {
>>>> - assert(!MacroExpandingLexersStack.empty());
>>>> - size_t tokIndex = MacroExpandingLexersStack.back().second;
>>>> - assert(tokIndex < MacroExpandedTokens.size());
>>>> - // Pop the cached macro expanded tokens from the end.
>>>> - MacroExpandedTokens.resize(tokIndex);
>>>> - MacroExpandingLexersStack.pop_back();
>>>> -}
>>>> -
>>>> -/// ComputeDATE_TIME - Compute the current time, enter it into the
>>>> specified
>>>> -/// scratch buffer, then return DATELoc/TIMELoc locations with the
>>>> position of
>>>> -/// the identifier tokens inserted.
>>>> -static void ComputeDATE_TIME(SourceLocation &DATELoc, SourceLocation
>>>> &TIMELoc,
>>>> - Preprocessor &PP) {
>>>> - time_t TT = time(0);
>>>> - struct tm *TM = localtime(&TT);
>>>> -
>>>> - static const char * const Months[] = {
>>>> -
>>>> "Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"
>>>> - };
>>>> -
>>>> - char TmpBuffer[32];
>>>> -#ifdef LLVM_ON_WIN32
>>>> - sprintf(TmpBuffer, "\"%s %2d %4d\"", Months[TM->tm_mon], TM->tm_mday,
>>>> - TM->tm_year+1900);
>>>> -#else
>>>> - snprintf(TmpBuffer, sizeof(TmpBuffer), "\"%s %2d %4d\"",
>>>> Months[TM->tm_mon], TM->tm_mday,
>>>> - TM->tm_year+1900);
>>>> -#endif
>>>> -
>>>> - Token TmpTok;
>>>> - TmpTok.startToken();
>>>> - PP.CreateString(TmpBuffer, strlen(TmpBuffer), TmpTok);
>>>> - DATELoc = TmpTok.getLocation();
>>>> -
>>>> -#ifdef LLVM_ON_WIN32
>>>> - sprintf(TmpBuffer, "\"%02d:%02d:%02d\"", TM->tm_hour, TM->tm_min,
>>>> TM->tm_sec);
>>>> -#else
>>>> - snprintf(TmpBuffer, sizeof(TmpBuffer), "\"%02d:%02d:%02d\"",
>>>> TM->tm_hour, TM->tm_min, TM->tm_sec);
>>>> -#endif
>>>> - PP.CreateString(TmpBuffer, strlen(TmpBuffer), TmpTok);
>>>> - TIMELoc = TmpTok.getLocation();
>>>> -}
>>>> -
>>>> -
>>>> -/// HasFeature - Return true if we recognize and implement the feature
>>>> -/// specified by the identifier as a standard language feature.
>>>> -static bool HasFeature(const Preprocessor &PP, const IdentifierInfo *II)
>>>> {
>>>> - const LangOptions &LangOpts = PP.getLangOpts();
>>>> - StringRef Feature = II->getName();
>>>> -
>>>> - // Normalize the feature name, __foo__ becomes foo.
>>>> - if (Feature.startswith("__") && Feature.endswith("__") &&
>>>> Feature.size() >= 4)
>>>> - Feature = Feature.substr(2, Feature.size() - 4);
>>>> -
>>>> - return llvm::StringSwitch<bool>(Feature)
>>>> - .Case("address_sanitizer", LangOpts.AddressSanitizer)
>>>> - .Case("attribute_analyzer_noreturn", true)
>>>> - .Case("attribute_availability", true)
>>>> - .Case("attribute_availability_with_message", true)
>>>> - .Case("attribute_cf_returns_not_retained", true)
>>>> - .Case("attribute_cf_returns_retained", true)
>>>> - .Case("attribute_deprecated_with_message", true)
>>>> - .Case("attribute_ext_vector_type", true)
>>>> - .Case("attribute_ns_returns_not_retained", true)
>>>> - .Case("attribute_ns_returns_retained", true)
>>>> - .Case("attribute_ns_consumes_self", true)
>>>> - .Case("attribute_ns_consumed", true)
>>>> - .Case("attribute_cf_consumed", true)
>>>> - .Case("attribute_objc_ivar_unused", true)
>>>> - .Case("attribute_objc_method_family", true)
>>>> - .Case("attribute_overloadable", true)
>>>> - .Case("attribute_unavailable_with_message", true)
>>>> - .Case("attribute_unused_on_fields", true)
>>>> - .Case("blocks", LangOpts.Blocks)
>>>> - .Case("cxx_exceptions", LangOpts.Exceptions)
>>>> - .Case("cxx_rtti", LangOpts.RTTI)
>>>> - .Case("enumerator_attributes", true)
>>>> - // Objective-C features
>>>> - .Case("objc_arr", LangOpts.ObjCAutoRefCount) // FIXME:
>>>> REMOVE?
>>>> - .Case("objc_arc", LangOpts.ObjCAutoRefCount)
>>>> - .Case("objc_arc_weak", LangOpts.ObjCARCWeak)
>>>> - .Case("objc_default_synthesize_properties", LangOpts.ObjC2)
>>>> - .Case("objc_fixed_enum", LangOpts.ObjC2)
>>>> - .Case("objc_instancetype", LangOpts.ObjC2)
>>>> - .Case("objc_modules", LangOpts.ObjC2 && LangOpts.Modules)
>>>> - .Case("objc_nonfragile_abi",
>>>> LangOpts.ObjCRuntime.isNonFragile())
>>>> - .Case("objc_weak_class",
>>>> LangOpts.ObjCRuntime.hasWeakClassImport())
>>>> - .Case("ownership_holds", true)
>>>> - .Case("ownership_returns", true)
>>>> - .Case("ownership_takes", true)
>>>> - .Case("objc_bool", true)
>>>> - .Case("objc_subscripting",
>>>> LangOpts.ObjCRuntime.isNonFragile())
>>>> - .Case("objc_array_literals", LangOpts.ObjC2)
>>>> - .Case("objc_dictionary_literals", LangOpts.ObjC2)
>>>> - .Case("objc_boxed_expressions", LangOpts.ObjC2)
>>>> - .Case("arc_cf_code_audited", true)
>>>> - // C11 features
>>>> - .Case("c_alignas", LangOpts.C11)
>>>> - .Case("c_atomic", LangOpts.C11)
>>>> - .Case("c_generic_selections", LangOpts.C11)
>>>> - .Case("c_static_assert", LangOpts.C11)
>>>> - // C++11 features
>>>> - .Case("cxx_access_control_sfinae", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_alias_templates", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_alignas", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_atomic", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_attributes", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_auto_type", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_constexpr", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_decltype", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_decltype_incomplete_return_types",
>>>> LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_default_function_template_args",
>>>> LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_defaulted_functions", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_delegating_constructors", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_deleted_functions", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_explicit_conversions", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_generalized_initializers", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_implicit_moves", LangOpts.CPlusPlus0x)
>>>> - //.Case("cxx_inheriting_constructors", false)
>>>> - .Case("cxx_inline_namespaces", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_lambdas", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_local_type_template_args", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_nonstatic_member_init", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_noexcept", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_nullptr", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_override_control", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_range_for", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_raw_string_literals", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_reference_qualified_functions",
>>>> LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_rvalue_references", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_strong_enums", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_static_assert", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_trailing_return", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_unicode_literals", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_unrestricted_unions", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_user_literals", LangOpts.CPlusPlus0x)
>>>> - .Case("cxx_variadic_templates", LangOpts.CPlusPlus0x)
>>>> - // Type traits
>>>> - .Case("has_nothrow_assign", LangOpts.CPlusPlus)
>>>> - .Case("has_nothrow_copy", LangOpts.CPlusPlus)
>>>> - .Case("has_nothrow_constructor", LangOpts.CPlusPlus)
>>>> - .Case("has_trivial_assign", LangOpts.CPlusPlus)
>>>> - .Case("has_trivial_copy", LangOpts.CPlusPlus)
>>>> - .Case("has_trivial_constructor", LangOpts.CPlusPlus)
>>>> - .Case("has_trivial_destructor", LangOpts.CPlusPlus)
>>>> - .Case("has_virtual_destructor", LangOpts.CPlusPlus)
>>>> - .Case("is_abstract", LangOpts.CPlusPlus)
>>>> - .Case("is_base_of", LangOpts.CPlusPlus)
>>>> - .Case("is_class", LangOpts.CPlusPlus)
>>>> - .Case("is_convertible_to", LangOpts.CPlusPlus)
>>>> - // __is_empty is available only if the horrible
>>>> - // "struct __is_empty" parsing hack hasn't been needed in
>>>> this
>>>> - // translation unit. If it has, __is_empty reverts to a
>>>> normal
>>>> - // identifier and __has_feature(is_empty) evaluates false.
>>>> - .Case("is_empty", LangOpts.CPlusPlus)
>>>> - .Case("is_enum", LangOpts.CPlusPlus)
>>>> - .Case("is_final", LangOpts.CPlusPlus)
>>>> - .Case("is_literal", LangOpts.CPlusPlus)
>>>> - .Case("is_standard_layout", LangOpts.CPlusPlus)
>>>> - .Case("is_pod", LangOpts.CPlusPlus)
>>>> - .Case("is_polymorphic", LangOpts.CPlusPlus)
>>>> - .Case("is_trivial", LangOpts.CPlusPlus)
>>>> - .Case("is_trivially_assignable", LangOpts.CPlusPlus)
>>>> - .Case("is_trivially_constructible", LangOpts.CPlusPlus)
>>>> - .Case("is_trivially_copyable", LangOpts.CPlusPlus)
>>>> - .Case("is_union", LangOpts.CPlusPlus)
>>>> - .Case("modules", LangOpts.Modules)
>>>> - .Case("tls", PP.getTargetInfo().isTLSSupported())
>>>> - .Case("underlying_type", LangOpts.CPlusPlus)
>>>> - .Default(false);
>>>> -}
>>>> -
>>>> -/// HasExtension - Return true if we recognize and implement the feature
>>>> -/// specified by the identifier, either as an extension or a standard
>>>> language
>>>> -/// feature.
>>>> -static bool HasExtension(const Preprocessor &PP, const IdentifierInfo
>>>> *II) {
>>>> - if (HasFeature(PP, II))
>>>> - return true;
>>>> -
>>>> - // If the use of an extension results in an error diagnostic,
>>>> extensions are
>>>> - // effectively unavailable, so just return false here.
>>>> - if (PP.getDiagnostics().getExtensionHandlingBehavior() ==
>>>> - DiagnosticsEngine::Ext_Error)
>>>> - return false;
>>>> -
>>>> - const LangOptions &LangOpts = PP.getLangOpts();
>>>> - StringRef Extension = II->getName();
>>>> -
>>>> - // Normalize the extension name, __foo__ becomes foo.
>>>> - if (Extension.startswith("__") && Extension.endswith("__") &&
>>>> - Extension.size() >= 4)
>>>> - Extension = Extension.substr(2, Extension.size() - 4);
>>>> -
>>>> - // Because we inherit the feature list from HasFeature, this string
>>>> switch
>>>> - // must be less restrictive than HasFeature's.
>>>> - return llvm::StringSwitch<bool>(Extension)
>>>> - // C11 features supported by other languages as extensions.
>>>> - .Case("c_alignas", true)
>>>> - .Case("c_atomic", true)
>>>> - .Case("c_generic_selections", true)
>>>> - .Case("c_static_assert", true)
>>>> - // C++0x features supported by other languages as extensions.
>>>> - .Case("cxx_atomic", LangOpts.CPlusPlus)
>>>> - .Case("cxx_deleted_functions", LangOpts.CPlusPlus)
>>>> - .Case("cxx_explicit_conversions", LangOpts.CPlusPlus)
>>>> - .Case("cxx_inline_namespaces", LangOpts.CPlusPlus)
>>>> - .Case("cxx_local_type_template_args", LangOpts.CPlusPlus)
>>>> - .Case("cxx_nonstatic_member_init", LangOpts.CPlusPlus)
>>>> - .Case("cxx_override_control", LangOpts.CPlusPlus)
>>>> - .Case("cxx_range_for", LangOpts.CPlusPlus)
>>>> - .Case("cxx_reference_qualified_functions",
>>>> LangOpts.CPlusPlus)
>>>> - .Case("cxx_rvalue_references", LangOpts.CPlusPlus)
>>>> - .Default(false);
>>>> -}
>>>> -
>>>> -/// HasAttribute - Return true if we recognize and implement the
>>>> attribute
>>>> -/// specified by the given identifier.
>>>> -static bool HasAttribute(const IdentifierInfo *II) {
>>>> - StringRef Name = II->getName();
>>>> - // Normalize the attribute name, __foo__ becomes foo.
>>>> - if (Name.startswith("__") && Name.endswith("__") && Name.size() >= 4)
>>>> - Name = Name.substr(2, Name.size() - 4);
>>>> -
>>>> - // FIXME: Do we need to handle namespaces here?
>>>> - return llvm::StringSwitch<bool>(Name)
>>>> -#include "clang/Lex/AttrSpellings.inc"
>>>> - .Default(false);
>>>> -}
>>>> -
>>>> -/// EvaluateHasIncludeCommon - Process a '__has_include("path")'
>>>> -/// or '__has_include_next("path")' expression.
>>>> -/// Returns true if successful.
>>>> -static bool EvaluateHasIncludeCommon(Token &Tok,
>>>> - IdentifierInfo *II, Preprocessor
>>>> &PP,
>>>> - const DirectoryLookup *LookupFrom)
>>>> {
>>>> - SourceLocation LParenLoc;
>>>> -
>>>> - // Get '('.
>>>> - PP.LexNonComment(Tok);
>>>> -
>>>> - // Ensure we have a '('.
>>>> - if (Tok.isNot(tok::l_paren)) {
>>>> - PP.Diag(Tok.getLocation(), diag::err_pp_missing_lparen) <<
>>>> II->getName();
>>>> - return false;
>>>> - }
>>>> -
>>>> - // Save '(' location for possible missing ')' message.
>>>> - LParenLoc = Tok.getLocation();
>>>> -
>>>> - // Get the file name.
>>>> - PP.getCurrentLexer()->LexIncludeFilename(Tok);
>>>> -
>>>> - // Reserve a buffer to get the spelling.
>>>> - SmallString<128> FilenameBuffer;
>>>> - StringRef Filename;
>>>> - SourceLocation EndLoc;
>>>> -
>>>> - switch (Tok.getKind()) {
>>>> - case tok::eod:
>>>> - // If the token kind is EOD, the error has already been diagnosed.
>>>> - return false;
>>>> -
>>>> - case tok::angle_string_literal:
>>>> - case tok::string_literal: {
>>>> - bool Invalid = false;
>>>> - Filename = PP.getSpelling(Tok, FilenameBuffer, &Invalid);
>>>> - if (Invalid)
>>>> - return false;
>>>> - break;
>>>> - }
>>>> -
>>>> - case tok::less:
>>>> - // This could be a <foo/bar.h> file coming from a macro expansion.
>>>> In this
>>>> - // case, glue the tokens together into FilenameBuffer and interpret
>>>> those.
>>>> - FilenameBuffer.push_back('<');
>>>> - if (PP.ConcatenateIncludeName(FilenameBuffer, EndLoc))
>>>> - return false; // Found <eod> but no ">"? Diagnostic already
>>>> emitted.
>>>> - Filename = FilenameBuffer.str();
>>>> - break;
>>>> - default:
>>>> - PP.Diag(Tok.getLocation(), diag::err_pp_expects_filename);
>>>> - return false;
>>>> - }
>>>> -
>>>> - // Get ')'.
>>>> - PP.LexNonComment(Tok);
>>>> -
>>>> - // Ensure we have a trailing ).
>>>> - if (Tok.isNot(tok::r_paren)) {
>>>> - PP.Diag(Tok.getLocation(), diag::err_pp_missing_rparen) <<
>>>> II->getName();
>>>> - PP.Diag(LParenLoc, diag::note_matching) << "(";
>>>> - return false;
>>>> - }
>>>> -
>>>> - bool isAngled = PP.GetIncludeFilenameSpelling(Tok.getLocation(),
>>>> Filename);
>>>> - // If GetIncludeFilenameSpelling set the start ptr to null, there was
>>>> an
>>>> - // error.
>>>> - if (Filename.empty())
>>>> - return false;
>>>> -
>>>> - // Search include directories.
>>>> - const DirectoryLookup *CurDir;
>>>> - const FileEntry *File =
>>>> - PP.LookupFile(Filename, isAngled, LookupFrom, CurDir, NULL, NULL,
>>>> NULL);
>>>> -
>>>> - // Get the result value. A result of true means the file exists.
>>>> - return File != 0;
>>>> -}
>>>> -
>>>> -/// EvaluateHasInclude - Process a '__has_include("path")' expression.
>>>> -/// Returns true if successful.
>>>> -static bool EvaluateHasInclude(Token &Tok, IdentifierInfo *II,
>>>> - Preprocessor &PP) {
>>>> - return EvaluateHasIncludeCommon(Tok, II, PP, NULL);
>>>> -}
>>>> -
>>>> -/// EvaluateHasIncludeNext - Process '__has_include_next("path")'
>>>> expression.
>>>> -/// Returns true if successful.
>>>> -static bool EvaluateHasIncludeNext(Token &Tok,
>>>> - IdentifierInfo *II, Preprocessor &PP)
>>>> {
>>>> - // __has_include_next is like __has_include, except that we start
>>>> - // searching after the current found directory. If we can't do this,
>>>> - // issue a diagnostic.
>>>> - const DirectoryLookup *Lookup = PP.GetCurDirLookup();
>>>> - if (PP.isInPrimaryFile()) {
>>>> - Lookup = 0;
>>>> - PP.Diag(Tok, diag::pp_include_next_in_primary);
>>>> - } else if (Lookup == 0) {
>>>> - PP.Diag(Tok, diag::pp_include_next_absolute_path);
>>>> - } else {
>>>> - // Start looking up in the next directory.
>>>> - ++Lookup;
>>>> - }
>>>> -
>>>> - return EvaluateHasIncludeCommon(Tok, II, PP, Lookup);
>>>> -}
>>>> -
>>>> -/// ExpandBuiltinMacro - If an identifier token is read that is to be
>>>> expanded
>>>> -/// as a builtin macro, handle it and return the next token as 'Tok'.
>>>> -void Preprocessor::ExpandBuiltinMacro(Token &Tok) {
>>>> - // Figure out which token this is.
>>>> - IdentifierInfo *II = Tok.getIdentifierInfo();
>>>> - assert(II && "Can't be a macro without id info!");
>>>> -
>>>> - // If this is an _Pragma or Microsoft __pragma directive, expand it,
>>>> - // invoke the pragma handler, then lex the token after it.
>>>> - if (II == Ident_Pragma)
>>>> - return Handle_Pragma(Tok);
>>>> - else if (II == Ident__pragma) // in non-MS mode this is null
>>>> - return HandleMicrosoft__pragma(Tok);
>>>> -
>>>> - ++NumBuiltinMacroExpanded;
>>>> -
>>>> - SmallString<128> TmpBuffer;
>>>> - llvm::raw_svector_ostream OS(TmpBuffer);
>>>> -
>>>> - // Set up the return result.
>>>> - Tok.setIdentifierInfo(0);
>>>> - Tok.clearFlag(Token::NeedsCleaning);
>>>> -
>>>> - if (II == Ident__LINE__) {
>>>> - // C99 6.10.8: "__LINE__: The presumed line number (within the
>>>> current
>>>> - // source file) of the current source line (an integer constant)".
>>>> This can
>>>> - // be affected by #line.
>>>> - SourceLocation Loc = Tok.getLocation();
>>>> -
>>>> - // Advance to the location of the first _, this might not be the
>>>> first byte
>>>> - // of the token if it starts with an escaped newline.
>>>> - Loc = AdvanceToTokenCharacter(Loc, 0);
>>>> -
>>>> - // One wrinkle here is that GCC expands __LINE__ to location of the
>>>> *end* of
>>>> - // a macro expansion. This doesn't matter for object-like macros,
>>>> but
>>>> - // can matter for a function-like macro that expands to contain
>>>> __LINE__.
>>>> - // Skip down through expansion points until we find a file loc for
>>>> the
>>>> - // end of the expansion history.
>>>> - Loc = SourceMgr.getExpansionRange(Loc).second;
>>>> - PresumedLoc PLoc = SourceMgr.getPresumedLoc(Loc);
>>>> -
>>>> - // __LINE__ expands to a simple numeric value.
>>>> - OS << (PLoc.isValid()? PLoc.getLine() : 1);
>>>> - Tok.setKind(tok::numeric_constant);
>>>> - } else if (II == Ident__FILE__ || II == Ident__BASE_FILE__) {
>>>> - // C99 6.10.8: "__FILE__: The presumed name of the current source
>>>> file (a
>>>> - // character string literal)". This can be affected by #line.
>>>> - PresumedLoc PLoc = SourceMgr.getPresumedLoc(Tok.getLocation());
>>>> -
>>>> - // __BASE_FILE__ is a GNU extension that returns the top of the
>>>> presumed
>>>> - // #include stack instead of the current file.
>>>> - if (II == Ident__BASE_FILE__ && PLoc.isValid()) {
>>>> - SourceLocation NextLoc = PLoc.getIncludeLoc();
>>>> - while (NextLoc.isValid()) {
>>>> - PLoc = SourceMgr.getPresumedLoc(NextLoc);
>>>> - if (PLoc.isInvalid())
>>>> - break;
>>>> -
>>>> - NextLoc = PLoc.getIncludeLoc();
>>>> - }
>>>> - }
>>>> -
>>>> - // Escape this filename. Turn '\' -> '\\' '"' -> '\"'
>>>> - SmallString<128> FN;
>>>> - if (PLoc.isValid()) {
>>>> - FN += PLoc.getFilename();
>>>> - Lexer::Stringify(FN);
>>>> - OS << '"' << FN.str() << '"';
>>>> - }
>>>> - Tok.setKind(tok::string_literal);
>>>> - } else if (II == Ident__DATE__) {
>>>> - if (!DATELoc.isValid())
>>>> - ComputeDATE_TIME(DATELoc, TIMELoc, *this);
>>>> - Tok.setKind(tok::string_literal);
>>>> - Tok.setLength(strlen("\"Mmm dd yyyy\""));
>>>> - Tok.setLocation(SourceMgr.createExpansionLoc(DATELoc,
>>>> Tok.getLocation(),
>>>> - Tok.getLocation(),
>>>> - Tok.getLength()));
>>>> - return;
>>>> - } else if (II == Ident__TIME__) {
>>>> - if (!TIMELoc.isValid())
>>>> - ComputeDATE_TIME(DATELoc, TIMELoc, *this);
>>>> - Tok.setKind(tok::string_literal);
>>>> - Tok.setLength(strlen("\"hh:mm:ss\""));
>>>> - Tok.setLocation(SourceMgr.createExpansionLoc(TIMELoc,
>>>> Tok.getLocation(),
>>>> - Tok.getLocation(),
>>>> - Tok.getLength()));
>>>> - return;
>>>> - } else if (II == Ident__INCLUDE_LEVEL__) {
>>>> - // Compute the presumed include depth of this token. This can be
>>>> affected
>>>> - // by GNU line markers.
>>>> - unsigned Depth = 0;
>>>> -
>>>> - PresumedLoc PLoc = SourceMgr.getPresumedLoc(Tok.getLocation());
>>>> - if (PLoc.isValid()) {
>>>> - PLoc = SourceMgr.getPresumedLoc(PLoc.getIncludeLoc());
>>>> - for (; PLoc.isValid(); ++Depth)
>>>> - PLoc = SourceMgr.getPresumedLoc(PLoc.getIncludeLoc());
>>>> - }
>>>> -
>>>> - // __INCLUDE_LEVEL__ expands to a simple numeric value.
>>>> - OS << Depth;
>>>> - Tok.setKind(tok::numeric_constant);
>>>> - } else if (II == Ident__TIMESTAMP__) {
>>>> - // MSVC, ICC, GCC, VisualAge C++ extension. The generated string
>>>> should be
>>>> - // of the form "Ddd Mmm dd hh::mm::ss yyyy", which is returned by
>>>> asctime.
>>>> -
>>>> - // Get the file that we are lexing out of. If we're currently
>>>> lexing from
>>>> - // a macro, dig into the include stack.
>>>> - const FileEntry *CurFile = 0;
>>>> - PreprocessorLexer *TheLexer = getCurrentFileLexer();
>>>> -
>>>> - if (TheLexer)
>>>> - CurFile = SourceMgr.getFileEntryForID(TheLexer->getFileID());
>>>> -
>>>> - const char *Result;
>>>> - if (CurFile) {
>>>> - time_t TT = CurFile->getModificationTime();
>>>> - struct tm *TM = localtime(&TT);
>>>> - Result = asctime(TM);
>>>> - } else {
>>>> - Result = "??? ??? ?? ??:??:?? ????\n";
>>>> - }
>>>> - // Surround the string with " and strip the trailing newline.
>>>> - OS << '"' << StringRef(Result, strlen(Result)-1) << '"';
>>>> - Tok.setKind(tok::string_literal);
>>>> - } else if (II == Ident__COUNTER__) {
>>>> - // __COUNTER__ expands to a simple numeric value.
>>>> - OS << CounterValue++;
>>>> - Tok.setKind(tok::numeric_constant);
>>>> - } else if (II == Ident__has_feature ||
>>>> - II == Ident__has_extension ||
>>>> - II == Ident__has_builtin ||
>>>> - II == Ident__has_attribute) {
>>>> - // The argument to these builtins should be a parenthesized
>>>> identifier.
>>>> - SourceLocation StartLoc = Tok.getLocation();
>>>> -
>>>> - bool IsValid = false;
>>>> - IdentifierInfo *FeatureII = 0;
>>>> -
>>>> - // Read the '('.
>>>> - Lex(Tok);
>>>> - if (Tok.is(tok::l_paren)) {
>>>> - // Read the identifier
>>>> - Lex(Tok);
>>>> - if (Tok.is(tok::identifier) || Tok.is(tok::kw_const)) {
>>>> - FeatureII = Tok.getIdentifierInfo();
>>>> -
>>>> - // Read the ')'.
>>>> - Lex(Tok);
>>>> - if (Tok.is(tok::r_paren))
>>>> - IsValid = true;
>>>> - }
>>>> - }
>>>> -
>>>> - bool Value = false;
>>>> - if (!IsValid)
>>>> - Diag(StartLoc, diag::err_feature_check_malformed);
>>>> - else if (II == Ident__has_builtin) {
>>>> - // Check for a builtin is trivial.
>>>> - Value = FeatureII->getBuiltinID() != 0;
>>>> - } else if (II == Ident__has_attribute)
>>>> - Value = HasAttribute(FeatureII);
>>>> - else if (II == Ident__has_extension)
>>>> - Value = HasExtension(*this, FeatureII);
>>>> - else {
>>>> - assert(II == Ident__has_feature && "Must be feature check");
>>>> - Value = HasFeature(*this, FeatureII);
>>>> - }
>>>> -
>>>> - OS << (int)Value;
>>>> - if (IsValid)
>>>> - Tok.setKind(tok::numeric_constant);
>>>> - } else if (II == Ident__has_include ||
>>>> - II == Ident__has_include_next) {
>>>> - // The argument to these two builtins should be a parenthesized
>>>> - // file name string literal using angle brackets (<>) or
>>>> - // double-quotes ("").
>>>> - bool Value;
>>>> - if (II == Ident__has_include)
>>>> - Value = EvaluateHasInclude(Tok, II, *this);
>>>> - else
>>>> - Value = EvaluateHasIncludeNext(Tok, II, *this);
>>>> - OS << (int)Value;
>>>> - Tok.setKind(tok::numeric_constant);
>>>> - } else if (II == Ident__has_warning) {
>>>> - // The argument should be a parenthesized string literal.
>>>> - // The argument to these builtins should be a parenthesized
>>>> identifier.
>>>> - SourceLocation StartLoc = Tok.getLocation();
>>>> - bool IsValid = false;
>>>> - bool Value = false;
>>>> - // Read the '('.
>>>> - Lex(Tok);
>>>> - do {
>>>> - if (Tok.is(tok::l_paren)) {
>>>> - // Read the string.
>>>> - Lex(Tok);
>>>> -
>>>> - // We need at least one string literal.
>>>> - if (!Tok.is(tok::string_literal)) {
>>>> - StartLoc = Tok.getLocation();
>>>> - IsValid = false;
>>>> - // Eat tokens until ')'.
>>>> - do Lex(Tok); while (!(Tok.is(tok::r_paren) ||
>>>> Tok.is(tok::eod)));
>>>> - break;
>>>> - }
>>>> -
>>>> - // String concatenation allows multiple strings, which can even
>>>> come
>>>> - // from macro expansion.
>>>> - SmallVector<Token, 4> StrToks;
>>>> - while (Tok.is(tok::string_literal)) {
>>>> - // Complain about, and drop, any ud-suffix.
>>>> - if (Tok.hasUDSuffix())
>>>> - Diag(Tok, diag::err_invalid_string_udl);
>>>> - StrToks.push_back(Tok);
>>>> - LexUnexpandedToken(Tok);
>>>> - }
>>>> -
>>>> - // Is the end a ')'?
>>>> - if (!(IsValid = Tok.is(tok::r_paren)))
>>>> - break;
>>>> -
>>>> - // Concatenate and parse the strings.
>>>> - StringLiteralParser Literal(&StrToks[0], StrToks.size(), *this);
>>>> - assert(Literal.isAscii() && "Didn't allow wide strings in");
>>>> - if (Literal.hadError)
>>>> - break;
>>>> - if (Literal.Pascal) {
>>>> - Diag(Tok, diag::warn_pragma_diagnostic_invalid);
>>>> - break;
>>>> - }
>>>> -
>>>> - StringRef WarningName(Literal.GetString());
>>>> -
>>>> - if (WarningName.size() < 3 || WarningName[0] != '-' ||
>>>> - WarningName[1] != 'W') {
>>>> - Diag(StrToks[0].getLocation(),
>>>> diag::warn_has_warning_invalid_option);
>>>> - break;
>>>> - }
>>>> -
>>>> - // Finally, check if the warning flags maps to a diagnostic
>>>> group.
>>>> - // We construct a SmallVector here to talk to
>>>> getDiagnosticIDs().
>>>> - // Although we don't use the result, this isn't a hot path, and
>>>> not
>>>> - // worth special casing.
>>>> - llvm::SmallVector<diag::kind, 10> Diags;
>>>> - Value = !getDiagnostics().getDiagnosticIDs()->
>>>> - getDiagnosticsInGroup(WarningName.substr(2), Diags);
>>>> - }
>>>> - } while (false);
>>>> -
>>>> - if (!IsValid)
>>>> - Diag(StartLoc, diag::err_warning_check_malformed);
>>>> -
>>>> - OS << (int)Value;
>>>> - Tok.setKind(tok::numeric_constant);
>>>> - } else {
>>>> - llvm_unreachable("Unknown identifier!");
>>>> - }
>>>> - CreateString(OS.str().data(), OS.str().size(), Tok,
>>>> - Tok.getLocation(), Tok.getLocation());
>>>> -}
>>>> -
>>>> -void Preprocessor::markMacroAsUsed(MacroInfo *MI) {
>>>> - // If the 'used' status changed, and the macro requires 'unused'
>>>> warning,
>>>> - // remove its SourceLocation from the warn-for-unused-macro locations.
>>>> - if (MI->isWarnIfUnused() && !MI->isUsed())
>>>> - WarnUnusedMacroLocs.erase(MI->getDefinitionLoc());
>>>> - MI->setIsUsed(true);
>>>> -}
>>>> +//===--- MacroExpansion.cpp - Top level Macro Expansion
>>>> -------------------===//
>>>> +//
>>>> +// The LLVM Compiler Infrastructure
>>>> +//
>>>> +// This file is distributed under the University of Illinois Open Source
>>>> +// License. See LICENSE.TXT for details.
>>>> +//
>>>>
>>>> +//===----------------------------------------------------------------------===//
>>>> +//
>>>> +// This file implements the top level handling of macro expasion for the
>>>> +// preprocessor.
>>>> +//
>>>>
>>>> +//===----------------------------------------------------------------------===//
>>>> +
>>>> +#include "clang/Lex/Preprocessor.h"
>>>> +#include "MacroArgs.h"
>>>> +#include "clang/Lex/MacroInfo.h"
>>>> +#include "clang/Basic/SourceManager.h"
>>>> +#include "clang/Basic/FileManager.h"
>>>> +#include "clang/Basic/TargetInfo.h"
>>>> +#include "clang/Lex/LexDiagnostic.h"
>>>> +#include "clang/Lex/CodeCompletionHandler.h"
>>>> +#include "clang/Lex/ExternalPreprocessorSource.h"
>>>> +#include "clang/Lex/LiteralSupport.h"
>>>> +#include "llvm/ADT/StringSwitch.h"
>>>> +#include "llvm/ADT/STLExtras.h"
>>>> +#include "llvm/Config/llvm-config.h"
>>>> +#include "llvm/Support/raw_ostream.h"
>>>> +#include "llvm/Support/ErrorHandling.h"
>>>> +#include <cstdio>
>>>> +#include <ctime>
>>>> +using namespace clang;
>>>> +
>>>> +MacroInfo *Preprocessor::getInfoForMacro(IdentifierInfo *II) const {
>>>> + assert(II->hasMacroDefinition() && "Identifier is not a macro!");
>>>> +
>>>> + macro_iterator Pos = Macros.find(II);
>>>> + if (Pos == Macros.end()) {
>>>> + // Load this macro from the external source.
>>>> + getExternalSource()->LoadMacroDefinition(II);
>>>> + Pos = Macros.find(II);
>>>> + }
>>>> + assert(Pos != Macros.end() && "Identifier macro info is missing!");
>>>> + assert(Pos->second->getUndefLoc().isInvalid() && "Macro is
>>>> undefined!");
>>>> + return Pos->second;
>>>> +}
>>>> +
>>>> +/// setMacroInfo - Specify a macro for this identifier.
>>>> +///
>>>> +void Preprocessor::setMacroInfo(IdentifierInfo *II, MacroInfo *MI,
>>>> + bool LoadedFromAST) {
>>>> + assert(MI && "MacroInfo should be non-zero!");
>>>> + MI->setPreviousDefinition(Macros[II]);
>>>> + Macros[II] = MI;
>>>> + II->setHasMacroDefinition(true);
>>>> + if (II->isFromAST() && !LoadedFromAST)
>>>> + II->setChangedSinceDeserialization();
>>>> +}
>>>> +
>>>> +/// \brief Undefine a macro for this identifier.
>>>> +void Preprocessor::clearMacroInfo(IdentifierInfo *II) {
>>>> + assert(II->hasMacroDefinition() && "Macro is not defined!");
>>>> + assert(Macros[II]->getUndefLoc().isValid() && "Macro is still
>>>> defined!");
>>>> + II->setHasMacroDefinition(false);
>>>> + if (II->isFromAST())
>>>> + II->setChangedSinceDeserialization();
>>>> +}
>>>> +
>>>> +/// RegisterBuiltinMacro - Register the specified identifier in the
>>>> identifier
>>>> +/// table and mark it as a builtin macro to be expanded.
>>>> +static IdentifierInfo *RegisterBuiltinMacro(Preprocessor &PP, const char
>>>> *Name){
>>>> + // Get the identifier.
>>>> + IdentifierInfo *Id = PP.getIdentifierInfo(Name);
>>>> +
>>>> + // Mark it as being a macro that is builtin.
>>>> + MacroInfo *MI = PP.AllocateMacroInfo(SourceLocation());
>>>> + MI->setIsBuiltinMacro();
>>>> + PP.setMacroInfo(Id, MI);
>>>> + return Id;
>>>> +}
>>>> +
>>>> +
>>>> +/// RegisterBuiltinMacros - Register builtin macros, such as __LINE__
>>>> with the
>>>> +/// identifier table.
>>>> +void Preprocessor::RegisterBuiltinMacros() {
>>>> + Ident__LINE__ = RegisterBuiltinMacro(*this, "__LINE__");
>>>> + Ident__FILE__ = RegisterBuiltinMacro(*this, "__FILE__");
>>>> + Ident__DATE__ = RegisterBuiltinMacro(*this, "__DATE__");
>>>> + Ident__TIME__ = RegisterBuiltinMacro(*this, "__TIME__");
>>>> + Ident__COUNTER__ = RegisterBuiltinMacro(*this, "__COUNTER__");
>>>> + Ident_Pragma = RegisterBuiltinMacro(*this, "_Pragma");
>>>> +
>>>> + // GCC Extensions.
>>>> + Ident__BASE_FILE__ = RegisterBuiltinMacro(*this, "__BASE_FILE__");
>>>> + Ident__INCLUDE_LEVEL__ = RegisterBuiltinMacro(*this,
>>>> "__INCLUDE_LEVEL__");
>>>> + Ident__TIMESTAMP__ = RegisterBuiltinMacro(*this, "__TIMESTAMP__");
>>>> +
>>>> + // Clang Extensions.
>>>> + Ident__has_feature = RegisterBuiltinMacro(*this,
>>>> "__has_feature");
>>>> + Ident__has_extension = RegisterBuiltinMacro(*this,
>>>> "__has_extension");
>>>> + Ident__has_builtin = RegisterBuiltinMacro(*this,
>>>> "__has_builtin");
>>>> + Ident__has_attribute = RegisterBuiltinMacro(*this,
>>>> "__has_attribute");
>>>> + Ident__has_include = RegisterBuiltinMacro(*this,
>>>> "__has_include");
>>>> + Ident__has_include_next = RegisterBuiltinMacro(*this,
>>>> "__has_include_next");
>>>> + Ident__has_warning = RegisterBuiltinMacro(*this,
>>>> "__has_warning");
>>>> +
>>>> + // Microsoft Extensions.
>>>> + if (LangOpts.MicrosoftExt)
>>>> + Ident__pragma = RegisterBuiltinMacro(*this, "__pragma");
>>>> + else
>>>> + Ident__pragma = 0;
>>>> +}
>>>> +
>>>> +/// isTrivialSingleTokenExpansion - Return true if MI, which has a
>>>> single token
>>>> +/// in its expansion, currently expands to that token literally.
>>>> +static bool isTrivialSingleTokenExpansion(const MacroInfo *MI,
>>>> + const IdentifierInfo
>>>> *MacroIdent,
>>>> + Preprocessor &PP) {
>>>> + IdentifierInfo *II = MI->getReplacementToken(0).getIdentifierInfo();
>>>> +
>>>> + // If the token isn't an identifier, it's always literally expanded.
>>>> + if (II == 0) return true;
>>>> +
>>>> + // If the information about this identifier is out of date, update it
>>>> from
>>>> + // the external source.
>>>> + if (II->isOutOfDate())
>>>> + PP.getExternalSource()->updateOutOfDateIdentifier(*II);
>>>> +
>>>> + // If the identifier is a macro, and if that macro is enabled, it may
>>>> be
>>>> + // expanded so it's not a trivial expansion.
>>>> + if (II->hasMacroDefinition() && PP.getMacroInfo(II)->isEnabled() &&
>>>> + // Fast expanding "#define X X" is ok, because X would be
>>>> disabled.
>>>> + II != MacroIdent)
>>>> + return false;
>>>> +
>>>> + // If this is an object-like macro invocation, it is safe to trivially
>>>> expand
>>>> + // it.
>>>> + if (MI->isObjectLike()) return true;
>>>> +
>>>> + // If this is a function-like macro invocation, it's safe to trivially
>>>> expand
>>>> + // as long as the identifier is not a macro argument.
>>>> + for (MacroInfo::arg_iterator I = MI->arg_begin(), E = MI->arg_end();
>>>> + I != E; ++I)
>>>> + if (*I == II)
>>>> + return false; // Identifier is a macro argument.
>>>> +
>>>> + return true;
>>>> +}
>>>> +
>>>> +
>>>> +/// isNextPPTokenLParen - Determine whether the next preprocessor token
>>>> to be
>>>> +/// lexed is a '('. If so, consume the token and return true, if not,
>>>> this
>>>> +/// method should have no observable side-effect on the lexed tokens.
>>>> +bool Preprocessor::isNextPPTokenLParen() {
>>>> + // Do some quick tests for rejection cases.
>>>> + unsigned Val;
>>>> + if (CurLexer)
>>>> + Val = CurLexer->isNextPPTokenLParen();
>>>> + else if (CurPTHLexer)
>>>> + Val = CurPTHLexer->isNextPPTokenLParen();
>>>> + else
>>>> + Val = CurTokenLexer->isNextTokenLParen();
>>>> +
>>>> + if (Val == 2) {
>>>> + // We have run off the end. If it's a source file we don't
>>>> + // examine enclosing ones (C99 5.1.1.2p4). Otherwise walk up the
>>>> + // macro stack.
>>>> + if (CurPPLexer)
>>>> + return false;
>>>> + for (unsigned i = IncludeMacroStack.size(); i != 0; --i) {
>>>> + IncludeStackInfo &Entry = IncludeMacroStack[i-1];
>>>> + if (Entry.TheLexer)
>>>> + Val = Entry.TheLexer->isNextPPTokenLParen();
>>>> + else if (Entry.ThePTHLexer)
>>>> + Val = Entry.ThePTHLexer->isNextPPTokenLParen();
>>>> + else
>>>> + Val = Entry.TheTokenLexer->isNextTokenLParen();
>>>> +
>>>> + if (Val != 2)
>>>> + break;
>>>> +
>>>> + // Ran off the end of a source file?
>>>> + if (Entry.ThePPLexer)
>>>> + return false;
>>>> + }
>>>> + }
>>>> +
>>>> + // Okay, if we know that the token is a '(', lex it and return.
>>>> Otherwise we
>>>> + // have found something that isn't a '(' or we found the end of the
>>>> + // translation unit. In either case, return false.
>>>> + return Val == 1;
>>>> +}
>>>> +
>>>> +/// HandleMacroExpandedIdentifier - If an identifier token is read that
>>>> is to be
>>>> +/// expanded as a macro, handle it and return the next token as
>>>> 'Identifier'.
>>>> +bool Preprocessor::HandleMacroExpandedIdentifier(Token &Identifier,
>>>> + MacroInfo *MI) {
>>>> + // If this is a macro expansion in the "#if !defined(x)" line for the
>>>> file,
>>>> + // then the macro could expand to different things in other contexts,
>>>> we need
>>>> + // to disable the optimization in this case.
>>>> + if (CurPPLexer) CurPPLexer->MIOpt.ExpandedMacro();
>>>> +
>>>> + // If this is a builtin macro, like __LINE__ or _Pragma, handle it
>>>> specially.
>>>> + if (MI->isBuiltinMacro()) {
>>>> + if (Callbacks) Callbacks->MacroExpands(Identifier, MI,
>>>> + Identifier.getLocation());
>>>> + ExpandBuiltinMacro(Identifier);
>>>> + return false;
>>>> + }
>>>> +
>>>> + /// Args - If this is a function-like macro expansion, this contains,
>>>> + /// for each macro argument, the list of tokens that were provided to
>>>> the
>>>> + /// invocation.
>>>> + MacroArgs *Args = 0;
>>>> +
>>>> + // Remember where the end of the expansion occurred. For an
>>>> object-like
>>>> + // macro, this is the identifier. For a function-like macro, this is
>>>> the ')'.
>>>> + SourceLocation ExpansionEnd = Identifier.getLocation();
>>>> +
>>>> + // If this is a function-like macro, read the arguments.
>>>> + if (MI->isFunctionLike()) {
>>>> + // C99 6.10.3p10: If the preprocessing token immediately after the
>>>> macro
>>>> + // name isn't a '(', this macro should not be expanded.
>>>> + if (!isNextPPTokenLParen())
>>>> + return true;
>>>> +
>>>> + // Remember that we are now parsing the arguments to a macro
>>>> invocation.
>>>> + // Preprocessor directives used inside macro arguments are not
>>>> portable, and
>>>> + // this enables the warning.
>>>> + InMacroArgs = true;
>>>> + Args = ReadFunctionLikeMacroArgs(Identifier, MI, ExpansionEnd);
>>>> +
>>>> + // Finished parsing args.
>>>> + InMacroArgs = false;
>>>> +
>>>> + // If there was an error parsing the arguments, bail out.
>>>> + if (Args == 0) return false;
>>>> +
>>>> + ++NumFnMacroExpanded;
>>>> + } else {
>>>> + ++NumMacroExpanded;
>>>> + }
>>>> +
>>>> + // Notice that this macro has been used.
>>>> + markMacroAsUsed(MI);
>>>> +
>>>> + // Remember where the token is expanded.
>>>> + SourceLocation ExpandLoc = Identifier.getLocation();
>>>> + SourceRange ExpansionRange(ExpandLoc, ExpansionEnd);
>>>> +
>>>> + if (Callbacks) {
>>>> + if (InMacroArgs) {
>>>> + // We can have macro expansion inside a conditional directive
>>>> while
>>>> + // reading the function macro arguments. To ensure, in that case,
>>>> that
>>>> + // MacroExpands callbacks still happen in source order, queue this
>>>> + // callback to have it happen after the function macro callback.
>>>> + DelayedMacroExpandsCallbacks.push_back(
>>>> + MacroExpandsInfo(Identifier, MI,
>>>> ExpansionRange));
>>>> + } else {
>>>> + Callbacks->MacroExpands(Identifier, MI, ExpansionRange);
>>>> + if (!DelayedMacroExpandsCallbacks.empty()) {
>>>> + for (unsigned i=0, e = DelayedMacroExpandsCallbacks.size();
>>>> i!=e; ++i) {
>>>> + MacroExpandsInfo &Info = DelayedMacroExpandsCallbacks[i];
>>>> + Callbacks->MacroExpands(Info.Tok, Info.MI, Info.Range);
>>>> + }
>>>> + DelayedMacroExpandsCallbacks.clear();
>>>> + }
>>>> + }
>>>> + }
>>>> +
>>>> + // If we started lexing a macro, enter the macro expansion body.
>>>> +
>>>> + // If this macro expands to no tokens, don't bother to push it onto
>>>> the
>>>> + // expansion stack, only to take it right back off.
>>>> + if (MI->getNumTokens() == 0) {
>>>> + // No need for arg info.
>>>> + if (Args) Args->destroy(*this);
>>>> +
>>>> + // Ignore this macro use, just return the next token in the current
>>>> + // buffer.
>>>> + bool HadLeadingSpace = Identifier.hasLeadingSpace();
>>>> + bool IsAtStartOfLine = Identifier.isAtStartOfLine();
>>>> +
>>>> + Lex(Identifier);
>>>> +
>>>> + // If the identifier isn't on some OTHER line, inherit the leading
>>>> + // whitespace/first-on-a-line property of this token. This handles
>>>> + // stuff like "! XX," -> "! ," and " XX," -> " ,", when XX is
>>>> + // empty.
>>>> + if (!Identifier.isAtStartOfLine()) {
>>>> + if (IsAtStartOfLine) Identifier.setFlag(Token::StartOfLine);
>>>> + if (HadLeadingSpace) Identifier.setFlag(Token::LeadingSpace);
>>>> + }
>>>> + Identifier.setFlag(Token::LeadingEmptyMacro);
>>>> + ++NumFastMacroExpanded;
>>>> + return false;
>>>> +
>>>> + } else if (MI->getNumTokens() == 1 &&
>>>> + isTrivialSingleTokenExpansion(MI,
>>>> Identifier.getIdentifierInfo(),
>>>> + *this)) {
>>>> + // Otherwise, if this macro expands into a single trivially-expanded
>>>> + // token: expand it now. This handles common cases like
>>>> + // "#define VAL 42".
>>>> +
>>>> + // No need for arg info.
>>>> + if (Args) Args->destroy(*this);
>>>> +
>>>> + // Propagate the isAtStartOfLine/hasLeadingSpace markers of the
>>>> macro
>>>> + // identifier to the expanded token.
>>>> + bool isAtStartOfLine = Identifier.isAtStartOfLine();
>>>> + bool hasLeadingSpace = Identifier.hasLeadingSpace();
>>>> +
>>>> + // Replace the result token.
>>>> + Identifier = MI->getReplacementToken(0);
>>>> +
>>>> + // Restore the StartOfLine/LeadingSpace markers.
>>>> + Identifier.setFlagValue(Token::StartOfLine , isAtStartOfLine);
>>>> + Identifier.setFlagValue(Token::LeadingSpace, hasLeadingSpace);
>>>> +
>>>> + // Update the tokens location to include both its expansion and
>>>> physical
>>>> + // locations.
>>>> + SourceLocation Loc =
>>>> + SourceMgr.createExpansionLoc(Identifier.getLocation(), ExpandLoc,
>>>> + ExpansionEnd,Identifier.getLength());
>>>> + Identifier.setLocation(Loc);
>>>> +
>>>> + // If this is a disabled macro or #define X X, we must mark the
>>>> result as
>>>> + // unexpandable.
>>>> + if (IdentifierInfo *NewII = Identifier.getIdentifierInfo()) {
>>>> + if (MacroInfo *NewMI = getMacroInfo(NewII))
>>>> + if (!NewMI->isEnabled() || NewMI == MI) {
>>>> + Identifier.setFlag(Token::DisableExpand);
>>>> + Diag(Identifier, diag::pp_disabled_macro_expansion);
>>>> + }
>>>> + }
>>>> +
>>>> + // Since this is not an identifier token, it can't be macro
>>>> expanded, so
>>>> + // we're done.
>>>> + ++NumFastMacroExpanded;
>>>> + return false;
>>>> + }
>>>> +
>>>> + // Start expanding the macro.
>>>> + EnterMacro(Identifier, ExpansionEnd, MI, Args);
>>>> +
>>>> + // Now that the macro is at the top of the include stack, ask the
>>>> + // preprocessor to read the next token from it.
>>>> + Lex(Identifier);
>>>> + return false;
>>>> +}
>>>> +
>>>> +/// ReadFunctionLikeMacroArgs - After reading "MACRO" and knowing that
>>>> the next
>>>> +/// token is the '(' of the macro, this method is invoked to read all of
>>>> the
>>>> +/// actual arguments specified for the macro invocation. This returns
>>>> null on
>>>> +/// error.
>>>> +MacroArgs *Preprocessor::ReadFunctionLikeMacroArgs(Token &MacroName,
>>>> + MacroInfo *MI,
>>>> + SourceLocation
>>>> &MacroEnd) {
>>>> + // The number of fixed arguments to parse.
>>>> + unsigned NumFixedArgsLeft = MI->getNumArgs();
>>>> + bool isVariadic = MI->isVariadic();
>>>> +
>>>> + // Outer loop, while there are more arguments, keep reading them.
>>>> + Token Tok;
>>>> +
>>>> + // Read arguments as unexpanded tokens. This avoids issues, e.g.,
>>>> where
>>>> + // an argument value in a macro could expand to ',' or '(' or ')'.
>>>> + LexUnexpandedToken(Tok);
>>>> + assert(Tok.is(tok::l_paren) && "Error computing l-paren-ness?");
>>>> +
>>>> + // ArgTokens - Build up a list of tokens that make up each argument.
>>>> Each
>>>> + // argument is separated by an EOF token. Use a SmallVector so we can
>>>> avoid
>>>> + // heap allocations in the common case.
>>>> + SmallVector<Token, 64> ArgTokens;
>>>> +
>>>> + unsigned NumActuals = 0;
>>>> + while (Tok.isNot(tok::r_paren)) {
>>>> + assert((Tok.is(tok::l_paren) || Tok.is(tok::comma)) &&
>>>> + "only expect argument separators here");
>>>> +
>>>> + unsigned ArgTokenStart = ArgTokens.size();
>>>> + SourceLocation ArgStartLoc = Tok.getLocation();
>>>> +
>>>> + // C99 6.10.3p11: Keep track of the number of l_parens we have seen.
>>>> Note
>>>> + // that we already consumed the first one.
>>>> + unsigned NumParens = 0;
>>>> +
>>>> + while (1) {
>>>> + // Read arguments as unexpanded tokens. This avoids issues, e.g.,
>>>> where
>>>> + // an argument value in a macro could expand to ',' or '(' or ')'.
>>>> + LexUnexpandedToken(Tok);
>>>> +
>>>> + if (Tok.is(tok::eof) || Tok.is(tok::eod)) { // "#if f(<eof>" &
>>>> "#if f(\n"
>>>> + Diag(MacroName, diag::err_unterm_macro_invoc);
>>>> + // Do not lose the EOF/EOD. Return it to the client.
>>>> + MacroName = Tok;
>>>> + return 0;
>>>> + } else if (Tok.is(tok::r_paren)) {
>>>> + // If we found the ) token, the macro arg list is done.
>>>> + if (NumParens-- == 0) {
>>>> + MacroEnd = Tok.getLocation();
>>>> + break;
>>>> + }
>>>> + } else if (Tok.is(tok::l_paren)) {
>>>> + ++NumParens;
>>>> + // In Microsoft-compatibility mode, commas from nested macro
>>>> expan-
>>>> + // sions should not be considered as argument separators. We test
>>>> + // for this with the IgnoredComma token flag.
>>>> + } else if (Tok.is(tok::comma)
>>>> + && !(Tok.getFlags() & Token::IgnoredComma) && NumParens == 0)
>>>> {
>>>> + // Comma ends this argument if there are more fixed arguments
>>>> expected.
>>>> + // However, if this is a variadic macro, and this is part of the
>>>> + // variadic part, then the comma is just an argument token.
>>>> + if (!isVariadic) break;
>>>> + if (NumFixedArgsLeft > 1)
>>>> + break;
>>>> + } else if (Tok.is(tok::comment) && !KeepMacroComments) {
>>>> + // If this is a comment token in the argument list and we're
>>>> just in
>>>> + // -C mode (not -CC mode), discard the comment.
>>>> + continue;
>>>> + } else if (Tok.getIdentifierInfo() != 0) {
>>>> + // Reading macro arguments can cause macros that we are
>>>> currently
>>>> + // expanding from to be popped off the expansion stack. Doing
>>>> so causes
>>>> + // them to be reenabled for expansion. Here we record whether
>>>> any
>>>> + // identifiers we lex as macro arguments correspond to disabled
>>>> macros.
>>>> + // If so, we mark the token as noexpand. This is a subtle
>>>> aspect of
>>>> + // C99 6.10.3.4p2.
>>>> + if (MacroInfo *MI = getMacroInfo(Tok.getIdentifierInfo()))
>>>> + if (!MI->isEnabled())
>>>> + Tok.setFlag(Token::DisableExpand);
>>>> + } else if (Tok.is(tok::code_completion)) {
>>>> + if (CodeComplete)
>>>> +
>>>> CodeComplete->CodeCompleteMacroArgument(MacroName.getIdentifierInfo(),
>>>> + MI, NumActuals);
>>>> + // Don't mark that we reached the code-completion point because
>>>> the
>>>> + // parser is going to handle the token and there will be another
>>>> + // code-completion callback.
>>>> + }
>>>> +
>>>> + ArgTokens.push_back(Tok);
>>>> + }
>>>> +
>>>> + // If this was an empty argument list foo(), don't add this as an
>>>> empty
>>>> + // argument.
>>>> + if (ArgTokens.empty() && Tok.getKind() == tok::r_paren)
>>>> + break;
>>>> +
>>>> + // If this is not a variadic macro, and too many args were
>>>> specified, emit
>>>> + // an error.
>>>> + if (!isVariadic && NumFixedArgsLeft == 0) {
>>>> + if (ArgTokens.size() != ArgTokenStart)
>>>> + ArgStartLoc = ArgTokens[ArgTokenStart].getLocation();
>>>> +
>>>> + // Emit the diagnostic at the macro name in case there is a
>>>> missing ).
>>>> + // Emitting it at the , could be far away from the macro name.
>>>> + Diag(ArgStartLoc, diag::err_too_many_args_in_macro_invoc);
>>>> + return 0;
>>>> + }
>>>> +
>>>> + // Empty arguments are standard in C99 and C++0x, and are supported
>>>> as an extension in
>>>> + // other modes.
>>>> + if (ArgTokens.size() == ArgTokenStart && !LangOpts.C99)
>>>> + Diag(Tok, LangOpts.CPlusPlus0x ?
>>>> + diag::warn_cxx98_compat_empty_fnmacro_arg :
>>>> + diag::ext_empty_fnmacro_arg);
>>>> +
>>>> + // Add a marker EOF token to the end of the token list for this
>>>> argument.
>>>> + Token EOFTok;
>>>> + EOFTok.startToken();
>>>> + EOFTok.setKind(tok::eof);
>>>> + EOFTok.setLocation(Tok.getLocation());
>>>> + EOFTok.setLength(0);
>>>> + ArgTokens.push_back(EOFTok);
>>>> + ++NumActuals;
>>>> + assert(NumFixedArgsLeft != 0 && "Too many arguments parsed");
>>>> + --NumFixedArgsLeft;
>>>> + }
>>>> +
>>>> + // Okay, we either found the r_paren. Check to see if we parsed too
>>>> few
>>>> + // arguments.
>>>> + unsigned MinArgsExpected = MI->getNumArgs();
>>>> +
>>>> + // See MacroArgs instance var for description of this.
>>>> + bool isVarargsElided = false;
>>>> +
>>>> + if (NumActuals < MinArgsExpected) {
>>>> + // There are several cases where too few arguments is ok, handle
>>>> them now.
>>>> + if (NumActuals == 0 && MinArgsExpected == 1) {
>>>> + // #define A(X) or #define A(...) ---> A()
>>>> +
>>>> + // If there is exactly one argument, and that argument is missing,
>>>> + // then we have an empty "()" argument empty list. This is fine,
>>>> even if
>>>> + // the macro expects one argument (the argument is just empty).
>>>> + isVarargsElided = MI->isVariadic();
>>>> + } else if (MI->isVariadic() &&
>>>> + (NumActuals+1 == MinArgsExpected || // A(x, ...) -> A(X)
>>>> + (NumActuals == 0 && MinArgsExpected == 2))) {// A(x,...)
>>>> -> A()
>>>> + // Varargs where the named vararg parameter is missing: OK as
>>>> extension.
>>>> + // #define A(x, ...)
>>>> + // A("blah")
>>>> + Diag(Tok, diag::ext_missing_varargs_arg);
>>>> + Diag(MI->getDefinitionLoc(), diag::note_macro_here)
>>>> + << MacroName.getIdentifierInfo();
>>>> +
>>>> + // Remember this occurred, allowing us to elide the comma when
>>>> used for
>>>> + // cases like:
>>>> + // #define A(x, foo...) blah(a, ## foo)
>>>> + // #define B(x, ...) blah(a, ## __VA_ARGS__)
>>>> + // #define C(...) blah(a, ## __VA_ARGS__)
>>>> + // A(x) B(x) C()
>>>> + isVarargsElided = true;
>>>> + } else {
>>>> + // Otherwise, emit the error.
>>>> + Diag(Tok, diag::err_too_few_args_in_macro_invoc);
>>>> + return 0;
>>>> + }
>>>> +
>>>> + // Add a marker EOF token to the end of the token list for this
>>>> argument.
>>>> + SourceLocation EndLoc = Tok.getLocation();
>>>> + Tok.startToken();
>>>> + Tok.setKind(tok::eof);
>>>> + Tok.setLocation(EndLoc);
>>>> + Tok.setLength(0);
>>>> + ArgTokens.push_back(Tok);
>>>> +
>>>> + // If we expect two arguments, add both as empty.
>>>> + if (NumActuals == 0 && MinArgsExpected == 2)
>>>> + ArgTokens.push_back(Tok);
>>>> +
>>>> + } else if (NumActuals > MinArgsExpected && !MI->isVariadic()) {
>>>> + // Emit the diagnostic at the macro name in case there is a missing
>>>> ).
>>>> + // Emitting it at the , could be far away from the macro name.
>>>> + Diag(MacroName, diag::err_too_many_args_in_macro_invoc);
>>>> + return 0;
>>>> + }
>>>> +
>>>> + return MacroArgs::create(MI, ArgTokens, isVarargsElided, *this);
>>>> +}
>>>> +
>>>> +/// \brief Keeps macro expanded tokens for TokenLexers.
>>>> +//
>>>> +/// Works like a stack; a TokenLexer adds the macro expanded tokens that
>>>> is
>>>> +/// going to lex in the cache and when it finishes the tokens are
>>>> removed
>>>> +/// from the end of the cache.
>>>> +Token *Preprocessor::cacheMacroExpandedTokens(TokenLexer *tokLexer,
>>>> + ArrayRef<Token> tokens) {
>>>> + assert(tokLexer);
>>>> + if (tokens.empty())
>>>> + return 0;
>>>> +
>>>> + size_t newIndex = MacroExpandedTokens.size();
>>>> + bool cacheNeedsToGrow = tokens.size() >
>>>> +
>>>> MacroExpandedTokens.capacity()-MacroExpandedTokens.size();
>>>> + MacroExpandedTokens.append(tokens.begin(), tokens.end());
>>>> +
>>>> + if (cacheNeedsToGrow) {
>>>> + // Go through all the TokenLexers whose 'Tokens' pointer points in
>>>> the
>>>> + // buffer and update the pointers to the (potential) new buffer
>>>> array.
>>>> + for (unsigned i = 0, e = MacroExpandingLexersStack.size(); i != e;
>>>> ++i) {
>>>> + TokenLexer *prevLexer;
>>>> + size_t tokIndex;
>>>> + llvm::tie(prevLexer, tokIndex) = MacroExpandingLexersStack[i];
>>>> + prevLexer->Tokens = MacroExpandedTokens.data() + tokIndex;
>>>> + }
>>>> + }
>>>> +
>>>> + MacroExpandingLexersStack.push_back(std::make_pair(tokLexer,
>>>> newIndex));
>>>> + return MacroExpandedTokens.data() + newIndex;
>>>> +}
>>>> +
>>>> +void Preprocessor::removeCachedMacroExpandedTokensOfLastLexer() {
>>>> + assert(!MacroExpandingLexersStack.empty());
>>>> + size_t tokIndex = MacroExpandingLexersStack.back().second;
>>>> + assert(tokIndex < MacroExpandedTokens.size());
>>>> + // Pop the cached macro expanded tokens from the end.
>>>> + MacroExpandedTokens.resize(tokIndex);
>>>> + MacroExpandingLexersStack.pop_back();
>>>> +}
>>>> +
>>>> +/// ComputeDATE_TIME - Compute the current time, enter it into the
>>>> specified
>>>> +/// scratch buffer, then return DATELoc/TIMELoc locations with the
>>>> position of
>>>> +/// the identifier tokens inserted.
>>>> +static void ComputeDATE_TIME(SourceLocation &DATELoc, SourceLocation
>>>> &TIMELoc,
>>>> + Preprocessor &PP) {
>>>> + time_t TT = time(0);
>>>> + struct tm *TM = localtime(&TT);
>>>> +
>>>> + static const char * const Months[] = {
>>>> +
>>>> "Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"
>>>> + };
>>>> +
>>>> + char TmpBuffer[32];
>>>> +#ifdef LLVM_ON_WIN32
>>>> + sprintf(TmpBuffer, "\"%s %2d %4d\"", Months[TM->tm_mon], TM->tm_mday,
>>>> + TM->tm_year+1900);
>>>> +#else
>>>> + snprintf(TmpBuffer, sizeof(TmpBuffer), "\"%s %2d %4d\"",
>>>> Months[TM->tm_mon], TM->tm_mday,
>>>> + TM->tm_year+1900);
>>>> +#endif
>>>> +
>>>> + Token TmpTok;
>>>> + TmpTok.startToken();
>>>> + PP.CreateString(TmpBuffer, strlen(TmpBuffer), TmpTok);
>>>> + DATELoc = TmpTok.getLocation();
>>>> +
>>>> +#ifdef LLVM_ON_WIN32
>>>> + sprintf(TmpBuffer, "\"%02d:%02d:%02d\"", TM->tm_hour, TM->tm_min,
>>>> TM->tm_sec);
>>>> +#else
>>>> + snprintf(TmpBuffer, sizeof(TmpBuffer), "\"%02d:%02d:%02d\"",
>>>> TM->tm_hour, TM->tm_min, TM->tm_sec);
>>>> +#endif
>>>> + PP.CreateString(TmpBuffer, strlen(TmpBuffer), TmpTok);
>>>> + TIMELoc = TmpTok.getLocation();
>>>> +}
>>>> +
>>>> +
>>>> +/// HasFeature - Return true if we recognize and implement the feature
>>>> +/// specified by the identifier as a standard language feature.
>>>> +static bool HasFeature(const Preprocessor &PP, const IdentifierInfo *II)
>>>> {
>>>> + const LangOptions &LangOpts = PP.getLangOpts();
>>>> + StringRef Feature = II->getName();
>>>> +
>>>> + // Normalize the feature name, __foo__ becomes foo.
>>>> + if (Feature.startswith("__") && Feature.endswith("__") &&
>>>> Feature.size() >= 4)
>>>> + Feature = Feature.substr(2, Feature.size() - 4);
>>>> +
>>>> + return llvm::StringSwitch<bool>(Feature)
>>>> + .Case("address_sanitizer", LangOpts.AddressSanitizer)
>>>> + .Case("attribute_analyzer_noreturn", true)
>>>> + .Case("attribute_availability", true)
>>>> + .Case("attribute_availability_with_message", true)
>>>> + .Case("attribute_cf_returns_not_retained", true)
>>>> + .Case("attribute_cf_returns_retained", true)
>>>> + .Case("attribute_deprecated_with_message", true)
>>>> + .Case("attribute_ext_vector_type", true)
>>>> + .Case("attribute_ns_returns_not_retained", true)
>>>> + .Case("attribute_ns_returns_retained", true)
>>>> + .Case("attribute_ns_consumes_self", true)
>>>> + .Case("attribute_ns_consumed", true)
>>>> + .Case("attribute_cf_consumed", true)
>>>> + .Case("attribute_objc_ivar_unused", true)
>>>> + .Case("attribute_objc_method_family", true)
>>>> + .Case("attribute_overloadable", true)
>>>> + .Case("attribute_unavailable_with_message", true)
>>>> + .Case("attribute_unused_on_fields", true)
>>>> + .Case("blocks", LangOpts.Blocks)
>>>> + .Case("cxx_exceptions", LangOpts.Exceptions)
>>>> + .Case("cxx_rtti", LangOpts.RTTI)
>>>> + .Case("enumerator_attributes", true)
>>>> + // Objective-C features
>>>> + .Case("objc_arr", LangOpts.ObjCAutoRefCount) // FIXME:
>>>> REMOVE?
>>>> + .Case("objc_arc", LangOpts.ObjCAutoRefCount)
>>>> + .Case("objc_arc_weak", LangOpts.ObjCARCWeak)
>>>> + .Case("objc_default_synthesize_properties", LangOpts.ObjC2)
>>>> + .Case("objc_fixed_enum", LangOpts.ObjC2)
>>>> + .Case("objc_instancetype", LangOpts.ObjC2)
>>>> + .Case("objc_modules", LangOpts.ObjC2 && LangOpts.Modules)
>>>> + .Case("objc_nonfragile_abi",
>>>> LangOpts.ObjCRuntime.isNonFragile())
>>>> + .Case("objc_weak_class",
>>>> LangOpts.ObjCRuntime.hasWeakClassImport())
>>>> + .Case("ownership_holds", true)
>>>> + .Case("ownership_returns", true)
>>>> + .Case("ownership_takes", true)
>>>> + .Case("objc_bool", true)
>>>> + .Case("objc_subscripting",
>>>> LangOpts.ObjCRuntime.isNonFragile())
>>>> + .Case("objc_array_literals", LangOpts.ObjC2)
>>>> + .Case("objc_dictionary_literals", LangOpts.ObjC2)
>>>> + .Case("objc_boxed_expressions", LangOpts.ObjC2)
>>>> + .Case("arc_cf_code_audited", true)
>>>> + // C11 features
>>>> + .Case("c_alignas", LangOpts.C11)
>>>> + .Case("c_atomic", LangOpts.C11)
>>>> + .Case("c_generic_selections", LangOpts.C11)
>>>> + .Case("c_static_assert", LangOpts.C11)
>>>> + // C++11 features
>>>> + .Case("cxx_access_control_sfinae", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_alias_templates", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_alignas", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_atomic", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_attributes", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_auto_type", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_constexpr", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_decltype", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_decltype_incomplete_return_types",
>>>> LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_default_function_template_args",
>>>> LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_defaulted_functions", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_delegating_constructors", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_deleted_functions", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_explicit_conversions", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_generalized_initializers", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_implicit_moves", LangOpts.CPlusPlus0x)
>>>> + //.Case("cxx_inheriting_constructors", false)
>>>> + .Case("cxx_inline_namespaces", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_lambdas", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_local_type_template_args", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_nonstatic_member_init", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_noexcept", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_nullptr", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_override_control", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_range_for", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_raw_string_literals", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_reference_qualified_functions",
>>>> LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_rvalue_references", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_strong_enums", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_static_assert", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_trailing_return", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_unicode_literals", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_unrestricted_unions", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_user_literals", LangOpts.CPlusPlus0x)
>>>> + .Case("cxx_variadic_templates", LangOpts.CPlusPlus0x)
>>>> + // Type traits
>>>> + .Case("has_nothrow_assign", LangOpts.CPlusPlus)
>>>> + .Case("has_nothrow_copy", LangOpts.CPlusPlus)
>>>> + .Case("has_nothrow_constructor", LangOpts.CPlusPlus)
>>>> + .Case("has_trivial_assign", LangOpts.CPlusPlus)
>>>> + .Case("has_trivial_copy", LangOpts.CPlusPlus)
>>>> + .Case("has_trivial_constructor", LangOpts.CPlusPlus)
>>>> + .Case("has_trivial_destructor", LangOpts.CPlusPlus)
>>>> + .Case("has_virtual_destructor", LangOpts.CPlusPlus)
>>>> + .Case("is_abstract", LangOpts.CPlusPlus)
>>>> + .Case("is_base_of", LangOpts.CPlusPlus)
>>>> + .Case("is_class", LangOpts.CPlusPlus)
>>>> + .Case("is_convertible_to", LangOpts.CPlusPlus)
>>>> + // __is_empty is available only if the horrible
>>>> + // "struct __is_empty" parsing hack hasn't been needed in
>>>> this
>>>> + // translation unit. If it has, __is_empty reverts to a
>>>> normal
>>>> + // identifier and __has_feature(is_empty) evaluates false.
>>>> + .Case("is_empty", LangOpts.CPlusPlus)
>>>> + .Case("is_enum", LangOpts.CPlusPlus)
>>>> + .Case("is_final", LangOpts.CPlusPlus)
>>>> + .Case("is_literal", LangOpts.CPlusPlus)
>>>> + .Case("is_standard_layout", LangOpts.CPlusPlus)
>>>> + .Case("is_pod", LangOpts.CPlusPlus)
>>>> + .Case("is_polymorphic", LangOpts.CPlusPlus)
>>>> + .Case("is_trivial", LangOpts.CPlusPlus)
>>>> + .Case("is_trivially_assignable", LangOpts.CPlusPlus)
>>>> + .Case("is_trivially_constructible", LangOpts.CPlusPlus)
>>>> + .Case("is_trivially_copyable", LangOpts.CPlusPlus)
>>>> + .Case("is_union", LangOpts.CPlusPlus)
>>>> + .Case("modules", LangOpts.Modules)
>>>> + .Case("tls", PP.getTargetInfo().isTLSSupported())
>>>> + .Case("underlying_type", LangOpts.CPlusPlus)
>>>> + .Default(false);
>>>> +}
>>>> +
>>>> +/// HasExtension - Return true if we recognize and implement the feature
>>>> +/// specified by the identifier, either as an extension or a standard
>>>> language
>>>> +/// feature.
>>>> +static bool HasExtension(const Preprocessor &PP, const IdentifierInfo
>>>> *II) {
>>>> + if (HasFeature(PP, II))
>>>> + return true;
>>>> +
>>>> + // If the use of an extension results in an error diagnostic,
>>>> extensions are
>>>> + // effectively unavailable, so just return false here.
>>>> + if (PP.getDiagnostics().getExtensionHandlingBehavior() ==
>>>> + DiagnosticsEngine::Ext_Error)
>>>> + return false;
>>>> +
>>>> + const LangOptions &LangOpts = PP.getLangOpts();
>>>> + StringRef Extension = II->getName();
>>>> +
>>>> + // Normalize the extension name, __foo__ becomes foo.
>>>> + if (Extension.startswith("__") && Extension.endswith("__") &&
>>>> + Extension.size() >= 4)
>>>> + Extension = Extension.substr(2, Extension.size() - 4);
>>>> +
>>>> + // Because we inherit the feature list from HasFeature, this string
>>>> switch
>>>> + // must be less restrictive than HasFeature's.
>>>> + return llvm::StringSwitch<bool>(Extension)
>>>> + // C11 features supported by other languages as extensions.
>>>> + .Case("c_alignas", true)
>>>> + .Case("c_atomic", true)
>>>> + .Case("c_generic_selections", true)
>>>> + .Case("c_static_assert", true)
>>>> + // C++0x features supported by other languages as extensions.
>>>> + .Case("cxx_atomic", LangOpts.CPlusPlus)
>>>> + .Case("cxx_deleted_functions", LangOpts.CPlusPlus)
>>>> + .Case("cxx_explicit_conversions", LangOpts.CPlusPlus)
>>>> + .Case("cxx_inline_namespaces", LangOpts.CPlusPlus)
>>>> + .Case("cxx_local_type_template_args", LangOpts.CPlusPlus)
>>>> + .Case("cxx_nonstatic_member_init", LangOpts.CPlusPlus)
>>>> + .Case("cxx_override_control", LangOpts.CPlusPlus)
>>>> + .Case("cxx_range_for", LangOpts.CPlusPlus)
>>>> + .Case("cxx_reference_qualified_functions",
>>>> LangOpts.CPlusPlus)
>>>> + .Case("cxx_rvalue_references", LangOpts.CPlusPlus)
>>>> + .Default(false);
>>>> +}
>>>> +
>>>> +/// HasAttribute - Return true if we recognize and implement the
>>>> attribute
>>>> +/// specified by the given identifier.
>>>> +static bool HasAttribute(const IdentifierInfo *II) {
>>>> + StringRef Name = II->getName();
>>>> + // Normalize the attribute name, __foo__ becomes foo.
>>>> + if (Name.startswith("__") && Name.endswith("__") && Name.size() >= 4)
>>>> + Name = Name.substr(2, Name.size() - 4);
>>>> +
>>>> + // FIXME: Do we need to handle namespaces here?
>>>> + return llvm::StringSwitch<bool>(Name)
>>>> +#include "clang/Lex/AttrSpellings.inc"
>>>> + .Default(false);
>>>> +}
>>>> +
>>>> +/// EvaluateHasIncludeCommon - Process a '__has_include("path")'
>>>> +/// or '__has_include_next("path")' expression.
>>>> +/// Returns true if successful.
>>>> +static bool EvaluateHasIncludeCommon(Token &Tok,
>>>> + IdentifierInfo *II, Preprocessor
>>>> &PP,
>>>> + const DirectoryLookup *LookupFrom)
>>>> {
>>>> + SourceLocation LParenLoc;
>>>> +
>>>> + // Get '('.
>>>> + PP.LexNonComment(Tok);
>>>> +
>>>> + // Ensure we have a '('.
>>>> + if (Tok.isNot(tok::l_paren)) {
>>>> + PP.Diag(Tok.getLocation(), diag::err_pp_missing_lparen) <<
>>>> II->getName();
>>>> + return false;
>>>> + }
>>>> +
>>>> + // Save '(' location for possible missing ')' message.
>>>> + LParenLoc = Tok.getLocation();
>>>> +
>>>> + // Get the file name.
>>>> + PP.getCurrentLexer()->LexIncludeFilename(Tok);
>>>> +
>>>> + // Reserve a buffer to get the spelling.
>>>> + SmallString<128> FilenameBuffer;
>>>> + StringRef Filename;
>>>> + SourceLocation EndLoc;
>>>> +
>>>> + switch (Tok.getKind()) {
>>>> + case tok::eod:
>>>> + // If the token kind is EOD, the error has already been diagnosed.
>>>> + return false;
>>>> +
>>>> + case tok::angle_string_literal:
>>>> + case tok::string_literal: {
>>>> + bool Invalid = false;
>>>> + Filename = PP.getSpelling(Tok, FilenameBuffer, &Invalid);
>>>> + if (Invalid)
>>>> + return false;
>>>> + break;
>>>> + }
>>>> +
>>>> + case tok::less:
>>>> + // This could be a <foo/bar.h> file coming from a macro expansion.
>>>> In this
>>>> + // case, glue the tokens together into FilenameBuffer and interpret
>>>> those.
>>>> + FilenameBuffer.push_back('<');
>>>> + if (PP.ConcatenateIncludeName(FilenameBuffer, EndLoc))
>>>> + return false; // Found <eod> but no ">"? Diagnostic already
>>>> emitted.
>>>> + Filename = FilenameBuffer.str();
>>>> + break;
>>>> + default:
>>>> + PP.Diag(Tok.getLocation(), diag::err_pp_expects_filename);
>>>> + return false;
>>>> + }
>>>> +
>>>> + // Get ')'.
>>>> + PP.LexNonComment(Tok);
>>>> +
>>>> + // Ensure we have a trailing ).
>>>> + if (Tok.isNot(tok::r_paren)) {
>>>> + PP.Diag(Tok.getLocation(), diag::err_pp_missing_rparen) <<
>>>> II->getName();
>>>> + PP.Diag(LParenLoc, diag::note_matching) << "(";
>>>> + return false;
>>>> + }
>>>> +
>>>> + bool isAngled = PP.GetIncludeFilenameSpelling(Tok.getLocation(),
>>>> Filename);
>>>> + // If GetIncludeFilenameSpelling set the start ptr to null, there was
>>>> an
>>>> + // error.
>>>> + if (Filename.empty())
>>>> + return false;
>>>> +
>>>> + // Search include directories.
>>>> + const DirectoryLookup *CurDir;
>>>> + const FileEntry *File =
>>>> + PP.LookupFile(Filename, isAngled, LookupFrom, CurDir, NULL, NULL,
>>>> NULL);
>>>> +
>>>> + // Get the result value. A result of true means the file exists.
>>>> + return File != 0;
>>>> +}
>>>> +
>>>> +/// EvaluateHasInclude - Process a '__has_include("path")' expression.
>>>> +/// Returns true if successful.
>>>> +static bool EvaluateHasInclude(Token &Tok, IdentifierInfo *II,
>>>> + Preprocessor &PP) {
>>>> + return EvaluateHasIncludeCommon(Tok, II, PP, NULL);
>>>> +}
>>>> +
>>>> +/// EvaluateHasIncludeNext - Process '__has_include_next("path")'
>>>> expression.
>>>> +/// Returns true if successful.
>>>> +static bool EvaluateHasIncludeNext(Token &Tok,
>>>> + IdentifierInfo *II, Preprocessor &PP)
>>>> {
>>>> + // __has_include_next is like __has_include, except that we start
>>>> + // searching after the current found directory. If we can't do this,
>>>> + // issue a diagnostic.
>>>> + const DirectoryLookup *Lookup = PP.GetCurDirLookup();
>>>> + if (PP.isInPrimaryFile()) {
>>>> + Lookup = 0;
>>>> + PP.Diag(Tok, diag::pp_include_next_in_primary);
>>>> + } else if (Lookup == 0) {
>>>> + PP.Diag(Tok, diag::pp_include_next_absolute_path);
>>>> + } else {
>>>> + // Start looking up in the next directory.
>>>> + ++Lookup;
>>>> + }
>>>> +
>>>> + return EvaluateHasIncludeCommon(Tok, II, PP, Lookup);
>>>> +}
>>>> +
>>>> +/// ExpandBuiltinMacro - If an identifier token is read that is to be
>>>> expanded
>>>> +/// as a builtin macro, handle it and return the next token as 'Tok'.
>>>> +void Preprocessor::ExpandBuiltinMacro(Token &Tok) {
>>>> + // Figure out which token this is.
>>>> + IdentifierInfo *II = Tok.getIdentifierInfo();
>>>> + assert(II && "Can't be a macro without id info!");
>>>> +
>>>> + // If this is an _Pragma or Microsoft __pragma directive, expand it,
>>>> + // invoke the pragma handler, then lex the token after it.
>>>> + if (II == Ident_Pragma)
>>>> + return Handle_Pragma(Tok);
>>>> + else if (II == Ident__pragma) // in non-MS mode this is null
>>>> + return HandleMicrosoft__pragma(Tok);
>>>> +
>>>> + ++NumBuiltinMacroExpanded;
>>>> +
>>>> + SmallString<128> TmpBuffer;
>>>> + llvm::raw_svector_ostream OS(TmpBuffer);
>>>> +
>>>> + // Set up the return result.
>>>> + Tok.setIdentifierInfo(0);
>>>> + Tok.clearFlag(Token::NeedsCleaning);
>>>> +
>>>> + if (II == Ident__LINE__) {
>>>> + // C99 6.10.8: "__LINE__: The presumed line number (within the
>>>> current
>>>> + // source file) of the current source line (an integer constant)".
>>>> This can
>>>> + // be affected by #line.
>>>> + SourceLocation Loc = Tok.getLocation();
>>>> +
>>>> + // Advance to the location of the first _, this might not be the
>>>> first byte
>>>> + // of the token if it starts with an escaped newline.
>>>> + Loc = AdvanceToTokenCharacter(Loc, 0);
>>>> +
>>>> + // One wrinkle here is that GCC expands __LINE__ to location of the
>>>> *end* of
>>>> + // a macro expansion. This doesn't matter for object-like macros,
>>>> but
>>>> + // can matter for a function-like macro that expands to contain
>>>> __LINE__.
>>>> + // Skip down through expansion points until we find a file loc for
>>>> the
>>>> + // end of the expansion history.
>>>> + Loc = SourceMgr.getExpansionRange(Loc).second;
>>>> + PresumedLoc PLoc = SourceMgr.getPresumedLoc(Loc);
>>>> +
>>>> + // __LINE__ expands to a simple numeric value.
>>>> + OS << (PLoc.isValid()? PLoc.getLine() : 1);
>>>> + Tok.setKind(tok::numeric_constant);
>>>> + } else if (II == Ident__FILE__ || II == Ident__BASE_FILE__) {
>>>> + // C99 6.10.8: "__FILE__: The presumed name of the current source
>>>> file (a
>>>> + // character string literal)". This can be affected by #line.
>>>> + PresumedLoc PLoc = SourceMgr.getPresumedLoc(Tok.getLocation());
>>>> +
>>>> + // __BASE_FILE__ is a GNU extension that returns the top of the
>>>> presumed
>>>> + // #include stack instead of the current file.
>>>> + if (II == Ident__BASE_FILE__ && PLoc.isValid()) {
>>>> + SourceLocation NextLoc = PLoc.getIncludeLoc();
>>>> + while (NextLoc.isValid()) {
>>>> + PLoc = SourceMgr.getPresumedLoc(NextLoc);
>>>> + if (PLoc.isInvalid())
>>>> + break;
>>>> +
>>>> + NextLoc = PLoc.getIncludeLoc();
>>>> + }
>>>> + }
>>>> +
>>>> + // Escape this filename. Turn '\' -> '\\' '"' -> '\"'
>>>> + SmallString<128> FN;
>>>> + if (PLoc.isValid()) {
>>>> + FN += PLoc.getFilename();
>>>> + Lexer::Stringify(FN);
>>>> + OS << '"' << FN.str() << '"';
>>>> + }
>>>> + Tok.setKind(tok::string_literal);
>>>> + } else if (II == Ident__DATE__) {
>>>> + if (!DATELoc.isValid())
>>>> + ComputeDATE_TIME(DATELoc, TIMELoc, *this);
>>>> + Tok.setKind(tok::string_literal);
>>>> + Tok.setLength(strlen("\"Mmm dd yyyy\""));
>>>> + Tok.setLocation(SourceMgr.createExpansionLoc(DATELoc,
>>>> Tok.getLocation(),
>>>> + Tok.getLocation(),
>>>> + Tok.getLength()));
>>>> + return;
>>>> + } else if (II == Ident__TIME__) {
>>>> + if (!TIMELoc.isValid())
>>>> + ComputeDATE_TIME(DATELoc, TIMELoc, *this);
>>>> + Tok.setKind(tok::string_literal);
>>>> + Tok.setLength(strlen("\"hh:mm:ss\""));
>>>> + Tok.setLocation(SourceMgr.createExpansionLoc(TIMELoc,
>>>> Tok.getLocation(),
>>>> + Tok.getLocation(),
>>>> + Tok.getLength()));
>>>> + return;
>>>> + } else if (II == Ident__INCLUDE_LEVEL__) {
>>>> + // Compute the presumed include depth of this token. This can be
>>>> affected
>>>> + // by GNU line markers.
>>>> + unsigned Depth = 0;
>>>> +
>>>> + PresumedLoc PLoc = SourceMgr.getPresumedLoc(Tok.getLocation());
>>>> + if (PLoc.isValid()) {
>>>> + PLoc = SourceMgr.getPresumedLoc(PLoc.getIncludeLoc());
>>>> + for (; PLoc.isValid(); ++Depth)
>>>> + PLoc = SourceMgr.getPresumedLoc(PLoc.getIncludeLoc());
>>>> + }
>>>> +
>>>> + // __INCLUDE_LEVEL__ expands to a simple numeric value.
>>>> + OS << Depth;
>>>> + Tok.setKind(tok::numeric_constant);
>>>> + } else if (II == Ident__TIMESTAMP__) {
>>>> + // MSVC, ICC, GCC, VisualAge C++ extension. The generated string
>>>> should be
>>>> + // of the form "Ddd Mmm dd hh::mm::ss yyyy", which is returned by
>>>> asctime.
>>>> +
>>>> + // Get the file that we are lexing out of. If we're currently
>>>> lexing from
>>>> + // a macro, dig into the include stack.
>>>> + const FileEntry *CurFile = 0;
>>>> + PreprocessorLexer *TheLexer = getCurrentFileLexer();
>>>> +
>>>> + if (TheLexer)
>>>> + CurFile = SourceMgr.getFileEntryForID(TheLexer->getFileID());
>>>> +
>>>> + const char *Result;
>>>> + if (CurFile) {
>>>> + time_t TT = CurFile->getModificationTime();
>>>> + struct tm *TM = localtime(&TT);
>>>> + Result = asctime(TM);
>>>> + } else {
>>>> + Result = "??? ??? ?? ??:??:?? ????\n";
>>>> + }
>>>> + // Surround the string with " and strip the trailing newline.
>>>> + OS << '"' << StringRef(Result, strlen(Result)-1) << '"';
>>>> + Tok.setKind(tok::string_literal);
>>>> + } else if (II == Ident__COUNTER__) {
>>>> + // __COUNTER__ expands to a simple numeric value.
>>>> + OS << CounterValue++;
>>>> + Tok.setKind(tok::numeric_constant);
>>>> + } else if (II == Ident__has_feature ||
>>>> + II == Ident__has_extension ||
>>>> + II == Ident__has_builtin ||
>>>> + II == Ident__has_attribute) {
>>>> + // The argument to these builtins should be a parenthesized
>>>> identifier.
>>>> + SourceLocation StartLoc = Tok.getLocation();
>>>> +
>>>> + bool IsValid = false;
>>>> + IdentifierInfo *FeatureII = 0;
>>>> +
>>>> + // Read the '('.
>>>> + Lex(Tok);
>>>> + if (Tok.is(tok::l_paren)) {
>>>> + // Read the identifier
>>>> + Lex(Tok);
>>>> + if (Tok.is(tok::identifier) || Tok.is(tok::kw_const)) {
>>>> + FeatureII = Tok.getIdentifierInfo();
>>>> +
>>>> + // Read the ')'.
>>>> + Lex(Tok);
>>>> + if (Tok.is(tok::r_paren))
>>>> + IsValid = true;
>>>> + }
>>>> + }
>>>> +
>>>> + bool Value = false;
>>>> + if (!IsValid)
>>>> + Diag(StartLoc, diag::err_feature_check_malformed);
>>>> + else if (II == Ident__has_builtin) {
>>>> + // Check for a builtin is trivial.
>>>> + Value = FeatureII->getBuiltinID() != 0;
>>>> + } else if (II == Ident__has_attribute)
>>>> + Value = HasAttribute(FeatureII);
>>>> + else if (II == Ident__has_extension)
>>>> + Value = HasExtension(*this, FeatureII);
>>>> + else {
>>>> + assert(II == Ident__has_feature && "Must be feature check");
>>>> + Value = HasFeature(*this, FeatureII);
>>>> + }
>>>> +
>>>> + OS << (int)Value;
>>>> + if (IsValid)
>>>> + Tok.setKind(tok::numeric_constant);
>>>> + } else if (II == Ident__has_include ||
>>>> + II == Ident__has_include_next) {
>>>> + // The argument to these two builtins should be a parenthesized
>>>> + // file name string literal using angle brackets (<>) or
>>>> + // double-quotes ("").
>>>> + bool Value;
>>>> + if (II == Ident__has_include)
>>>> + Value = EvaluateHasInclude(Tok, II, *this);
>>>> + else
>>>> + Value = EvaluateHasIncludeNext(Tok, II, *this);
>>>> + OS << (int)Value;
>>>> + Tok.setKind(tok::numeric_constant);
>>>> + } else if (II == Ident__has_warning) {
>>>> + // The argument should be a parenthesized string literal.
>>>> + // The argument to these builtins should be a parenthesized
>>>> identifier.
>>>> + SourceLocation StartLoc = Tok.getLocation();
>>>> + bool IsValid = false;
>>>> + bool Value = false;
>>>> + // Read the '('.
>>>> + Lex(Tok);
>>>> + do {
>>>> + if (Tok.is(tok::l_paren)) {
>>>> + // Read the string.
>>>> + Lex(Tok);
>>>> +
>>>> + // We need at least one string literal.
>>>> + if (!Tok.is(tok::string_literal)) {
>>>> + StartLoc = Tok.getLocation();
>>>> + IsValid = false;
>>>> + // Eat tokens until ')'.
>>>> + do Lex(Tok); while (!(Tok.is(tok::r_paren) ||
>>>> Tok.is(tok::eod)));
>>>> + break;
>>>> + }
>>>> +
>>>> + // String concatenation allows multiple strings, which can even
>>>> come
>>>> + // from macro expansion.
>>>> + SmallVector<Token, 4> StrToks;
>>>> + while (Tok.is(tok::string_literal)) {
>>>> + // Complain about, and drop, any ud-suffix.
>>>> + if (Tok.hasUDSuffix())
>>>> + Diag(Tok, diag::err_invalid_string_udl);
>>>> + StrToks.push_back(Tok);
>>>> + LexUnexpandedToken(Tok);
>>>> + }
>>>> +
>>>> + // Is the end a ')'?
>>>> + if (!(IsValid = Tok.is(tok::r_paren)))
>>>> + break;
>>>> +
>>>> + // Concatenate and parse the strings.
>>>> + StringLiteralParser Literal(&StrToks[0], StrToks.size(), *this);
>>>> + assert(Literal.isAscii() && "Didn't allow wide strings in");
>>>> + if (Literal.hadError)
>>>> + break;
>>>> + if (Literal.Pascal) {
>>>> + Diag(Tok, diag::warn_pragma_diagnostic_invalid);
>>>> + break;
>>>> + }
>>>> +
>>>> + StringRef WarningName(Literal.GetString());
>>>> +
>>>> + if (WarningName.size() < 3 || WarningName[0] != '-' ||
>>>> + WarningName[1] != 'W') {
>>>> + Diag(StrToks[0].getLocation(),
>>>> diag::warn_has_warning_invalid_option);
>>>> + break;
>>>> + }
>>>> +
>>>> + // Finally, check if the warning flags maps to a diagnostic
>>>> group.
>>>> + // We construct a SmallVector here to talk to
>>>> getDiagnosticIDs().
>>>> + // Although we don't use the result, this isn't a hot path, and
>>>> not
>>>> + // worth special casing.
>>>> + llvm::SmallVector<diag::kind, 10> Diags;
>>>> + Value = !getDiagnostics().getDiagnosticIDs()->
>>>> + getDiagnosticsInGroup(WarningName.substr(2), Diags);
>>>> + }
>>>> + } while (false);
>>>> +
>>>> + if (!IsValid)
>>>> + Diag(StartLoc, diag::err_warning_check_malformed);
>>>> +
>>>> + OS << (int)Value;
>>>> + Tok.setKind(tok::numeric_constant);
>>>> + } else {
>>>> + llvm_unreachable("Unknown identifier!");
>>>> + }
>>>> + CreateString(OS.str().data(), OS.str().size(), Tok,
>>>> + Tok.getLocation(), Tok.getLocation());
>>>> +}
>>>> +
>>>> +void Preprocessor::markMacroAsUsed(MacroInfo *MI) {
>>>> + // If the 'used' status changed, and the macro requires 'unused'
>>>> warning,
>>>> + // remove its SourceLocation from the warn-for-unused-macro locations.
>>>> + if (MI->isWarnIfUnused() && !MI->isUsed())
>>>> + WarnUnusedMacroLocs.erase(MI->getDefinitionLoc());
>>>> + MI->setIsUsed(true);
>>>> +}
>>>>
>>>> Modified: cfe/trunk/lib/Lex/TokenLexer.cpp
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/TokenLexer.cpp?rev=163022&r1=163021&r2=163022&view=diff
>>>>
>>>> ==============================================================================
>>>> --- cfe/trunk/lib/Lex/TokenLexer.cpp (original)
>>>> +++ cfe/trunk/lib/Lex/TokenLexer.cpp Fri Aug 31 16:10:54 2012
>>>> @@ -225,6 +225,12 @@
>>>> Token &Tok = ResultToks[i];
>>>> if (Tok.is(tok::hashhash))
>>>> Tok.setKind(tok::unknown);
>>>> + // In Microsoft-compatibility mode, we follow MSVC's
>>>> preprocessing
>>>> + // behaviour by not considering commas from nested macro
>>>> expansions
>>>> + // as argument separators. Set a flag on the token so we can
>>>> test
>>>> + // for this later when the macro expansion is processed.
>>>> + if (Tok.is(tok::comma) && PP.getLangOpts().MicrosoftMode)
>>>> + Tok.setFlag(Token::IgnoredComma);
>>>> }
>>>>
>>>> if(ExpandLocStart.isValid()) {
>>>>
>>>> Added: cfe/trunk/test/Preprocessor/microsoft-ext.c
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Preprocessor/microsoft-ext.c?rev=163022&view=auto
>>>>
>>>> ==============================================================================
>>>> --- cfe/trunk/test/Preprocessor/microsoft-ext.c (added)
>>>> +++ cfe/trunk/test/Preprocessor/microsoft-ext.c Fri Aug 31 16:10:54 2012
>>>> @@ -0,0 +1,7 @@
>>>> +// RUN: %clang_cc1 -E -fms-compatibility %s | FileCheck %s
>>>> +
>>>> +# define M2(x, y) x + y
>>>> +# define P(x, y) {x, y}
>>>> +# define M(x, y) M2(x, P(x, y))
>>>> +M(a, b) // CHECK: a + {a, b}
>>>> +
>>>>
>>>>
>>>> _______________________________________________
>>>> cfe-commits mailing list
>>>> cfe-commits at cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>>>
>>>
>>
>>
>>
>> --
>> João Matos
>>
>> _______________________________________________
>> cfe-commits mailing list
>> cfe-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>>
More information about the cfe-commits
mailing list