[cfe-commits] r163022 - in /cfe/trunk: include/clang/Lex/Token.h lib/Lex/PPMacroExpansion.cpp lib/Lex/TokenLexer.cpp test/Preprocessor/microsoft-ext.c
João Matos
ripzonetriton at gmail.com
Wed Sep 26 02:26:22 PDT 2012
Can you add a new test testing for what regressed?
On Wed, Sep 26, 2012 at 9:21 AM, Nico Weber <thakis at chromium.org> wrote:
> Reverted in r164672. The test you added surprisingly passes even with
> the code changes reverted, so I kept the test around.
>
> Nico
>
> On Wed, Sep 26, 2012 at 4:56 PM, Nico Weber <thakis at chromium.org> wrote:
> > In case anyone was wondering, here's the actual diff to
> PPMacroExpansion.cpp:
> >
> > Nicos-MacBook-Pro:clang thakis$ diff -u ~/triton-before.txt
> ~/triton-after.txt
> > --- /Users/thakis/triton-before.txt 2012-09-26 16:54:53.000000000
> +0900
> > +++ /Users/thakis/triton-after.txt 2012-09-26 16:55:04.000000000
> +0900
> > @@ -403,7 +403,11 @@
> > }
> > } else if (Tok.is(tok::l_paren)) {
> > ++NumParens;
> > - } else if (Tok.is(tok::comma) && NumParens == 0) {
> > + // In Microsoft-compatibility mode, commas from nested macro
> expan-
> > + // sions should not be considered as argument separators. We test
> > + // for this with the IgnoredComma token flag.
> > + } else if (Tok.is(tok::comma)
> > + && !(Tok.getFlags() & Token::IgnoredComma) && NumParens == 0)
> {
> > // Comma ends this argument if there are more fixed arguments
> expected.
> > // However, if this is a variadic macro, and this is part of the
> > // variadic part, then the comma is just an argument token.
> > @@ -1171,3 +1175,4 @@
> > WarnUnusedMacroLocs.erase(MI->getDefinitionLoc());
> > MI->setIsUsed(true);
> > }
> >
> >
> >
> > This change caused PR13924, I might roll it out.
> >
> > On Wed, Sep 5, 2012 at 2:10 AM, João Matos <ripzonetriton at gmail.com>
> wrote:
> >> Will do. Yes, the this line endings problem was really annoying. I was
> not
> >> aware VS didn't respect them. I've introduced some measures locally so
> >> hopefully this won't happen in the future.
> >>
> >> Regarding your concerns, I didn't benchmark this, but I did had the
> >> performance nature of the preprocessor when writing the code, and tried
> to
> >> make minimal changes to not affect it.
> >>
> >>
> >> On Tue, Sep 4, 2012 at 5:56 PM, Chandler Carruth <chandlerc at google.com>
> >> wrote:
> >>>
> >>> On Fri, Aug 31, 2012 at 5:10 PM, Joao Matos <ripzonetriton at gmail.com>
> >>> wrote:
> >>>>
> >>>> Author: triton
> >>>> Date: Fri Aug 31 16:10:54 2012
> >>>> New Revision: 163022
> >>>>
> >>>> URL: http://llvm.org/viewvc/llvm-project?rev=163022&view=rev
> >>>> Log:
> >>>> Emulate MSVC's preprocessor macro argument separator behavior by not
> >>>> considering commas from nested macro expansions as argument
> separators.
> >>>> Fixes parsing of VS 2012 headers.
> >>>
> >>>
> >>> Arg, no one can review this patch now because the line endings got
> >>> thrashed.
> >>>
> >>> In the future, if you have trouble with line endings, *please* revert
> the
> >>> patch first, and then commit a new patch with only the intended edits.
> That
> >>> way our post-commit review can proceed normally.
> >>>
> >>> That said, I have some significant concerns about this patch that
> didn't
> >>> come up in the initial review, and I think might merit reverting the
> patch
> >>> temporarily until we understand them.
> >>>
> >>> What is the performance impact of this patch? The performance of the
> >>> preprocessor is *incredibly* sensitive. Have you benchmarked the patch
> in
> >>> microsoft mode and non-microsoft mode, before and after, with some of
> the
> >>> heavy users of preprocess macros? The single-source GCC version, or
> some of
> >>> the Boost preprocessor libraries might make excellent benchmarks.
> >>>
> >>> Also, more test cases would seem to be in order. How does this interact
> >>> with token pasting? How does it interact with variadic macros which
> also
> >>> have strange comma behavior?
> >>>
> >>>>
> >>>>
> >>>> Added:
> >>>> cfe/trunk/test/Preprocessor/microsoft-ext.c
> >>>> Modified:
> >>>> cfe/trunk/include/clang/Lex/Token.h
> >>>> cfe/trunk/lib/Lex/PPMacroExpansion.cpp
> >>>> cfe/trunk/lib/Lex/TokenLexer.cpp
> >>>>
> >>>> Modified: cfe/trunk/include/clang/Lex/Token.h
> >>>> URL:
> >>>>
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/Token.h?rev=163022&r1=163021&r2=163022&view=diff
> >>>>
> >>>>
> ==============================================================================
> >>>> --- cfe/trunk/include/clang/Lex/Token.h (original)
> >>>> +++ cfe/trunk/include/clang/Lex/Token.h Fri Aug 31 16:10:54 2012
> >>>> @@ -76,7 +76,8 @@
> >>>> DisableExpand = 0x04, // This identifier may never be macro
> >>>> expanded.
> >>>> NeedsCleaning = 0x08, // Contained an escaped newline or
> trigraph.
> >>>> LeadingEmptyMacro = 0x10, // Empty macro exists before this
> token.
> >>>> - HasUDSuffix = 0x20 // This string or character literal has a
> >>>> ud-suffix.
> >>>> + HasUDSuffix = 0x20, // This string or character literal has a
> >>>> ud-suffix.
> >>>> + IgnoredComma = 0x40 // Flags ignored commas from nested macro
> >>>> expansions.
> >>>> };
> >>>>
> >>>> tok::TokenKind getKind() const { return (tok::TokenKind)Kind; }
> >>>>
> >>>> Modified: cfe/trunk/lib/Lex/PPMacroExpansion.cpp
> >>>> URL:
> >>>>
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPMacroExpansion.cpp?rev=163022&r1=163021&r2=163022&view=diff
> >>>>
> >>>>
> ==============================================================================
> >>>> --- cfe/trunk/lib/Lex/PPMacroExpansion.cpp (original)
> >>>> +++ cfe/trunk/lib/Lex/PPMacroExpansion.cpp Fri Aug 31 16:10:54 2012
> >>>> @@ -1,1173 +1,1177 @@
> >>>> -//===--- MacroExpansion.cpp - Top level Macro Expansion
> >>>> -------------------===//
> >>>> -//
> >>>> -// The LLVM Compiler Infrastructure
> >>>> -//
> >>>> -// This file is distributed under the University of Illinois Open
> Source
> >>>> -// License. See LICENSE.TXT for details.
> >>>> -//
> >>>>
> >>>>
> -//===----------------------------------------------------------------------===//
> >>>> -//
> >>>> -// This file implements the top level handling of macro expasion for
> the
> >>>> -// preprocessor.
> >>>> -//
> >>>>
> >>>>
> -//===----------------------------------------------------------------------===//
> >>>> -
> >>>> -#include "clang/Lex/Preprocessor.h"
> >>>> -#include "MacroArgs.h"
> >>>> -#include "clang/Lex/MacroInfo.h"
> >>>> -#include "clang/Basic/SourceManager.h"
> >>>> -#include "clang/Basic/FileManager.h"
> >>>> -#include "clang/Basic/TargetInfo.h"
> >>>> -#include "clang/Lex/LexDiagnostic.h"
> >>>> -#include "clang/Lex/CodeCompletionHandler.h"
> >>>> -#include "clang/Lex/ExternalPreprocessorSource.h"
> >>>> -#include "clang/Lex/LiteralSupport.h"
> >>>> -#include "llvm/ADT/StringSwitch.h"
> >>>> -#include "llvm/ADT/STLExtras.h"
> >>>> -#include "llvm/Config/llvm-config.h"
> >>>> -#include "llvm/Support/raw_ostream.h"
> >>>> -#include "llvm/Support/ErrorHandling.h"
> >>>> -#include <cstdio>
> >>>> -#include <ctime>
> >>>> -using namespace clang;
> >>>> -
> >>>> -MacroInfo *Preprocessor::getInfoForMacro(IdentifierInfo *II) const {
> >>>> - assert(II->hasMacroDefinition() && "Identifier is not a macro!");
> >>>> -
> >>>> - macro_iterator Pos = Macros.find(II);
> >>>> - if (Pos == Macros.end()) {
> >>>> - // Load this macro from the external source.
> >>>> - getExternalSource()->LoadMacroDefinition(II);
> >>>> - Pos = Macros.find(II);
> >>>> - }
> >>>> - assert(Pos != Macros.end() && "Identifier macro info is missing!");
> >>>> - assert(Pos->second->getUndefLoc().isInvalid() && "Macro is
> >>>> undefined!");
> >>>> - return Pos->second;
> >>>> -}
> >>>> -
> >>>> -/// setMacroInfo - Specify a macro for this identifier.
> >>>> -///
> >>>> -void Preprocessor::setMacroInfo(IdentifierInfo *II, MacroInfo *MI,
> >>>> - bool LoadedFromAST) {
> >>>> - assert(MI && "MacroInfo should be non-zero!");
> >>>> - MI->setPreviousDefinition(Macros[II]);
> >>>> - Macros[II] = MI;
> >>>> - II->setHasMacroDefinition(true);
> >>>> - if (II->isFromAST() && !LoadedFromAST)
> >>>> - II->setChangedSinceDeserialization();
> >>>> -}
> >>>> -
> >>>> -/// \brief Undefine a macro for this identifier.
> >>>> -void Preprocessor::clearMacroInfo(IdentifierInfo *II) {
> >>>> - assert(II->hasMacroDefinition() && "Macro is not defined!");
> >>>> - assert(Macros[II]->getUndefLoc().isValid() && "Macro is still
> >>>> defined!");
> >>>> - II->setHasMacroDefinition(false);
> >>>> - if (II->isFromAST())
> >>>> - II->setChangedSinceDeserialization();
> >>>> -}
> >>>> -
> >>>> -/// RegisterBuiltinMacro - Register the specified identifier in the
> >>>> identifier
> >>>> -/// table and mark it as a builtin macro to be expanded.
> >>>> -static IdentifierInfo *RegisterBuiltinMacro(Preprocessor &PP, const
> char
> >>>> *Name){
> >>>> - // Get the identifier.
> >>>> - IdentifierInfo *Id = PP.getIdentifierInfo(Name);
> >>>> -
> >>>> - // Mark it as being a macro that is builtin.
> >>>> - MacroInfo *MI = PP.AllocateMacroInfo(SourceLocation());
> >>>> - MI->setIsBuiltinMacro();
> >>>> - PP.setMacroInfo(Id, MI);
> >>>> - return Id;
> >>>> -}
> >>>> -
> >>>> -
> >>>> -/// RegisterBuiltinMacros - Register builtin macros, such as __LINE__
> >>>> with the
> >>>> -/// identifier table.
> >>>> -void Preprocessor::RegisterBuiltinMacros() {
> >>>> - Ident__LINE__ = RegisterBuiltinMacro(*this, "__LINE__");
> >>>> - Ident__FILE__ = RegisterBuiltinMacro(*this, "__FILE__");
> >>>> - Ident__DATE__ = RegisterBuiltinMacro(*this, "__DATE__");
> >>>> - Ident__TIME__ = RegisterBuiltinMacro(*this, "__TIME__");
> >>>> - Ident__COUNTER__ = RegisterBuiltinMacro(*this, "__COUNTER__");
> >>>> - Ident_Pragma = RegisterBuiltinMacro(*this, "_Pragma");
> >>>> -
> >>>> - // GCC Extensions.
> >>>> - Ident__BASE_FILE__ = RegisterBuiltinMacro(*this,
> "__BASE_FILE__");
> >>>> - Ident__INCLUDE_LEVEL__ = RegisterBuiltinMacro(*this,
> >>>> "__INCLUDE_LEVEL__");
> >>>> - Ident__TIMESTAMP__ = RegisterBuiltinMacro(*this,
> "__TIMESTAMP__");
> >>>> -
> >>>> - // Clang Extensions.
> >>>> - Ident__has_feature = RegisterBuiltinMacro(*this,
> >>>> "__has_feature");
> >>>> - Ident__has_extension = RegisterBuiltinMacro(*this,
> >>>> "__has_extension");
> >>>> - Ident__has_builtin = RegisterBuiltinMacro(*this,
> >>>> "__has_builtin");
> >>>> - Ident__has_attribute = RegisterBuiltinMacro(*this,
> >>>> "__has_attribute");
> >>>> - Ident__has_include = RegisterBuiltinMacro(*this,
> >>>> "__has_include");
> >>>> - Ident__has_include_next = RegisterBuiltinMacro(*this,
> >>>> "__has_include_next");
> >>>> - Ident__has_warning = RegisterBuiltinMacro(*this,
> >>>> "__has_warning");
> >>>> -
> >>>> - // Microsoft Extensions.
> >>>> - if (LangOpts.MicrosoftExt)
> >>>> - Ident__pragma = RegisterBuiltinMacro(*this, "__pragma");
> >>>> - else
> >>>> - Ident__pragma = 0;
> >>>> -}
> >>>> -
> >>>> -/// isTrivialSingleTokenExpansion - Return true if MI, which has a
> >>>> single token
> >>>> -/// in its expansion, currently expands to that token literally.
> >>>> -static bool isTrivialSingleTokenExpansion(const MacroInfo *MI,
> >>>> - const IdentifierInfo
> >>>> *MacroIdent,
> >>>> - Preprocessor &PP) {
> >>>> - IdentifierInfo *II =
> MI->getReplacementToken(0).getIdentifierInfo();
> >>>> -
> >>>> - // If the token isn't an identifier, it's always literally
> expanded.
> >>>> - if (II == 0) return true;
> >>>> -
> >>>> - // If the information about this identifier is out of date, update
> it
> >>>> from
> >>>> - // the external source.
> >>>> - if (II->isOutOfDate())
> >>>> - PP.getExternalSource()->updateOutOfDateIdentifier(*II);
> >>>> -
> >>>> - // If the identifier is a macro, and if that macro is enabled, it
> may
> >>>> be
> >>>> - // expanded so it's not a trivial expansion.
> >>>> - if (II->hasMacroDefinition() && PP.getMacroInfo(II)->isEnabled() &&
> >>>> - // Fast expanding "#define X X" is ok, because X would be
> >>>> disabled.
> >>>> - II != MacroIdent)
> >>>> - return false;
> >>>> -
> >>>> - // If this is an object-like macro invocation, it is safe to
> trivially
> >>>> expand
> >>>> - // it.
> >>>> - if (MI->isObjectLike()) return true;
> >>>> -
> >>>> - // If this is a function-like macro invocation, it's safe to
> trivially
> >>>> expand
> >>>> - // as long as the identifier is not a macro argument.
> >>>> - for (MacroInfo::arg_iterator I = MI->arg_begin(), E =
> MI->arg_end();
> >>>> - I != E; ++I)
> >>>> - if (*I == II)
> >>>> - return false; // Identifier is a macro argument.
> >>>> -
> >>>> - return true;
> >>>> -}
> >>>> -
> >>>> -
> >>>> -/// isNextPPTokenLParen - Determine whether the next preprocessor
> token
> >>>> to be
> >>>> -/// lexed is a '('. If so, consume the token and return true, if
> not,
> >>>> this
> >>>> -/// method should have no observable side-effect on the lexed tokens.
> >>>> -bool Preprocessor::isNextPPTokenLParen() {
> >>>> - // Do some quick tests for rejection cases.
> >>>> - unsigned Val;
> >>>> - if (CurLexer)
> >>>> - Val = CurLexer->isNextPPTokenLParen();
> >>>> - else if (CurPTHLexer)
> >>>> - Val = CurPTHLexer->isNextPPTokenLParen();
> >>>> - else
> >>>> - Val = CurTokenLexer->isNextTokenLParen();
> >>>> -
> >>>> - if (Val == 2) {
> >>>> - // We have run off the end. If it's a source file we don't
> >>>> - // examine enclosing ones (C99 5.1.1.2p4). Otherwise walk up the
> >>>> - // macro stack.
> >>>> - if (CurPPLexer)
> >>>> - return false;
> >>>> - for (unsigned i = IncludeMacroStack.size(); i != 0; --i) {
> >>>> - IncludeStackInfo &Entry = IncludeMacroStack[i-1];
> >>>> - if (Entry.TheLexer)
> >>>> - Val = Entry.TheLexer->isNextPPTokenLParen();
> >>>> - else if (Entry.ThePTHLexer)
> >>>> - Val = Entry.ThePTHLexer->isNextPPTokenLParen();
> >>>> - else
> >>>> - Val = Entry.TheTokenLexer->isNextTokenLParen();
> >>>> -
> >>>> - if (Val != 2)
> >>>> - break;
> >>>> -
> >>>> - // Ran off the end of a source file?
> >>>> - if (Entry.ThePPLexer)
> >>>> - return false;
> >>>> - }
> >>>> - }
> >>>> -
> >>>> - // Okay, if we know that the token is a '(', lex it and return.
> >>>> Otherwise we
> >>>> - // have found something that isn't a '(' or we found the end of the
> >>>> - // translation unit. In either case, return false.
> >>>> - return Val == 1;
> >>>> -}
> >>>> -
> >>>> -/// HandleMacroExpandedIdentifier - If an identifier token is read
> that
> >>>> is to be
> >>>> -/// expanded as a macro, handle it and return the next token as
> >>>> 'Identifier'.
> >>>> -bool Preprocessor::HandleMacroExpandedIdentifier(Token &Identifier,
> >>>> - MacroInfo *MI) {
> >>>> - // If this is a macro expansion in the "#if !defined(x)" line for
> the
> >>>> file,
> >>>> - // then the macro could expand to different things in other
> contexts,
> >>>> we need
> >>>> - // to disable the optimization in this case.
> >>>> - if (CurPPLexer) CurPPLexer->MIOpt.ExpandedMacro();
> >>>> -
> >>>> - // If this is a builtin macro, like __LINE__ or _Pragma, handle it
> >>>> specially.
> >>>> - if (MI->isBuiltinMacro()) {
> >>>> - if (Callbacks) Callbacks->MacroExpands(Identifier, MI,
> >>>> - Identifier.getLocation());
> >>>> - ExpandBuiltinMacro(Identifier);
> >>>> - return false;
> >>>> - }
> >>>> -
> >>>> - /// Args - If this is a function-like macro expansion, this
> contains,
> >>>> - /// for each macro argument, the list of tokens that were provided
> to
> >>>> the
> >>>> - /// invocation.
> >>>> - MacroArgs *Args = 0;
> >>>> -
> >>>> - // Remember where the end of the expansion occurred. For an
> >>>> object-like
> >>>> - // macro, this is the identifier. For a function-like macro, this
> is
> >>>> the ')'.
> >>>> - SourceLocation ExpansionEnd = Identifier.getLocation();
> >>>> -
> >>>> - // If this is a function-like macro, read the arguments.
> >>>> - if (MI->isFunctionLike()) {
> >>>> - // C99 6.10.3p10: If the preprocessing token immediately after
> the
> >>>> macro
> >>>> - // name isn't a '(', this macro should not be expanded.
> >>>> - if (!isNextPPTokenLParen())
> >>>> - return true;
> >>>> -
> >>>> - // Remember that we are now parsing the arguments to a macro
> >>>> invocation.
> >>>> - // Preprocessor directives used inside macro arguments are not
> >>>> portable, and
> >>>> - // this enables the warning.
> >>>> - InMacroArgs = true;
> >>>> - Args = ReadFunctionLikeMacroArgs(Identifier, MI, ExpansionEnd);
> >>>> -
> >>>> - // Finished parsing args.
> >>>> - InMacroArgs = false;
> >>>> -
> >>>> - // If there was an error parsing the arguments, bail out.
> >>>> - if (Args == 0) return false;
> >>>> -
> >>>> - ++NumFnMacroExpanded;
> >>>> - } else {
> >>>> - ++NumMacroExpanded;
> >>>> - }
> >>>> -
> >>>> - // Notice that this macro has been used.
> >>>> - markMacroAsUsed(MI);
> >>>> -
> >>>> - // Remember where the token is expanded.
> >>>> - SourceLocation ExpandLoc = Identifier.getLocation();
> >>>> - SourceRange ExpansionRange(ExpandLoc, ExpansionEnd);
> >>>> -
> >>>> - if (Callbacks) {
> >>>> - if (InMacroArgs) {
> >>>> - // We can have macro expansion inside a conditional directive
> >>>> while
> >>>> - // reading the function macro arguments. To ensure, in that
> case,
> >>>> that
> >>>> - // MacroExpands callbacks still happen in source order, queue
> this
> >>>> - // callback to have it happen after the function macro
> callback.
> >>>> - DelayedMacroExpandsCallbacks.push_back(
> >>>> - MacroExpandsInfo(Identifier, MI,
> >>>> ExpansionRange));
> >>>> - } else {
> >>>> - Callbacks->MacroExpands(Identifier, MI, ExpansionRange);
> >>>> - if (!DelayedMacroExpandsCallbacks.empty()) {
> >>>> - for (unsigned i=0, e = DelayedMacroExpandsCallbacks.size();
> >>>> i!=e; ++i) {
> >>>> - MacroExpandsInfo &Info = DelayedMacroExpandsCallbacks[i];
> >>>> - Callbacks->MacroExpands(Info.Tok, Info.MI, Info.Range);
> >>>> - }
> >>>> - DelayedMacroExpandsCallbacks.clear();
> >>>> - }
> >>>> - }
> >>>> - }
> >>>> -
> >>>> - // If we started lexing a macro, enter the macro expansion body.
> >>>> -
> >>>> - // If this macro expands to no tokens, don't bother to push it onto
> >>>> the
> >>>> - // expansion stack, only to take it right back off.
> >>>> - if (MI->getNumTokens() == 0) {
> >>>> - // No need for arg info.
> >>>> - if (Args) Args->destroy(*this);
> >>>> -
> >>>> - // Ignore this macro use, just return the next token in the
> current
> >>>> - // buffer.
> >>>> - bool HadLeadingSpace = Identifier.hasLeadingSpace();
> >>>> - bool IsAtStartOfLine = Identifier.isAtStartOfLine();
> >>>> -
> >>>> - Lex(Identifier);
> >>>> -
> >>>> - // If the identifier isn't on some OTHER line, inherit the
> leading
> >>>> - // whitespace/first-on-a-line property of this token. This
> handles
> >>>> - // stuff like "! XX," -> "! ," and " XX," -> " ,", when XX
> is
> >>>> - // empty.
> >>>> - if (!Identifier.isAtStartOfLine()) {
> >>>> - if (IsAtStartOfLine) Identifier.setFlag(Token::StartOfLine);
> >>>> - if (HadLeadingSpace) Identifier.setFlag(Token::LeadingSpace);
> >>>> - }
> >>>> - Identifier.setFlag(Token::LeadingEmptyMacro);
> >>>> - ++NumFastMacroExpanded;
> >>>> - return false;
> >>>> -
> >>>> - } else if (MI->getNumTokens() == 1 &&
> >>>> - isTrivialSingleTokenExpansion(MI,
> >>>> Identifier.getIdentifierInfo(),
> >>>> - *this)) {
> >>>> - // Otherwise, if this macro expands into a single
> trivially-expanded
> >>>> - // token: expand it now. This handles common cases like
> >>>> - // "#define VAL 42".
> >>>> -
> >>>> - // No need for arg info.
> >>>> - if (Args) Args->destroy(*this);
> >>>> -
> >>>> - // Propagate the isAtStartOfLine/hasLeadingSpace markers of the
> >>>> macro
> >>>> - // identifier to the expanded token.
> >>>> - bool isAtStartOfLine = Identifier.isAtStartOfLine();
> >>>> - bool hasLeadingSpace = Identifier.hasLeadingSpace();
> >>>> -
> >>>> - // Replace the result token.
> >>>> - Identifier = MI->getReplacementToken(0);
> >>>> -
> >>>> - // Restore the StartOfLine/LeadingSpace markers.
> >>>> - Identifier.setFlagValue(Token::StartOfLine , isAtStartOfLine);
> >>>> - Identifier.setFlagValue(Token::LeadingSpace, hasLeadingSpace);
> >>>> -
> >>>> - // Update the tokens location to include both its expansion and
> >>>> physical
> >>>> - // locations.
> >>>> - SourceLocation Loc =
> >>>> - SourceMgr.createExpansionLoc(Identifier.getLocation(),
> ExpandLoc,
> >>>> -
> ExpansionEnd,Identifier.getLength());
> >>>> - Identifier.setLocation(Loc);
> >>>> -
> >>>> - // If this is a disabled macro or #define X X, we must mark the
> >>>> result as
> >>>> - // unexpandable.
> >>>> - if (IdentifierInfo *NewII = Identifier.getIdentifierInfo()) {
> >>>> - if (MacroInfo *NewMI = getMacroInfo(NewII))
> >>>> - if (!NewMI->isEnabled() || NewMI == MI) {
> >>>> - Identifier.setFlag(Token::DisableExpand);
> >>>> - Diag(Identifier, diag::pp_disabled_macro_expansion);
> >>>> - }
> >>>> - }
> >>>> -
> >>>> - // Since this is not an identifier token, it can't be macro
> >>>> expanded, so
> >>>> - // we're done.
> >>>> - ++NumFastMacroExpanded;
> >>>> - return false;
> >>>> - }
> >>>> -
> >>>> - // Start expanding the macro.
> >>>> - EnterMacro(Identifier, ExpansionEnd, MI, Args);
> >>>> -
> >>>> - // Now that the macro is at the top of the include stack, ask the
> >>>> - // preprocessor to read the next token from it.
> >>>> - Lex(Identifier);
> >>>> - return false;
> >>>> -}
> >>>> -
> >>>> -/// ReadFunctionLikeMacroArgs - After reading "MACRO" and knowing
> that
> >>>> the next
> >>>> -/// token is the '(' of the macro, this method is invoked to read
> all of
> >>>> the
> >>>> -/// actual arguments specified for the macro invocation. This
> returns
> >>>> null on
> >>>> -/// error.
> >>>> -MacroArgs *Preprocessor::ReadFunctionLikeMacroArgs(Token &MacroName,
> >>>> - MacroInfo *MI,
> >>>> - SourceLocation
> >>>> &MacroEnd) {
> >>>> - // The number of fixed arguments to parse.
> >>>> - unsigned NumFixedArgsLeft = MI->getNumArgs();
> >>>> - bool isVariadic = MI->isVariadic();
> >>>> -
> >>>> - // Outer loop, while there are more arguments, keep reading them.
> >>>> - Token Tok;
> >>>> -
> >>>> - // Read arguments as unexpanded tokens. This avoids issues, e.g.,
> >>>> where
> >>>> - // an argument value in a macro could expand to ',' or '(' or ')'.
> >>>> - LexUnexpandedToken(Tok);
> >>>> - assert(Tok.is(tok::l_paren) && "Error computing l-paren-ness?");
> >>>> -
> >>>> - // ArgTokens - Build up a list of tokens that make up each
> argument.
> >>>> Each
> >>>> - // argument is separated by an EOF token. Use a SmallVector so we
> can
> >>>> avoid
> >>>> - // heap allocations in the common case.
> >>>> - SmallVector<Token, 64> ArgTokens;
> >>>> -
> >>>> - unsigned NumActuals = 0;
> >>>> - while (Tok.isNot(tok::r_paren)) {
> >>>> - assert((Tok.is(tok::l_paren) || Tok.is(tok::comma)) &&
> >>>> - "only expect argument separators here");
> >>>> -
> >>>> - unsigned ArgTokenStart = ArgTokens.size();
> >>>> - SourceLocation ArgStartLoc = Tok.getLocation();
> >>>> -
> >>>> - // C99 6.10.3p11: Keep track of the number of l_parens we have
> seen.
> >>>> Note
> >>>> - // that we already consumed the first one.
> >>>> - unsigned NumParens = 0;
> >>>> -
> >>>> - while (1) {
> >>>> - // Read arguments as unexpanded tokens. This avoids issues,
> e.g.,
> >>>> where
> >>>> - // an argument value in a macro could expand to ',' or '(' or
> ')'.
> >>>> - LexUnexpandedToken(Tok);
> >>>> -
> >>>> - if (Tok.is(tok::eof) || Tok.is(tok::eod)) { // "#if f(<eof>" &
> >>>> "#if f(\n"
> >>>> - Diag(MacroName, diag::err_unterm_macro_invoc);
> >>>> - // Do not lose the EOF/EOD. Return it to the client.
> >>>> - MacroName = Tok;
> >>>> - return 0;
> >>>> - } else if (Tok.is(tok::r_paren)) {
> >>>> - // If we found the ) token, the macro arg list is done.
> >>>> - if (NumParens-- == 0) {
> >>>> - MacroEnd = Tok.getLocation();
> >>>> - break;
> >>>> - }
> >>>> - } else if (Tok.is(tok::l_paren)) {
> >>>> - ++NumParens;
> >>>> - } else if (Tok.is(tok::comma) && NumParens == 0) {
> >>>> - // Comma ends this argument if there are more fixed arguments
> >>>> expected.
> >>>> - // However, if this is a variadic macro, and this is part of
> the
> >>>> - // variadic part, then the comma is just an argument token.
> >>>> - if (!isVariadic) break;
> >>>> - if (NumFixedArgsLeft > 1)
> >>>> - break;
> >>>> - } else if (Tok.is(tok::comment) && !KeepMacroComments) {
> >>>> - // If this is a comment token in the argument list and we're
> >>>> just in
> >>>> - // -C mode (not -CC mode), discard the comment.
> >>>> - continue;
> >>>> - } else if (Tok.getIdentifierInfo() != 0) {
> >>>> - // Reading macro arguments can cause macros that we are
> >>>> currently
> >>>> - // expanding from to be popped off the expansion stack.
> Doing
> >>>> so causes
> >>>> - // them to be reenabled for expansion. Here we record
> whether
> >>>> any
> >>>> - // identifiers we lex as macro arguments correspond to
> disabled
> >>>> macros.
> >>>> - // If so, we mark the token as noexpand. This is a subtle
> >>>> aspect of
> >>>> - // C99 6.10.3.4p2.
> >>>> - if (MacroInfo *MI = getMacroInfo(Tok.getIdentifierInfo()))
> >>>> - if (!MI->isEnabled())
> >>>> - Tok.setFlag(Token::DisableExpand);
> >>>> - } else if (Tok.is(tok::code_completion)) {
> >>>> - if (CodeComplete)
> >>>> -
> >>>> CodeComplete->CodeCompleteMacroArgument(MacroName.getIdentifierInfo(),
> >>>> - MI, NumActuals);
> >>>> - // Don't mark that we reached the code-completion point
> because
> >>>> the
> >>>> - // parser is going to handle the token and there will be
> another
> >>>> - // code-completion callback.
> >>>> - }
> >>>> -
> >>>> - ArgTokens.push_back(Tok);
> >>>> - }
> >>>> -
> >>>> - // If this was an empty argument list foo(), don't add this as an
> >>>> empty
> >>>> - // argument.
> >>>> - if (ArgTokens.empty() && Tok.getKind() == tok::r_paren)
> >>>> - break;
> >>>> -
> >>>> - // If this is not a variadic macro, and too many args were
> >>>> specified, emit
> >>>> - // an error.
> >>>> - if (!isVariadic && NumFixedArgsLeft == 0) {
> >>>> - if (ArgTokens.size() != ArgTokenStart)
> >>>> - ArgStartLoc = ArgTokens[ArgTokenStart].getLocation();
> >>>> -
> >>>> - // Emit the diagnostic at the macro name in case there is a
> >>>> missing ).
> >>>> - // Emitting it at the , could be far away from the macro name.
> >>>> - Diag(ArgStartLoc, diag::err_too_many_args_in_macro_invoc);
> >>>> - return 0;
> >>>> - }
> >>>> -
> >>>> - // Empty arguments are standard in C99 and C++0x, and are
> supported
> >>>> as an extension in
> >>>> - // other modes.
> >>>> - if (ArgTokens.size() == ArgTokenStart && !LangOpts.C99)
> >>>> - Diag(Tok, LangOpts.CPlusPlus0x ?
> >>>> - diag::warn_cxx98_compat_empty_fnmacro_arg :
> >>>> - diag::ext_empty_fnmacro_arg);
> >>>> -
> >>>> - // Add a marker EOF token to the end of the token list for this
> >>>> argument.
> >>>> - Token EOFTok;
> >>>> - EOFTok.startToken();
> >>>> - EOFTok.setKind(tok::eof);
> >>>> - EOFTok.setLocation(Tok.getLocation());
> >>>> - EOFTok.setLength(0);
> >>>> - ArgTokens.push_back(EOFTok);
> >>>> - ++NumActuals;
> >>>> - assert(NumFixedArgsLeft != 0 && "Too many arguments parsed");
> >>>> - --NumFixedArgsLeft;
> >>>> - }
> >>>> -
> >>>> - // Okay, we either found the r_paren. Check to see if we parsed
> too
> >>>> few
> >>>> - // arguments.
> >>>> - unsigned MinArgsExpected = MI->getNumArgs();
> >>>> -
> >>>> - // See MacroArgs instance var for description of this.
> >>>> - bool isVarargsElided = false;
> >>>> -
> >>>> - if (NumActuals < MinArgsExpected) {
> >>>> - // There are several cases where too few arguments is ok, handle
> >>>> them now.
> >>>> - if (NumActuals == 0 && MinArgsExpected == 1) {
> >>>> - // #define A(X) or #define A(...) ---> A()
> >>>> -
> >>>> - // If there is exactly one argument, and that argument is
> missing,
> >>>> - // then we have an empty "()" argument empty list. This is
> fine,
> >>>> even if
> >>>> - // the macro expects one argument (the argument is just empty).
> >>>> - isVarargsElided = MI->isVariadic();
> >>>> - } else if (MI->isVariadic() &&
> >>>> - (NumActuals+1 == MinArgsExpected || // A(x, ...) ->
> A(X)
> >>>> - (NumActuals == 0 && MinArgsExpected == 2))) {//
> A(x,...)
> >>>> -> A()
> >>>> - // Varargs where the named vararg parameter is missing: OK as
> >>>> extension.
> >>>> - // #define A(x, ...)
> >>>> - // A("blah")
> >>>> - Diag(Tok, diag::ext_missing_varargs_arg);
> >>>> - Diag(MI->getDefinitionLoc(), diag::note_macro_here)
> >>>> - << MacroName.getIdentifierInfo();
> >>>> -
> >>>> - // Remember this occurred, allowing us to elide the comma when
> >>>> used for
> >>>> - // cases like:
> >>>> - // #define A(x, foo...) blah(a, ## foo)
> >>>> - // #define B(x, ...) blah(a, ## __VA_ARGS__)
> >>>> - // #define C(...) blah(a, ## __VA_ARGS__)
> >>>> - // A(x) B(x) C()
> >>>> - isVarargsElided = true;
> >>>> - } else {
> >>>> - // Otherwise, emit the error.
> >>>> - Diag(Tok, diag::err_too_few_args_in_macro_invoc);
> >>>> - return 0;
> >>>> - }
> >>>> -
> >>>> - // Add a marker EOF token to the end of the token list for this
> >>>> argument.
> >>>> - SourceLocation EndLoc = Tok.getLocation();
> >>>> - Tok.startToken();
> >>>> - Tok.setKind(tok::eof);
> >>>> - Tok.setLocation(EndLoc);
> >>>> - Tok.setLength(0);
> >>>> - ArgTokens.push_back(Tok);
> >>>> -
> >>>> - // If we expect two arguments, add both as empty.
> >>>> - if (NumActuals == 0 && MinArgsExpected == 2)
> >>>> - ArgTokens.push_back(Tok);
> >>>> -
> >>>> - } else if (NumActuals > MinArgsExpected && !MI->isVariadic()) {
> >>>> - // Emit the diagnostic at the macro name in case there is a
> missing
> >>>> ).
> >>>> - // Emitting it at the , could be far away from the macro name.
> >>>> - Diag(MacroName, diag::err_too_many_args_in_macro_invoc);
> >>>> - return 0;
> >>>> - }
> >>>> -
> >>>> - return MacroArgs::create(MI, ArgTokens, isVarargsElided, *this);
> >>>> -}
> >>>> -
> >>>> -/// \brief Keeps macro expanded tokens for TokenLexers.
> >>>> -//
> >>>> -/// Works like a stack; a TokenLexer adds the macro expanded tokens
> that
> >>>> is
> >>>> -/// going to lex in the cache and when it finishes the tokens are
> >>>> removed
> >>>> -/// from the end of the cache.
> >>>> -Token *Preprocessor::cacheMacroExpandedTokens(TokenLexer *tokLexer,
> >>>> - ArrayRef<Token>
> tokens) {
> >>>> - assert(tokLexer);
> >>>> - if (tokens.empty())
> >>>> - return 0;
> >>>> -
> >>>> - size_t newIndex = MacroExpandedTokens.size();
> >>>> - bool cacheNeedsToGrow = tokens.size() >
> >>>> -
> >>>> MacroExpandedTokens.capacity()-MacroExpandedTokens.size();
> >>>> - MacroExpandedTokens.append(tokens.begin(), tokens.end());
> >>>> -
> >>>> - if (cacheNeedsToGrow) {
> >>>> - // Go through all the TokenLexers whose 'Tokens' pointer points
> in
> >>>> the
> >>>> - // buffer and update the pointers to the (potential) new buffer
> >>>> array.
> >>>> - for (unsigned i = 0, e = MacroExpandingLexersStack.size(); i !=
> e;
> >>>> ++i) {
> >>>> - TokenLexer *prevLexer;
> >>>> - size_t tokIndex;
> >>>> - llvm::tie(prevLexer, tokIndex) = MacroExpandingLexersStack[i];
> >>>> - prevLexer->Tokens = MacroExpandedTokens.data() + tokIndex;
> >>>> - }
> >>>> - }
> >>>> -
> >>>> - MacroExpandingLexersStack.push_back(std::make_pair(tokLexer,
> >>>> newIndex));
> >>>> - return MacroExpandedTokens.data() + newIndex;
> >>>> -}
> >>>> -
> >>>> -void Preprocessor::removeCachedMacroExpandedTokensOfLastLexer() {
> >>>> - assert(!MacroExpandingLexersStack.empty());
> >>>> - size_t tokIndex = MacroExpandingLexersStack.back().second;
> >>>> - assert(tokIndex < MacroExpandedTokens.size());
> >>>> - // Pop the cached macro expanded tokens from the end.
> >>>> - MacroExpandedTokens.resize(tokIndex);
> >>>> - MacroExpandingLexersStack.pop_back();
> >>>> -}
> >>>> -
> >>>> -/// ComputeDATE_TIME - Compute the current time, enter it into the
> >>>> specified
> >>>> -/// scratch buffer, then return DATELoc/TIMELoc locations with the
> >>>> position of
> >>>> -/// the identifier tokens inserted.
> >>>> -static void ComputeDATE_TIME(SourceLocation &DATELoc, SourceLocation
> >>>> &TIMELoc,
> >>>> - Preprocessor &PP) {
> >>>> - time_t TT = time(0);
> >>>> - struct tm *TM = localtime(&TT);
> >>>> -
> >>>> - static const char * const Months[] = {
> >>>> -
> >>>>
> "Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"
> >>>> - };
> >>>> -
> >>>> - char TmpBuffer[32];
> >>>> -#ifdef LLVM_ON_WIN32
> >>>> - sprintf(TmpBuffer, "\"%s %2d %4d\"", Months[TM->tm_mon],
> TM->tm_mday,
> >>>> - TM->tm_year+1900);
> >>>> -#else
> >>>> - snprintf(TmpBuffer, sizeof(TmpBuffer), "\"%s %2d %4d\"",
> >>>> Months[TM->tm_mon], TM->tm_mday,
> >>>> - TM->tm_year+1900);
> >>>> -#endif
> >>>> -
> >>>> - Token TmpTok;
> >>>> - TmpTok.startToken();
> >>>> - PP.CreateString(TmpBuffer, strlen(TmpBuffer), TmpTok);
> >>>> - DATELoc = TmpTok.getLocation();
> >>>> -
> >>>> -#ifdef LLVM_ON_WIN32
> >>>> - sprintf(TmpBuffer, "\"%02d:%02d:%02d\"", TM->tm_hour, TM->tm_min,
> >>>> TM->tm_sec);
> >>>> -#else
> >>>> - snprintf(TmpBuffer, sizeof(TmpBuffer), "\"%02d:%02d:%02d\"",
> >>>> TM->tm_hour, TM->tm_min, TM->tm_sec);
> >>>> -#endif
> >>>> - PP.CreateString(TmpBuffer, strlen(TmpBuffer), TmpTok);
> >>>> - TIMELoc = TmpTok.getLocation();
> >>>> -}
> >>>> -
> >>>> -
> >>>> -/// HasFeature - Return true if we recognize and implement the
> feature
> >>>> -/// specified by the identifier as a standard language feature.
> >>>> -static bool HasFeature(const Preprocessor &PP, const IdentifierInfo
> *II)
> >>>> {
> >>>> - const LangOptions &LangOpts = PP.getLangOpts();
> >>>> - StringRef Feature = II->getName();
> >>>> -
> >>>> - // Normalize the feature name, __foo__ becomes foo.
> >>>> - if (Feature.startswith("__") && Feature.endswith("__") &&
> >>>> Feature.size() >= 4)
> >>>> - Feature = Feature.substr(2, Feature.size() - 4);
> >>>> -
> >>>> - return llvm::StringSwitch<bool>(Feature)
> >>>> - .Case("address_sanitizer", LangOpts.AddressSanitizer)
> >>>> - .Case("attribute_analyzer_noreturn", true)
> >>>> - .Case("attribute_availability", true)
> >>>> - .Case("attribute_availability_with_message", true)
> >>>> - .Case("attribute_cf_returns_not_retained", true)
> >>>> - .Case("attribute_cf_returns_retained", true)
> >>>> - .Case("attribute_deprecated_with_message", true)
> >>>> - .Case("attribute_ext_vector_type", true)
> >>>> - .Case("attribute_ns_returns_not_retained", true)
> >>>> - .Case("attribute_ns_returns_retained", true)
> >>>> - .Case("attribute_ns_consumes_self", true)
> >>>> - .Case("attribute_ns_consumed", true)
> >>>> - .Case("attribute_cf_consumed", true)
> >>>> - .Case("attribute_objc_ivar_unused", true)
> >>>> - .Case("attribute_objc_method_family", true)
> >>>> - .Case("attribute_overloadable", true)
> >>>> - .Case("attribute_unavailable_with_message", true)
> >>>> - .Case("attribute_unused_on_fields", true)
> >>>> - .Case("blocks", LangOpts.Blocks)
> >>>> - .Case("cxx_exceptions", LangOpts.Exceptions)
> >>>> - .Case("cxx_rtti", LangOpts.RTTI)
> >>>> - .Case("enumerator_attributes", true)
> >>>> - // Objective-C features
> >>>> - .Case("objc_arr", LangOpts.ObjCAutoRefCount) // FIXME:
> >>>> REMOVE?
> >>>> - .Case("objc_arc", LangOpts.ObjCAutoRefCount)
> >>>> - .Case("objc_arc_weak", LangOpts.ObjCARCWeak)
> >>>> - .Case("objc_default_synthesize_properties",
> LangOpts.ObjC2)
> >>>> - .Case("objc_fixed_enum", LangOpts.ObjC2)
> >>>> - .Case("objc_instancetype", LangOpts.ObjC2)
> >>>> - .Case("objc_modules", LangOpts.ObjC2 && LangOpts.Modules)
> >>>> - .Case("objc_nonfragile_abi",
> >>>> LangOpts.ObjCRuntime.isNonFragile())
> >>>> - .Case("objc_weak_class",
> >>>> LangOpts.ObjCRuntime.hasWeakClassImport())
> >>>> - .Case("ownership_holds", true)
> >>>> - .Case("ownership_returns", true)
> >>>> - .Case("ownership_takes", true)
> >>>> - .Case("objc_bool", true)
> >>>> - .Case("objc_subscripting",
> >>>> LangOpts.ObjCRuntime.isNonFragile())
> >>>> - .Case("objc_array_literals", LangOpts.ObjC2)
> >>>> - .Case("objc_dictionary_literals", LangOpts.ObjC2)
> >>>> - .Case("objc_boxed_expressions", LangOpts.ObjC2)
> >>>> - .Case("arc_cf_code_audited", true)
> >>>> - // C11 features
> >>>> - .Case("c_alignas", LangOpts.C11)
> >>>> - .Case("c_atomic", LangOpts.C11)
> >>>> - .Case("c_generic_selections", LangOpts.C11)
> >>>> - .Case("c_static_assert", LangOpts.C11)
> >>>> - // C++11 features
> >>>> - .Case("cxx_access_control_sfinae", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_alias_templates", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_alignas", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_atomic", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_attributes", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_auto_type", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_constexpr", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_decltype", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_decltype_incomplete_return_types",
> >>>> LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_default_function_template_args",
> >>>> LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_defaulted_functions", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_delegating_constructors", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_deleted_functions", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_explicit_conversions", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_generalized_initializers",
> LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_implicit_moves", LangOpts.CPlusPlus0x)
> >>>> - //.Case("cxx_inheriting_constructors", false)
> >>>> - .Case("cxx_inline_namespaces", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_lambdas", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_local_type_template_args",
> LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_nonstatic_member_init", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_noexcept", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_nullptr", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_override_control", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_range_for", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_raw_string_literals", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_reference_qualified_functions",
> >>>> LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_rvalue_references", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_strong_enums", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_static_assert", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_trailing_return", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_unicode_literals", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_unrestricted_unions", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_user_literals", LangOpts.CPlusPlus0x)
> >>>> - .Case("cxx_variadic_templates", LangOpts.CPlusPlus0x)
> >>>> - // Type traits
> >>>> - .Case("has_nothrow_assign", LangOpts.CPlusPlus)
> >>>> - .Case("has_nothrow_copy", LangOpts.CPlusPlus)
> >>>> - .Case("has_nothrow_constructor", LangOpts.CPlusPlus)
> >>>> - .Case("has_trivial_assign", LangOpts.CPlusPlus)
> >>>> - .Case("has_trivial_copy", LangOpts.CPlusPlus)
> >>>> - .Case("has_trivial_constructor", LangOpts.CPlusPlus)
> >>>> - .Case("has_trivial_destructor", LangOpts.CPlusPlus)
> >>>> - .Case("has_virtual_destructor", LangOpts.CPlusPlus)
> >>>> - .Case("is_abstract", LangOpts.CPlusPlus)
> >>>> - .Case("is_base_of", LangOpts.CPlusPlus)
> >>>> - .Case("is_class", LangOpts.CPlusPlus)
> >>>> - .Case("is_convertible_to", LangOpts.CPlusPlus)
> >>>> - // __is_empty is available only if the horrible
> >>>> - // "struct __is_empty" parsing hack hasn't been needed in
> >>>> this
> >>>> - // translation unit. If it has, __is_empty reverts to a
> >>>> normal
> >>>> - // identifier and __has_feature(is_empty) evaluates
> false.
> >>>> - .Case("is_empty", LangOpts.CPlusPlus)
> >>>> - .Case("is_enum", LangOpts.CPlusPlus)
> >>>> - .Case("is_final", LangOpts.CPlusPlus)
> >>>> - .Case("is_literal", LangOpts.CPlusPlus)
> >>>> - .Case("is_standard_layout", LangOpts.CPlusPlus)
> >>>> - .Case("is_pod", LangOpts.CPlusPlus)
> >>>> - .Case("is_polymorphic", LangOpts.CPlusPlus)
> >>>> - .Case("is_trivial", LangOpts.CPlusPlus)
> >>>> - .Case("is_trivially_assignable", LangOpts.CPlusPlus)
> >>>> - .Case("is_trivially_constructible", LangOpts.CPlusPlus)
> >>>> - .Case("is_trivially_copyable", LangOpts.CPlusPlus)
> >>>> - .Case("is_union", LangOpts.CPlusPlus)
> >>>> - .Case("modules", LangOpts.Modules)
> >>>> - .Case("tls", PP.getTargetInfo().isTLSSupported())
> >>>> - .Case("underlying_type", LangOpts.CPlusPlus)
> >>>> - .Default(false);
> >>>> -}
> >>>> -
> >>>> -/// HasExtension - Return true if we recognize and implement the
> feature
> >>>> -/// specified by the identifier, either as an extension or a standard
> >>>> language
> >>>> -/// feature.
> >>>> -static bool HasExtension(const Preprocessor &PP, const IdentifierInfo
> >>>> *II) {
> >>>> - if (HasFeature(PP, II))
> >>>> - return true;
> >>>> -
> >>>> - // If the use of an extension results in an error diagnostic,
> >>>> extensions are
> >>>> - // effectively unavailable, so just return false here.
> >>>> - if (PP.getDiagnostics().getExtensionHandlingBehavior() ==
> >>>> - DiagnosticsEngine::Ext_Error)
> >>>> - return false;
> >>>> -
> >>>> - const LangOptions &LangOpts = PP.getLangOpts();
> >>>> - StringRef Extension = II->getName();
> >>>> -
> >>>> - // Normalize the extension name, __foo__ becomes foo.
> >>>> - if (Extension.startswith("__") && Extension.endswith("__") &&
> >>>> - Extension.size() >= 4)
> >>>> - Extension = Extension.substr(2, Extension.size() - 4);
> >>>> -
> >>>> - // Because we inherit the feature list from HasFeature, this string
> >>>> switch
> >>>> - // must be less restrictive than HasFeature's.
> >>>> - return llvm::StringSwitch<bool>(Extension)
> >>>> - // C11 features supported by other languages as
> extensions.
> >>>> - .Case("c_alignas", true)
> >>>> - .Case("c_atomic", true)
> >>>> - .Case("c_generic_selections", true)
> >>>> - .Case("c_static_assert", true)
> >>>> - // C++0x features supported by other languages as
> extensions.
> >>>> - .Case("cxx_atomic", LangOpts.CPlusPlus)
> >>>> - .Case("cxx_deleted_functions", LangOpts.CPlusPlus)
> >>>> - .Case("cxx_explicit_conversions", LangOpts.CPlusPlus)
> >>>> - .Case("cxx_inline_namespaces", LangOpts.CPlusPlus)
> >>>> - .Case("cxx_local_type_template_args", LangOpts.CPlusPlus)
> >>>> - .Case("cxx_nonstatic_member_init", LangOpts.CPlusPlus)
> >>>> - .Case("cxx_override_control", LangOpts.CPlusPlus)
> >>>> - .Case("cxx_range_for", LangOpts.CPlusPlus)
> >>>> - .Case("cxx_reference_qualified_functions",
> >>>> LangOpts.CPlusPlus)
> >>>> - .Case("cxx_rvalue_references", LangOpts.CPlusPlus)
> >>>> - .Default(false);
> >>>> -}
> >>>> -
> >>>> -/// HasAttribute - Return true if we recognize and implement the
> >>>> attribute
> >>>> -/// specified by the given identifier.
> >>>> -static bool HasAttribute(const IdentifierInfo *II) {
> >>>> - StringRef Name = II->getName();
> >>>> - // Normalize the attribute name, __foo__ becomes foo.
> >>>> - if (Name.startswith("__") && Name.endswith("__") && Name.size() >=
> 4)
> >>>> - Name = Name.substr(2, Name.size() - 4);
> >>>> -
> >>>> - // FIXME: Do we need to handle namespaces here?
> >>>> - return llvm::StringSwitch<bool>(Name)
> >>>> -#include "clang/Lex/AttrSpellings.inc"
> >>>> - .Default(false);
> >>>> -}
> >>>> -
> >>>> -/// EvaluateHasIncludeCommon - Process a '__has_include("path")'
> >>>> -/// or '__has_include_next("path")' expression.
> >>>> -/// Returns true if successful.
> >>>> -static bool EvaluateHasIncludeCommon(Token &Tok,
> >>>> - IdentifierInfo *II, Preprocessor
> >>>> &PP,
> >>>> - const DirectoryLookup
> *LookupFrom)
> >>>> {
> >>>> - SourceLocation LParenLoc;
> >>>> -
> >>>> - // Get '('.
> >>>> - PP.LexNonComment(Tok);
> >>>> -
> >>>> - // Ensure we have a '('.
> >>>> - if (Tok.isNot(tok::l_paren)) {
> >>>> - PP.Diag(Tok.getLocation(), diag::err_pp_missing_lparen) <<
> >>>> II->getName();
> >>>> - return false;
> >>>> - }
> >>>> -
> >>>> - // Save '(' location for possible missing ')' message.
> >>>> - LParenLoc = Tok.getLocation();
> >>>> -
> >>>> - // Get the file name.
> >>>> - PP.getCurrentLexer()->LexIncludeFilename(Tok);
> >>>> -
> >>>> - // Reserve a buffer to get the spelling.
> >>>> - SmallString<128> FilenameBuffer;
> >>>> - StringRef Filename;
> >>>> - SourceLocation EndLoc;
> >>>> -
> >>>> - switch (Tok.getKind()) {
> >>>> - case tok::eod:
> >>>> - // If the token kind is EOD, the error has already been
> diagnosed.
> >>>> - return false;
> >>>> -
> >>>> - case tok::angle_string_literal:
> >>>> - case tok::string_literal: {
> >>>> - bool Invalid = false;
> >>>> - Filename = PP.getSpelling(Tok, FilenameBuffer, &Invalid);
> >>>> - if (Invalid)
> >>>> - return false;
> >>>> - break;
> >>>> - }
> >>>> -
> >>>> - case tok::less:
> >>>> - // This could be a <foo/bar.h> file coming from a macro
> expansion.
> >>>> In this
> >>>> - // case, glue the tokens together into FilenameBuffer and
> interpret
> >>>> those.
> >>>> - FilenameBuffer.push_back('<');
> >>>> - if (PP.ConcatenateIncludeName(FilenameBuffer, EndLoc))
> >>>> - return false; // Found <eod> but no ">"? Diagnostic already
> >>>> emitted.
> >>>> - Filename = FilenameBuffer.str();
> >>>> - break;
> >>>> - default:
> >>>> - PP.Diag(Tok.getLocation(), diag::err_pp_expects_filename);
> >>>> - return false;
> >>>> - }
> >>>> -
> >>>> - // Get ')'.
> >>>> - PP.LexNonComment(Tok);
> >>>> -
> >>>> - // Ensure we have a trailing ).
> >>>> - if (Tok.isNot(tok::r_paren)) {
> >>>> - PP.Diag(Tok.getLocation(), diag::err_pp_missing_rparen) <<
> >>>> II->getName();
> >>>> - PP.Diag(LParenLoc, diag::note_matching) << "(";
> >>>> - return false;
> >>>> - }
> >>>> -
> >>>> - bool isAngled = PP.GetIncludeFilenameSpelling(Tok.getLocation(),
> >>>> Filename);
> >>>> - // If GetIncludeFilenameSpelling set the start ptr to null, there
> was
> >>>> an
> >>>> - // error.
> >>>> - if (Filename.empty())
> >>>> - return false;
> >>>> -
> >>>> - // Search include directories.
> >>>> - const DirectoryLookup *CurDir;
> >>>> - const FileEntry *File =
> >>>> - PP.LookupFile(Filename, isAngled, LookupFrom, CurDir, NULL,
> NULL,
> >>>> NULL);
> >>>> -
> >>>> - // Get the result value. A result of true means the file exists.
> >>>> - return File != 0;
> >>>> -}
> >>>> -
> >>>> -/// EvaluateHasInclude - Process a '__has_include("path")'
> expression.
> >>>> -/// Returns true if successful.
> >>>> -static bool EvaluateHasInclude(Token &Tok, IdentifierInfo *II,
> >>>> - Preprocessor &PP) {
> >>>> - return EvaluateHasIncludeCommon(Tok, II, PP, NULL);
> >>>> -}
> >>>> -
> >>>> -/// EvaluateHasIncludeNext - Process '__has_include_next("path")'
> >>>> expression.
> >>>> -/// Returns true if successful.
> >>>> -static bool EvaluateHasIncludeNext(Token &Tok,
> >>>> - IdentifierInfo *II, Preprocessor
> &PP)
> >>>> {
> >>>> - // __has_include_next is like __has_include, except that we start
> >>>> - // searching after the current found directory. If we can't do
> this,
> >>>> - // issue a diagnostic.
> >>>> - const DirectoryLookup *Lookup = PP.GetCurDirLookup();
> >>>> - if (PP.isInPrimaryFile()) {
> >>>> - Lookup = 0;
> >>>> - PP.Diag(Tok, diag::pp_include_next_in_primary);
> >>>> - } else if (Lookup == 0) {
> >>>> - PP.Diag(Tok, diag::pp_include_next_absolute_path);
> >>>> - } else {
> >>>> - // Start looking up in the next directory.
> >>>> - ++Lookup;
> >>>> - }
> >>>> -
> >>>> - return EvaluateHasIncludeCommon(Tok, II, PP, Lookup);
> >>>> -}
> >>>> -
> >>>> -/// ExpandBuiltinMacro - If an identifier token is read that is to be
> >>>> expanded
> >>>> -/// as a builtin macro, handle it and return the next token as 'Tok'.
> >>>> -void Preprocessor::ExpandBuiltinMacro(Token &Tok) {
> >>>> - // Figure out which token this is.
> >>>> - IdentifierInfo *II = Tok.getIdentifierInfo();
> >>>> - assert(II && "Can't be a macro without id info!");
> >>>> -
> >>>> - // If this is an _Pragma or Microsoft __pragma directive, expand
> it,
> >>>> - // invoke the pragma handler, then lex the token after it.
> >>>> - if (II == Ident_Pragma)
> >>>> - return Handle_Pragma(Tok);
> >>>> - else if (II == Ident__pragma) // in non-MS mode this is null
> >>>> - return HandleMicrosoft__pragma(Tok);
> >>>> -
> >>>> - ++NumBuiltinMacroExpanded;
> >>>> -
> >>>> - SmallString<128> TmpBuffer;
> >>>> - llvm::raw_svector_ostream OS(TmpBuffer);
> >>>> -
> >>>> - // Set up the return result.
> >>>> - Tok.setIdentifierInfo(0);
> >>>> - Tok.clearFlag(Token::NeedsCleaning);
> >>>> -
> >>>> - if (II == Ident__LINE__) {
> >>>> - // C99 6.10.8: "__LINE__: The presumed line number (within the
> >>>> current
> >>>> - // source file) of the current source line (an integer
> constant)".
> >>>> This can
> >>>> - // be affected by #line.
> >>>> - SourceLocation Loc = Tok.getLocation();
> >>>> -
> >>>> - // Advance to the location of the first _, this might not be the
> >>>> first byte
> >>>> - // of the token if it starts with an escaped newline.
> >>>> - Loc = AdvanceToTokenCharacter(Loc, 0);
> >>>> -
> >>>> - // One wrinkle here is that GCC expands __LINE__ to location of
> the
> >>>> *end* of
> >>>> - // a macro expansion. This doesn't matter for object-like
> macros,
> >>>> but
> >>>> - // can matter for a function-like macro that expands to contain
> >>>> __LINE__.
> >>>> - // Skip down through expansion points until we find a file loc
> for
> >>>> the
> >>>> - // end of the expansion history.
> >>>> - Loc = SourceMgr.getExpansionRange(Loc).second;
> >>>> - PresumedLoc PLoc = SourceMgr.getPresumedLoc(Loc);
> >>>> -
> >>>> - // __LINE__ expands to a simple numeric value.
> >>>> - OS << (PLoc.isValid()? PLoc.getLine() : 1);
> >>>> - Tok.setKind(tok::numeric_constant);
> >>>> - } else if (II == Ident__FILE__ || II == Ident__BASE_FILE__) {
> >>>> - // C99 6.10.8: "__FILE__: The presumed name of the current source
> >>>> file (a
> >>>> - // character string literal)". This can be affected by #line.
> >>>> - PresumedLoc PLoc = SourceMgr.getPresumedLoc(Tok.getLocation());
> >>>> -
> >>>> - // __BASE_FILE__ is a GNU extension that returns the top of the
> >>>> presumed
> >>>> - // #include stack instead of the current file.
> >>>> - if (II == Ident__BASE_FILE__ && PLoc.isValid()) {
> >>>> - SourceLocation NextLoc = PLoc.getIncludeLoc();
> >>>> - while (NextLoc.isValid()) {
> >>>> - PLoc = SourceMgr.getPresumedLoc(NextLoc);
> >>>> - if (PLoc.isInvalid())
> >>>> - break;
> >>>> -
> >>>> - NextLoc = PLoc.getIncludeLoc();
> >>>> - }
> >>>> - }
> >>>> -
> >>>> - // Escape this filename. Turn '\' -> '\\' '"' -> '\"'
> >>>> - SmallString<128> FN;
> >>>> - if (PLoc.isValid()) {
> >>>> - FN += PLoc.getFilename();
> >>>> - Lexer::Stringify(FN);
> >>>> - OS << '"' << FN.str() << '"';
> >>>> - }
> >>>> - Tok.setKind(tok::string_literal);
> >>>> - } else if (II == Ident__DATE__) {
> >>>> - if (!DATELoc.isValid())
> >>>> - ComputeDATE_TIME(DATELoc, TIMELoc, *this);
> >>>> - Tok.setKind(tok::string_literal);
> >>>> - Tok.setLength(strlen("\"Mmm dd yyyy\""));
> >>>> - Tok.setLocation(SourceMgr.createExpansionLoc(DATELoc,
> >>>> Tok.getLocation(),
> >>>> - Tok.getLocation(),
> >>>> - Tok.getLength()));
> >>>> - return;
> >>>> - } else if (II == Ident__TIME__) {
> >>>> - if (!TIMELoc.isValid())
> >>>> - ComputeDATE_TIME(DATELoc, TIMELoc, *this);
> >>>> - Tok.setKind(tok::string_literal);
> >>>> - Tok.setLength(strlen("\"hh:mm:ss\""));
> >>>> - Tok.setLocation(SourceMgr.createExpansionLoc(TIMELoc,
> >>>> Tok.getLocation(),
> >>>> - Tok.getLocation(),
> >>>> - Tok.getLength()));
> >>>> - return;
> >>>> - } else if (II == Ident__INCLUDE_LEVEL__) {
> >>>> - // Compute the presumed include depth of this token. This can be
> >>>> affected
> >>>> - // by GNU line markers.
> >>>> - unsigned Depth = 0;
> >>>> -
> >>>> - PresumedLoc PLoc = SourceMgr.getPresumedLoc(Tok.getLocation());
> >>>> - if (PLoc.isValid()) {
> >>>> - PLoc = SourceMgr.getPresumedLoc(PLoc.getIncludeLoc());
> >>>> - for (; PLoc.isValid(); ++Depth)
> >>>> - PLoc = SourceMgr.getPresumedLoc(PLoc.getIncludeLoc());
> >>>> - }
> >>>> -
> >>>> - // __INCLUDE_LEVEL__ expands to a simple numeric value.
> >>>> - OS << Depth;
> >>>> - Tok.setKind(tok::numeric_constant);
> >>>> - } else if (II == Ident__TIMESTAMP__) {
> >>>> - // MSVC, ICC, GCC, VisualAge C++ extension. The generated string
> >>>> should be
> >>>> - // of the form "Ddd Mmm dd hh::mm::ss yyyy", which is returned by
> >>>> asctime.
> >>>> -
> >>>> - // Get the file that we are lexing out of. If we're currently
> >>>> lexing from
> >>>> - // a macro, dig into the include stack.
> >>>> - const FileEntry *CurFile = 0;
> >>>> - PreprocessorLexer *TheLexer = getCurrentFileLexer();
> >>>> -
> >>>> - if (TheLexer)
> >>>> - CurFile = SourceMgr.getFileEntryForID(TheLexer->getFileID());
> >>>> -
> >>>> - const char *Result;
> >>>> - if (CurFile) {
> >>>> - time_t TT = CurFile->getModificationTime();
> >>>> - struct tm *TM = localtime(&TT);
> >>>> - Result = asctime(TM);
> >>>> - } else {
> >>>> - Result = "??? ??? ?? ??:??:?? ????\n";
> >>>> - }
> >>>> - // Surround the string with " and strip the trailing newline.
> >>>> - OS << '"' << StringRef(Result, strlen(Result)-1) << '"';
> >>>> - Tok.setKind(tok::string_literal);
> >>>> - } else if (II == Ident__COUNTER__) {
> >>>> - // __COUNTER__ expands to a simple numeric value.
> >>>> - OS << CounterValue++;
> >>>> - Tok.setKind(tok::numeric_constant);
> >>>> - } else if (II == Ident__has_feature ||
> >>>> - II == Ident__has_extension ||
> >>>> - II == Ident__has_builtin ||
> >>>> - II == Ident__has_attribute) {
> >>>> - // The argument to these builtins should be a parenthesized
> >>>> identifier.
> >>>> - SourceLocation StartLoc = Tok.getLocation();
> >>>> -
> >>>> - bool IsValid = false;
> >>>> - IdentifierInfo *FeatureII = 0;
> >>>> -
> >>>> - // Read the '('.
> >>>> - Lex(Tok);
> >>>> - if (Tok.is(tok::l_paren)) {
> >>>> - // Read the identifier
> >>>> - Lex(Tok);
> >>>> - if (Tok.is(tok::identifier) || Tok.is(tok::kw_const)) {
> >>>> - FeatureII = Tok.getIdentifierInfo();
> >>>> -
> >>>> - // Read the ')'.
> >>>> - Lex(Tok);
> >>>> - if (Tok.is(tok::r_paren))
> >>>> - IsValid = true;
> >>>> - }
> >>>> - }
> >>>> -
> >>>> - bool Value = false;
> >>>> - if (!IsValid)
> >>>> - Diag(StartLoc, diag::err_feature_check_malformed);
> >>>> - else if (II == Ident__has_builtin) {
> >>>> - // Check for a builtin is trivial.
> >>>> - Value = FeatureII->getBuiltinID() != 0;
> >>>> - } else if (II == Ident__has_attribute)
> >>>> - Value = HasAttribute(FeatureII);
> >>>> - else if (II == Ident__has_extension)
> >>>> - Value = HasExtension(*this, FeatureII);
> >>>> - else {
> >>>> - assert(II == Ident__has_feature && "Must be feature check");
> >>>> - Value = HasFeature(*this, FeatureII);
> >>>> - }
> >>>> -
> >>>> - OS << (int)Value;
> >>>> - if (IsValid)
> >>>> - Tok.setKind(tok::numeric_constant);
> >>>> - } else if (II == Ident__has_include ||
> >>>> - II == Ident__has_include_next) {
> >>>> - // The argument to these two builtins should be a parenthesized
> >>>> - // file name string literal using angle brackets (<>) or
> >>>> - // double-quotes ("").
> >>>> - bool Value;
> >>>> - if (II == Ident__has_include)
> >>>> - Value = EvaluateHasInclude(Tok, II, *this);
> >>>> - else
> >>>> - Value = EvaluateHasIncludeNext(Tok, II, *this);
> >>>> - OS << (int)Value;
> >>>> - Tok.setKind(tok::numeric_constant);
> >>>> - } else if (II == Ident__has_warning) {
> >>>> - // The argument should be a parenthesized string literal.
> >>>> - // The argument to these builtins should be a parenthesized
> >>>> identifier.
> >>>> - SourceLocation StartLoc = Tok.getLocation();
> >>>> - bool IsValid = false;
> >>>> - bool Value = false;
> >>>> - // Read the '('.
> >>>> - Lex(Tok);
> >>>> - do {
> >>>> - if (Tok.is(tok::l_paren)) {
> >>>> - // Read the string.
> >>>> - Lex(Tok);
> >>>> -
> >>>> - // We need at least one string literal.
> >>>> - if (!Tok.is(tok::string_literal)) {
> >>>> - StartLoc = Tok.getLocation();
> >>>> - IsValid = false;
> >>>> - // Eat tokens until ')'.
> >>>> - do Lex(Tok); while (!(Tok.is(tok::r_paren) ||
> >>>> Tok.is(tok::eod)));
> >>>> - break;
> >>>> - }
> >>>> -
> >>>> - // String concatenation allows multiple strings, which can
> even
> >>>> come
> >>>> - // from macro expansion.
> >>>> - SmallVector<Token, 4> StrToks;
> >>>> - while (Tok.is(tok::string_literal)) {
> >>>> - // Complain about, and drop, any ud-suffix.
> >>>> - if (Tok.hasUDSuffix())
> >>>> - Diag(Tok, diag::err_invalid_string_udl);
> >>>> - StrToks.push_back(Tok);
> >>>> - LexUnexpandedToken(Tok);
> >>>> - }
> >>>> -
> >>>> - // Is the end a ')'?
> >>>> - if (!(IsValid = Tok.is(tok::r_paren)))
> >>>> - break;
> >>>> -
> >>>> - // Concatenate and parse the strings.
> >>>> - StringLiteralParser Literal(&StrToks[0], StrToks.size(),
> *this);
> >>>> - assert(Literal.isAscii() && "Didn't allow wide strings in");
> >>>> - if (Literal.hadError)
> >>>> - break;
> >>>> - if (Literal.Pascal) {
> >>>> - Diag(Tok, diag::warn_pragma_diagnostic_invalid);
> >>>> - break;
> >>>> - }
> >>>> -
> >>>> - StringRef WarningName(Literal.GetString());
> >>>> -
> >>>> - if (WarningName.size() < 3 || WarningName[0] != '-' ||
> >>>> - WarningName[1] != 'W') {
> >>>> - Diag(StrToks[0].getLocation(),
> >>>> diag::warn_has_warning_invalid_option);
> >>>> - break;
> >>>> - }
> >>>> -
> >>>> - // Finally, check if the warning flags maps to a diagnostic
> >>>> group.
> >>>> - // We construct a SmallVector here to talk to
> >>>> getDiagnosticIDs().
> >>>> - // Although we don't use the result, this isn't a hot path,
> and
> >>>> not
> >>>> - // worth special casing.
> >>>> - llvm::SmallVector<diag::kind, 10> Diags;
> >>>> - Value = !getDiagnostics().getDiagnosticIDs()->
> >>>> - getDiagnosticsInGroup(WarningName.substr(2), Diags);
> >>>> - }
> >>>> - } while (false);
> >>>> -
> >>>> - if (!IsValid)
> >>>> - Diag(StartLoc, diag::err_warning_check_malformed);
> >>>> -
> >>>> - OS << (int)Value;
> >>>> - Tok.setKind(tok::numeric_constant);
> >>>> - } else {
> >>>> - llvm_unreachable("Unknown identifier!");
> >>>> - }
> >>>> - CreateString(OS.str().data(), OS.str().size(), Tok,
> >>>> - Tok.getLocation(), Tok.getLocation());
> >>>> -}
> >>>> -
> >>>> -void Preprocessor::markMacroAsUsed(MacroInfo *MI) {
> >>>> - // If the 'used' status changed, and the macro requires 'unused'
> >>>> warning,
> >>>> - // remove its SourceLocation from the warn-for-unused-macro
> locations.
> >>>> - if (MI->isWarnIfUnused() && !MI->isUsed())
> >>>> - WarnUnusedMacroLocs.erase(MI->getDefinitionLoc());
> >>>> - MI->setIsUsed(true);
> >>>> -}
> >>>> +//===--- MacroExpansion.cpp - Top level Macro Expansion
> >>>> -------------------===//
> >>>> +//
> >>>> +// The LLVM Compiler Infrastructure
> >>>> +//
> >>>> +// This file is distributed under the University of Illinois Open
> Source
> >>>> +// License. See LICENSE.TXT for details.
> >>>> +//
> >>>>
> >>>>
> +//===----------------------------------------------------------------------===//
> >>>> +//
> >>>> +// This file implements the top level handling of macro expasion for
> the
> >>>> +// preprocessor.
> >>>> +//
> >>>>
> >>>>
> +//===----------------------------------------------------------------------===//
> >>>> +
> >>>> +#include "clang/Lex/Preprocessor.h"
> >>>> +#include "MacroArgs.h"
> >>>> +#include "clang/Lex/MacroInfo.h"
> >>>> +#include "clang/Basic/SourceManager.h"
> >>>> +#include "clang/Basic/FileManager.h"
> >>>> +#include "clang/Basic/TargetInfo.h"
> >>>> +#include "clang/Lex/LexDiagnostic.h"
> >>>> +#include "clang/Lex/CodeCompletionHandler.h"
> >>>> +#include "clang/Lex/ExternalPreprocessorSource.h"
> >>>> +#include "clang/Lex/LiteralSupport.h"
> >>>> +#include "llvm/ADT/StringSwitch.h"
> >>>> +#include "llvm/ADT/STLExtras.h"
> >>>> +#include "llvm/Config/llvm-config.h"
> >>>> +#include "llvm/Support/raw_ostream.h"
> >>>> +#include "llvm/Support/ErrorHandling.h"
> >>>> +#include <cstdio>
> >>>> +#include <ctime>
> >>>> +using namespace clang;
> >>>> +
> >>>> +MacroInfo *Preprocessor::getInfoForMacro(IdentifierInfo *II) const {
> >>>> + assert(II->hasMacroDefinition() && "Identifier is not a macro!");
> >>>> +
> >>>> + macro_iterator Pos = Macros.find(II);
> >>>> + if (Pos == Macros.end()) {
> >>>> + // Load this macro from the external source.
> >>>> + getExternalSource()->LoadMacroDefinition(II);
> >>>> + Pos = Macros.find(II);
> >>>> + }
> >>>> + assert(Pos != Macros.end() && "Identifier macro info is missing!");
> >>>> + assert(Pos->second->getUndefLoc().isInvalid() && "Macro is
> >>>> undefined!");
> >>>> + return Pos->second;
> >>>> +}
> >>>> +
> >>>> +/// setMacroInfo - Specify a macro for this identifier.
> >>>> +///
> >>>> +void Preprocessor::setMacroInfo(IdentifierInfo *II, MacroInfo *MI,
> >>>> + bool LoadedFromAST) {
> >>>> + assert(MI && "MacroInfo should be non-zero!");
> >>>> + MI->setPreviousDefinition(Macros[II]);
> >>>> + Macros[II] = MI;
> >>>> + II->setHasMacroDefinition(true);
> >>>> + if (II->isFromAST() && !LoadedFromAST)
> >>>> + II->setChangedSinceDeserialization();
> >>>> +}
> >>>> +
> >>>> +/// \brief Undefine a macro for this identifier.
> >>>> +void Preprocessor::clearMacroInfo(IdentifierInfo *II) {
> >>>> + assert(II->hasMacroDefinition() && "Macro is not defined!");
> >>>> + assert(Macros[II]->getUndefLoc().isValid() && "Macro is still
> >>>> defined!");
> >>>> + II->setHasMacroDefinition(false);
> >>>> + if (II->isFromAST())
> >>>> + II->setChangedSinceDeserialization();
> >>>> +}
> >>>> +
> >>>> +/// RegisterBuiltinMacro - Register the specified identifier in the
> >>>> identifier
> >>>> +/// table and mark it as a builtin macro to be expanded.
> >>>> +static IdentifierInfo *RegisterBuiltinMacro(Preprocessor &PP, const
> char
> >>>> *Name){
> >>>> + // Get the identifier.
> >>>> + IdentifierInfo *Id = PP.getIdentifierInfo(Name);
> >>>> +
> >>>> + // Mark it as being a macro that is builtin.
> >>>> + MacroInfo *MI = PP.AllocateMacroInfo(SourceLocation());
> >>>> + MI->setIsBuiltinMacro();
> >>>> + PP.setMacroInfo(Id, MI);
> >>>> + return Id;
> >>>> +}
> >>>> +
> >>>> +
> >>>> +/// RegisterBuiltinMacros - Register builtin macros, such as __LINE__
> >>>> with the
> >>>> +/// identifier table.
> >>>> +void Preprocessor::RegisterBuiltinMacros() {
> >>>> + Ident__LINE__ = RegisterBuiltinMacro(*this, "__LINE__");
> >>>> + Ident__FILE__ = RegisterBuiltinMacro(*this, "__FILE__");
> >>>> + Ident__DATE__ = RegisterBuiltinMacro(*this, "__DATE__");
> >>>> + Ident__TIME__ = RegisterBuiltinMacro(*this, "__TIME__");
> >>>> + Ident__COUNTER__ = RegisterBuiltinMacro(*this, "__COUNTER__");
> >>>> + Ident_Pragma = RegisterBuiltinMacro(*this, "_Pragma");
> >>>> +
> >>>> + // GCC Extensions.
> >>>> + Ident__BASE_FILE__ = RegisterBuiltinMacro(*this,
> "__BASE_FILE__");
> >>>> + Ident__INCLUDE_LEVEL__ = RegisterBuiltinMacro(*this,
> >>>> "__INCLUDE_LEVEL__");
> >>>> + Ident__TIMESTAMP__ = RegisterBuiltinMacro(*this,
> "__TIMESTAMP__");
> >>>> +
> >>>> + // Clang Extensions.
> >>>> + Ident__has_feature = RegisterBuiltinMacro(*this,
> >>>> "__has_feature");
> >>>> + Ident__has_extension = RegisterBuiltinMacro(*this,
> >>>> "__has_extension");
> >>>> + Ident__has_builtin = RegisterBuiltinMacro(*this,
> >>>> "__has_builtin");
> >>>> + Ident__has_attribute = RegisterBuiltinMacro(*this,
> >>>> "__has_attribute");
> >>>> + Ident__has_include = RegisterBuiltinMacro(*this,
> >>>> "__has_include");
> >>>> + Ident__has_include_next = RegisterBuiltinMacro(*this,
> >>>> "__has_include_next");
> >>>> + Ident__has_warning = RegisterBuiltinMacro(*this,
> >>>> "__has_warning");
> >>>> +
> >>>> + // Microsoft Extensions.
> >>>> + if (LangOpts.MicrosoftExt)
> >>>> + Ident__pragma = RegisterBuiltinMacro(*this, "__pragma");
> >>>> + else
> >>>> + Ident__pragma = 0;
> >>>> +}
> >>>> +
> >>>> +/// isTrivialSingleTokenExpansion - Return true if MI, which has a
> >>>> single token
> >>>> +/// in its expansion, currently expands to that token literally.
> >>>> +static bool isTrivialSingleTokenExpansion(const MacroInfo *MI,
> >>>> + const IdentifierInfo
> >>>> *MacroIdent,
> >>>> + Preprocessor &PP) {
> >>>> + IdentifierInfo *II =
> MI->getReplacementToken(0).getIdentifierInfo();
> >>>> +
> >>>> + // If the token isn't an identifier, it's always literally
> expanded.
> >>>> + if (II == 0) return true;
> >>>> +
> >>>> + // If the information about this identifier is out of date, update
> it
> >>>> from
> >>>> + // the external source.
> >>>> + if (II->isOutOfDate())
> >>>> + PP.getExternalSource()->updateOutOfDateIdentifier(*II);
> >>>> +
> >>>> + // If the identifier is a macro, and if that macro is enabled, it
> may
> >>>> be
> >>>> + // expanded so it's not a trivial expansion.
> >>>> + if (II->hasMacroDefinition() && PP.getMacroInfo(II)->isEnabled() &&
> >>>> + // Fast expanding "#define X X" is ok, because X would be
> >>>> disabled.
> >>>> + II != MacroIdent)
> >>>> + return false;
> >>>> +
> >>>> + // If this is an object-like macro invocation, it is safe to
> trivially
> >>>> expand
> >>>> + // it.
> >>>> + if (MI->isObjectLike()) return true;
> >>>> +
> >>>> + // If this is a function-like macro invocation, it's safe to
> trivially
> >>>> expand
> >>>> + // as long as the identifier is not a macro argument.
> >>>> + for (MacroInfo::arg_iterator I = MI->arg_begin(), E =
> MI->arg_end();
> >>>> + I != E; ++I)
> >>>> + if (*I == II)
> >>>> + return false; // Identifier is a macro argument.
> >>>> +
> >>>> + return true;
> >>>> +}
> >>>> +
> >>>> +
> >>>> +/// isNextPPTokenLParen - Determine whether the next preprocessor
> token
> >>>> to be
> >>>> +/// lexed is a '('. If so, consume the token and return true, if
> not,
> >>>> this
> >>>> +/// method should have no observable side-effect on the lexed tokens.
> >>>> +bool Preprocessor::isNextPPTokenLParen() {
> >>>> + // Do some quick tests for rejection cases.
> >>>> + unsigned Val;
> >>>> + if (CurLexer)
> >>>> + Val = CurLexer->isNextPPTokenLParen();
> >>>> + else if (CurPTHLexer)
> >>>> + Val = CurPTHLexer->isNextPPTokenLParen();
> >>>> + else
> >>>> + Val = CurTokenLexer->isNextTokenLParen();
> >>>> +
> >>>> + if (Val == 2) {
> >>>> + // We have run off the end. If it's a source file we don't
> >>>> + // examine enclosing ones (C99 5.1.1.2p4). Otherwise walk up the
> >>>> + // macro stack.
> >>>> + if (CurPPLexer)
> >>>> + return false;
> >>>> + for (unsigned i = IncludeMacroStack.size(); i != 0; --i) {
> >>>> + IncludeStackInfo &Entry = IncludeMacroStack[i-1];
> >>>> + if (Entry.TheLexer)
> >>>> + Val = Entry.TheLexer->isNextPPTokenLParen();
> >>>> + else if (Entry.ThePTHLexer)
> >>>> + Val = Entry.ThePTHLexer->isNextPPTokenLParen();
> >>>> + else
> >>>> + Val = Entry.TheTokenLexer->isNextTokenLParen();
> >>>> +
> >>>> + if (Val != 2)
> >>>> + break;
> >>>> +
> >>>> + // Ran off the end of a source file?
> >>>> + if (Entry.ThePPLexer)
> >>>> + return false;
> >>>> + }
> >>>> + }
> >>>> +
> >>>> + // Okay, if we know that the token is a '(', lex it and return.
> >>>> Otherwise we
> >>>> + // have found something that isn't a '(' or we found the end of the
> >>>> + // translation unit. In either case, return false.
> >>>> + return Val == 1;
> >>>> +}
> >>>> +
> >>>> +/// HandleMacroExpandedIdentifier - If an identifier token is read
> that
> >>>> is to be
> >>>> +/// expanded as a macro, handle it and return the next token as
> >>>> 'Identifier'.
> >>>> +bool Preprocessor::HandleMacroExpandedIdentifier(Token &Identifier,
> >>>> + MacroInfo *MI) {
> >>>> + // If this is a macro expansion in the "#if !defined(x)" line for
> the
> >>>> file,
> >>>> + // then the macro could expand to different things in other
> contexts,
> >>>> we need
> >>>> + // to disable the optimization in this case.
> >>>> + if (CurPPLexer) CurPPLexer->MIOpt.ExpandedMacro();
> >>>> +
> >>>> + // If this is a builtin macro, like __LINE__ or _Pragma, handle it
> >>>> specially.
> >>>> + if (MI->isBuiltinMacro()) {
> >>>> + if (Callbacks) Callbacks->MacroExpands(Identifier, MI,
> >>>> + Identifier.getLocation());
> >>>> + ExpandBuiltinMacro(Identifier);
> >>>> + return false;
> >>>> + }
> >>>> +
> >>>> + /// Args - If this is a function-like macro expansion, this
> contains,
> >>>> + /// for each macro argument, the list of tokens that were provided
> to
> >>>> the
> >>>> + /// invocation.
> >>>> + MacroArgs *Args = 0;
> >>>> +
> >>>> + // Remember where the end of the expansion occurred. For an
> >>>> object-like
> >>>> + // macro, this is the identifier. For a function-like macro, this
> is
> >>>> the ')'.
> >>>> + SourceLocation ExpansionEnd = Identifier.getLocation();
> >>>> +
> >>>> + // If this is a function-like macro, read the arguments.
> >>>> + if (MI->isFunctionLike()) {
> >>>> + // C99 6.10.3p10: If the preprocessing token immediately after
> the
> >>>> macro
> >>>> + // name isn't a '(', this macro should not be expanded.
> >>>> + if (!isNextPPTokenLParen())
> >>>> + return true;
> >>>> +
> >>>> + // Remember that we are now parsing the arguments to a macro
> >>>> invocation.
> >>>> + // Preprocessor directives used inside macro arguments are not
> >>>> portable, and
> >>>> + // this enables the warning.
> >>>> + InMacroArgs = true;
> >>>> + Args = ReadFunctionLikeMacroArgs(Identifier, MI, ExpansionEnd);
> >>>> +
> >>>> + // Finished parsing args.
> >>>> + InMacroArgs = false;
> >>>> +
> >>>> + // If there was an error parsing the arguments, bail out.
> >>>> + if (Args == 0) return false;
> >>>> +
> >>>> + ++NumFnMacroExpanded;
> >>>> + } else {
> >>>> + ++NumMacroExpanded;
> >>>> + }
> >>>> +
> >>>> + // Notice that this macro has been used.
> >>>> + markMacroAsUsed(MI);
> >>>> +
> >>>> + // Remember where the token is expanded.
> >>>> + SourceLocation ExpandLoc = Identifier.getLocation();
> >>>> + SourceRange ExpansionRange(ExpandLoc, ExpansionEnd);
> >>>> +
> >>>> + if (Callbacks) {
> >>>> + if (InMacroArgs) {
> >>>> + // We can have macro expansion inside a conditional directive
> >>>> while
> >>>> + // reading the function macro arguments. To ensure, in that
> case,
> >>>> that
> >>>> + // MacroExpands callbacks still happen in source order, queue
> this
> >>>> + // callback to have it happen after the function macro
> callback.
> >>>> + DelayedMacroExpandsCallbacks.push_back(
> >>>> + MacroExpandsInfo(Identifier, MI,
> >>>> ExpansionRange));
> >>>> + } else {
> >>>> + Callbacks->MacroExpands(Identifier, MI, ExpansionRange);
> >>>> + if (!DelayedMacroExpandsCallbacks.empty()) {
> >>>> + for (unsigned i=0, e = DelayedMacroExpandsCallbacks.size();
> >>>> i!=e; ++i) {
> >>>> + MacroExpandsInfo &Info = DelayedMacroExpandsCallbacks[i];
> >>>> + Callbacks->MacroExpands(Info.Tok, Info.MI, Info.Range);
> >>>> + }
> >>>> + DelayedMacroExpandsCallbacks.clear();
> >>>> + }
> >>>> + }
> >>>> + }
> >>>> +
> >>>> + // If we started lexing a macro, enter the macro expansion body.
> >>>> +
> >>>> + // If this macro expands to no tokens, don't bother to push it onto
> >>>> the
> >>>> + // expansion stack, only to take it right back off.
> >>>> + if (MI->getNumTokens() == 0) {
> >>>> + // No need for arg info.
> >>>> + if (Args) Args->destroy(*this);
> >>>> +
> >>>> + // Ignore this macro use, just return the next token in the
> current
> >>>> + // buffer.
> >>>> + bool HadLeadingSpace = Identifier.hasLeadingSpace();
> >>>> + bool IsAtStartOfLine = Identifier.isAtStartOfLine();
> >>>> +
> >>>> + Lex(Identifier);
> >>>> +
> >>>> + // If the identifier isn't on some OTHER line, inherit the
> leading
> >>>> + // whitespace/first-on-a-line property of this token. This
> handles
> >>>> + // stuff like "! XX," -> "! ," and " XX," -> " ,", when XX
> is
> >>>> + // empty.
> >>>> + if (!Identifier.isAtStartOfLine()) {
> >>>> + if (IsAtStartOfLine) Identifier.setFlag(Token::StartOfLine);
> >>>> + if (HadLeadingSpace) Identifier.setFlag(Token::LeadingSpace);
> >>>> + }
> >>>> + Identifier.setFlag(Token::LeadingEmptyMacro);
> >>>> + ++NumFastMacroExpanded;
> >>>> + return false;
> >>>> +
> >>>> + } else if (MI->getNumTokens() == 1 &&
> >>>> + isTrivialSingleTokenExpansion(MI,
> >>>> Identifier.getIdentifierInfo(),
> >>>> + *this)) {
> >>>> + // Otherwise, if this macro expands into a single
> trivially-expanded
> >>>> + // token: expand it now. This handles common cases like
> >>>> + // "#define VAL 42".
> >>>> +
> >>>> + // No need for arg info.
> >>>> + if (Args) Args->destroy(*this);
> >>>> +
> >>>> + // Propagate the isAtStartOfLine/hasLeadingSpace markers of the
> >>>> macro
> >>>> + // identifier to the expanded token.
> >>>> + bool isAtStartOfLine = Identifier.isAtStartOfLine();
> >>>> + bool hasLeadingSpace = Identifier.hasLeadingSpace();
> >>>> +
> >>>> + // Replace the result token.
> >>>> + Identifier = MI->getReplacementToken(0);
> >>>> +
> >>>> + // Restore the StartOfLine/LeadingSpace markers.
> >>>> + Identifier.setFlagValue(Token::StartOfLine , isAtStartOfLine);
> >>>> + Identifier.setFlagValue(Token::LeadingSpace, hasLeadingSpace);
> >>>> +
> >>>> + // Update the tokens location to include both its expansion and
> >>>> physical
> >>>> + // locations.
> >>>> + SourceLocation Loc =
> >>>> + SourceMgr.createExpansionLoc(Identifier.getLocation(),
> ExpandLoc,
> >>>> +
> ExpansionEnd,Identifier.getLength());
> >>>> + Identifier.setLocation(Loc);
> >>>> +
> >>>> + // If this is a disabled macro or #define X X, we must mark the
> >>>> result as
> >>>> + // unexpandable.
> >>>> + if (IdentifierInfo *NewII = Identifier.getIdentifierInfo()) {
> >>>> + if (MacroInfo *NewMI = getMacroInfo(NewII))
> >>>> + if (!NewMI->isEnabled() || NewMI == MI) {
> >>>> + Identifier.setFlag(Token::DisableExpand);
> >>>> + Diag(Identifier, diag::pp_disabled_macro_expansion);
> >>>> + }
> >>>> + }
> >>>> +
> >>>> + // Since this is not an identifier token, it can't be macro
> >>>> expanded, so
> >>>> + // we're done.
> >>>> + ++NumFastMacroExpanded;
> >>>> + return false;
> >>>> + }
> >>>> +
> >>>> + // Start expanding the macro.
> >>>> + EnterMacro(Identifier, ExpansionEnd, MI, Args);
> >>>> +
> >>>> + // Now that the macro is at the top of the include stack, ask the
> >>>> + // preprocessor to read the next token from it.
> >>>> + Lex(Identifier);
> >>>> + return false;
> >>>> +}
> >>>> +
> >>>> +/// ReadFunctionLikeMacroArgs - After reading "MACRO" and knowing
> that
> >>>> the next
> >>>> +/// token is the '(' of the macro, this method is invoked to read
> all of
> >>>> the
> >>>> +/// actual arguments specified for the macro invocation. This
> returns
> >>>> null on
> >>>> +/// error.
> >>>> +MacroArgs *Preprocessor::ReadFunctionLikeMacroArgs(Token &MacroName,
> >>>> + MacroInfo *MI,
> >>>> + SourceLocation
> >>>> &MacroEnd) {
> >>>> + // The number of fixed arguments to parse.
> >>>> + unsigned NumFixedArgsLeft = MI->getNumArgs();
> >>>> + bool isVariadic = MI->isVariadic();
> >>>> +
> >>>> + // Outer loop, while there are more arguments, keep reading them.
> >>>> + Token Tok;
> >>>> +
> >>>> + // Read arguments as unexpanded tokens. This avoids issues, e.g.,
> >>>> where
> >>>> + // an argument value in a macro could expand to ',' or '(' or ')'.
> >>>> + LexUnexpandedToken(Tok);
> >>>> + assert(Tok.is(tok::l_paren) && "Error computing l-paren-ness?");
> >>>> +
> >>>> + // ArgTokens - Build up a list of tokens that make up each
> argument.
> >>>> Each
> >>>> + // argument is separated by an EOF token. Use a SmallVector so we
> can
> >>>> avoid
> >>>> + // heap allocations in the common case.
> >>>> + SmallVector<Token, 64> ArgTokens;
> >>>> +
> >>>> + unsigned NumActuals = 0;
> >>>> + while (Tok.isNot(tok::r_paren)) {
> >>>> + assert((Tok.is(tok::l_paren) || Tok.is(tok::comma)) &&
> >>>> + "only expect argument separators here");
> >>>> +
> >>>> + unsigned ArgTokenStart = ArgTokens.size();
> >>>> + SourceLocation ArgStartLoc = Tok.getLocation();
> >>>> +
> >>>> + // C99 6.10.3p11: Keep track of the number of l_parens we have
> seen.
> >>>> Note
> >>>> + // that we already consumed the first one.
> >>>> + unsigned NumParens = 0;
> >>>> +
> >>>> + while (1) {
> >>>> + // Read arguments as unexpanded tokens. This avoids issues,
> e.g.,
> >>>> where
> >>>> + // an argument value in a macro could expand to ',' or '(' or
> ')'.
> >>>> + LexUnexpandedToken(Tok);
> >>>> +
> >>>> + if (Tok.is(tok::eof) || Tok.is(tok::eod)) { // "#if f(<eof>" &
> >>>> "#if f(\n"
> >>>> + Diag(MacroName, diag::err_unterm_macro_invoc);
> >>>> + // Do not lose the EOF/EOD. Return it to the client.
> >>>> + MacroName = Tok;
> >>>> + return 0;
> >>>> + } else if (Tok.is(tok::r_paren)) {
> >>>> + // If we found the ) token, the macro arg list is done.
> >>>> + if (NumParens-- == 0) {
> >>>> + MacroEnd = Tok.getLocation();
> >>>> + break;
> >>>> + }
> >>>> + } else if (Tok.is(tok::l_paren)) {
> >>>> + ++NumParens;
> >>>> + // In Microsoft-compatibility mode, commas from nested macro
> >>>> expan-
> >>>> + // sions should not be considered as argument separators. We
> test
> >>>> + // for this with the IgnoredComma token flag.
> >>>> + } else if (Tok.is(tok::comma)
> >>>> + && !(Tok.getFlags() & Token::IgnoredComma) && NumParens ==
> 0)
> >>>> {
> >>>> + // Comma ends this argument if there are more fixed arguments
> >>>> expected.
> >>>> + // However, if this is a variadic macro, and this is part of
> the
> >>>> + // variadic part, then the comma is just an argument token.
> >>>> + if (!isVariadic) break;
> >>>> + if (NumFixedArgsLeft > 1)
> >>>> + break;
> >>>> + } else if (Tok.is(tok::comment) && !KeepMacroComments) {
> >>>> + // If this is a comment token in the argument list and we're
> >>>> just in
> >>>> + // -C mode (not -CC mode), discard the comment.
> >>>> + continue;
> >>>> + } else if (Tok.getIdentifierInfo() != 0) {
> >>>> + // Reading macro arguments can cause macros that we are
> >>>> currently
> >>>> + // expanding from to be popped off the expansion stack.
> Doing
> >>>> so causes
> >>>> + // them to be reenabled for expansion. Here we record
> whether
> >>>> any
> >>>> + // identifiers we lex as macro arguments correspond to
> disabled
> >>>> macros.
> >>>> + // If so, we mark the token as noexpand. This is a subtle
> >>>> aspect of
> >>>> + // C99 6.10.3.4p2.
> >>>> + if (MacroInfo *MI = getMacroInfo(Tok.getIdentifierInfo()))
> >>>> + if (!MI->isEnabled())
> >>>> + Tok.setFlag(Token::DisableExpand);
> >>>> + } else if (Tok.is(tok::code_completion)) {
> >>>> + if (CodeComplete)
> >>>> +
> >>>> CodeComplete->CodeCompleteMacroArgument(MacroName.getIdentifierInfo(),
> >>>> + MI, NumActuals);
> >>>> + // Don't mark that we reached the code-completion point
> because
> >>>> the
> >>>> + // parser is going to handle the token and there will be
> another
> >>>> + // code-completion callback.
> >>>> + }
> >>>> +
> >>>> + ArgTokens.push_back(Tok);
> >>>> + }
> >>>> +
> >>>> + // If this was an empty argument list foo(), don't add this as an
> >>>> empty
> >>>> + // argument.
> >>>> + if (ArgTokens.empty() && Tok.getKind() == tok::r_paren)
> >>>> + break;
> >>>> +
> >>>> + // If this is not a variadic macro, and too many args were
> >>>> specified, emit
> >>>> + // an error.
> >>>> + if (!isVariadic && NumFixedArgsLeft == 0) {
> >>>> + if (ArgTokens.size() != ArgTokenStart)
> >>>> + ArgStartLoc = ArgTokens[ArgTokenStart].getLocation();
> >>>> +
> >>>> + // Emit the diagnostic at the macro name in case there is a
> >>>> missing ).
> >>>> + // Emitting it at the , could be far away from the macro name.
> >>>> + Diag(ArgStartLoc, diag::err_too_many_args_in_macro_invoc);
> >>>> + return 0;
> >>>> + }
> >>>> +
> >>>> + // Empty arguments are standard in C99 and C++0x, and are
> supported
> >>>> as an extension in
> >>>> + // other modes.
> >>>> + if (ArgTokens.size() == ArgTokenStart && !LangOpts.C99)
> >>>> + Diag(Tok, LangOpts.CPlusPlus0x ?
> >>>> + diag::warn_cxx98_compat_empty_fnmacro_arg :
> >>>> + diag::ext_empty_fnmacro_arg);
> >>>> +
> >>>> + // Add a marker EOF token to the end of the token list for this
> >>>> argument.
> >>>> + Token EOFTok;
> >>>> + EOFTok.startToken();
> >>>> + EOFTok.setKind(tok::eof);
> >>>> + EOFTok.setLocation(Tok.getLocation());
> >>>> + EOFTok.setLength(0);
> >>>> + ArgTokens.push_back(EOFTok);
> >>>> + ++NumActuals;
> >>>> + assert(NumFixedArgsLeft != 0 && "Too many arguments parsed");
> >>>> + --NumFixedArgsLeft;
> >>>> + }
> >>>> +
> >>>> + // Okay, we either found the r_paren. Check to see if we parsed
> too
> >>>> few
> >>>> + // arguments.
> >>>> + unsigned MinArgsExpected = MI->getNumArgs();
> >>>> +
> >>>> + // See MacroArgs instance var for description of this.
> >>>> + bool isVarargsElided = false;
> >>>> +
> >>>> + if (NumActuals < MinArgsExpected) {
> >>>> + // There are several cases where too few arguments is ok, handle
> >>>> them now.
> >>>> + if (NumActuals == 0 && MinArgsExpected == 1) {
> >>>> + // #define A(X) or #define A(...) ---> A()
> >>>> +
> >>>> + // If there is exactly one argument, and that argument is
> missing,
> >>>> + // then we have an empty "()" argument empty list. This is
> fine,
> >>>> even if
> >>>> + // the macro expects one argument (the argument is just empty).
> >>>> + isVarargsElided = MI->isVariadic();
> >>>> + } else if (MI->isVariadic() &&
> >>>> + (NumActuals+1 == MinArgsExpected || // A(x, ...) ->
> A(X)
> >>>> + (NumActuals == 0 && MinArgsExpected == 2))) {//
> A(x,...)
> >>>> -> A()
> >>>> + // Varargs where the named vararg parameter is missing: OK as
> >>>> extension.
> >>>> + // #define A(x, ...)
> >>>> + // A("blah")
> >>>> + Diag(Tok, diag::ext_missing_varargs_arg);
> >>>> + Diag(MI->getDefinitionLoc(), diag::note_macro_here)
> >>>> + << MacroName.getIdentifierInfo();
> >>>> +
> >>>> + // Remember this occurred, allowing us to elide the comma when
> >>>> used for
> >>>> + // cases like:
> >>>> + // #define A(x, foo...) blah(a, ## foo)
> >>>> + // #define B(x, ...) blah(a, ## __VA_ARGS__)
> >>>> + // #define C(...) blah(a, ## __VA_ARGS__)
> >>>> + // A(x) B(x) C()
> >>>> + isVarargsElided = true;
> >>>> + } else {
> >>>> + // Otherwise, emit the error.
> >>>> + Diag(Tok, diag::err_too_few_args_in_macro_invoc);
> >>>> + return 0;
> >>>> + }
> >>>> +
> >>>> + // Add a marker EOF token to the end of the token list for this
> >>>> argument.
> >>>> + SourceLocation EndLoc = Tok.getLocation();
> >>>> + Tok.startToken();
> >>>> + Tok.setKind(tok::eof);
> >>>> + Tok.setLocation(EndLoc);
> >>>> + Tok.setLength(0);
> >>>> + ArgTokens.push_back(Tok);
> >>>> +
> >>>> + // If we expect two arguments, add both as empty.
> >>>> + if (NumActuals == 0 && MinArgsExpected == 2)
> >>>> + ArgTokens.push_back(Tok);
> >>>> +
> >>>> + } else if (NumActuals > MinArgsExpected && !MI->isVariadic()) {
> >>>> + // Emit the diagnostic at the macro name in case there is a
> missing
> >>>> ).
> >>>> + // Emitting it at the , could be far away from the macro name.
> >>>> + Diag(MacroName, diag::err_too_many_args_in_macro_invoc);
> >>>> + return 0;
> >>>> + }
> >>>> +
> >>>> + return MacroArgs::create(MI, ArgTokens, isVarargsElided, *this);
> >>>> +}
> >>>> +
> >>>> +/// \brief Keeps macro expanded tokens for TokenLexers.
> >>>> +//
> >>>> +/// Works like a stack; a TokenLexer adds the macro expanded tokens
> that
> >>>> is
> >>>> +/// going to lex in the cache and when it finishes the tokens are
> >>>> removed
> >>>> +/// from the end of the cache.
> >>>> +Token *Preprocessor::cacheMacroExpandedTokens(TokenLexer *tokLexer,
> >>>> + ArrayRef<Token>
> tokens) {
> >>>> + assert(tokLexer);
> >>>> + if (tokens.empty())
> >>>> + return 0;
> >>>> +
> >>>> + size_t newIndex = MacroExpandedTokens.size();
> >>>> + bool cacheNeedsToGrow = tokens.size() >
> >>>> +
> >>>> MacroExpandedTokens.capacity()-MacroExpandedTokens.size();
> >>>> + MacroExpandedTokens.append(tokens.begin(), tokens.end());
> >>>> +
> >>>> + if (cacheNeedsToGrow) {
> >>>> + // Go through all the TokenLexers whose 'Tokens' pointer points
> in
> >>>> the
> >>>> + // buffer and update the pointers to the (potential) new buffer
> >>>> array.
> >>>> + for (unsigned i = 0, e = MacroExpandingLexersStack.size(); i !=
> e;
> >>>> ++i) {
> >>>> + TokenLexer *prevLexer;
> >>>> + size_t tokIndex;
> >>>> + llvm::tie(prevLexer, tokIndex) = MacroExpandingLexersStack[i];
> >>>> + prevLexer->Tokens = MacroExpandedTokens.data() + tokIndex;
> >>>> + }
> >>>> + }
> >>>> +
> >>>> + MacroExpandingLexersStack.push_back(std::make_pair(tokLexer,
> >>>> newIndex));
> >>>> + return MacroExpandedTokens.data() + newIndex;
> >>>> +}
> >>>> +
> >>>> +void Preprocessor::removeCachedMacroExpandedTokensOfLastLexer() {
> >>>> + assert(!MacroExpandingLexersStack.empty());
> >>>> + size_t tokIndex = MacroExpandingLexersStack.back().second;
> >>>> + assert(tokIndex < MacroExpandedTokens.size());
> >>>> + // Pop the cached macro expanded tokens from the end.
> >>>> + MacroExpandedTokens.resize(tokIndex);
> >>>> + MacroExpandingLexersStack.pop_back();
> >>>> +}
> >>>> +
> >>>> +/// ComputeDATE_TIME - Compute the current time, enter it into the
> >>>> specified
> >>>> +/// scratch buffer, then return DATELoc/TIMELoc locations with the
> >>>> position of
> >>>> +/// the identifier tokens inserted.
> >>>> +static void ComputeDATE_TIME(SourceLocation &DATELoc, SourceLocation
> >>>> &TIMELoc,
> >>>> + Preprocessor &PP) {
> >>>> + time_t TT = time(0);
> >>>> + struct tm *TM = localtime(&TT);
> >>>> +
> >>>> + static const char * const Months[] = {
> >>>> +
> >>>>
> "Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"
> >>>> + };
> >>>> +
> >>>> + char TmpBuffer[32];
> >>>> +#ifdef LLVM_ON_WIN32
> >>>> + sprintf(TmpBuffer, "\"%s %2d %4d\"", Months[TM->tm_mon],
> TM->tm_mday,
> >>>> + TM->tm_year+1900);
> >>>> +#else
> >>>> + snprintf(TmpBuffer, sizeof(TmpBuffer), "\"%s %2d %4d\"",
> >>>> Months[TM->tm_mon], TM->tm_mday,
> >>>> + TM->tm_year+1900);
> >>>> +#endif
> >>>> +
> >>>> + Token TmpTok;
> >>>> + TmpTok.startToken();
> >>>> + PP.CreateString(TmpBuffer, strlen(TmpBuffer), TmpTok);
> >>>> + DATELoc = TmpTok.getLocation();
> >>>> +
> >>>> +#ifdef LLVM_ON_WIN32
> >>>> + sprintf(TmpBuffer, "\"%02d:%02d:%02d\"", TM->tm_hour, TM->tm_min,
> >>>> TM->tm_sec);
> >>>> +#else
> >>>> + snprintf(TmpBuffer, sizeof(TmpBuffer), "\"%02d:%02d:%02d\"",
> >>>> TM->tm_hour, TM->tm_min, TM->tm_sec);
> >>>> +#endif
> >>>> + PP.CreateString(TmpBuffer, strlen(TmpBuffer), TmpTok);
> >>>> + TIMELoc = TmpTok.getLocation();
> >>>> +}
> >>>> +
> >>>> +
> >>>> +/// HasFeature - Return true if we recognize and implement the
> feature
> >>>> +/// specified by the identifier as a standard language feature.
> >>>> +static bool HasFeature(const Preprocessor &PP, const IdentifierInfo
> *II)
> >>>> {
> >>>> + const LangOptions &LangOpts = PP.getLangOpts();
> >>>> + StringRef Feature = II->getName();
> >>>> +
> >>>> + // Normalize the feature name, __foo__ becomes foo.
> >>>> + if (Feature.startswith("__") && Feature.endswith("__") &&
> >>>> Feature.size() >= 4)
> >>>> + Feature = Feature.substr(2, Feature.size() - 4);
> >>>> +
> >>>> + return llvm::StringSwitch<bool>(Feature)
> >>>> + .Case("address_sanitizer", LangOpts.AddressSanitizer)
> >>>> + .Case("attribute_analyzer_noreturn", true)
> >>>> + .Case("attribute_availability", true)
> >>>> + .Case("attribute_availability_with_message", true)
> >>>> + .Case("attribute_cf_returns_not_retained", true)
> >>>> + .Case("attribute_cf_returns_retained", true)
> >>>> + .Case("attribute_deprecated_with_message", true)
> >>>> + .Case("attribute_ext_vector_type", true)
> >>>> + .Case("attribute_ns_returns_not_retained", true)
> >>>> + .Case("attribute_ns_returns_retained", true)
> >>>> + .Case("attribute_ns_consumes_self", true)
> >>>> + .Case("attribute_ns_consumed", true)
> >>>> + .Case("attribute_cf_consumed", true)
> >>>> + .Case("attribute_objc_ivar_unused", true)
> >>>> + .Case("attribute_objc_method_family", true)
> >>>> + .Case("attribute_overloadable", true)
> >>>> + .Case("attribute_unavailable_with_message", true)
> >>>> + .Case("attribute_unused_on_fields", true)
> >>>> + .Case("blocks", LangOpts.Blocks)
> >>>> + .Case("cxx_exceptions", LangOpts.Exceptions)
> >>>> + .Case("cxx_rtti", LangOpts.RTTI)
> >>>> + .Case("enumerator_attributes", true)
> >>>> + // Objective-C features
> >>>> + .Case("objc_arr", LangOpts.ObjCAutoRefCount) // FIXME:
> >>>> REMOVE?
> >>>> + .Case("objc_arc", LangOpts.ObjCAutoRefCount)
> >>>> + .Case("objc_arc_weak", LangOpts.ObjCARCWeak)
> >>>> + .Case("objc_default_synthesize_properties",
> LangOpts.ObjC2)
> >>>> + .Case("objc_fixed_enum", LangOpts.ObjC2)
> >>>> + .Case("objc_instancetype", LangOpts.ObjC2)
> >>>> + .Case("objc_modules", LangOpts.ObjC2 && LangOpts.Modules)
> >>>> + .Case("objc_nonfragile_abi",
> >>>> LangOpts.ObjCRuntime.isNonFragile())
> >>>> + .Case("objc_weak_class",
> >>>> LangOpts.ObjCRuntime.hasWeakClassImport())
> >>>> + .Case("ownership_holds", true)
> >>>> + .Case("ownership_returns", true)
> >>>> + .Case("ownership_takes", true)
> >>>> + .Case("objc_bool", true)
> >>>> + .Case("objc_subscripting",
> >>>> LangOpts.ObjCRuntime.isNonFragile())
> >>>> + .Case("objc_array_literals", LangOpts.ObjC2)
> >>>> + .Case("objc_dictionary_literals", LangOpts.ObjC2)
> >>>> + .Case("objc_boxed_expressions", LangOpts.ObjC2)
> >>>> + .Case("arc_cf_code_audited", true)
> >>>> + // C11 features
> >>>> + .Case("c_alignas", LangOpts.C11)
> >>>> + .Case("c_atomic", LangOpts.C11)
> >>>> + .Case("c_generic_selections", LangOpts.C11)
> >>>> + .Case("c_static_assert", LangOpts.C11)
> >>>> + // C++11 features
> >>>> + .Case("cxx_access_control_sfinae", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_alias_templates", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_alignas", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_atomic", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_attributes", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_auto_type", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_constexpr", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_decltype", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_decltype_incomplete_return_types",
> >>>> LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_default_function_template_args",
> >>>> LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_defaulted_functions", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_delegating_constructors", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_deleted_functions", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_explicit_conversions", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_generalized_initializers",
> LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_implicit_moves", LangOpts.CPlusPlus0x)
> >>>> + //.Case("cxx_inheriting_constructors", false)
> >>>> + .Case("cxx_inline_namespaces", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_lambdas", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_local_type_template_args",
> LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_nonstatic_member_init", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_noexcept", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_nullptr", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_override_control", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_range_for", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_raw_string_literals", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_reference_qualified_functions",
> >>>> LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_rvalue_references", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_strong_enums", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_static_assert", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_trailing_return", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_unicode_literals", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_unrestricted_unions", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_user_literals", LangOpts.CPlusPlus0x)
> >>>> + .Case("cxx_variadic_templates", LangOpts.CPlusPlus0x)
> >>>> + // Type traits
> >>>> + .Case("has_nothrow_assign", LangOpts.CPlusPlus)
> >>>> + .Case("has_nothrow_copy", LangOpts.CPlusPlus)
> >>>> + .Case("has_nothrow_constructor", LangOpts.CPlusPlus)
> >>>> + .Case("has_trivial_assign", LangOpts.CPlusPlus)
> >>>> + .Case("has_trivial_copy", LangOpts.CPlusPlus)
> >>>> + .Case("has_trivial_constructor", LangOpts.CPlusPlus)
> >>>> + .Case("has_trivial_destructor", LangOpts.CPlusPlus)
> >>>> + .Case("has_virtual_destructor", LangOpts.CPlusPlus)
> >>>> + .Case("is_abstract", LangOpts.CPlusPlus)
> >>>> + .Case("is_base_of", LangOpts.CPlusPlus)
> >>>> + .Case("is_class", LangOpts.CPlusPlus)
> >>>> + .Case("is_convertible_to", LangOpts.CPlusPlus)
> >>>> + // __is_empty is available only if the horrible
> >>>> + // "struct __is_empty" parsing hack hasn't been needed in
> >>>> this
> >>>> + // translation unit. If it has, __is_empty reverts to a
> >>>> normal
> >>>> + // identifier and __has_feature(is_empty) evaluates
> false.
> >>>> + .Case("is_empty", LangOpts.CPlusPlus)
> >>>> + .Case("is_enum", LangOpts.CPlusPlus)
> >>>> + .Case("is_final", LangOpts.CPlusPlus)
> >>>> + .Case("is_literal", LangOpts.CPlusPlus)
> >>>> + .Case("is_standard_layout", LangOpts.CPlusPlus)
> >>>> + .Case("is_pod", LangOpts.CPlusPlus)
> >>>> + .Case("is_polymorphic", LangOpts.CPlusPlus)
> >>>> + .Case("is_trivial", LangOpts.CPlusPlus)
> >>>> + .Case("is_trivially_assignable", LangOpts.CPlusPlus)
> >>>> + .Case("is_trivially_constructible", LangOpts.CPlusPlus)
> >>>> + .Case("is_trivially_copyable", LangOpts.CPlusPlus)
> >>>> + .Case("is_union", LangOpts.CPlusPlus)
> >>>> + .Case("modules", LangOpts.Modules)
> >>>> + .Case("tls", PP.getTargetInfo().isTLSSupported())
> >>>> + .Case("underlying_type", LangOpts.CPlusPlus)
> >>>> + .Default(false);
> >>>> +}
> >>>> +
> >>>> +/// HasExtension - Return true if we recognize and implement the
> feature
> >>>> +/// specified by the identifier, either as an extension or a standard
> >>>> language
> >>>> +/// feature.
> >>>> +static bool HasExtension(const Preprocessor &PP, const IdentifierInfo
> >>>> *II) {
> >>>> + if (HasFeature(PP, II))
> >>>> + return true;
> >>>> +
> >>>> + // If the use of an extension results in an error diagnostic,
> >>>> extensions are
> >>>> + // effectively unavailable, so just return false here.
> >>>> + if (PP.getDiagnostics().getExtensionHandlingBehavior() ==
> >>>> + DiagnosticsEngine::Ext_Error)
> >>>> + return false;
> >>>> +
> >>>> + const LangOptions &LangOpts = PP.getLangOpts();
> >>>> + StringRef Extension = II->getName();
> >>>> +
> >>>> + // Normalize the extension name, __foo__ becomes foo.
> >>>> + if (Extension.startswith("__") && Extension.endswith("__") &&
> >>>> + Extension.size() >= 4)
> >>>> + Extension = Extension.substr(2, Extension.size() - 4);
> >>>> +
> >>>> + // Because we inherit the feature list from HasFeature, this string
> >>>> switch
> >>>> + // must be less restrictive than HasFeature's.
> >>>> + return llvm::StringSwitch<bool>(Extension)
> >>>> + // C11 features supported by other languages as
> extensions.
> >>>> + .Case("c_alignas", true)
> >>>> + .Case("c_atomic", true)
> >>>> + .Case("c_generic_selections", true)
> >>>> + .Case("c_static_assert", true)
> >>>> + // C++0x features supported by other languages as
> extensions.
> >>>> + .Case("cxx_atomic", LangOpts.CPlusPlus)
> >>>> + .Case("cxx_deleted_functions", LangOpts.CPlusPlus)
> >>>> + .Case("cxx_explicit_conversions", LangOpts.CPlusPlus)
> >>>> + .Case("cxx_inline_namespaces", LangOpts.CPlusPlus)
> >>>> + .Case("cxx_local_type_template_args", LangOpts.CPlusPlus)
> >>>> + .Case("cxx_nonstatic_member_init", LangOpts.CPlusPlus)
> >>>> + .Case("cxx_override_control", LangOpts.CPlusPlus)
> >>>> + .Case("cxx_range_for", LangOpts.CPlusPlus)
> >>>> + .Case("cxx_reference_qualified_functions",
> >>>> LangOpts.CPlusPlus)
> >>>> + .Case("cxx_rvalue_references", LangOpts.CPlusPlus)
> >>>> + .Default(false);
> >>>> +}
> >>>> +
> >>>> +/// HasAttribute - Return true if we recognize and implement the
> >>>> attribute
> >>>> +/// specified by the given identifier.
> >>>> +static bool HasAttribute(const IdentifierInfo *II) {
> >>>> + StringRef Name = II->getName();
> >>>> + // Normalize the attribute name, __foo__ becomes foo.
> >>>> + if (Name.startswith("__") && Name.endswith("__") && Name.size() >=
> 4)
> >>>> + Name = Name.substr(2, Name.size() - 4);
> >>>> +
> >>>> + // FIXME: Do we need to handle namespaces here?
> >>>> + return llvm::StringSwitch<bool>(Name)
> >>>> +#include "clang/Lex/AttrSpellings.inc"
> >>>> + .Default(false);
> >>>> +}
> >>>> +
> >>>> +/// EvaluateHasIncludeCommon - Process a '__has_include("path")'
> >>>> +/// or '__has_include_next("path")' expression.
> >>>> +/// Returns true if successful.
> >>>> +static bool EvaluateHasIncludeCommon(Token &Tok,
> >>>> + IdentifierInfo *II, Preprocessor
> >>>> &PP,
> >>>> + const DirectoryLookup
> *LookupFrom)
> >>>> {
> >>>> + SourceLocation LParenLoc;
> >>>> +
> >>>> + // Get '('.
> >>>> + PP.LexNonComment(Tok);
> >>>> +
> >>>> + // Ensure we have a '('.
> >>>> + if (Tok.isNot(tok::l_paren)) {
> >>>> + PP.Diag(Tok.getLocation(), diag::err_pp_missing_lparen) <<
> >>>> II->getName();
> >>>> + return false;
> >>>> + }
> >>>> +
> >>>> + // Save '(' location for possible missing ')' message.
> >>>> + LParenLoc = Tok.getLocation();
> >>>> +
> >>>> + // Get the file name.
> >>>> + PP.getCurrentLexer()->LexIncludeFilename(Tok);
> >>>> +
> >>>> + // Reserve a buffer to get the spelling.
> >>>> + SmallString<128> FilenameBuffer;
> >>>> + StringRef Filename;
> >>>> + SourceLocation EndLoc;
> >>>> +
> >>>> + switch (Tok.getKind()) {
> >>>> + case tok::eod:
> >>>> + // If the token kind is EOD, the error has already been
> diagnosed.
> >>>> + return false;
> >>>> +
> >>>> + case tok::angle_string_literal:
> >>>> + case tok::string_literal: {
> >>>> + bool Invalid = false;
> >>>> + Filename = PP.getSpelling(Tok, FilenameBuffer, &Invalid);
> >>>> + if (Invalid)
> >>>> + return false;
> >>>> + break;
> >>>> + }
> >>>> +
> >>>> + case tok::less:
> >>>> + // This could be a <foo/bar.h> file coming from a macro
> expansion.
> >>>> In this
> >>>> + // case, glue the tokens together into FilenameBuffer and
> interpret
> >>>> those.
> >>>> + FilenameBuffer.push_back('<');
> >>>> + if (PP.ConcatenateIncludeName(FilenameBuffer, EndLoc))
> >>>> + return false; // Found <eod> but no ">"? Diagnostic already
> >>>> emitted.
> >>>> + Filename = FilenameBuffer.str();
> >>>> + break;
> >>>> + default:
> >>>> + PP.Diag(Tok.getLocation(), diag::err_pp_expects_filename);
> >>>> + return false;
> >>>> + }
> >>>> +
> >>>> + // Get ')'.
> >>>> + PP.LexNonComment(Tok);
> >>>> +
> >>>> + // Ensure we have a trailing ).
> >>>> + if (Tok.isNot(tok::r_paren)) {
> >>>> + PP.Diag(Tok.getLocation(), diag::err_pp_missing_rparen) <<
> >>>> II->getName();
> >>>> + PP.Diag(LParenLoc, diag::note_matching) << "(";
> >>>> + return false;
> >>>> + }
> >>>> +
> >>>> + bool isAngled = PP.GetIncludeFilenameSpelling(Tok.getLocation(),
> >>>> Filename);
> >>>> + // If GetIncludeFilenameSpelling set the start ptr to null, there
> was
> >>>> an
> >>>> + // error.
> >>>> + if (Filename.empty())
> >>>> + return false;
> >>>> +
> >>>> + // Search include directories.
> >>>> + const DirectoryLookup *CurDir;
> >>>> + const FileEntry *File =
> >>>> + PP.LookupFile(Filename, isAngled, LookupFrom, CurDir, NULL,
> NULL,
> >>>> NULL);
> >>>> +
> >>>> + // Get the result value. A result of true means the file exists.
> >>>> + return File != 0;
> >>>> +}
> >>>> +
> >>>> +/// EvaluateHasInclude - Process a '__has_include("path")'
> expression.
> >>>> +/// Returns true if successful.
> >>>> +static bool EvaluateHasInclude(Token &Tok, IdentifierInfo *II,
> >>>> + Preprocessor &PP) {
> >>>> + return EvaluateHasIncludeCommon(Tok, II, PP, NULL);
> >>>> +}
> >>>> +
> >>>> +/// EvaluateHasIncludeNext - Process '__has_include_next("path")'
> >>>> expression.
> >>>> +/// Returns true if successful.
> >>>> +static bool EvaluateHasIncludeNext(Token &Tok,
> >>>> + IdentifierInfo *II, Preprocessor
> &PP)
> >>>> {
> >>>> + // __has_include_next is like __has_include, except that we start
> >>>> + // searching after the current found directory. If we can't do
> this,
> >>>> + // issue a diagnostic.
> >>>> + const DirectoryLookup *Lookup = PP.GetCurDirLookup();
> >>>> + if (PP.isInPrimaryFile()) {
> >>>> + Lookup = 0;
> >>>> + PP.Diag(Tok, diag::pp_include_next_in_primary);
> >>>> + } else if (Lookup == 0) {
> >>>> + PP.Diag(Tok, diag::pp_include_next_absolute_path);
> >>>> + } else {
> >>>> + // Start looking up in the next directory.
> >>>> + ++Lookup;
> >>>> + }
> >>>> +
> >>>> + return EvaluateHasIncludeCommon(Tok, II, PP, Lookup);
> >>>> +}
> >>>> +
> >>>> +/// ExpandBuiltinMacro - If an identifier token is read that is to be
> >>>> expanded
> >>>> +/// as a builtin macro, handle it and return the next token as 'Tok'.
> >>>> +void Preprocessor::ExpandBuiltinMacro(Token &Tok) {
> >>>> + // Figure out which token this is.
> >>>> + IdentifierInfo *II = Tok.getIdentifierInfo();
> >>>> + assert(II && "Can't be a macro without id info!");
> >>>> +
> >>>> + // If this is an _Pragma or Microsoft __pragma directive, expand
> it,
> >>>> + // invoke the pragma handler, then lex the token after it.
> >>>> + if (II == Ident_Pragma)
> >>>> + return Handle_Pragma(Tok);
> >>>> + else if (II == Ident__pragma) // in non-MS mode this is null
> >>>> + return HandleMicrosoft__pragma(Tok);
> >>>> +
> >>>> + ++NumBuiltinMacroExpanded;
> >>>> +
> >>>> + SmallString<128> TmpBuffer;
> >>>> + llvm::raw_svector_ostream OS(TmpBuffer);
> >>>> +
> >>>> + // Set up the return result.
> >>>> + Tok.setIdentifierInfo(0);
> >>>> + Tok.clearFlag(Token::NeedsCleaning);
> >>>> +
> >>>> + if (II == Ident__LINE__) {
> >>>> + // C99 6.10.8: "__LINE__: The presumed line number (within the
> >>>> current
> >>>> + // source file) of the current source line (an integer
> constant)".
> >>>> This can
> >>>> + // be affected by #line.
> >>>> + SourceLocation Loc = Tok.getLocation();
> >>>> +
> >>>> + // Advance to the location of the first _, this might not be the
> >>>> first byte
> >>>> + // of the token if it starts with an escaped newline.
> >>>> + Loc = AdvanceToTokenCharacter(Loc, 0);
> >>>> +
> >>>> + // One wrinkle here is that GCC expands __LINE__ to location of
> the
> >>>> *end* of
> >>>> + // a macro expansion. This doesn't matter for object-like
> macros,
> >>>> but
> >>>> + // can matter for a function-like macro that expands to contain
> >>>> __LINE__.
> >>>> + // Skip down through expansion points until we find a file loc
> for
> >>>> the
> >>>> + // end of the expansion history.
> >>>> + Loc = SourceMgr.getExpansionRange(Loc).second;
> >>>> + PresumedLoc PLoc = SourceMgr.getPresumedLoc(Loc);
> >>>> +
> >>>> + // __LINE__ expands to a simple numeric value.
> >>>> + OS << (PLoc.isValid()? PLoc.getLine() : 1);
> >>>> + Tok.setKind(tok::numeric_constant);
> >>>> + } else if (II == Ident__FILE__ || II == Ident__BASE_FILE__) {
> >>>> + // C99 6.10.8: "__FILE__: The presumed name of the current source
> >>>> file (a
> >>>> + // character string literal)". This can be affected by #line.
> >>>> + PresumedLoc PLoc = SourceMgr.getPresumedLoc(Tok.getLocation());
> >>>> +
> >>>> + // __BASE_FILE__ is a GNU extension that returns the top of the
> >>>> presumed
> >>>> + // #include stack instead of the current file.
> >>>> + if (II == Ident__BASE_FILE__ && PLoc.isValid()) {
> >>>> + SourceLocation NextLoc = PLoc.getIncludeLoc();
> >>>> + while (NextLoc.isValid()) {
> >>>> + PLoc = SourceMgr.getPresumedLoc(NextLoc);
> >>>> + if (PLoc.isInvalid())
> >>>> + break;
> >>>> +
> >>>> + NextLoc = PLoc.getIncludeLoc();
> >>>> + }
> >>>> + }
> >>>> +
> >>>> + // Escape this filename. Turn '\' -> '\\' '"' -> '\"'
> >>>> + SmallString<128> FN;
> >>>> + if (PLoc.isValid()) {
> >>>> + FN += PLoc.getFilename();
> >>>> + Lexer::Stringify(FN);
> >>>> + OS << '"' << FN.str() << '"';
> >>>> + }
> >>>> + Tok.setKind(tok::string_literal);
> >>>> + } else if (II == Ident__DATE__) {
> >>>> + if (!DATELoc.isValid())
> >>>> + ComputeDATE_TIME(DATELoc, TIMELoc, *this);
> >>>> + Tok.setKind(tok::string_literal);
> >>>> + Tok.setLength(strlen("\"Mmm dd yyyy\""));
> >>>> + Tok.setLocation(SourceMgr.createExpansionLoc(DATELoc,
> >>>> Tok.getLocation(),
> >>>> + Tok.getLocation(),
> >>>> + Tok.getLength()));
> >>>> + return;
> >>>> + } else if (II == Ident__TIME__) {
> >>>> + if (!TIMELoc.isValid())
> >>>> + ComputeDATE_TIME(DATELoc, TIMELoc, *this);
> >>>> + Tok.setKind(tok::string_literal);
> >>>> + Tok.setLength(strlen("\"hh:mm:ss\""));
> >>>> + Tok.setLocation(SourceMgr.createExpansionLoc(TIMELoc,
> >>>> Tok.getLocation(),
> >>>> + Tok.getLocation(),
> >>>> + Tok.getLength()));
> >>>> + return;
> >>>> + } else if (II == Ident__INCLUDE_LEVEL__) {
> >>>> + // Compute the presumed include depth of this token. This can be
> >>>> affected
> >>>> + // by GNU line markers.
> >>>> + unsigned Depth = 0;
> >>>> +
> >>>> + PresumedLoc PLoc = SourceMgr.getPresumedLoc(Tok.getLocation());
> >>>> + if (PLoc.isValid()) {
> >>>> + PLoc = SourceMgr.getPresumedLoc(PLoc.getIncludeLoc());
> >>>> + for (; PLoc.isValid(); ++Depth)
> >>>> + PLoc = SourceMgr.getPresumedLoc(PLoc.getIncludeLoc());
> >>>> + }
> >>>> +
> >>>> + // __INCLUDE_LEVEL__ expands to a simple numeric value.
> >>>> + OS << Depth;
> >>>> + Tok.setKind(tok::numeric_constant);
> >>>> + } else if (II == Ident__TIMESTAMP__) {
> >>>> + // MSVC, ICC, GCC, VisualAge C++ extension. The generated string
> >>>> should be
> >>>> + // of the form "Ddd Mmm dd hh::mm::ss yyyy", which is returned by
> >>>> asctime.
> >>>> +
> >>>> + // Get the file that we are lexing out of. If we're currently
> >>>> lexing from
> >>>> + // a macro, dig into the include stack.
> >>>> + const FileEntry *CurFile = 0;
> >>>> + PreprocessorLexer *TheLexer = getCurrentFileLexer();
> >>>> +
> >>>> + if (TheLexer)
> >>>> + CurFile = SourceMgr.getFileEntryForID(TheLexer->getFileID());
> >>>> +
> >>>> + const char *Result;
> >>>> + if (CurFile) {
> >>>> + time_t TT = CurFile->getModificationTime();
> >>>> + struct tm *TM = localtime(&TT);
> >>>> + Result = asctime(TM);
> >>>> + } else {
> >>>> + Result = "??? ??? ?? ??:??:?? ????\n";
> >>>> + }
> >>>> + // Surround the string with " and strip the trailing newline.
> >>>> + OS << '"' << StringRef(Result, strlen(Result)-1) << '"';
> >>>> + Tok.setKind(tok::string_literal);
> >>>> + } else if (II == Ident__COUNTER__) {
> >>>> + // __COUNTER__ expands to a simple numeric value.
> >>>> + OS << CounterValue++;
> >>>> + Tok.setKind(tok::numeric_constant);
> >>>> + } else if (II == Ident__has_feature ||
> >>>> + II == Ident__has_extension ||
> >>>> + II == Ident__has_builtin ||
> >>>> + II == Ident__has_attribute) {
> >>>> + // The argument to these builtins should be a parenthesized
> >>>> identifier.
> >>>> + SourceLocation StartLoc = Tok.getLocation();
> >>>> +
> >>>> + bool IsValid = false;
> >>>> + IdentifierInfo *FeatureII = 0;
> >>>> +
> >>>> + // Read the '('.
> >>>> + Lex(Tok);
> >>>> + if (Tok.is(tok::l_paren)) {
> >>>> + // Read the identifier
> >>>> + Lex(Tok);
> >>>> + if (Tok.is(tok::identifier) || Tok.is(tok::kw_const)) {
> >>>> + FeatureII = Tok.getIdentifierInfo();
> >>>> +
> >>>> + // Read the ')'.
> >>>> + Lex(Tok);
> >>>> + if (Tok.is(tok::r_paren))
> >>>> + IsValid = true;
> >>>> + }
> >>>> + }
> >>>> +
> >>>> + bool Value = false;
> >>>> + if (!IsValid)
> >>>> + Diag(StartLoc, diag::err_feature_check_malformed);
> >>>> + else if (II == Ident__has_builtin) {
> >>>> + // Check for a builtin is trivial.
> >>>> + Value = FeatureII->getBuiltinID() != 0;
> >>>> + } else if (II == Ident__has_attribute)
> >>>> + Value = HasAttribute(FeatureII);
> >>>> + else if (II == Ident__has_extension)
> >>>> + Value = HasExtension(*this, FeatureII);
> >>>> + else {
> >>>> + assert(II == Ident__has_feature && "Must be feature check");
> >>>> + Value = HasFeature(*this, FeatureII);
> >>>> + }
> >>>> +
> >>>> + OS << (int)Value;
> >>>> + if (IsValid)
> >>>> + Tok.setKind(tok::numeric_constant);
> >>>> + } else if (II == Ident__has_include ||
> >>>> + II == Ident__has_include_next) {
> >>>> + // The argument to these two builtins should be a parenthesized
> >>>> + // file name string literal using angle brackets (<>) or
> >>>> + // double-quotes ("").
> >>>> + bool Value;
> >>>> + if (II == Ident__has_include)
> >>>> + Value = EvaluateHasInclude(Tok, II, *this);
> >>>> + else
> >>>> + Value = EvaluateHasIncludeNext(Tok, II, *this);
> >>>> + OS << (int)Value;
> >>>> + Tok.setKind(tok::numeric_constant);
> >>>> + } else if (II == Ident__has_warning) {
> >>>> + // The argument should be a parenthesized string literal.
> >>>> + // The argument to these builtins should be a parenthesized
> >>>> identifier.
> >>>> + SourceLocation StartLoc = Tok.getLocation();
> >>>> + bool IsValid = false;
> >>>> + bool Value = false;
> >>>> + // Read the '('.
> >>>> + Lex(Tok);
> >>>> + do {
> >>>> + if (Tok.is(tok::l_paren)) {
> >>>> + // Read the string.
> >>>> + Lex(Tok);
> >>>> +
> >>>> + // We need at least one string literal.
> >>>> + if (!Tok.is(tok::string_literal)) {
> >>>> + StartLoc = Tok.getLocation();
> >>>> + IsValid = false;
> >>>> + // Eat tokens until ')'.
> >>>> + do Lex(Tok); while (!(Tok.is(tok::r_paren) ||
> >>>> Tok.is(tok::eod)));
> >>>> + break;
> >>>> + }
> >>>> +
> >>>> + // String concatenation allows multiple strings, which can
> even
> >>>> come
> >>>> + // from macro expansion.
> >>>> + SmallVector<Token, 4> StrToks;
> >>>> + while (Tok.is(tok::string_literal)) {
> >>>> + // Complain about, and drop, any ud-suffix.
> >>>> + if (Tok.hasUDSuffix())
> >>>> + Diag(Tok, diag::err_invalid_string_udl);
> >>>> + StrToks.push_back(Tok);
> >>>> + LexUnexpandedToken(Tok);
> >>>> + }
> >>>> +
> >>>> + // Is the end a ')'?
> >>>> + if (!(IsValid = Tok.is(tok::r_paren)))
> >>>> + break;
> >>>> +
> >>>> + // Concatenate and parse the strings.
> >>>> + StringLiteralParser Literal(&StrToks[0], StrToks.size(),
> *this);
> >>>> + assert(Literal.isAscii() && "Didn't allow wide strings in");
> >>>> + if (Literal.hadError)
> >>>> + break;
> >>>> + if (Literal.Pascal) {
> >>>> + Diag(Tok, diag::warn_pragma_diagnostic_invalid);
> >>>> + break;
> >>>> + }
> >>>> +
> >>>> + StringRef WarningName(Literal.GetString());
> >>>> +
> >>>> + if (WarningName.size() < 3 || WarningName[0] != '-' ||
> >>>> + WarningName[1] != 'W') {
> >>>> + Diag(StrToks[0].getLocation(),
> >>>> diag::warn_has_warning_invalid_option);
> >>>> + break;
> >>>> + }
> >>>> +
> >>>> + // Finally, check if the warning flags maps to a diagnostic
> >>>> group.
> >>>> + // We construct a SmallVector here to talk to
> >>>> getDiagnosticIDs().
> >>>> + // Although we don't use the result, this isn't a hot path,
> and
> >>>> not
> >>>> + // worth special casing.
> >>>> + llvm::SmallVector<diag::kind, 10> Diags;
> >>>> + Value = !getDiagnostics().getDiagnosticIDs()->
> >>>> + getDiagnosticsInGroup(WarningName.substr(2), Diags);
> >>>> + }
> >>>> + } while (false);
> >>>> +
> >>>> + if (!IsValid)
> >>>> + Diag(StartLoc, diag::err_warning_check_malformed);
> >>>> +
> >>>> + OS << (int)Value;
> >>>> + Tok.setKind(tok::numeric_constant);
> >>>> + } else {
> >>>> + llvm_unreachable("Unknown identifier!");
> >>>> + }
> >>>> + CreateString(OS.str().data(), OS.str().size(), Tok,
> >>>> + Tok.getLocation(), Tok.getLocation());
> >>>> +}
> >>>> +
> >>>> +void Preprocessor::markMacroAsUsed(MacroInfo *MI) {
> >>>> + // If the 'used' status changed, and the macro requires 'unused'
> >>>> warning,
> >>>> + // remove its SourceLocation from the warn-for-unused-macro
> locations.
> >>>> + if (MI->isWarnIfUnused() && !MI->isUsed())
> >>>> + WarnUnusedMacroLocs.erase(MI->getDefinitionLoc());
> >>>> + MI->setIsUsed(true);
> >>>> +}
> >>>>
> >>>> Modified: cfe/trunk/lib/Lex/TokenLexer.cpp
> >>>> URL:
> >>>>
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/TokenLexer.cpp?rev=163022&r1=163021&r2=163022&view=diff
> >>>>
> >>>>
> ==============================================================================
> >>>> --- cfe/trunk/lib/Lex/TokenLexer.cpp (original)
> >>>> +++ cfe/trunk/lib/Lex/TokenLexer.cpp Fri Aug 31 16:10:54 2012
> >>>> @@ -225,6 +225,12 @@
> >>>> Token &Tok = ResultToks[i];
> >>>> if (Tok.is(tok::hashhash))
> >>>> Tok.setKind(tok::unknown);
> >>>> + // In Microsoft-compatibility mode, we follow MSVC's
> >>>> preprocessing
> >>>> + // behaviour by not considering commas from nested macro
> >>>> expansions
> >>>> + // as argument separators. Set a flag on the token so we
> can
> >>>> test
> >>>> + // for this later when the macro expansion is processed.
> >>>> + if (Tok.is(tok::comma) && PP.getLangOpts().MicrosoftMode)
> >>>> + Tok.setFlag(Token::IgnoredComma);
> >>>> }
> >>>>
> >>>> if(ExpandLocStart.isValid()) {
> >>>>
> >>>> Added: cfe/trunk/test/Preprocessor/microsoft-ext.c
> >>>> URL:
> >>>>
> http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Preprocessor/microsoft-ext.c?rev=163022&view=auto
> >>>>
> >>>>
> ==============================================================================
> >>>> --- cfe/trunk/test/Preprocessor/microsoft-ext.c (added)
> >>>> +++ cfe/trunk/test/Preprocessor/microsoft-ext.c Fri Aug 31 16:10:54
> 2012
> >>>> @@ -0,0 +1,7 @@
> >>>> +// RUN: %clang_cc1 -E -fms-compatibility %s | FileCheck %s
> >>>> +
> >>>> +# define M2(x, y) x + y
> >>>> +# define P(x, y) {x, y}
> >>>> +# define M(x, y) M2(x, P(x, y))
> >>>> +M(a, b) // CHECK: a + {a, b}
> >>>> +
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> cfe-commits mailing list
> >>>> cfe-commits at cs.uiuc.edu
> >>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
> >>>
> >>>
> >>
> >>
> >>
> >> --
> >> João Matos
> >>
> >> _______________________________________________
> >> cfe-commits mailing list
> >> cfe-commits at cs.uiuc.edu
> >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
> >>
>
--
João Matos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20120926/48771062/attachment.html>
More information about the cfe-commits
mailing list