[cfe-commits] r91391 - in /cfe/trunk: include/clang/Lex/Preprocessor.h lib/Lex/MacroArgs.cpp lib/Lex/MacroArgs.h lib/Lex/Preprocessor.cpp

[Daniel Dunbar]

Hi Chris,

On Mon, Dec 14, 2009 at 5:51 PM, Chris Lattner <sabre at nondot.org> wrote:
> Author: lattner
> Date: Mon Dec 14 19:51:03 2009
> New Revision: 91391
>
> URL: http://llvm.org/viewvc/llvm-project?rev=91391&view=rev
> Log:
> set up the machinery for a MacroArgs cache hanging off Preprocessor.
> We creating and free thousands of MacroArgs objects (and the related
> std::vectors hanging off them) for the testcase in PR5610 even though
> there are only ~20 live at a time.  This doesn't actually use the
> cache yet.

It looks like this is just adding a free list, would it make sense to
factor out a generic implementation? It seems generally useful,
especially if done as a generic allocator, for example one can imagine
FIFO / doubly linked, batch free'd free lists.

 - Daniel

>
> Modified:
>    cfe/trunk/include/clang/Lex/Preprocessor.h
>    cfe/trunk/lib/Lex/MacroArgs.cpp
>    cfe/trunk/lib/Lex/MacroArgs.h
>    cfe/trunk/lib/Lex/Preprocessor.cpp
>
> Modified: cfe/trunk/include/clang/Lex/Preprocessor.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/Preprocessor.h?rev=91391&r1=91390&r2=91391&view=diff
>
> ==============================================================================
> --- cfe/trunk/include/clang/Lex/Preprocessor.h (original)
> +++ cfe/trunk/include/clang/Lex/Preprocessor.h Mon Dec 14 19:51:03 2009
> @@ -139,7 +139,7 @@
>   /// CurPPLexer - This is the current top of the stack what we're lexing from
>   ///  if not expanding a macro.  This is an alias for either CurLexer or
>   ///  CurPTHLexer.
> -  PreprocessorLexer* CurPPLexer;
> +  PreprocessorLexer *CurPPLexer;
>
>   /// CurLookup - The DirectoryLookup structure used to find the current
>   /// FileEntry, if CurLexer is non-null and if applicable.  This allows us to
> @@ -176,8 +176,14 @@
>   llvm::DenseMap<IdentifierInfo*, MacroInfo*> Macros;
>
>   /// MICache - A "freelist" of MacroInfo objects that can be reused for quick
> -  ///  allocation.
> +  /// allocation.
> +  /// FIXME: why not use a singly linked list?
>   std::vector<MacroInfo*> MICache;
> +
> +  /// MacroArgCache - This is a "freelist" of MacroArg objects that can be
> +  /// reused for quick allocation.
> +  MacroArgs *MacroArgCache;
> +  friend class MacroArgs;
>
>   // Various statistics we track for performance analysis.
>   unsigned NumDirectives, NumIncluded, NumDefined, NumUndefined, NumPragma;
>
> Modified: cfe/trunk/lib/Lex/MacroArgs.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/MacroArgs.cpp?rev=91391&r1=91390&r2=91391&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/Lex/MacroArgs.cpp (original)
> +++ cfe/trunk/lib/Lex/MacroArgs.cpp Mon Dec 14 19:51:03 2009
> @@ -48,6 +48,19 @@
>   free(this);
>  }
>
> +/// deallocate - This should only be called by the Preprocessor when managing
> +/// its freelist.
> +MacroArgs *MacroArgs::deallocate() {
> +  MacroArgs *Next = ArgCache;
> +
> +  // Run the dtor to deallocate the vectors.
> +  this->~MacroArgs();
> +  // Release the memory for the object.
> +  free(this);
> +
> +  return Next;
> +}
> +
>
>  /// getArgLength - Given a pointer to an expanded or unexpanded argument,
>  /// return the number of tokens, not counting the EOF, that make up the
>
> Modified: cfe/trunk/lib/Lex/MacroArgs.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/MacroArgs.h?rev=91391&r1=91390&r2=91391&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/Lex/MacroArgs.h (original)
> +++ cfe/trunk/lib/Lex/MacroArgs.h Mon Dec 14 19:51:03 2009
> @@ -46,8 +46,12 @@
>   /// stringified form of an argument has not yet been computed, this is empty.
>   std::vector<Token> StringifiedArgs;
>
> +  /// ArgCache - This is a linked list of MacroArgs objects that the
> +  /// Preprocessor owns which we use to avoid thrashing malloc/free.
> +  MacroArgs *ArgCache;
> +
>   MacroArgs(unsigned NumToks, bool varargsElided)
> -    : NumUnexpArgTokens(NumToks), VarargsElided(varargsElided) {}
> +    : NumUnexpArgTokens(NumToks), VarargsElided(varargsElided), ArgCache(0) {}
>   ~MacroArgs() {}
>  public:
>   /// MacroArgs ctor function - Create a new MacroArgs object with the specified
> @@ -103,6 +107,11 @@
>   ///
>   static Token StringifyArgument(const Token *ArgToks,
>                                  Preprocessor &PP, bool Charify = false);
> +
> +
> +  /// deallocate - This should only be called by the Preprocessor when managing
> +  /// its freelist.
> +  MacroArgs *deallocate();
>  };
>
>  }  // end namespace clang
>
> Modified: cfe/trunk/lib/Lex/Preprocessor.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Preprocessor.cpp?rev=91391&r1=91390&r2=91391&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/Lex/Preprocessor.cpp (original)
> +++ cfe/trunk/lib/Lex/Preprocessor.cpp Mon Dec 14 19:51:03 2009
> @@ -26,6 +26,7 @@
>  //===----------------------------------------------------------------------===//
>
>  #include "clang/Lex/Preprocessor.h"
> +#include "MacroArgs.h"
>  #include "clang/Lex/HeaderSearch.h"
>  #include "clang/Lex/MacroInfo.h"
>  #include "clang/Lex/Pragma.h"
> @@ -51,7 +52,7 @@
>   : Diags(&diags), Features(opts), Target(target),FileMgr(Headers.getFileMgr()),
>     SourceMgr(SM), HeaderInfo(Headers), Identifiers(opts, IILookup),
>     BuiltinInfo(Target), CodeCompletionFile(0), CurPPLexer(0), CurDirLookup(0),
> -    Callbacks(0) {
> +    Callbacks(0), MacroArgCache(0) {
>   ScratchBuf = new ScratchBuffer(SourceMgr);
>   CounterValue = 0; // __COUNTER__ starts at 0.
>   OwnsHeaderSearch = OwnsHeaders;
> @@ -111,6 +112,10 @@
>   // Free any cached macro expanders.
>   for (unsigned i = 0, e = NumCachedTokenLexers; i != e; ++i)
>     delete TokenLexerCache[i];
> +
> +  // Free any cached MacroArgs.
> +  for (MacroArgs *ArgList = MacroArgCache; ArgList; )
> +    ArgList = ArgList->deallocate();
>
>   // Release pragma information.
>   delete PragmaHandlers;
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>