[cfe-commits] r74704 - in /cfe/trunk: include/clang/AST/ include/clang/Frontend/ include/clang/Lex/ include/clang/Parse/ lib/AST/ lib/Frontend/ lib/Lex/ lib/Parse/ lib/Sema/ test/ test/Index/ tools/index-test/
Argyrios Kyrtzidis
kyrtzidis at apple.com
Thu Jul 2 20:36:12 PDT 2009
This looks good and, btw, is totally awesome!
-Argiris
On Jul 2, 2009, at 10:08 AM, Douglas Gregor wrote:
> Author: dgregor
> Date: Thu Jul 2 12:08:52 2009
> New Revision: 74704
>
> URL: http://llvm.org/viewvc/llvm-project?rev=74704&view=rev
> Log:
> Add support for retrieving the Doxygen comment associated with a given
> declaration in the AST.
>
> The new ASTContext::getCommentForDecl function searches for a comment
> that is attached to the given declaration, and returns that comment,
> which may be composed of several comment blocks.
>
> Comments are always available in an AST. However, to avoid harming
> performance, we don't actually parse the comments. Rather, we keep the
> source ranges of all of the comments within a large, sorted vector,
> then lazily extract comments via a binary search in that vector only
> when needed (which never occurs in a "normal" compile).
>
> Comments are written to a precompiled header/AST file as a blob of
> source ranges. That blob is only lazily loaded when one requests a
> comment for a declaration (this never occurs in a "normal" compile).
>
> The indexer testbed now supports comment extraction. When the
> -point-at location points to a declaration with a Doxygen-style
> comment, the indexer testbed prints the associated comment
> block(s). See test/Index/comments.c for an example.
>
> Some notes:
> - We don't actually attempt to parse the comment blocks themselves,
> beyond identifying them as Doxygen comment blocks to associate them
> with a declaration.
> - We won't find comment blocks that aren't adjacent to the
> declaration, because we start our search based on the location of
> the declaration.
> - We don't go through the necessary hops to find, for example,
> whether some redeclaration of a declaration has comments when our
> current declaration does not. Similarly, we don't attempt to
> associate a \param Foo marker in a function body comment with the
> parameter named Foo (although that is certainly possible).
> - Verification of my "no performance impact" claims is still "to be
> done".
>
>
> Added:
> cfe/trunk/test/Index/comments.c (with props)
> Modified:
> cfe/trunk/include/clang/AST/ASTContext.h
> cfe/trunk/include/clang/AST/ExternalASTSource.h
> cfe/trunk/include/clang/Frontend/PCHBitCodes.h
> cfe/trunk/include/clang/Frontend/PCHReader.h
> cfe/trunk/include/clang/Frontend/PCHWriter.h
> cfe/trunk/include/clang/Lex/Preprocessor.h
> cfe/trunk/include/clang/Parse/Action.h
> cfe/trunk/include/clang/Parse/Parser.h
> cfe/trunk/lib/AST/ASTContext.cpp
> cfe/trunk/lib/Frontend/PCHReader.cpp
> cfe/trunk/lib/Frontend/PCHWriter.cpp
> cfe/trunk/lib/Lex/Lexer.cpp
> cfe/trunk/lib/Lex/Pragma.cpp
> cfe/trunk/lib/Lex/Preprocessor.cpp
> cfe/trunk/lib/Parse/Parser.cpp
> cfe/trunk/lib/Sema/Sema.cpp
> cfe/trunk/lib/Sema/Sema.h
> cfe/trunk/test/CMakeLists.txt
> cfe/trunk/tools/index-test/index-test.cpp
>
> Modified: cfe/trunk/include/clang/AST/ASTContext.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/AST/ASTContext.h?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/include/clang/AST/ASTContext.h (original)
> +++ cfe/trunk/include/clang/AST/ASTContext.h Thu Jul 2 12:08:52 2009
> @@ -141,10 +141,19 @@
> /// this ASTContext object.
> LangOptions LangOpts;
>
> + /// \brief Whether we have already loaded comment source ranges
> from an
> + /// external source.
> + bool LoadedExternalComments;
> +
> /// MallocAlloc/BumpAlloc - The allocator objects used to create
> AST objects.
> bool FreeMemory;
> llvm::MallocAllocator MallocAlloc;
> llvm::BumpPtrAllocator BumpAlloc;
> +
> + /// \brief Mapping from declarations to their comments, once we
> have
> + /// already looked up the comment associated with a given
> declaration.
> + llvm::DenseMap<const Decl *, std::string> DeclComments;
> +
> public:
> TargetInfo &Target;
> IdentifierTable &Idents;
> @@ -154,6 +163,10 @@
> llvm::OwningPtr<ExternalASTSource> ExternalSource;
> clang::PrintingPolicy PrintingPolicy;
>
> + /// \brief Source ranges for all of the comments in the source
> file,
> + /// sorted in order of appearance in the translation unit.
> + std::vector<SourceRange> Comments;
> +
> SourceManager& getSourceManager() { return SourceMgr; }
> const SourceManager& getSourceManager() const { return SourceMgr; }
> void *Allocate(unsigned Size, unsigned Align = 8) {
> @@ -178,7 +191,8 @@
>
> TranslationUnitDecl *getTranslationUnitDecl() const { return
> TUDecl; }
>
> -
> + const char *getCommentForDecl(const Decl *D);
> +
> // Builtin Types.
> QualType VoidTy;
> QualType BoolTy;
>
> Modified: cfe/trunk/include/clang/AST/ExternalASTSource.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/AST/ExternalASTSource.h?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/include/clang/AST/ExternalASTSource.h (original)
> +++ cfe/trunk/include/clang/AST/ExternalASTSource.h Thu Jul 2
> 12:08:52 2009
> @@ -18,6 +18,7 @@
> #include "clang/AST/Type.h"
> #include "llvm/ADT/SmallVector.h"
> #include <cassert>
> +#include <vector>
> namespace clang {
>
> class ASTConsumer;
> @@ -57,6 +58,14 @@
>
> virtual ~ExternalASTSource();
>
> + /// \brief Reads the source ranges that correspond to comments from
> + /// an external AST source.
> + ///
> + /// \param Comments the contents of this vector will be
> + /// replaced with the sorted set of source ranges corresponding to
> + /// comments in the source code.
> + virtual void ReadComments(std::vector<SourceRange> &Comments) = 0;
> +
> /// \brief Resolve a type ID into a type, potentially building a new
> /// type.
> virtual QualType GetType(uint32_t ID) = 0;
>
> Modified: cfe/trunk/include/clang/Frontend/PCHBitCodes.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/PCHBitCodes.h?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/include/clang/Frontend/PCHBitCodes.h (original)
> +++ cfe/trunk/include/clang/Frontend/PCHBitCodes.h Thu Jul 2
> 12:08:52 2009
> @@ -218,7 +218,11 @@
>
> /// \brief Record code for the original file that was used to
> /// generate the precompiled header.
> - ORIGINAL_FILE_NAME = 20
> + ORIGINAL_FILE_NAME = 20,
> +
> + /// \brief Record code for the sorted array of source ranges
> where
> + /// comments were encountered in the source code.
> + COMMENT_RANGES = 21
> };
>
> /// \brief Record types used within a source manager block.
>
> Modified: cfe/trunk/include/clang/Frontend/PCHReader.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/PCHReader.h?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/include/clang/Frontend/PCHReader.h (original)
> +++ cfe/trunk/include/clang/Frontend/PCHReader.h Thu Jul 2 12:08:52
> 2009
> @@ -279,6 +279,12 @@
> /// been loaded.
> llvm::SmallVector<Selector, 16> SelectorsLoaded;
>
> + /// \brief A sorted array of source ranges containing comments.
> + SourceRange *Comments;
> +
> + /// \brief The number of source ranges in the Comments array.
> + unsigned NumComments;
> +
> /// \brief The set of external definitions stored in the the PCH
> /// file.
> llvm::SmallVector<uint64_t, 16> ExternalDefinitions;
> @@ -452,6 +458,14 @@
> /// build prior to including the precompiled header.
> const std::string &getSuggestedPredefines() { return
> SuggestedPredefines; }
>
> + /// \brief Reads the source ranges that correspond to comments from
> + /// an external AST source.
> + ///
> + /// \param Comments the contents of this vector will be
> + /// replaced with the sorted set of source ranges corresponding to
> + /// comments in the source code.
> + virtual void ReadComments(std::vector<SourceRange> &Comments);
> +
> /// \brief Resolve a type ID into a type, potentially building a new
> /// type.
> virtual QualType GetType(pch::TypeID ID);
>
> Modified: cfe/trunk/include/clang/Frontend/PCHWriter.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/PCHWriter.h?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/include/clang/Frontend/PCHWriter.h (original)
> +++ cfe/trunk/include/clang/Frontend/PCHWriter.h Thu Jul 2 12:08:52
> 2009
> @@ -166,6 +166,7 @@
> void WriteSourceManagerBlock(SourceManager &SourceMgr,
> const Preprocessor &PP);
> void WritePreprocessor(const Preprocessor &PP);
> + void WriteComments(ASTContext &Context);
> void WriteType(const Type *T);
> void WriteTypesBlock(ASTContext &Context);
> uint64_t WriteDeclContextLexicalBlock(ASTContext &Context,
> DeclContext *DC);
>
> Modified: cfe/trunk/include/clang/Lex/Preprocessor.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/Preprocessor.h?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/include/clang/Lex/Preprocessor.h (original)
> +++ cfe/trunk/include/clang/Lex/Preprocessor.h Thu Jul 2 12:08:52
> 2009
> @@ -26,6 +26,7 @@
> #include "llvm/ADT/DenseMap.h"
> #include "llvm/ADT/OwningPtr.h"
> #include "llvm/Support/Allocator.h"
> +#include <vector>
>
> namespace clang {
>
> @@ -35,6 +36,7 @@
> class HeaderSearch;
> class PragmaNamespace;
> class PragmaHandler;
> +class CommentHandler;
> class ScratchBuffer;
> class TargetInfo;
> class PPCallbacks;
> @@ -109,6 +111,10 @@
> /// with this preprocessor.
> PragmaNamespace *PragmaHandlers;
>
> + /// \brief Tracks all of the comment handlers that the client
> registered
> + /// with this preprocessor.
> + std::vector<CommentHandler *> CommentHandlers;
> +
> /// CurLexer - This is the current top of the stack that we're
> lexing from if
> /// not expanding a macro and we are lexing directly from source
> code.
> /// Only one of CurLexer, CurPTHLexer, or CurTokenLexer will be
> non-null.
> @@ -301,6 +307,14 @@
> /// to remove a handler that has not been registered.
> void RemovePragmaHandler(const char *Namespace, PragmaHandler
> *Handler);
>
> + /// \brief Add the specified comment handler to the preprocessor.
> + void AddCommentHandler(CommentHandler *Handler);
> +
> + /// \brief Remove the specified comment handler.
> + ///
> + /// It is an error to remove a handler that has not been
> registered.
> + void RemoveCommentHandler(CommentHandler *Handler);
> +
> /// EnterMainSourceFile - Enter the specified FileID as the main
> source file,
> /// which implicitly adds the builtin defines etc.
> void EnterMainSourceFile();
> @@ -791,6 +805,7 @@
> void HandlePragmaSystemHeader(Token &SysHeaderTok);
> void HandlePragmaDependency(Token &DependencyTok);
> void HandlePragmaComment(Token &CommentTok);
> + void HandleComment(SourceRange Comment);
> };
>
> /// PreprocessorFactory - A generic factory interface for lazily
> creating
> @@ -801,6 +816,15 @@
> virtual Preprocessor* CreatePreprocessor() = 0;
> };
>
> +/// \brief Abstract base class that describes a handler that will
> receive
> +/// source ranges for each of the comments encountered in the
> source file.
> +class CommentHandler {
> +public:
> + virtual ~CommentHandler();
> +
> + virtual void HandleComment(Preprocessor &PP, SourceRange Comment)
> = 0;
> +};
> +
> } // end namespace clang
>
> #endif
>
> Modified: cfe/trunk/include/clang/Parse/Action.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Parse/Action.h?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/include/clang/Parse/Action.h (original)
> +++ cfe/trunk/include/clang/Parse/Action.h Thu Jul 2 12:08:52 2009
> @@ -152,6 +152,10 @@
> /// an empty string if not. This is used for pretty crash
> reporting.
> virtual std::string getDeclName(DeclPtrTy D) { return ""; }
>
> + /// \brief Invoked for each comment in the source code, providing
> the source
> + /// range that contains the comment.
> + virtual void ActOnComment(SourceRange Comment) { }
> +
> //
> =
> =
> =--------------------------------------------------------------------
> ===//
> // Declaration Tracking Callbacks.
> //
> =
> =
> =--------------------------------------------------------------------
> ===//
>
> Modified: cfe/trunk/include/clang/Parse/Parser.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Parse/Parser.h?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/include/clang/Parse/Parser.h (original)
> +++ cfe/trunk/include/clang/Parse/Parser.h Thu Jul 2 12:08:52 2009
> @@ -82,7 +82,8 @@
> llvm::OwningPtr<PragmaHandler> PackHandler;
> llvm::OwningPtr<PragmaHandler> UnusedHandler;
> llvm::OwningPtr<PragmaHandler> WeakHandler;
> -
> + llvm::OwningPtr<clang::CommentHandler> CommentHandler;
> +
> /// Whether the '>' token acts as an operator or not. This will be
> /// true except when we are parsing an expression within a C++
> /// template argument list, where the '>' closes the template
>
> Modified: cfe/trunk/lib/AST/ASTContext.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/AST/ASTContext.cpp?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/lib/AST/ASTContext.cpp (original)
> +++ cfe/trunk/lib/AST/ASTContext.cpp Thu Jul 2 12:08:52 2009
> @@ -37,7 +37,8 @@
> bool FreeMem, unsigned size_reserve) :
> GlobalNestedNameSpecifier(0), CFConstantStringTypeDecl(0),
> ObjCFastEnumerationStateTypeDecl(0), SourceMgr(SM), LangOpts(LOpts),
> - FreeMemory(FreeMem), Target(t), Idents(idents), Selectors(sels),
> + LoadedExternalComments(false), FreeMemory(FreeMem), Target(t),
> + Idents(idents), Selectors(sels),
> BuiltinInfo(builtins), ExternalSource(0), PrintingPolicy(LOpts) {
> if (size_reserve > 0) Types.reserve(size_reserve);
> InitBuiltinTypes();
> @@ -202,6 +203,207 @@
> InitBuiltinType(NullPtrTy, BuiltinType::NullPtr);
> }
>
> +namespace {
> + class BeforeInTranslationUnit
> + : std::binary_function<SourceRange, SourceRange, bool> {
> + SourceManager *SourceMgr;
> +
> + public:
> + explicit BeforeInTranslationUnit(SourceManager *SM) : SourceMgr
> (SM) { }
> +
> + bool operator()(SourceRange X, SourceRange Y) {
> + return SourceMgr->isBeforeInTranslationUnit(X.getBegin(),
> Y.getBegin());
> + }
> + };
> +}
> +
> +/// \brief Determine whether the given comment is a Doxygen-style
> comment.
> +///
> +/// \param Start the start of the comment text.
> +///
> +/// \param End the end of the comment text.
> +///
> +/// \param Member whether we want to check whether this is a member
> comment
> +/// (which requires a < after the Doxygen-comment delimiter).
> Otherwise,
> +/// we only return true when we find a non-member comment.
> +static bool
> +isDoxygenComment(SourceManager &SourceMgr, SourceRange Comment,
> + bool Member = false) {
> + const char *BufferStart
> + = SourceMgr.getBufferData(SourceMgr.getFileID(Comment.getBegin
> ())).first;
> + const char *Start = BufferStart + SourceMgr.getFileOffset
> (Comment.getBegin());
> + const char* End = BufferStart + SourceMgr.getFileOffset
> (Comment.getEnd());
> +
> + if (End - Start < 4)
> + return false;
> +
> + assert(Start[0] == '/' && "Not a comment?");
> + if (Start[1] == '*' && !(Start[2] == '!' || Start[2] == '*'))
> + return false;
> + if (Start[1] == '/' && !(Start[2] == '!' || Start[2] == '/'))
> + return false;
> +
> + return (Start[3] == '<') == Member;
> +}
> +
> +/// \brief Retrieve the comment associated with the given
> declaration, if
> +/// it has one.
> +const char *ASTContext::getCommentForDecl(const Decl *D) {
> + if (!D)
> + return 0;
> +
> + // Check whether we have cached a comment string for this
> declaration
> + // already.
> + llvm::DenseMap<const Decl *, std::string>::iterator Pos
> + = DeclComments.find(D);
> + if (Pos != DeclComments.end())
> + return Pos->second.c_str();
> +
> + // If we have an external AST source and have not yet loaded
> comments from
> + // that source, do so now.
> + if (ExternalSource && !LoadedExternalComments) {
> + std::vector<SourceRange> LoadedComments;
> + ExternalSource->ReadComments(LoadedComments);
> +
> + if (!LoadedComments.empty())
> + Comments.insert(Comments.begin(), LoadedComments.begin(),
> + LoadedComments.end());
> +
> + LoadedExternalComments = true;
> + }
> +
> + // If there are no comments anywhere, we won't find anything.
> + if (Comments.empty())
> + return 0;
> +
> + // If the declaration doesn't map directly to a location in a
> file, we
> + // can't find the comment.
> + SourceLocation DeclStartLoc = D->getLocStart();
> + if (DeclStartLoc.isInvalid() || !DeclStartLoc.isFileID())
> + return 0;
> +
> + // Find the comment that occurs just before this declaration.
> + std::vector<SourceRange>::iterator LastComment
> + = std::lower_bound(Comments.begin(), Comments.end(),
> + SourceRange(DeclStartLoc),
> + BeforeInTranslationUnit(&SourceMgr));
> +
> + // Decompose the location for the start of the declaration and
> find the
> + // beginning of the file buffer.
> + std::pair<FileID, unsigned> DeclStartDecomp
> + = SourceMgr.getDecomposedLoc(DeclStartLoc);
> + const char *FileBufferStart
> + = SourceMgr.getBufferData(DeclStartDecomp.first).first;
> +
> + // First check whether we have a comment for a member.
> + if (LastComment != Comments.end() &&
> + !isa<TagDecl>(D) && !isa<NamespaceDecl>(D) &&
> + isDoxygenComment(SourceMgr, *LastComment, true)) {
> + std::pair<FileID, unsigned> LastCommentEndDecomp
> + = SourceMgr.getDecomposedLoc(LastComment->getEnd());
> + if (DeclStartDecomp.first == LastCommentEndDecomp.first &&
> + SourceMgr.getLineNumber(DeclStartDecomp.first,
> DeclStartDecomp.second)
> + == SourceMgr.getLineNumber(LastCommentEndDecomp.first,
> + LastCommentEndDecomp.second)) {
> + // The Doxygen member comment comes after the declaration
> starts and
> + // is on the same line and in the same file as the
> declaration. This
> + // is the comment we want.
> + std::string &Result = DeclComments[D];
> + Result.append(FileBufferStart +
> + SourceMgr.getFileOffset(LastComment->getBegin
> ()),
> + FileBufferStart + LastCommentEndDecomp.second +
> 1);
> + return Result.c_str();
> + }
> + }
> +
> + if (LastComment == Comments.begin())
> + return 0;
> + --LastComment;
> +
> + // Decompose the end of the comment.
> + std::pair<FileID, unsigned> LastCommentEndDecomp
> + = SourceMgr.getDecomposedLoc(LastComment->getEnd());
> +
> + // If the comment and the declaration aren't in the same file,
> then they
> + // aren't related.
> + if (DeclStartDecomp.first != LastCommentEndDecomp.first)
> + return 0;
> +
> + // Check that we actually have a Doxygen comment.
> + if (!isDoxygenComment(SourceMgr, *LastComment))
> + return 0;
> +
> + // Compute the starting line for the declaration and for the end
> of the
> + // comment (this is expensive).
> + unsigned DeclStartLine
> + = SourceMgr.getLineNumber(DeclStartDecomp.first,
> DeclStartDecomp.second);
> + unsigned CommentEndLine
> + = SourceMgr.getLineNumber(LastCommentEndDecomp.first,
> + LastCommentEndDecomp.second);
> +
> + // If the comment does not end on the line prior to the
> declaration, then
> + // the comment is not associated with the declaration at all.
> + if (CommentEndLine + 1 != DeclStartLine)
> + return 0;
> +
> + // We have a comment, but there may be more comments on the
> previous lines.
> + // Keep looking so long as the comments are still Doxygen
> comments and are
> + // still adjacent.
> + unsigned ExpectedLine
> + = SourceMgr.getSpellingLineNumber(LastComment->getBegin()) - 1;
> + std::vector<SourceRange>::iterator FirstComment = LastComment;
> + while (FirstComment != Comments.begin()) {
> + // Look at the previous comment
> + --FirstComment;
> + std::pair<FileID, unsigned> Decomp
> + = SourceMgr.getDecomposedLoc(FirstComment->getEnd());
> +
> + // If this previous comment is in a different file, we're done.
> + if (Decomp.first != DeclStartDecomp.first) {
> + ++FirstComment;
> + break;
> + }
> +
> + // If this comment is not a Doxygen comment, we're done.
> + if (!isDoxygenComment(SourceMgr, *FirstComment)) {
> + ++FirstComment;
> + break;
> + }
> +
> + // If the line number is not what we expected, we're done.
> + unsigned Line = SourceMgr.getLineNumber(Decomp.first,
> Decomp.second);
> + if (Line != ExpectedLine) {
> + ++FirstComment;
> + break;
> + }
> +
> + // Set the next expected line number.
> + ExpectedLine
> + = SourceMgr.getSpellingLineNumber(FirstComment->getBegin()) -
> 1;
> + }
> +
> + // The iterator range [FirstComment, LastComment] contains all of
> the
> + // BCPL comments that, together, are associated with this
> declaration.
> + // Form a single comment block string for this declaration that
> concatenates
> + // all of these comments.
> + std::string &Result = DeclComments[D];
> + while (FirstComment != LastComment) {
> + std::pair<FileID, unsigned> DecompStart
> + = SourceMgr.getDecomposedLoc(FirstComment->getBegin());
> + std::pair<FileID, unsigned> DecompEnd
> + = SourceMgr.getDecomposedLoc(FirstComment->getEnd());
> + Result.append(FileBufferStart + DecompStart.second,
> + FileBufferStart + DecompEnd.second + 1);
> + ++FirstComment;
> + }
> +
> + // Append the last comment line.
> + Result.append(FileBufferStart +
> + SourceMgr.getFileOffset(LastComment->getBegin()),
> + FileBufferStart + LastCommentEndDecomp.second + 1);
> + return Result.c_str();
> +}
> +
> //
> =
> =
> =
> ----------------------------------------------------------------------=
> ==//
> // Type Sizing and Analysis
> //
> =
> =
> =
> ----------------------------------------------------------------------=
> ==//
>
> Modified: cfe/trunk/lib/Frontend/PCHReader.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/PCHReader.cpp?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/lib/Frontend/PCHReader.cpp (original)
> +++ cfe/trunk/lib/Frontend/PCHReader.cpp Thu Jul 2 12:08:52 2009
> @@ -344,7 +344,8 @@
> IdentifierOffsets(0),
> MethodPoolLookupTable(0), MethodPoolLookupTableData(0),
> TotalSelectorsInMethodPool(0), SelectorOffsets(0),
> - TotalNumSelectors(0), NumStatHits(0), NumStatMisses(0),
> + TotalNumSelectors(0), Comments(0), NumComments(0),
> + NumStatHits(0), NumStatMisses(0),
> NumSLocEntriesRead(0), NumStatementsRead(0),
> NumMacrosRead(0), NumMethodPoolSelectorsRead(0),
> NumMethodPoolMisses(0),
> NumLexicalDeclContextsRead(0), NumVisibleDeclContextsRead(0) { }
> @@ -1350,6 +1351,11 @@
> case pch::ORIGINAL_FILE_NAME:
> OriginalFileName.assign(BlobStart, BlobLen);
> break;
> +
> + case pch::COMMENT_RANGES:
> + Comments = (SourceRange *)BlobStart;
> + NumComments = BlobLen / sizeof(SourceRange);
> + break;
> }
> }
> Error("premature end of bitstream in PCH file");
> @@ -1664,6 +1670,12 @@
> return false;
> }
>
> +void PCHReader::ReadComments(std::vector<SourceRange> &Comments) {
> + Comments.resize(NumComments);
> + std::copy(this->Comments, this->Comments + NumComments,
> + Comments.begin());
> +}
> +
> /// \brief Read and return the type at the given offset.
> ///
> /// This routine actually reads the record corresponding to the type
>
> Modified: cfe/trunk/lib/Frontend/PCHWriter.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/PCHWriter.cpp?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/lib/Frontend/PCHWriter.cpp (original)
> +++ cfe/trunk/lib/Frontend/PCHWriter.cpp Thu Jul 2 12:08:52 2009
> @@ -374,7 +374,8 @@
> RECORD(STAT_CACHE);
> RECORD(EXT_VECTOR_DECLS);
> RECORD(OBJC_CATEGORY_IMPLEMENTATIONS);
> -
> + RECORD(COMMENT_RANGES);
> +
> // SourceManager Block.
> BLOCK(SOURCE_MANAGER_BLOCK);
> RECORD(SM_SLOC_FILE_ENTRY);
> @@ -989,6 +990,24 @@
> Stream.ExitBlock();
> }
>
> +void PCHWriter::WriteComments(ASTContext &Context) {
> + using namespace llvm;
> +
> + if (Context.Comments.empty())
> + return;
> +
> + BitCodeAbbrev *CommentAbbrev = new BitCodeAbbrev();
> + CommentAbbrev->Add(BitCodeAbbrevOp(pch::COMMENT_RANGES));
> + CommentAbbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob));
> + unsigned CommentCode = Stream.EmitAbbrev(CommentAbbrev);
> +
> + RecordData Record;
> + Record.push_back(pch::COMMENT_RANGES);
> + Stream.EmitRecordWithBlob(CommentCode, Record,
> + (const char*)&Context.Comments[0],
> + Context.Comments.size() * sizeof
> (SourceRange));
> +}
> +
> //
> =
> =
> =
> ----------------------------------------------------------------------=
> ==//
> // Type Serialization
> //
> =
> =
> =
> ----------------------------------------------------------------------=
> ==//
> @@ -1746,7 +1765,8 @@
> WriteStatCache(*StatCalls);
> WriteSourceManagerBlock(Context.getSourceManager(), PP);
> WritePreprocessor(PP);
> -
> + WriteComments(Context);
> +
> // Keep writing types and declarations until all types and
> // declarations have been written.
> do {
>
> Modified: cfe/trunk/lib/Lex/Lexer.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Lexer.cpp?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/lib/Lex/Lexer.cpp (original)
> +++ cfe/trunk/lib/Lex/Lexer.cpp Thu Jul 2 12:08:52 2009
> @@ -903,7 +903,10 @@
> } while (C != '\n' && C != '\r');
>
> // Found but did not consume the newline.
> -
> + if (PP)
> + PP->HandleComment(SourceRange(getSourceLocation(BufferPtr),
> + getSourceLocation(CurPtr)));
> +
> // If we are returning comments as tokens, return this comment as
> a token.
> if (inKeepCommentMode())
> return SaveBCPLComment(Result, CurPtr);
> @@ -1146,6 +1149,10 @@
> C = *CurPtr++;
> }
>
> + if (PP)
> + PP->HandleComment(SourceRange(getSourceLocation(BufferPtr),
> + getSourceLocation(CurPtr)));
> +
> // If we are returning comments as tokens, return this comment as
> a token.
> if (inKeepCommentMode()) {
> FormTokenWithChars(Result, CurPtr, tok::comment);
>
> Modified: cfe/trunk/lib/Lex/Pragma.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Pragma.cpp?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/lib/Lex/Pragma.cpp (original)
> +++ cfe/trunk/lib/Lex/Pragma.cpp Thu Jul 2 12:08:52 2009
> @@ -19,6 +19,7 @@
> #include "clang/Lex/LexDiagnostic.h"
> #include "clang/Basic/FileManager.h"
> #include "clang/Basic/SourceManager.h"
> +#include <algorithm>
> using namespace clang;
>
> // Out-of-line destructor to provide a home for the class.
>
> Modified: cfe/trunk/lib/Lex/Preprocessor.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Preprocessor.cpp?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/lib/Lex/Preprocessor.cpp (original)
> +++ cfe/trunk/lib/Lex/Preprocessor.cpp Thu Jul 2 12:08:52 2009
> @@ -476,3 +476,26 @@
> if (II.isExtensionToken() && !DisableMacroExpansion)
> Diag(Identifier, diag::ext_token_used);
> }
> +
> +void Preprocessor::AddCommentHandler(CommentHandler *Handler) {
> + assert(Handler && "NULL comment handler");
> + assert(std::find(CommentHandlers.begin(), CommentHandlers.end(),
> Handler) ==
> + CommentHandlers.end() && "Comment handler already
> registered");
> + CommentHandlers.push_back(Handler);
> +}
> +
> +void Preprocessor::RemoveCommentHandler(CommentHandler *Handler) {
> + std::vector<CommentHandler *>::iterator Pos
> + = std::find(CommentHandlers.begin(), CommentHandlers.end(),
> Handler);
> + assert(Pos != CommentHandlers.end() && "Comment handler not
> registered");
> + CommentHandlers.erase(Pos);
> +}
> +
> +void Preprocessor::HandleComment(SourceRange Comment) {
> + for (std::vector<CommentHandler *>::iterator H =
> CommentHandlers.begin(),
> + HEnd = CommentHandlers.end();
> + H != HEnd; ++H)
> + (*H)->HandleComment(*this, Comment);
> +}
> +
> +CommentHandler::~CommentHandler() { }
>
> Modified: cfe/trunk/lib/Parse/Parser.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/Parser.cpp?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/lib/Parse/Parser.cpp (original)
> +++ cfe/trunk/lib/Parse/Parser.cpp Thu Jul 2 12:08:52 2009
> @@ -20,6 +20,19 @@
> #include "ParsePragma.h"
> using namespace clang;
>
> +/// \brief A comment handler that passes comments found by the
> preprocessor
> +/// to the parser action.
> +class ActionCommentHandler : public CommentHandler {
> + Action &Actions;
> +
> +public:
> + explicit ActionCommentHandler(Action &Actions) : Actions(Actions)
> { }
> +
> + virtual void HandleComment(Preprocessor &PP, SourceRange Comment) {
> + Actions.ActOnComment(Comment);
> + }
> +};
> +
> Parser::Parser(Preprocessor &pp, Action &actions)
> : CrashInfo(*this), PP(pp), Actions(actions), Diags
> (PP.getDiagnostics()),
> GreaterThanIsOperator(true) {
> @@ -43,6 +56,9 @@
> WeakHandler.reset(new
> PragmaWeakHandler(&PP.getIdentifierTable().get("weak"),
> actions));
> PP.AddPragmaHandler(0, WeakHandler.get());
> +
> + CommentHandler.reset(new ActionCommentHandler(actions));
> + PP.AddCommentHandler(CommentHandler.get());
> }
>
> /// If a crash happens while the parser is active, print out a line
> indicating
> @@ -294,6 +310,7 @@
> UnusedHandler.reset();
> PP.RemovePragmaHandler(0, WeakHandler.get());
> WeakHandler.reset();
> + PP.RemoveCommentHandler(CommentHandler.get());
> }
>
> /// Initialize - Warm up the parser.
>
> Modified: cfe/trunk/lib/Sema/Sema.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/Sema.cpp?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/lib/Sema/Sema.cpp (original)
> +++ cfe/trunk/lib/Sema/Sema.cpp Thu Jul 2 12:08:52 2009
> @@ -345,3 +345,7 @@
> = SemaRef.ActiveTemplateInstantiations.back();
> }
> }
> +
> +void Sema::ActOnComment(SourceRange Comment) {
> + Context.Comments.push_back(Comment);
> +}
>
> Modified: cfe/trunk/lib/Sema/Sema.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/Sema.h?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/lib/Sema/Sema.h (original)
> +++ cfe/trunk/lib/Sema/Sema.h Thu Jul 2 12:08:52 2009
> @@ -362,6 +362,8 @@
> return CurBlock ? CurBlock->SwitchStack : FunctionSwitchStack;
> }
>
> + virtual void ActOnComment(SourceRange Comment);
> +
> //
> =
> =
> =--------------------------------------------------------------------
> ===//
> // Type Analysis / Processing: SemaType.cpp.
> //
>
> Modified: cfe/trunk/test/CMakeLists.txt
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CMakeLists.txt?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/test/CMakeLists.txt (original)
> +++ cfe/trunk/test/CMakeLists.txt Thu Jul 2 12:08:52 2009
> @@ -8,6 +8,7 @@
> "Driver"
> "FixIt"
> "Frontend"
> + "Index"
> "Lexer"
> "Misc"
> "PCH"
> @@ -54,6 +55,6 @@
> --clang-cc=${LLVM_TOOLS_PATH}/${CMAKE_CFG_INTDIR}/clang-cc
> ${all_testdirs}
> WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
> - DEPENDS clang clang-cc
> + DEPENDS clang clang-cc index-test
> COMMENT "Running Clang regression tests")
> endif()
>
> Added: cfe/trunk/test/Index/comments.c
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Index/comments.c?rev=74704&view=auto
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/test/Index/comments.c (added)
> +++ cfe/trunk/test/Index/comments.c Thu Jul 2 12:08:52 2009
> @@ -0,0 +1,30 @@
> +// RUN: clang-cc -emit-pch -o %t.ast %s &&
> +// RUN: index-test %t.ast -point-at %s:22:6 | grep "starts here" &&
> +// RUN: index-test %t.ast -point-at %s:22:6 | grep "block comment" &&
> +// RUN: index-test %t.ast -point-at %s:28:6 | grep "BCPL" &&
> +// RUN: index-test %t.ast -point-at %s:28:6 | grep "But" &&
> +// RUN: index-test %t.ast -point-at %s:28:6 | grep "NOT" | count 0 &&
> +// RUN: index-test %t.ast -point-at %s:30:6 | grep "member"
> +
> +
> +
> +
> +
> +
> +//! It all starts here.
> +/*! It's a little odd to continue line this,
> + *
> + * but we need more multi-line comments. */
> +/// This comment comes before my other comments
> +/** This is a block comment that is associated with the function f.
> It
> + * runs for three lines.
> + */
> +void f(int, int);
> +
> +// NOT IN THE COMMENT
> +/// This is a BCPL comment that is associated with the function g.
> +/// It has only two lines.
> +/** But there are other blocks that are part of the comment, too. */
> +void g(int);
> +
> +void h(int); ///< This is a member comment.
> \ No newline at end of file
>
> Propchange: cfe/trunk/test/Index/comments.c
>
> ------------------------------------------------------------------------------
> svn:eol-style = native
>
> Propchange: cfe/trunk/test/Index/comments.c
>
> ------------------------------------------------------------------------------
> svn:keywords = Id
>
> Propchange: cfe/trunk/test/Index/comments.c
>
> ------------------------------------------------------------------------------
> svn:mime-type = text/plain
>
> Modified: cfe/trunk/tools/index-test/index-test.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/index-test/index-test.cpp?rev=74704&r1=74703&r2=74704&view=diff
>
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- cfe/trunk/tools/index-test/index-test.cpp (original)
> +++ cfe/trunk/tools/index-test/index-test.cpp Thu Jul 2 12:08:52 2009
> @@ -130,6 +130,9 @@
> OS << ND->getNameAsString();
> OS << "\n";
>
> + if (const char *Comment = AST->getASTContext().getCommentForDecl
> (Point.D))
> + OS << "Comment associated with this declaration:\n" <<
> Comment << "\n";
> +
> if (Point.Node) {
> OS << "Statement node at point: " << Point.Node-
> >getStmtClassName()
> << " ";
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
More information about the cfe-commits
mailing list