r305044 - [libclang] Introduce a new parsing option 'CXTranslationUnit_SingleFileParse' that puts preprocessor in a mode for parsing a single file only.

Argyrios Kyrtzidis via cfe-commits cfe-commits at lists.llvm.org
Fri Jun 9 08:24:09 PDT 2017


Hi Manuel,

Apologies, I will start a RFC giving more context. Also I’ll ask for feedback for another preprocessor change related to this I’d propose as follow-up.

> On Jun 9, 2017, at 3:07 AM, Manuel Klimek <klimek at google.com> wrote:
> 
> Hey ho,
> 
> this looks like a really cool new feature, and the CL looks good to me; I was wondering whether in the future we can send CLs like this through pre-commit review, or have an RFC email thread. If you have more stuff landing, a single RFC email thread describing how the features work together would also be super useful, as it makes it easier to follow along the patches later.
> 
> Cheers,
> /Manuel
> 
> On Fri, Jun 9, 2017 at 3:21 AM Argyrios Kyrtzidis via cfe-commits <cfe-commits at lists.llvm.org <mailto:cfe-commits at lists.llvm.org>> wrote:
> Author: akirtzidis
> Date: Thu Jun  8 20:20:48 2017
> New Revision: 305044
> 
> URL: http://llvm.org/viewvc/llvm-project?rev=305044&view=rev <http://llvm.org/viewvc/llvm-project?rev=305044&view=rev>
> Log:
> [libclang] Introduce a new parsing option 'CXTranslationUnit_SingleFileParse' that puts preprocessor in a mode for parsing a single file only.
> 
> This is useful for parsing a single file, as a fast/inaccurate 'mode' that can still provide declarations from the file, like the classes and their methods.
> 
> Added:
>     cfe/trunk/test/Index/singe-file-parse.m
> Modified:
>     cfe/trunk/include/clang-c/Index.h
>     cfe/trunk/include/clang/Frontend/ASTUnit.h
>     cfe/trunk/include/clang/Lex/PreprocessorOptions.h
>     cfe/trunk/lib/Frontend/ASTUnit.cpp
>     cfe/trunk/lib/Lex/PPDirectives.cpp
>     cfe/trunk/tools/c-index-test/c-index-test.c
>     cfe/trunk/tools/libclang/CIndex.cpp
> 
> Modified: cfe/trunk/include/clang-c/Index.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang-c/Index.h?rev=305044&r1=305043&r2=305044&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang-c/Index.h?rev=305044&r1=305043&r2=305044&view=diff>
> ==============================================================================
> --- cfe/trunk/include/clang-c/Index.h (original)
> +++ cfe/trunk/include/clang-c/Index.h Thu Jun  8 20:20:48 2017
> @@ -32,7 +32,7 @@
>   * compatible, thus CINDEX_VERSION_MAJOR is expected to remain stable.
>   */
>  #define CINDEX_VERSION_MAJOR 0
> -#define CINDEX_VERSION_MINOR 42
> +#define CINDEX_VERSION_MINOR 43
> 
>  #define CINDEX_VERSION_ENCODE(major, minor) ( \
>        ((major) * 10000)                       \
> @@ -1234,7 +1234,12 @@ enum CXTranslationUnit_Flags {
>     * purposes of an IDE, this is undesirable behavior and as much information
>     * as possible should be reported. Use this flag to enable this behavior.
>     */
> -  CXTranslationUnit_KeepGoing = 0x200
> +  CXTranslationUnit_KeepGoing = 0x200,
> +
> +  /**
> +   * \brief Sets the preprocessor in a mode for parsing a single file only.
> +   */
> +  CXTranslationUnit_SingleFileParse = 0x400
>  };
> 
>  /**
> 
> Modified: cfe/trunk/include/clang/Frontend/ASTUnit.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/ASTUnit.h?rev=305044&r1=305043&r2=305044&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/ASTUnit.h?rev=305044&r1=305043&r2=305044&view=diff>
> ==============================================================================
> --- cfe/trunk/include/clang/Frontend/ASTUnit.h (original)
> +++ cfe/trunk/include/clang/Frontend/ASTUnit.h Thu Jun  8 20:20:48 2017
> @@ -871,6 +871,7 @@ public:
>        bool CacheCodeCompletionResults = false,
>        bool IncludeBriefCommentsInCodeCompletion = false,
>        bool AllowPCHWithCompilerErrors = false, bool SkipFunctionBodies = false,
> +      bool SingleFileParse = false,
>        bool UserFilesAreVolatile = false, bool ForSerialization = false,
>        llvm::Optional<StringRef> ModuleFormat = llvm::None,
>        std::unique_ptr<ASTUnit> *ErrAST = nullptr,
> 
> Modified: cfe/trunk/include/clang/Lex/PreprocessorOptions.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/PreprocessorOptions.h?rev=305044&r1=305043&r2=305044&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/PreprocessorOptions.h?rev=305044&r1=305043&r2=305044&view=diff>
> ==============================================================================
> --- cfe/trunk/include/clang/Lex/PreprocessorOptions.h (original)
> +++ cfe/trunk/include/clang/Lex/PreprocessorOptions.h Thu Jun  8 20:20:48 2017
> @@ -95,6 +95,9 @@ public:
>    /// If given, a PTH cache file to use for speeding up header parsing.
>    std::string TokenCache;
> 
> +  /// When enabled, preprocessor is in a mode for parsing a single file only.
> +  bool SingleFileParseMode = false;
> +
>    /// \brief True if the SourceManager should report the original file name for
>    /// contents of files that were remapped to other files. Defaults to true.
>    bool RemappedFilesKeepOriginalName;
> @@ -181,6 +184,7 @@ public:
>      ImplicitPCHInclude.clear();
>      ImplicitPTHInclude.clear();
>      TokenCache.clear();
> +    SingleFileParseMode = false;
>      RetainRemappedFileBuffers = true;
>      PrecompiledPreambleBytes.first = 0;
>      PrecompiledPreambleBytes.second = 0;
> 
> Modified: cfe/trunk/lib/Frontend/ASTUnit.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/ASTUnit.cpp?rev=305044&r1=305043&r2=305044&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/ASTUnit.cpp?rev=305044&r1=305043&r2=305044&view=diff>
> ==============================================================================
> --- cfe/trunk/lib/Frontend/ASTUnit.cpp (original)
> +++ cfe/trunk/lib/Frontend/ASTUnit.cpp Thu Jun  8 20:20:48 2017
> @@ -1982,7 +1982,7 @@ ASTUnit *ASTUnit::LoadFromCommandLine(
>      unsigned PrecompilePreambleAfterNParses, TranslationUnitKind TUKind,
>      bool CacheCodeCompletionResults, bool IncludeBriefCommentsInCodeCompletion,
>      bool AllowPCHWithCompilerErrors, bool SkipFunctionBodies,
> -    bool UserFilesAreVolatile, bool ForSerialization,
> +    bool SingleFileParse, bool UserFilesAreVolatile, bool ForSerialization,
>      llvm::Optional<StringRef> ModuleFormat, std::unique_ptr<ASTUnit> *ErrAST,
>      IntrusiveRefCntPtr<vfs::FileSystem> VFS) {
>    assert(Diags.get() && "no DiagnosticsEngine was provided");
> @@ -2011,6 +2011,7 @@ ASTUnit *ASTUnit::LoadFromCommandLine(
>    PPOpts.RemappedFilesKeepOriginalName = RemappedFilesKeepOriginalName;
>    PPOpts.AllowPCHWithCompilerErrors = AllowPCHWithCompilerErrors;
>    PPOpts.GeneratePreamble = PrecompilePreambleAfterNParses != 0;
> +  PPOpts.SingleFileParseMode = SingleFileParse;
> 
>    // Override the resources path.
>    CI->getHeaderSearchOpts().ResourceDir = ResourceFilesPath;
> 
> Modified: cfe/trunk/lib/Lex/PPDirectives.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPDirectives.cpp?rev=305044&r1=305043&r2=305044&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPDirectives.cpp?rev=305044&r1=305043&r2=305044&view=diff>
> ==============================================================================
> --- cfe/trunk/lib/Lex/PPDirectives.cpp (original)
> +++ cfe/trunk/lib/Lex/PPDirectives.cpp Thu Jun  8 20:20:48 2017
> @@ -30,6 +30,7 @@
>  #include "clang/Lex/PPCallbacks.h"
>  #include "clang/Lex/Pragma.h"
>  #include "clang/Lex/Preprocessor.h"
> +#include "clang/Lex/PreprocessorOptions.h"
>  #include "clang/Lex/PTHLexer.h"
>  #include "clang/Lex/Token.h"
>  #include "llvm/ADT/ArrayRef.h"
> @@ -1845,10 +1846,13 @@ void Preprocessor::HandleIncludeDirectiv
>    // we've imported or already built.
>    bool ShouldEnter = true;
> 
> +  if (PPOpts->SingleFileParseMode)
> +    ShouldEnter = false;
> +
>    // Determine whether we should try to import the module for this #include, if
>    // there is one. Don't do so if precompiled module support is disabled or we
>    // are processing this module textually (because we're building the module).
> -  if (File && SuggestedModule && getLangOpts().Modules &&
> +  if (ShouldEnter && File && SuggestedModule && getLangOpts().Modules &&
>        SuggestedModule.getModule()->getTopLevelModuleName() !=
>            getLangOpts().CurrentModule) {
>      // If this include corresponds to a module but that module is
> 
> Added: cfe/trunk/test/Index/singe-file-parse.m
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Index/singe-file-parse.m?rev=305044&view=auto <http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Index/singe-file-parse.m?rev=305044&view=auto>
> ==============================================================================
> --- cfe/trunk/test/Index/singe-file-parse.m (added)
> +++ cfe/trunk/test/Index/singe-file-parse.m Thu Jun  8 20:20:48 2017
> @@ -0,0 +1,11 @@
> +// RUN: c-index-test -single-file-parse %s | FileCheck %s
> +
> +#include <stdint.h>
> +
> +// CHECK-NOT: TypedefDecl=intptr_t
> +
> +// CHECK: [[@LINE+1]]:12: ObjCInterfaceDecl=MyCls
> + at interface MyCls
> +// CHECK: [[@LINE+1]]:8: ObjCInstanceMethodDecl=some_meth
> +-(void)some_meth;
> + at end
> 
> Modified: cfe/trunk/tools/c-index-test/c-index-test.c
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/c-index-test/c-index-test.c?rev=305044&r1=305043&r2=305044&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/c-index-test/c-index-test.c?rev=305044&r1=305043&r2=305044&view=diff>
> ==============================================================================
> --- cfe/trunk/tools/c-index-test/c-index-test.c (original)
> +++ cfe/trunk/tools/c-index-test/c-index-test.c Thu Jun  8 20:20:48 2017
> @@ -1850,6 +1850,34 @@ int perform_test_reparse_source(int argc
>    return result;
>  }
> 
> +static int perform_single_file_parse(const char *filename) {
> +  CXIndex Idx;
> +  CXTranslationUnit TU;
> +  enum CXErrorCode Err;
> +  int result;
> +
> +  Idx = clang_createIndex(/* excludeDeclsFromPCH */1,
> +                          /* displayDiagnostics=*/1);
> +
> +  Err = clang_parseTranslationUnit2(Idx, filename,
> +                                    /*command_line_args=*/NULL,
> +                                    /*num_command_line_args=*/0,
> +                                    /*unsaved_files=*/NULL,
> +                                    /*num_unsaved_files=*/0,
> +                                    CXTranslationUnit_SingleFileParse, &TU);
> +  if (Err != CXError_Success) {
> +    fprintf(stderr, "Unable to load translation unit!\n");
> +    describeLibclangFailure(Err);
> +    clang_disposeIndex(Idx);
> +    return 1;
> +  }
> +
> +  result = perform_test_load(Idx, TU, /*filter=*/"all", /*prefix=*/NULL, FilteredPrintingVisitor, /*PostVisit=*/NULL,
> +                             /*CommentSchemaFile=*/NULL);
> +  clang_disposeIndex(Idx);
> +  return result;
> +}
> +
>  /******************************************************************************/
>  /* Logic for testing clang_getCursor().                                       */
>  /******************************************************************************/
> @@ -4439,6 +4467,8 @@ int cindextest_main(int argc, const char
>        return perform_test_load_source(argc - 3, argv + 3, argv[2], I,
>                                        postVisit);
>    }
> +  else if (argc >= 3 && strcmp(argv[1], "-single-file-parse") == 0)
> +    return perform_single_file_parse(argv[2]);
>    else if (argc >= 4 && strcmp(argv[1], "-test-file-scan") == 0)
>      return perform_file_scan(argv[2], argv[3],
>                               argc >= 5 ? argv[4] : 0);
> 
> Modified: cfe/trunk/tools/libclang/CIndex.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/libclang/CIndex.cpp?rev=305044&r1=305043&r2=305044&view=diff <http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/libclang/CIndex.cpp?rev=305044&r1=305043&r2=305044&view=diff>
> ==============================================================================
> --- cfe/trunk/tools/libclang/CIndex.cpp (original)
> +++ cfe/trunk/tools/libclang/CIndex.cpp Thu Jun  8 20:20:48 2017
> @@ -3300,12 +3300,14 @@ clang_parseTranslationUnit_Impl(CXIndex
>        options & CXTranslationUnit_CreatePreambleOnFirstParse;
>    // FIXME: Add a flag for modules.
>    TranslationUnitKind TUKind
> -    = (options & CXTranslationUnit_Incomplete)? TU_Prefix : TU_Complete;
> +    = (options & (CXTranslationUnit_Incomplete |
> +                  CXTranslationUnit_SingleFileParse))? TU_Prefix : TU_Complete;
>    bool CacheCodeCompletionResults
>      = options & CXTranslationUnit_CacheCompletionResults;
>    bool IncludeBriefCommentsInCodeCompletion
>      = options & CXTranslationUnit_IncludeBriefCommentsInCodeCompletion;
>    bool SkipFunctionBodies = options & CXTranslationUnit_SkipFunctionBodies;
> +  bool SingleFileParse = options & CXTranslationUnit_SingleFileParse;
>    bool ForSerialization = options & CXTranslationUnit_ForSerialization;
> 
>    // Configure the diagnostics.
> @@ -3390,7 +3392,7 @@ clang_parseTranslationUnit_Impl(CXIndex
>        /*CaptureDiagnostics=*/true, *RemappedFiles.get(),
>        /*RemappedFilesKeepOriginalName=*/true, PrecompilePreambleAfterNParses,
>        TUKind, CacheCodeCompletionResults, IncludeBriefCommentsInCodeCompletion,
> -      /*AllowPCHWithCompilerErrors=*/true, SkipFunctionBodies,
> +      /*AllowPCHWithCompilerErrors=*/true, SkipFunctionBodies, SingleFileParse,
>        /*UserFilesAreVolatile=*/true, ForSerialization,
>        CXXIdx->getPCHContainerOperations()->getRawReader().getFormat(),
>        &ErrUnit));
> 
> 
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at lists.llvm.org <mailto:cfe-commits at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20170609/892d6cd7/attachment-0001.html>


More information about the cfe-commits mailing list