[clang-tools-extra] r291446 - [include-fixer] Load symbol index asynchronously.
Bill Seurer via cfe-commits
cfe-commits at lists.llvm.org
Thu Jan 12 15:00:19 PST 2017
On 01/10/2017 01:53 PM, Benjamin Kramer wrote:
> I didn't manage to reproduce this. Does adding ${PTHREAD_LIB} to
> LINK_LIBS of tools/clang/tools/extra/include-fixer/plugin/CMakeLists.txt
> help?
That does seem to make it work.
> On Tue, Jan 10, 2017 at 8:31 PM, Bill Seurer <seurer at linux.vnet.ibm.com> wrote:
>> On 01/09/2017 09:18 AM, Benjamin Kramer via cfe-commits wrote:
>>>
>>> Author: d0k
>>> Date: Mon Jan 9 09:18:28 2017
>>> New Revision: 291446
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=291446&view=rev
>>> Log:
>>> [include-fixer] Load symbol index asynchronously.
>>>
>>> We don't actually need the index until parse time, so fetch it in the
>>> background and start parsing. By the time it is actually needed it's
>>> likely that the loading phase has completed in the background.
>>
>>
>> This update causes a linker error on ppc64le on a Release+Asserts+Shared
>> build.
>>
>> cmake with -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON
>> -DBUILD_SHARED_LIBS=ON
>>
>> http://lab.llvm.org:8011/builders/clang-ppc64le-linux-multistage/builds/754
>>
>>
>> Details:
>>
>> [117/123] Linking CXX shared library
>> lib/libclangIncludeFixerPlugin.so.40.0svn
>> FAILED: : && /home/seurer/gcc/install/gcc-5.4.0/bin/g++ -fPIC -fPIC
>> -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings
>> -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long
>> -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment
>> -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -fno-common
>> -Woverloaded-virtual -fno-strict-aliasing -O3
>> -L/home/seurer/gcc/install/gcc-5.4.0/lib64 -Wl,-z,defs
>> -Wl,-rpath-link,/home/seurer/llvm/build/llvm-test2/./lib -Wl,-O3
>> -Wl,--gc-sections -shared -Wl,-soname,libclangIncludeFixerPlugin.so.40 -o
>> lib/libclangIncludeFixerPlugin.so.40.0svn
>> tools/clang/tools/extra/include-fixer/plugin/CMakeFiles/clangIncludeFixerPlugin.dir/IncludeFixerPlugin.cpp.o
>> lib/libclangAST.so.40.0svn lib/libclangBasic.so.40.0svn
>> lib/libclangFrontend.so.40.0svn lib/libclangIncludeFixer.so.40.0svn
>> lib/libclangParse.so.40.0svn lib/libclangSema.so.40.0svn
>> lib/libclangTooling.so.40.0svn lib/libLLVMSupport.so.40.0svn
>> -Wl,-rpath,"\$ORIGIN/../lib" && :
>> tools/clang/tools/extra/include-fixer/plugin/CMakeFiles/clangIncludeFixerPlugin.dir/IncludeFixerPlugin.cpp.o:(.toc+0xb0):
>> undefined reference to `pthread_create'
>> collect2: error: ld returned 1 exit status
>> [117/123] Building CXX object
>> tools/clang/tools/extra/include-fixer/tool/CMakeFiles/clang-include-fixer.dir/ClangIncludeFixer.cpp.o
>> ninja: build stopped: subcommand failed.
>> [1/7] Linking CXX shared library lib/libclangIncludeFixerPlugin.so.40.0svn
>> FAILED: : && /home/seurer/gcc/install/gcc-5.4.0/bin/g++ -fPIC -fPIC
>> -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings
>> -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long
>> -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment
>> -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -fno-common
>> -Woverloaded-virtual -fno-strict-aliasing -O3
>> -L/home/seurer/gcc/install/gcc-5.4.0/lib64 -Wl,-z,defs
>> -Wl,-rpath-link,/home/seurer/llvm/build/llvm-test2/./lib -Wl,-O3
>> -Wl,--gc-sections -shared -Wl,-soname,libclangIncludeFixerPlugin.so.40 -o
>> lib/libclangIncludeFixerPlugin.so.40.0svn
>> tools/clang/tools/extra/include-fixer/plugin/CMakeFiles/clangIncludeFixerPlugin.dir/IncludeFixerPlugin.cpp.o
>> lib/libclangAST.so.40.0svn lib/libclangBasic.so.40.0svn
>> lib/libclangFrontend.so.40.0svn lib/libclangIncludeFixer.so.40.0svn
>> lib/libclangParse.so.40.0svn lib/libclangSema.so.40.0svn
>> lib/libclangTooling.so.40.0svn lib/libLLVMSupport.so.40.0svn
>> -Wl,-rpath,"\$ORIGIN/../lib" && :
>> tools/clang/tools/extra/include-fixer/plugin/CMakeFiles/clangIncludeFixerPlugin.dir/IncludeFixerPlugin.cpp.o:(.toc+0xb0):
>> undefined reference to `pthread_create'
>> collect2: error: ld returned 1 exit status
>>
>>
>>
>>
>>
>>> Modified:
>>> clang-tools-extra/trunk/include-fixer/SymbolIndexManager.cpp
>>> clang-tools-extra/trunk/include-fixer/SymbolIndexManager.h
>>> clang-tools-extra/trunk/include-fixer/plugin/IncludeFixerPlugin.cpp
>>> clang-tools-extra/trunk/include-fixer/tool/ClangIncludeFixer.cpp
>>> clang-tools-extra/trunk/unittests/include-fixer/IncludeFixerTest.cpp
>>>
>>> Modified: clang-tools-extra/trunk/include-fixer/SymbolIndexManager.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/include-fixer/SymbolIndexManager.cpp?rev=291446&r1=291445&r2=291446&view=diff
>>>
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/include-fixer/SymbolIndexManager.cpp
>>> (original)
>>> +++ clang-tools-extra/trunk/include-fixer/SymbolIndexManager.cpp Mon Jan
>>> 9 09:18:28 2017
>>> @@ -64,7 +64,7 @@ SymbolIndexManager::search(llvm::StringR
>>> do {
>>> std::vector<clang::find_all_symbols::SymbolInfo> Symbols;
>>> for (const auto &DB : SymbolIndices) {
>>> - auto Res = DB->search(Names.back().str());
>>> + auto Res = DB.get()->search(Names.back());
>>> Symbols.insert(Symbols.end(), Res.begin(), Res.end());
>>> }
>>>
>>>
>>> Modified: clang-tools-extra/trunk/include-fixer/SymbolIndexManager.h
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/include-fixer/SymbolIndexManager.h?rev=291446&r1=291445&r2=291446&view=diff
>>>
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/include-fixer/SymbolIndexManager.h (original)
>>> +++ clang-tools-extra/trunk/include-fixer/SymbolIndexManager.h Mon Jan 9
>>> 09:18:28 2017
>>> @@ -13,6 +13,7 @@
>>> #include "SymbolIndex.h"
>>> #include "find-all-symbols/SymbolInfo.h"
>>> #include "llvm/ADT/StringRef.h"
>>> +#include <future>
>>>
>>> namespace clang {
>>> namespace include_fixer {
>>> @@ -21,8 +22,8 @@ namespace include_fixer {
>>> /// to an indentifier in the source code from multiple symbol databases.
>>> class SymbolIndexManager {
>>> public:
>>> - void addSymbolIndex(std::unique_ptr<SymbolIndex> DB) {
>>> - SymbolIndices.push_back(std::move(DB));
>>> + void addSymbolIndex(std::function<std::unique_ptr<SymbolIndex>()> F) {
>>> + SymbolIndices.push_back(std::async(std::launch::async, F));
>>> }
>>>
>>> /// Search for header files to be included for an identifier.
>>> @@ -39,7 +40,7 @@ public:
>>> search(llvm::StringRef Identifier, bool IsNestedSearch = true) const;
>>>
>>> private:
>>> - std::vector<std::unique_ptr<SymbolIndex>> SymbolIndices;
>>> + std::vector<std::shared_future<std::unique_ptr<SymbolIndex>>>
>>> SymbolIndices;
>>> };
>>>
>>> } // namespace include_fixer
>>>
>>> Modified:
>>> clang-tools-extra/trunk/include-fixer/plugin/IncludeFixerPlugin.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/include-fixer/plugin/IncludeFixerPlugin.cpp?rev=291446&r1=291445&r2=291446&view=diff
>>>
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/include-fixer/plugin/IncludeFixerPlugin.cpp
>>> (original)
>>> +++ clang-tools-extra/trunk/include-fixer/plugin/IncludeFixerPlugin.cpp
>>> Mon Jan 9 09:18:28 2017
>>> @@ -61,23 +61,26 @@ public:
>>> Input = Arg.substr(strlen("-input="));
>>> }
>>>
>>> - llvm::ErrorOr<std::unique_ptr<include_fixer::YamlSymbolIndex>>
>>> SymbolIdx(
>>> - nullptr);
>>> - if (DB == "yaml") {
>>> - if (!Input.empty()) {
>>> - SymbolIdx =
>>> include_fixer::YamlSymbolIndex::createFromFile(Input);
>>> - } else {
>>> - // If we don't have any input file, look in the directory of the
>>> first
>>> - // file and its parents.
>>> - const FrontendOptions &FO = CI.getFrontendOpts();
>>> - SmallString<128> AbsolutePath(
>>> - tooling::getAbsolutePath(FO.Inputs[0].getFile()));
>>> - StringRef Directory = llvm::sys::path::parent_path(AbsolutePath);
>>> - SymbolIdx = include_fixer::YamlSymbolIndex::createFromDirectory(
>>> - Directory, "find_all_symbols_db.yaml");
>>> + std::string InputFile = CI.getFrontendOpts().Inputs[0].getFile();
>>> + auto CreateYamlIdx = [=]() ->
>>> std::unique_ptr<include_fixer::SymbolIndex> {
>>> + llvm::ErrorOr<std::unique_ptr<include_fixer::YamlSymbolIndex>>
>>> SymbolIdx(
>>> + nullptr);
>>> + if (DB == "yaml") {
>>> + if (!Input.empty()) {
>>> + SymbolIdx =
>>> include_fixer::YamlSymbolIndex::createFromFile(Input);
>>> + } else {
>>> + // If we don't have any input file, look in the directory of
>>> the first
>>> + // file and its parents.
>>> + SmallString<128>
>>> AbsolutePath(tooling::getAbsolutePath(InputFile));
>>> + StringRef Directory =
>>> llvm::sys::path::parent_path(AbsolutePath);
>>> + SymbolIdx =
>>> include_fixer::YamlSymbolIndex::createFromDirectory(
>>> + Directory, "find_all_symbols_db.yaml");
>>> + }
>>> }
>>> - }
>>> - SymbolIndexMgr->addSymbolIndex(std::move(*SymbolIdx));
>>> + return std::move(*SymbolIdx);
>>> + };
>>> +
>>> + SymbolIndexMgr->addSymbolIndex(std::move(CreateYamlIdx));
>>> return true;
>>> }
>>>
>>>
>>> Modified: clang-tools-extra/trunk/include-fixer/tool/ClangIncludeFixer.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/include-fixer/tool/ClangIncludeFixer.cpp?rev=291446&r1=291445&r2=291446&view=diff
>>>
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/include-fixer/tool/ClangIncludeFixer.cpp
>>> (original)
>>> +++ clang-tools-extra/trunk/include-fixer/tool/ClangIncludeFixer.cpp Mon
>>> Jan 9 09:18:28 2017
>>> @@ -179,30 +179,36 @@ createSymbolIndexManager(StringRef FileP
>>> find_all_symbols::SymbolInfo::SymbolKind::Unknown,
>>> CommaSplits[I].trim(), 1, {}, /*NumOccurrences=*/E - I));
>>> }
>>> - SymbolIndexMgr->addSymbolIndex(
>>> - llvm::make_unique<include_fixer::InMemorySymbolIndex>(Symbols));
>>> + SymbolIndexMgr->addSymbolIndex([=]() {
>>> + return
>>> llvm::make_unique<include_fixer::InMemorySymbolIndex>(Symbols);
>>> + });
>>> break;
>>> }
>>> case yaml: {
>>> - llvm::ErrorOr<std::unique_ptr<include_fixer::YamlSymbolIndex>>
>>> DB(nullptr);
>>> - if (!Input.empty()) {
>>> - DB = include_fixer::YamlSymbolIndex::createFromFile(Input);
>>> - } else {
>>> - // If we don't have any input file, look in the directory of the
>>> first
>>> - // file and its parents.
>>> - SmallString<128> AbsolutePath(tooling::getAbsolutePath(FilePath));
>>> - StringRef Directory = llvm::sys::path::parent_path(AbsolutePath);
>>> - DB = include_fixer::YamlSymbolIndex::createFromDirectory(
>>> - Directory, "find_all_symbols_db.yaml");
>>> - }
>>> + auto CreateYamlIdx = [=]() ->
>>> std::unique_ptr<include_fixer::SymbolIndex> {
>>> + llvm::ErrorOr<std::unique_ptr<include_fixer::YamlSymbolIndex>> DB(
>>> + nullptr);
>>> + if (!Input.empty()) {
>>> + DB = include_fixer::YamlSymbolIndex::createFromFile(Input);
>>> + } else {
>>> + // If we don't have any input file, look in the directory of the
>>> + // first
>>> + // file and its parents.
>>> + SmallString<128>
>>> AbsolutePath(tooling::getAbsolutePath(FilePath));
>>> + StringRef Directory = llvm::sys::path::parent_path(AbsolutePath);
>>> + DB = include_fixer::YamlSymbolIndex::createFromDirectory(
>>> + Directory, "find_all_symbols_db.yaml");
>>> + }
>>>
>>> - if (!DB) {
>>> - llvm::errs() << "Couldn't find YAML db: " <<
>>> DB.getError().message()
>>> - << '\n';
>>> - return nullptr;
>>> - }
>>> + if (!DB) {
>>> + llvm::errs() << "Couldn't find YAML db: " <<
>>> DB.getError().message()
>>> + << '\n';
>>> + return nullptr;
>>> + }
>>> + return std::move(*DB);
>>> + };
>>>
>>> - SymbolIndexMgr->addSymbolIndex(std::move(*DB));
>>> + SymbolIndexMgr->addSymbolIndex(std::move(CreateYamlIdx));
>>> break;
>>> }
>>> }
>>>
>>> Modified:
>>> clang-tools-extra/trunk/unittests/include-fixer/IncludeFixerTest.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/include-fixer/IncludeFixerTest.cpp?rev=291446&r1=291445&r2=291446&view=diff
>>>
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/unittests/include-fixer/IncludeFixerTest.cpp
>>> (original)
>>> +++ clang-tools-extra/trunk/unittests/include-fixer/IncludeFixerTest.cpp
>>> Mon Jan 9 09:18:28 2017
>>> @@ -85,8 +85,9 @@ static std::string runIncludeFixer(
>>> 1, {}),
>>> };
>>> auto SymbolIndexMgr =
>>> llvm::make_unique<include_fixer::SymbolIndexManager>();
>>> - SymbolIndexMgr->addSymbolIndex(
>>> - llvm::make_unique<include_fixer::InMemorySymbolIndex>(Symbols));
>>> + SymbolIndexMgr->addSymbolIndex([=]() {
>>> + return
>>> llvm::make_unique<include_fixer::InMemorySymbolIndex>(Symbols);
>>> + });
>>>
>>> std::vector<IncludeFixerContext> FixerContexts;
>>> IncludeFixerActionFactory Factory(*SymbolIndexMgr, FixerContexts,
>>> "llvm");
>>>
>>>
>>> _______________________________________________
>>> cfe-commits mailing list
>>> cfe-commits at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>>>
>>
>>
>> --
>>
>> -Bill Seurer
>>
>
--
-Bill Seurer
More information about the cfe-commits
mailing list