[llvm-dev] Opt plugin linkage
Viktor Was BSc via llvm-dev
llvm-dev at lists.llvm.org
Sat Apr 20 07:24:20 PDT 2019
Hi Philip,
unfortunately in the case of ExecutionEngine it doesn't have the symbols
needed. I was hoping to be able to do this by linking ExecutionEngine
privately (target_link_libraries( pluginName PRIVATE LLVMExecutionEngine
)), for testing purposes at least. How does the plugin loading work that
the symbols are seemingly omitted/stripped/inaccessible?
Cheers,
Viktor
On 4/19/19 12:13 PM, Philip Pfaffe wrote:
> Hi Viktor,
>
> Though, I feel like this should also be possible with statically
> linking the ExecutionEngine into the plugin shared object.
>
> Unfortunately it won't. In general, you don't link any LLVM libraries
> into opt plugins because opt already has all the symbols you need. If
> you were to link in the libraries, all sorts of problems arise, most
> of them because we have lot's of global state everywhere. With dynamic
> linking, most of the problems can be avoided because the dependencies
> won't be loaded twice.
>
> Cheers,
> Philip
>
>
> I don't quite understand how the concrete Interpreter/MCJIT
> linking via llvm/ExecutionEngine/Interpreter.h and
> llvm/ExecutionEngine/MCJIT.h works. So far I was only using
> ExecutionEngine.h without Interpreter or MCJIT, so that could have
> been the problem why static linking didn't work.
>
> After playing around with Zhangs solution I've found out that I
> get the same undefined symbol to llvm::EngineBuilder error when
> omitting the MCJIT header and lib (since I actually only need an
> interpreter), my guess is that this is due to Interpreter.h
> missing the getenv trick used in MCJIT.h to prevent it being
> optimized away by the compiler? I might take a closer look at this
> at some point, right now I'm just happy to finally be able to work
> on my opt pass.
>
>
> Thanks,
>
> Viktor
>
>
> On 4/18/19 7:38 PM, John Brawn wrote:
>>
>> The fundamental problem here is that opt doesn’t use
>> ExecutionEngine (because it has no need to), so trying
>>
>> to use ExecutionEngine (or any other bit of llvm that opt doesn’t
>> use for that matter) in an opt plugin isn’t
>>
>> going to work.
>>
>> The solution I’d go with would be to build llvm with shared
>> libraries (use –DBUILD_SHARED_LIBS=ON on the
>>
>> cmake command) then link the plugin against ExecutionEngine. That
>> shouldn’t require any changes anywhere
>>
>> (when I gave it a quick try it seemed to work).
>>
>> John
>>
>> *From:*llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On
>> Behalf Of *Zhang via llvm-dev
>> *Sent:* 16 April 2019 22:04
>> *To:* Viktor Was BSc; llvm-dev
>> *Subject:* Re: [llvm-dev] Opt plugin linkage
>>
>> Just another follow-up, seems like ExecutionEngine is still not
>> usable without editing CMakeLists, on my setup I had to edit
>> CMakeLists.txt as well to get the example working.
>>
>> Below is the modifications I made:
>>
>> Modified Hello.cpp for testing:
>>
>> ```
>>
>> bool runOnFunction(Function &F) override {
>>
>> ++HelloCounter;
>>
>> std::string errStr;
>>
>> EngineBuilder eb;
>>
>> eb.setErrorStr(&errStr);
>>
>> eb.setEngineKind(EngineKind::Kind::JIT);
>>
>> ExecutionEngine* ee=eb.create();
>>
>> errs()<<"ExecutionEngine:"<<ee<<" Error:"<<errStr<<"\n";
>>
>> delete ee;
>>
>> errs() << "Hello: ";
>>
>> errs().write_escaped(F.getName()) << '\n';
>>
>> return false;
>>
>> }
>>
>> ```
>>
>> Modified opt's CMakeLists.txt:
>>
>> ```
>>
>> set(LLVM_LINK_COMPONENTS
>>
>> ${LLVM_TARGETS_TO_BUILD}
>>
>> AggressiveInstCombine
>>
>> Analysis
>>
>> BitWriter
>>
>> CodeGen
>>
>> Core
>>
>> MC
>>
>> MCJIT
>>
>> Object
>>
>> OrcJIT
>>
>> Interpreter
>>
>> RuntimeDyld
>>
>> Coroutines
>>
>> IPO
>>
>> IRReader
>>
>> InstCombine
>>
>> Instrumentation
>>
>> MC
>>
>> ObjCARCOpts
>>
>> ScalarOpts
>>
>> Support
>>
>> Target
>>
>> TransformUtils
>>
>> Vectorize
>>
>> Passes
>>
>> ExecutionEngine
>>
>> )
>>
>> # Support plugins.
>>
>> set(LLVM_NO_DEAD_STRIP 1)
>>
>> add_llvm_tool(opt
>>
>> AnalysisWrappers.cpp
>>
>> BreakpointPrinter.cpp
>>
>> Debugify.cpp
>>
>> GraphPrinters.cpp
>>
>> NewPMDriver.cpp
>>
>> PassPrinters.cpp
>>
>> PrintSCC.cpp
>>
>> opt.cpp
>>
>> DEPENDS
>>
>> intrinsics_gen
>>
>> )
>>
>> export_executable_symbols(opt)
>>
>> if(WITH_POLLY AND LINK_POLLY_INTO_TOOLS)
>>
>> target_link_libraries(opt PRIVATE Polly)
>>
>> endif(WITH_POLLY AND LINK_POLLY_INTO_TOOLS)
>>
>> target_link_libraries(opt PUBLIC LLVMExecutionEngine)
>>
>> ```
>>
>> Modified opt's LLVMBuild.txt:
>>
>> ```
>>
>> ;===- ./tools/opt/LLVMBuild.txt
>> --------------------------------*- Conf -*--===;
>>
>> ;
>>
>> ; The LLVM Compiler Infrastructure
>>
>> ;
>>
>> ; This file is distributed under the University of Illinois Open
>> Source
>>
>> ; License. See LICENSE.TXT for details.
>>
>> ;
>>
>> ;===------------------------------------------------------------------------===;
>>
>> ;
>>
>> ; This is an LLVMBuild description file for the components in
>> this subdirectory.
>>
>> ;
>>
>> ; For more information on the LLVMBuild system, please see:
>>
>> ;
>>
>> ; http://llvm.org/docs/LLVMBuild.html
>>
>> ;
>>
>> ;===------------------------------------------------------------------------===;
>>
>> [component_0]
>>
>> type = Tool
>>
>> name = opt
>>
>> parent = Tools
>>
>> required_libraries =
>>
>> AsmParser
>>
>> BitReader
>>
>> BitWriter
>>
>> CodeGen
>>
>> IRReader
>>
>> IPO
>>
>> Instrumentation
>>
>> Scalar
>>
>> ObjCARC
>>
>> Passes
>>
>> ExecutionEngine
>>
>> Interpreter
>>
>> MCJIT
>>
>> Native
>>
>> NativeCodeGen
>>
>> all-targets
>>
>> ```
>>
>> On top of that I also added these lines to the beginning of main
>> function in opt.cpp to force linking ExecutionEngine:
>>
>> ```
>>
>> if(argc==-1){
>>
>> EngineBuilder eb;
>>
>> ExecutionEngine* ee=eb.create();
>>
>> delete ee;
>>
>> }
>>
>> ```
>>
>> as well as force linking MCJIT and Interpreter by including
>> headers in opt:
>>
>> ```
>>
>> #include "llvm/ExecutionEngine/ExecutionEngine.h"
>>
>> #include "llvm/ExecutionEngine/Interpreter.h"
>>
>> #include "llvm/ExecutionEngine/MCJIT.h"
>>
>> ```
>>
>> Moving cl::parseCommandLineOptions doesn't seem to be related to
>> the issue, my previous assumption was wrong.
>>
>> Those modifications works at least on my setup with the following
>> output:
>>
>> ```
>>
>> λ : >>> bin/opt -load lib/LLVMHello.dylib hw.ll -hello
>>
>> WARNING: You're attempting to print out a bitcode file.
>>
>> This is inadvisable as it may cause display problems. If
>>
>> you REALLY want to taste LLVM bitcode first-hand, you
>>
>> can force output with the `-f' option.
>>
>> Assertion failed: (M && "Module is null?"), function Init, file
>> /Users/naville/Development/Hikari/lib/ExecutionEngine/ExecutionEngine.cpp,
>> line 80.
>>
>> ```
>>
>> Which is due to i didn't initialize the EEBuilder properly, but
>> at least the pass now loads and executes properly
>>
>> Zhang
>>
>> ------------------ Original ------------------
>>
>> *From: * "Zhang via llvm-dev"<llvm-dev at lists.llvm.org>
>> <mailto:llvm-dev at lists.llvm.org>;
>>
>> *Date: * Wed, Apr 17, 2019 04:35 AM
>>
>> *To: * "Viktor Was BSc"<gs15m015 at technikum-wien.at>
>> <mailto:gs15m015 at technikum-wien.at>;
>> "llvm-dev"<llvm-dev at lists.llvm.org>
>> <mailto:llvm-dev at lists.llvm.org>;
>>
>> *Subject: * Re: [llvm-dev] Opt plugin linkage
>>
>> Hey:
>>
>> I spent sometime debugging this, it seems like editing
>> ``llvm/tools/opt.cpp`` and move
>> ``cl::ParseCommandLineOptions(argc, argv,
>>
>> "llvm .bc -> .bc modular optimizer and analysis
>> printer\n");`` to the beginning of main() solved it for me. I'm
>> not sure if this is a bug on LLVM side
>>
>> Zhang
>>
>> ------------------ Original ------------------
>>
>> *From: * "Viktor Was BSc via llvm-dev"<llvm-dev at lists.llvm.org>
>> <mailto:llvm-dev at lists.llvm.org>;
>>
>> *Date: * Wed, Apr 17, 2019 03:09 AM
>>
>> *To: * "llvm-dev"<llvm-dev at lists.llvm.org>
>> <mailto:llvm-dev at lists.llvm.org>;
>>
>> *Subject: * Re: [llvm-dev] Opt plugin linkage
>>
>> How come the hello pass example is so totally useless as a
>> starting point? Why is this not using any library/compontent
>> which could conflict with opt or clang and showing how to handle
>> this? I have no clue as to how I have to setup llvm to get this
>> to work or why it doesn't work in the first place with the setup
>> described in "Getting started" and "writing an llvm pass" pages etc.
>>
>> Also there is basically no documentation for the custom cmake
>> commands.
>>
>> Can please somebody help me with this issue? How do I get
>> dynamically loaded llvm pass plugins to work? Am I the only one
>> ever having this issue?
>>
>> Thanks
>>
>> Viktor
>>
>> On Apr 16, 2019, at 05:38, Viktor Was BSc via llvm-dev
>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>
>> Hi,
>>
>> I have a dynamically loaded llvm pass built in-tree with ninja
>> (generated with cmake, basically a copy of the hallo pass plugin,
>> linux, llvm/clang version 6.0.1).
>>
>> It uses the ExecutionEngine.
>>
>> Building it without linking against LLVMExecutionEngine library
>> results in an undefined symbol to the vtable of the EngineBuilder
>> when loaded to opt. Linking the plugin with LLVMExecutionEngine
>> results in the pass simply not being executable with giving "opt:
>> Unkown command line argument '-passArg'."
>>
>> For a minimal example add set(LLVM_LINK_COMPONENTS Core) to the
>> CMakeLists.txt of the Hello llvm pass.
>>
>> There is no error or warning at any point when linking or loading
>> a plugin linked against some libs.
>>
>> How do I find out which llvm libs I can't link with a dynamically
>> loaded plugin?
>>
>> How can I use the EngineBuilder in my plugin with proper symbol
>> resolution?
>>
>> For reproductivity:
>>
>> cmake -G "Sublime Text 2 - Ninja"
>> -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
>> -DLLVM_EXPORT_SYMBOLS_FOR_PLUGINS=ON -DLLVM_BUILD_TESTS=ON
>> -DLLVM_BUILD_EXAMPLES=ON
>> -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra"
>> -DLLVM_TOOL_CLANG_BUILD=ON -DLLVM_TOOL_CLANG_TOOLS_EXTRA=ON
>> -DLLVM_OPTIMIZED_TABLEGEN=ON -DCLANG_BUILD_EXAMPLES=ON
>> -DCLANG_PLUGIN_SUPPORT=ON
>>
>> Hope someone can help me out.
>>
>> Viktor
>>
>> ------------------------------------------------------------------------
>> LLVM Developers mailing list llvm-dev at lists.llvm.org
>> <mailto:llvm-dev at lists.llvm.org>
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190420/dcdc1b44/attachment.html>
More information about the llvm-dev
mailing list