[llvm-dev] Orc JIT v2 breaks OpenMP in 11.x branch?
Geoff Levner via llvm-dev
llvm-dev at lists.llvm.org
Mon Aug 10 01:34:31 PDT 2020
Thanks, Lang.
I think I have identified the commit that breaks our application,
although I haven't quite finished verifying (it takes a while to
compile and test a debug version of LLVM). It's commit
1a4fb2edcb908d6c9141036d29b46a347b1b6f18, dated June 1:
-----
Summary: This changes Clang's generation of OpenMP runtime functions
to use the types and functions defined in OpenMPKinds and
OpenMPConstants. New OpenMP runtime function information should now be
added to OMPKinds.def. This patch also changed the definitions of
__kmpc_push_num_teams and __kmpc_copyprivate to match those found in
the runtime.
-----
The review of the changes is here: https://reviews.llvm.org/D80222
I am CCing the author.
Geoff
On Mon, Aug 10, 2020 at 6:54 AM Lang Hames <lhames at gmail.com> wrote:
>
> Hi Geoff,
>
> Nothing in that backtrace leaps out at me. Based on the stack trace and description my first guess would be a clang misconfiguration rather than a JIT bug.
>
> How is that clang invocation being made? Is it from inside a callback from ORC, or is it before you add your module to the JIT?
>
> -- Lang.
>
> On Mon, Aug 3, 2020 at 5:41 AM Geoff Levner <glevner at gmail.com> wrote:
>>
>> Here, by the way, is the stack trace leading to the failed assertion.
>> Note #9: clang::CodeGen::CGOpenMPRuntime::emitForStaticFinish().
>>
>> #0 0x00007fb70fa44870 in abort () from /usr/lib64/libc.so.6
>> #1 0x00007fb70fa3c0e6 in __assert_fail_base () from /usr/lib64/libc.so.6
>> #2 0x00007fb70fa3c192 in __assert_fail () from /usr/lib64/libc.so.6
>> #3 0x00007fb715ca1a68 in llvm::CallInst::init (this=0x51f6e10,
>> FTy=0x51fd280, Func=0x51f6d28, Args=..., Bundles=...,
>> NameStr=...) at
>> /u/geoff/llvmrepo/llvm-project/llvm/lib/IR/Instructions.cpp:453
>> #4 0x00007fb7116a9e4e in llvm::CallInst::CallInst (this=0x51f6e10,
>> Ty=0x51fd280, Func=0x51f6d28, Args=..., Bundles=...,
>> NameStr=..., InsertBefore=0x0) at
>> /u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/Instructions.h:1681
>> #5 0x00007fb7116a9cf2 in llvm::CallInst::Create (Ty=0x51fd280,
>> Func=0x51f6d28, Args=..., Bundles=..., NameStr=...,
>> InsertBefore=0x0) at
>> /u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/Instructions.h:1489
>> #6 0x00007fb71171f1e0 in llvm::IRBuilderBase::CreateCall
>> (this=0x7fffd4fdaac8, FTy=0x51fd280, Callee=0x51f6d28, Args=...,
>> OpBundles=..., Name=..., FPMathTag=0x0) at
>> /u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/IRBuilder.h:2338
>> #7 0x00007fb71171f2d5 in llvm::IRBuilderBase::CreateCall
>> (this=0x7fffd4fdaac8, Callee=..., Args=..., OpBundles=..., Name=...,
>> FPMathTag=0x0) at
>> /u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/IRBuilder.h:2356
>> #8 0x00007fb7117124eb in
>> clang::CodeGen::CodeGenFunction::EmitRuntimeCall (this=0x7fffd4fda9d0,
>> callee=..., args=...,
>> name=...) at
>> /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGCall.cpp:4051
>> #9 0x00007fb7118b601f in
>> clang::CodeGen::CGOpenMPRuntime::emitForStaticFinish (this=0x4614460,
>> CGF=..., Loc=...,
>> DKind=llvm::omp::OMPD_for) at
>> /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:2875
>> #10 0x00007fb71197c294 in
>> clang::CodeGen::CodeGenFunction::<lambda(clang::CodeGen::CodeGenFunction&)>::operator()(clang::CodeGen::CodeGenFunction
>> &) const (__closure=0x7fffd4fd8340, CGF=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3048
>> #11 0x00007fb71199754d in
>> llvm::function_ref<void(clang::CodeGen::CodeGenFunction&)>::callback_fn<clang::CodeGen::CodeGenFunction::EmitOMPWorksharingLoop(const
>> clang::OMPLoopDirective&, clang::Expr*, const CodeGenLoopBoundsTy&,
>> const CodeGenDispatchBoundsTy&)::<lambda(clang::CodeGen::CodeGenFunction&)>
>> >(intptr_t, clang::CodeGen::CodeGenFunction &)
>> (callable=140736766772032,
>> params#0=...) at
>> /u/geoff/llvmrepo/llvm-project/llvm/include/llvm/ADT/STLExtras.h:186
>> #12 0x00007fb71199ef91 in llvm::function_ref<void
>> (clang::CodeGen::CodeGenFunction&)>::operator()(clang::CodeGen::CodeGenFunction&)
>> const (this=0x7fffd4fd7b40, params#0=...) at
>> /u/geoff/llvmrepo/llvm-project/llvm/include/llvm/ADT/STLExtras.h:203
>> #13 0x00007fb71199d79a in
>> clang::CodeGen::CodeGenFunction::OpenMPCancelExitStack::emitExit(clang::CodeGen::CodeGenFunction&,
>> llvm::omp::Directive, llvm::function_ref<void
>> (clang::CodeGen::CodeGenFunction&)>) (this=0x7fffd4fdb678, CGF=...,
>> Kind=llvm::omp::OMPD_for, CodeGen=...) at
>> /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenFunction.h:1361
>> #14 0x00007fb71197d41e in
>> clang::CodeGen::CodeGenFunction::EmitOMPWorksharingLoop(clang::OMPLoopDirective
>> const&, clang::Expr*,
>> llvm::function_ref<std::pair<clang::CodeGen::LValue,
>> clang::CodeGen::LValue> (clang::CodeGen::CodeGenFunction&,
>> clang::OMPExecutableDirective const&)> const&,
>> llvm::function_ref<std::pair<llvm::Value*, llvm::Value*>
>> (clang::CodeGen::CodeGenFunction&, clang::OMPExecutableDirective
>> const&, clang::CodeGen::Address, clang::CodeGen::Address)> const&)
>> (this=0x7fffd4fda9d0, S=...,
>> EUB=0x4cb7250, CodeGenLoopBounds=..., CGDispatchBounds=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3050
>> #15 0x00007fb71197f746 in emitWorksharingDirective (CGF=..., S=...,
>> HasCancel=false)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3335
>> #16 0x00007fb71197f7b0 in
>> clang::CodeGen::CodeGenFunction::<lambda(clang::CodeGen::CodeGenFunction&,
>> clang::CodeGen::PrePostActionTy&)>::operator()(clang::CodeGen::CodeGenFunction
>> &, clang::CodeGen::PrePostActionTy &) const (__closure=0x7fffd4fd8940,
>> CGF=...) at
>> /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3346
>> #17 0x00007fb71199771f in
>> clang::CodeGen::RegionCodeGenTy::CallbackFn<clang::CodeGen::CodeGenFunction::EmitOMPForDirective(const
>> clang::OMPForDirective&)::<lambda(clang::CodeGen::CodeGenFunction&,
>> clang::CodeGen::PrePostActionTy&)> >(intptr_t,
>> clang::CodeGen::CodeGenFunction &, clang::CodeGen::PrePostActionTy &)
>> (CodeGen=140736766773568, CGF=..., Action=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.h:80
>> #18 0x00007fb7118a8787 in clang::CodeGen::RegionCodeGenTy::operator()
>> (this=0x4d7e948, CGF=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:595
>> #19 0x00007fb7118ab598 in (anonymous
>> namespace)::CGOpenMPRegionInfo::EmitBody (this=0x4d7e8d0, CGF=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1041
>> #20 0x00007fb7118cce36 in
>> clang::CodeGen::CGOpenMPRuntime::emitInlinedDirective (this=0x4614460,
>> CGF=...,
>> InnerKind=llvm::omp::OMPD_for, CodeGen=..., HasCancel=false)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:6222
>> #21 0x00007fb71197f8bd in
>> clang::CodeGen::CodeGenFunction::EmitOMPForDirective
>> (this=0x7fffd4fda9d0, S=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3353
>> #22 0x00007fb711952aeb in clang::CodeGen::CodeGenFunction::EmitStmt
>> (this=0x7fffd4fda9d0, S=0x46c3b18, Attrs=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:202
>> #23 0x00007fb7119537d3 in
>> clang::CodeGen::CodeGenFunction::EmitCompoundStmtWithoutScope
>> (this=0x7fffd4fda9d0, S=...,
>> GetLast=false, AggSlot=...) at
>> /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:453
>> #24 0x00007fb7119534b6 in
>> clang::CodeGen::CodeGenFunction::EmitCompoundStmt
>> (this=0x7fffd4fda9d0, S=..., GetLast=false,
>> AggSlot=...) at
>> /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:404
>> #25 0x00007fb7119532b8 in
>> clang::CodeGen::CodeGenFunction::EmitSimpleStmt (this=0x7fffd4fda9d0,
>> S=0x46c4750)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:377
>> #26 0x00007fb711952547 in clang::CodeGen::CodeGenFunction::EmitStmt
>> (this=0x7fffd4fda9d0, S=0x46c4750, Attrs=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:53
>> #27 0x00007fb7119757e9 in
>> clang::CodeGen::CodeGenFunction::<lambda(clang::CodeGen::CodeGenFunction&,
>> clang::CodeGen::PrePostActionTy&)>::operator()(clang::CodeGen::CodeGenFunction
>> &, clang::CodeGen::PrePostActionTy &) const (__closure=0x7fffd4fdc360,
>> CGF=..., Action=...) at
>> /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:1733
>> #28 0x00007fb711996ecf in
>> clang::CodeGen::RegionCodeGenTy::CallbackFn<clang::CodeGen::CodeGenFunction::EmitOMPParallelDirective(const
>> clang::OMPParallelDirective&)::<lambda(clang::CodeGen::CodeGenFunction&,
>> clang::CodeGen::PrePostActionTy&)> >(intptr_t,
>> clang::CodeGen::CodeGenFunction &, clang::CodeGen::PrePostActionTy &)
>> (CodeGen=140736766788448, CGF=..., Action=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.h:80
>> #29 0x00007fb7118a8787 in clang::CodeGen::RegionCodeGenTy::operator()
>> (this=0x7fffd4fda988, CGF=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:595
>> #30 0x00007fb7118ab598 in (anonymous
>> namespace)::CGOpenMPRegionInfo::EmitBody (this=0x7fffd4fda910,
>> CGF=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1041
>> #31 0x00007fb71196e1e1 in
>> clang::CodeGen::CodeGenFunction::GenerateOpenMPCapturedStmtFunction
>> (this=0x7fffd4fda9d0, S=...,
>> Loc=...) at
>> /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:615
>> #32 0x00007fb7118ace11 in emitParallelOrTeamsOutlinedFunction
>> (CGM=..., D=..., CS=0x4a693a8, ThreadIDVar=0x49fe398,
>> InnermostKind=llvm::omp::OMPD_parallel, OutlinedHelperName=..., CodeGen=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1289
>> #33 0x00007fb7118acec7 in
>> clang::CodeGen::CGOpenMPRuntime::emitParallelOutlinedFunction
>> (this=0x4614460, D=...,
>> ThreadIDVar=0x49fe398, InnermostKind=llvm::omp::OMPD_parallel, CodeGen=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1297
>> #34 0x00007fb7119745c9 in emitCommonOMPParallelDirective (CGF=...,
>> S=..., InnermostKind=llvm::omp::OMPD_parallel, CodeGen=...,
>> CodeGenBoundParameters=...) at
>> /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:1530
>> #35 0x00007fb711975bd9 in
>> clang::CodeGen::CodeGenFunction::EmitOMPParallelDirective
>> (this=0x7fffd4fdc7e0, S=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:1740
>> #36 0x00007fb711952aa5 in clang::CodeGen::CodeGenFunction::EmitStmt
>> (this=0x7fffd4fdc7e0, S=0x4a695a8, Attrs=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:196
>> #37 0x00007fb7119537d3 in
>> clang::CodeGen::CodeGenFunction::EmitCompoundStmtWithoutScope
>> (this=0x7fffd4fdc7e0, S=...,
>> GetLast=false, AggSlot=...) at
>> /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:453
>> #38 0x00007fb7119d69ec in
>> clang::CodeGen::CodeGenFunction::EmitFunctionBody
>> (this=0x7fffd4fdc7e0, Body=0x4a695e0)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenFunction.cpp:1147
>> #39 0x00007fb7119d75bc in
>> clang::CodeGen::CodeGenFunction::GenerateCode (this=0x7fffd4fdc7e0,
>> GD=..., Fn=0x4699718, FnInfo=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenFunction.cpp:1312
>> #40 0x00007fb7119facdf in
>> clang::CodeGen::CodeGenModule::EmitGlobalFunctionDefinition
>> (this=0x4b103a0, GD=..., GV=0x4699718)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:4528
>> #41 0x00007fb7119f3f73 in
>> clang::CodeGen::CodeGenModule::EmitGlobalDefinition (this=0x4b103a0,
>> GD=..., GV=0x4699718)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:2890
>> #42 0x00007fb7119f0ec2 in clang::CodeGen::CodeGenModule::EmitDeferred
>> (this=0x4b103a0)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:2208
>> #43 0x00007fb7119e8448 in clang::CodeGen::CodeGenModule::Release
>> (this=0x4b103a0)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:399
>> ---Type <return> to continue, or q <return> to quit---
>> #44 0x00007fb71166b523 in (anonymous
>> namespace)::CodeGeneratorImpl::HandleTranslationUnit (this=0x4ceb9c0,
>> Ctx=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/ModuleBuilder.cpp:267
>> #45 0x00007fb71163ff48 in
>> clang::BackendConsumer::HandleTranslationUnit (this=0x49fc8d0, C=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenAction.cpp:283
>> #46 0x00007fb711dbf5f5 in
>> clang::MultiplexConsumer::HandleTranslationUnit (this=0x2cc6290,
>> Ctx=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/MultiplexConsumer.cpp:292
>> #47 0x00007fb71203fb15 in clang::ParseAST (S=..., PrintStats=false,
>> SkipFunctionBodies=false)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/Parse/ParseAST.cpp:171
>> #48 0x00007fb711d7fee7 in clang::ASTFrontendAction::ExecuteAction
>> (this=0x41bcf20)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1058
>> #49 0x00007fb71163b730 in clang::CodeGenAction::ExecuteAction (this=0x41bcf20)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenAction.cpp:1184
>> #50 0x00007fb711d80137 in clang::WrapperFrontendAction::ExecuteAction
>> (this=0x4cb8cd0)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1089
>> #51 0x00007fb711d7f83f in clang::FrontendAction::Execute (this=0x4cb8cd0)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/FrontendAction.cpp:950
>> #52 0x00007fb711d0d96f in clang::CompilerInstance::ExecuteAction
>> (this=0x7fffd4fe1ed0, Act=...)
>> at /u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/CompilerInstance.cpp:984
>>
>> On Mon, Aug 3, 2020 at 2:28 PM Geoff Levner <glevner at gmail.com> wrote:
>> >
>> > Greetings, Lang and other JITters,
>> >
>> > Last week I moved our Orc v2-based app from top-of-tree to the new
>> > 11.x branch, and it no longer handles C++ code containing OpenMP
>> > directives correctly.
>> >
>> > More specifically, if I JIT compile a function containing OpenMP
>> > parallel/for pragmas, using a release version of LLVM, the code
>> > compiles and seems to work, but if I try to write the module's bitcode
>> > to file, I get the following error message:
>> >
>> > Invalid record (Producer: 'LLVM11.0.0' Reader: 'LLVM 11.0.0').
>> >
>> > If I use a debug version of LLVM, compilation fails with a broken assertion:
>> >
>> > .../llvm/lib/IR/Instructions.cpp:455: void
>> > llvm::CallInst::init(llvm::FunctionType*, llvm::Va
>> > lue*, llvm::ArrayRef<llvm::Value*>,
>> > llvm::ArrayRef<llvm::OperandBundleDefT<llvm::Value*> >, const
>> > llvm::Twine&): Assertion `(i >
>> > = FTy->getNumParams() || FTy->getParamType(i) == Args[i]->getType())
>> > && "Calling a function with a bad signature!"' failed.
>> >
>> > The same code compiles correctly using clang from the command line. It
>> > also works fine using Orc v2 in the master branch of LLVM from the end
>> > of May.
>> >
>> > The problems are way too deep in the LLVM code for me to make any
>> > sense of them... Does anybody else have a clue what might have
>> > happened to Orc and/or OpenMP in the past couple months to trigger
>> > this?
>> >
>> > Geoff
More information about the llvm-dev
mailing list