[Openmp-dev] Task-based application aborting in kmp_alloc.cpp

Mon Jan 4 04:27:22 PST 2021

Dear devs,

Happy new year to all of you!

I am seeing the following assert triggering in the Clang OpenMP runtime 
when running an application using OpenMP tasks and taskloops:

Assertion failure at kmp_alloc.cpp(2012): size > 128 * 64.
OMP: Error #13: Assertion failure at kmp_alloc.cpp(2012).

backtrace from DDT:

#19 __kmp_launch_worker (thr=0xa24500) at 
src/clang/llvm-project/openmp/runtime/src/z_Linux_util.cpp:590 (at 
0x0000155554d589ec)
#18 __kmp_launch_thread (this_thr=this_thr at entry=0xa24500) at 
src/clang/llvm-project/openmp/runtime/src/kmp_runtime.cpp:5742 (at 
0x0000155554cf03e2)
#17 __kmp_fork_barrier (gtid=gtid at entry=26, tid=tid at entry=-2) at 
src/clang/llvm-project/openmp/runtime/src/kmp_barrier.cpp:1979 (at 
0x0000155554d25f0c)
#16 __kmp_hyper_barrier_release (bt=bt at entry=bs_forkjoin_barrier, 
this_thr=this_thr at entry=0xa24500, gtid=gtid at entry=26, tid=tid at entry=-2, 
propagate_icvs=propagate_icvs at entry=1, 
itt_sync_obj=itt_sync_obj at entry=0x0) at 
src/clang/llvm-project/openmp/runtime/src/kmp_barrier.cpp:672 (at 
0x0000155554d1c15e)
#15 kmp_flag_64<false, true>::wait (itt_sync_obj=0x0, final_spin=1, 
this_thr=0xa24500, this=0x15514afdba10) at 
src/clang/llvm-project/openmp/runtime/src/kmp_wait_release.h:899 (at 
0x0000155554d1c15e)
#14 __kmp_wait_template<kmp_flag_64<>, true, false, true>(kmp_info_t *, 
kmp_flag_64<false, true> *, void *) (this_thr=this_thr at entry=0xa24500, 
flag=flag at entry=0x15514afdba10, itt_sync_obj=itt_sync_obj at entry=0x0) at 
src/clang/llvm-project/openmp/runtime/src/kmp_wait_release.h:345 (at 
0x0000155554d1bb29)
#13 kmp_flag_64<false, true>::execute_tasks (is_constrained=0, 
itt_sync_obj=0x0, thread_finished=0x15514afdb94c, final_spin=1, gtid=26, 
this_thr=0xa24500, this=0x15514afdba10) at 
src/clang/llvm-project/openmp/runtime/src/kmp_wait_release.h:892 (at 
0x0000155554d1bb29)
#12 __kmp_execute_tasks_64<false, true> (thread=thread at entry=0xa24500, 
gtid=gtid at entry=26, flag=flag at entry=0x15514afdba10, 
final_spin=final_spin at entry=1, 
thread_finished=thread_finished at entry=0x15514afdb94c, 
itt_sync_obj=itt_sync_obj at entry=0x0, is_constrained=0) at 
src/clang/llvm-project/openmp/runtime/src/kmp_tasking.cpp:3029 (at 
0x0000155554d176a9)
#11 __kmp_execute_tasks_template<kmp_flag_64<> > (is_constrained=0, 
itt_sync_obj=0x0, thread_finished=<optimized out>, final_spin=<optimized 
out>, flag=0x15514afdba10, gtid=26, thread=thread at entry=0xa24500) at 
/sw/hawk-rh8/hlrs/non-spack/compiler/gcc/10.2.0/include/c++/10.2.0/bits/atomic_base.h:420 
(at 0x0000155554d176a9)
#10 __kmp_invoke_task (gtid=gtid at entry=26, task=0xada040, 
current_task=current_task at entry=0x9b2280) at 
src/clang/llvm-project/openmp/runtime/src/kmp_tasking.cpp:1589 (at 
0x0000155554d10da7)
#9 __kmp_task_finish<false> (gtid=gtid at entry=26, 
task=task at entry=0xada040, resumed_task=resumed_task at entry=0x9b2280) at 
/sw/hawk-rh8/hlrs/non-spack/compiler/gcc/10.2.0/include/c++/10.2.0/bits/atomic_base.h:556 
(at 0x0000155554d108e7)
#8 __kmp_release_deps (gtid=gtid at entry=26, task=task at entry=0xad9f00) at 
src/clang/llvm-project/openmp/runtime/src/kmp_taskdeps.h:106 (at 
0x0000155554d0fa53)
#7 __kmp_dephash_free (h=<optimized out>, thread=0xa24500) at 
src/clang/llvm-project/openmp/runtime/src/kmp_taskdeps.h:80 (at 
0x0000155554d0fa53)
#6 __kmp_dephash_free_entries (h=<optimized out>, thread=0xa24500) at 
src/clang/llvm-project/openmp/runtime/src/kmp_taskdeps.h:62 (at 
0x0000155554d0fa53)
#5 __kmp_depnode_list_free (list=0x15513affd580, thread=0xa24500) at 
src/clang/llvm-project/openmp/runtime/src/kmp_taskdeps.h:47 (at 
0x0000155554d0fa53)
#4 ___kmp_fast_free (this_thr=this_thr at entry=0xa24500, 
ptr=ptr at entry=0x15513affd580, _file_=_file_ at entry=0x155554d8b710 
"src/clang/llvm-project/openmp/runtime/src/kmp_taskdeps.h", 
_line_=_line_ at entry=47) at 
src/clang/llvm-project/openmp/runtime/src/kmp_alloc.cpp:2012 (at 
0x0000155554cc9098)
#3 __kmp_debug_assert (msg=msg at entry=0x155554d7e46c "size > 128 * 64", 
file=0x155554d7d087 "kmp_alloc.cpp", file at entry=0x155554d7d038 
"src/clang/llvm-project/openmp/runtime/src/kmp_alloc.cpp", 
line=line at entry=2012) at 
src/clang/llvm-project/openmp/runtime/src/kmp_debug.cpp:74 (at 
0x0000155554ce2b55)
#2 __kmp_fatal (message=...) at 
src/clang/llvm-project/openmp/runtime/src/kmp_i18n.cpp:868 (at 
0x0000155554ce77e0)
#1 __kmp_abort_process () at 
src/clang/llvm-project/openmp/runtime/src/kmp_runtime.cpp:444 (at 
0x0000155554cf1abc)
#0 abort () from /lib64/libc.so.6 (at 0x00001555546e2bce)

Any idea what might cause this? I am having a hard time reproducing this 
in a small example so I'd appreciate if someone could hint at what might 
trigger this assertion. This error is reproducible in my C++ port of the 
NPB BT-MZ benchmark using OpenMP tasks (both tied and untied, no 
detached tasks or offloading), starting from class E (class D runs fine 
but is too small for my experiments). I have removed all stack-based 
array allocations to rule out stack overflows inside tasks.

This is llvmorg-12-init-15664-gba82c0b315 (pulled last night) compiled 
with GCC 10.2.0 on an AMD EPYC system running 32 threads per process (16 
processes in total using MPI).

I can compile and run the application with no issues with GCC 10.2.0. I 
can reproduce the assert if imposing Clang's libgomp after compiling the 
application with GCC. I used GCC's ASAN both with GCC's and Clang's 
libgomp on the application and didn't see any errors. Any advice on what 
else I could try?

Many thanks in advance,
Joseph
-- 
Dipl.-Inf. Joseph Schuchart
High Performance Computing Center Stuttgart (HLRS)
Nobelstr. 19
D-70569 Stuttgart

Tel.: +49(0)711-68565890
Fax: +49(0)711-6856832
E-Mail: schuchart at hlrs.de