[llvm-dev] Linux/ARM: Segfault issue when we build clang sources including __thread variable using -O2 flag

Geunsik Lim via llvm-dev llvm-dev at lists.llvm.org
Tue May 3 03:04:21 PDT 2016


In order to share more information, below is the machine instructions of
the "PAL_ThrowExceptionFromContext" function including __thread variables
from the CoreCLR's seh.cpp source.  I wonder why Clan/LLVM can not
guarantee the normal execution of the application code including __thread
variables in case of the aggressive optimization levels (e.g., -O2 and -O3)



========= source: __thread variable in the src/pal/src/exception/seh.cpp
===============================
 *
https://github.com/dotnet/coreclr/blob/08e1b2f05ba15efff366c4fd70ff475359022bb3/src/pal/src/exception/seh.cpp


171 --*/
172 VOID

173 PALAPI
174 PAL_ThrowExceptionFromContext(CONTEXT* context, PAL_SEHException* ex)
175 {
176     // We need to make a copy of the exception off stack, since the
"ex" is located in one     of the stack
177     // frames that will become obsolete by the
ThrowExceptionFromContextInternal and the Th    rowExceptionHelper
178     // could overwrite the "ex" object by stack e.g. when allocating
the low level exceptio    n object for "throw".
179     static __thread BYTE
threadLocalExceptionStorage[sizeof(PAL_SEHException)];
180     ThrowExceptionFromContextInternal(context, new
(threadLocalExceptionStorage) PAL_SEHExc    eption(*ex));
181 }

========  debug: -O0  <- Build Ok, Running Success
=============================================
3561791
3561792 0085bc14 <PAL_ThrowExceptionFromContext>:
3561793   85bc14:       b580            push    {r7, lr}
3561794   85bc16:       466f            mov     r7, sp
3561795   85bc18:       b08a            sub     sp, #40 ; 0x28
3561796   85bc1a:       460a            mov     r2, r1
3561797   85bc1c:       4603            mov     r3, r0
3561798   85bc1e:       9009            str     r0, [sp, #36]   ; 0x24
3561799   85bc20:       9108            str     r1, [sp, #32]
3561800   85bc22:       2000            movs    r0, #0
3561801   85bc24:       4601            mov     r1, r0
3561802   85bc26:       f8dd c024       ldr.w   ip, [sp, #36]   ; 0x24
3561803   85bc2a:       2800            cmp     r0, #0
3561804   85bc2c:       9207            str     r2, [sp, #28]
3561805   85bc2e:       9306            str     r3, [sp, #24]
3561806   85bc30:       f8cd c014       str.w   ip, [sp, #20]
3561807   85bc34:       9104            str     r1, [sp, #16]
3561808   85bc36:       d10d            bne.n   85bc54
<PAL_ThrowExceptionFromContext+0x40>
3561809   85bc38:       e7ff            b.n     85bc3a
<PAL_ThrowExceptionFromContext+0x26>
3561810   85bc3a:       9908            ldr     r1, [sp, #32]
3561811   85bc3c:       480a            ldr     r0, [pc, #40]   ; (85bc68
<PAL_ThrowExceptionFromContext+0x54>)
3561812   85bc3e:       4478            add     r0, pc
3561813   85bc40:       9103            str     r1, [sp, #12]
3561814   85bc42:       f7db cf96       blx     37b70 <_init+0xb34>
3561815   85bc46:       9002            str     r0, [sp, #8]
3561816   85bc48:       9903            ldr     r1, [sp, #12]
3561817   85bc4a:       f66d f1ef       bl      2c902c
<_ZN16PAL_SEHExceptionC2ERKS_>
3561818   85bc4e:       9802            ldr     r0, [sp, #8]
3561819   85bc50:       9004            str     r0, [sp, #16]
3561820   85bc52:       e7ff            b.n     85bc54
<PAL_ThrowExceptionFromContext+0x40>
3561821   85bc54:       9804            ldr     r0, [sp, #16]
3561822   85bc56:       9905            ldr     r1, [sp, #20]
3561823   85bc58:       9001            str     r0, [sp, #4]
3561824   85bc5a:       4608            mov     r0, r1
3561825   85bc5c:       9901            ldr     r1, [sp, #4]
3561826   85bc5e:       f0aa fa47       bl      9060f0
<ThrowExceptionFromContextInternal>
3561827   85bc62:       b00a            add     sp, #40 ; 0x28
3561828   85bc64:       bd80            pop     {r7, pc}
3561829   85bc66:       bf00            nop
3561830   85bc68:       00293806        .word   0x00293806
3561831



========== debug: -O1  <- Build Ok, Running Success
====================================

1955139
1955140 005103f0 <PAL_ThrowExceptionFromContext>:
1955141   5103f0:       e92d 48f0       stmdb   sp!, {r4, r5, r6, r7, fp,
lr}
1955142   5103f4:       af03            add     r7, sp, #12
1955143   5103f6:       4605            mov     r5, r0
1955144   5103f8:       4807            ldr     r0, [pc, #28]   ; (510418
<PAL_ThrowExceptionFromContext+0x28>)
1955145   5103fa:       460c            mov     r4, r1
1955146   5103fc:       4478            add     r0, pc
1955147   5103fe:       f727 e06e       blx     374dc <_init+0xba0>
1955148   510402:       4606            mov     r6, r0
1955149   510404:       4621            mov     r1, r4
1955150   510406:       f49f f8cf       bl      1af5a8
<_ZN16PAL_SEHExceptionC2ERKS_>
1955151   51040a:       4628            mov     r0, r5
1955152   51040c:       4631            mov     r1, r6
1955153   51040e:       e8bd 48f0       ldmia.w sp!, {r4, r5, r6, r7, fp,
lr}
1955154   510412:       f073 bb87       b.w     583b24
<ThrowExceptionFromContextInternal>
1955155   510416:       bf00            nop
1955156   510418:       00270038        .word   0x00270038
1955157

========== debug: -O2  <- Build Ok, Running Failed
======================================

2557238
2557239 006ba338 <PAL_ThrowExceptionFromContext>:
2557240   6ba338:       e92d 41f0       stmdb   sp!, {r4, r5, r6, r7, r8,
lr}
2557241   6ba33c:       af03            add     r7, sp, #12
2557242   6ba33e:       4680            mov     r8, r0
2557243   6ba340:       4812            ldr     r0, [pc, #72]   ; (6ba38c
<PAL_ThrowExceptionFromContext+0x54>)
2557244   6ba342:       460c            mov     r4, r1
2557245   6ba344:       4478            add     r0, pc
2557246   6ba346:       f577 e13a       blx     315bc <_init+0xca4>
2557247   6ba34a:       4606            mov     r6, r0
2557248   6ba34c:       f04f 31ff       mov.w   r1, #4294967295 ; 0xffffffff
2557249   6ba350:       6031            str     r1, [r6, #0]
2557250   6ba352:       f106 000c       add.w   r0, r6, #12
2557251   6ba356:       f104 010c       add.w   r1, r4, #12
2557252   6ba35a:       2250            movs    r2, #80 ; 0x50
2557253   6ba35c:       f106 0560       add.w   r5, r6, #96     ; 0x60
2557254   6ba360:       6070            str     r0, [r6, #4]
2557255   6ba362:       60b5            str     r5, [r6, #8]
2557256   6ba364:       f576 e7ea       blx     3133c <_init+0xa24>
2557257   6ba368:       f104 0160       add.w   r1, r4, #96     ; 0x60
2557258   6ba36c:       4628            mov     r0, r5
2557259   6ba36e:       f44f 72d0       mov.w   r2, #416        ; 0x1a0
2557260   6ba372:       f576 e7e4       blx     3133c <_init+0xa24>
2557261   6ba376:       f8d4 0200       ldr.w   r0, [r4, #512]  ; 0x200
2557262   6ba37a:       4631            mov     r1, r6
2557263   6ba37c:       f8c6 0200       str.w   r0, [r6, #512]  ; 0x200
2557264   6ba380:       4640            mov     r0, r8
2557265   6ba382:       e8bd 41f0       ldmia.w sp!, {r4, r5, r6, r7, r8,
lr}
2557266   6ba386:       f080 bd45       b.w     73ae14
<ThrowExceptionFromContextInternal>
2557267   6ba38a:       bf00            nop
2557268   6ba38c:       002820e4        .word   0x002820e4
2557269


========== debug: -O3  <- Build Ok, Running Failed
=====================================

2642852 006f3d54 <PAL_ThrowExceptionFromContext>:
2642853   6f3d54:       e92d 41f0       stmdb   sp!, {r4, r5, r6, r7, r8,
lr}
2642854   6f3d58:       af03            add     r7, sp, #12
2642855   6f3d5a:       4680            mov     r8, r0
2642856   6f3d5c:       4812            ldr     r0, [pc, #72]   ; (6f3da8
<PAL_ThrowExceptionFromContext+0x54>)
2642857   6f3d5e:       460c            mov     r4, r1
2642858   6f3d60:       4478            add     r0, pc
2642859   6f3d62:       f53d e410       blx     31584 <_init+0xca4>
2642860   6f3d66:       4606            mov     r6, r0
2642861   6f3d68:       f04f 31ff       mov.w   r1, #4294967295 ; 0xffffffff
2642862   6f3d6c:       6031            str     r1, [r6, #0]
2642863   6f3d6e:       f106 000c       add.w   r0, r6, #12
2642864   6f3d72:       f104 010c       add.w   r1, r4, #12
2642865   6f3d76:       2250            movs    r2, #80 ; 0x50
2642866   6f3d78:       f106 0560       add.w   r5, r6, #96     ; 0x60
2642867   6f3d7c:       6070            str     r0, [r6, #4]
2642868   6f3d7e:       60b5            str     r5, [r6, #8]
2642869   6f3d80:       f53d e2c0       blx     31304 <_init+0xa24>
2642870   6f3d84:       f104 0160       add.w   r1, r4, #96     ; 0x60
2642871   6f3d88:       4628            mov     r0, r5
2642872   6f3d8a:       f44f 72d0       mov.w   r2, #416        ; 0x1a0
2642873   6f3d8e:       f53d e2ba       blx     31304 <_init+0xa24>
2642874   6f3d92:       f8d4 0200       ldr.w   r0, [r4, #512]  ; 0x200
2642875   6f3d96:       4631            mov     r1, r6
2642876   6f3d98:       f8c6 0200       str.w   r0, [r6, #512]  ; 0x200
2642877   6f3d9c:       4640            mov     r0, r8
2642878   6f3d9e:       e8bd 41f0       ldmia.w sp!, {r4, r5, r6, r7, r8,
lr}
2642879   6f3da2:       f081 bee1       b.w     775b68
<ThrowExceptionFromContextInternal>
2642880   6f3da6:       bf00            nop
2642881   6f3da8:       002906c8        .word   0x002906c8


End of line.

On Tue, May 3, 2016 at 5:55 PM, Geunsik Lim <leemgs at gmail.com> wrote:

> Now, There are four thread local storage (TLS) models in Clang/LLVM as
> following:
> 1) global-dynamic TLS model
> 2) local-dynamic   TLS model
> 3) local-exec         TLS model
> 4) initial-exec        TLS model
> and emulated-TLS (for Android S/W platform)??
>
> Even though, We can build run normally with the static relocation method
> by selecting the initial-exec TLS model instead of global-dynamic TLS
> model (by default) , We need to run the clang application code with
> global-dynamic (or local-dynamic) TLS model in order that we consider an
> application code is working with dlopen(3) library call.
>
> If someone have already found the appropriate solution for some clang/LLVM
> applications including 1) __thread variables and 2) -O2/-O3 of the clang
> language, Could you share us?
>
>
>
>
> On Tue, May 3, 2016 at 5:14 PM, Geunsik Lim <leemgs at gmail.com> wrote:
>
>> A few days ago, I tried to change the optimization flag from -O0 to -O2
>> to speed up the execution of the application on Ubuntu/ARM 14.04 32 bit.
>> When I compiled the source code with  -O2 flag instead of -O0 flag, I
>> could not run the application normally by getting always the segmentation
>> fault.
>>
>> Here is debugging information with GDB command in case of that. As you
>> can see, we could not execute simple hello!! console application
>> because of the bug of clang/LLVM when we try to use "static __thread"
>> variable with -O2 flag.
>>
>> *root:/dotnet/runtime.release.20160322.2/example> gdb ../corerun  ./core.20160322*
>> GNU gdb (GDB) 7.9.1
>> Copyright (C) 2015 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>> and "show warranty" for details.
>> This GDB was configured as "armv7l-tizen-linux-gnueabi".
>> Type "show configuration" for configuration details.
>> For bug reporting instructions, please see:
>> <http://www.gnu.org/software/gdb/bugs/>.
>> Find the GDB manual and other documentation resources online at:
>> <http://www.gnu.org/software/gdb/documentation/>.
>> For help, type "help".
>> Type "apropos word" to search for commands related to "word"...
>> Reading symbols from ../corerun...(no debugging symbols found)...done.
>> [New LWP 742]
>> [New LWP 745]
>> [New LWP 744]
>> [New LWP 743]
>> [New LWP 746]
>>
>> warning: File "/lib/arm-linux-gnueabihf/libthread_db-1.0.so" auto-loading has been declined
>> by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
>> To enable execution of this file add
>>     add-auto-load-safe-path /lib/arm-linux-gnueabihf/libthread_db-1.0.so
>> line to your configuration file "//.gdbinit".
>> To completely disable this security protection add
>>     set auto-load safe-path /
>> line to your configuration file "//.gdbinit".
>> For more information about this security protection see the
>> "Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
>>     info "(gdb)Auto-loading safe path"
>>
>> warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
>>
>> warning: File "/lib/arm-linux-gnueabihf/libthread_db-1.0.so" auto-loading has been declined
>> by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
>>
>> warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
>> ---Type <return> to continue, or q <return> to quit---
>> Core was generated by `../corerun -c .. ./hello.exe'.
>> Program terminated with signal SIGSEGV, Segmentation fault.
>> #0  0x76f7a410 in __tls_get_addr () from /lib/ld-linux-armhf.so.3
>> (gdb) thread info
>> No symbol "info" in current context.
>> (gdb) bt
>> #0  0x76f7a410 in __tls_get_addr () from /lib/ld-linux-armhf.so.3
>> #1  0x7699787c in ClassLoader::LoadTypeHandleForTypeKey_Body(TypeKey*, TypeHandle, ClassLoadLevel) () from /dotnet/runtime.release.20160222.2/libcoreclr.so
>> #2  0x76994c4a in ClassLoader::LoadTypeHandleForTypeKey(TypeKey*, TypeHandle, ClassLoadLevel, InstantiationContext const*) ()
>>    from /dotnet/runtime.release.20160222.2/libcoreclr.so
>> #3  0x76995a38 in ClassLoader::LoadTypeDefThrowing(Module*, unsigned int, ClassLoader::NotFoundAction, ClassLoader::PermitUninstantiatedFlag, unsigned int, ClassLoadLevel, Instantiation*) ()
>>    from /dotnet/runtime.release.20160222.2/libcoreclr.so
>> #4  0x769939f2 in ClassLoader::LoadTypeHandleThrowing(NameHandle*, ClassLoadLevel, Module*) () from /dotnet/runtime.release.20160222.2/libcoreclr.so
>> #5  0x769937c2 in ClassLoader::LoadTypeByNameThrowing(Assembly*, char const*, char const*, ClassLoader::NotFoundAction, ClassLoader::LoadTypesFlag, ClassLoadLevel) () from /dotnet/runtime.release.20160222.2/libcoreclr.so
>> #6  0x76980918 in MscorlibBinder::LookupClass(BinderClassID) ()
>>    from /dotnet/runtime.release.20160222.2/libcoreclr.so
>> #7  0x769675ca in SystemDomain::LoadBaseSystemClasses() ()
>>    from /dotnet/runtime.release.20160222.2/libcoreclr.so
>> #8  0x769673ac in SystemDomain::Init() ()
>>    from /dotnet/runtime.release.20160222.2/libcoreclr.so
>> #9  0x768dbc16 in EEStartupHelper(tagCOINITEE) ()
>>    from /dotnet/runtime.release.20160222.2/libcoreclr.so
>> ---Type <return> to continue, or q <return> to quit---
>> #10 0x768db070 in EEStartup(tagCOINITEE) ()
>>    from /dotnet/runtime.release.20160222.2/libcoreclr.so
>> #11 0x768daf56 in EnsureEEStarted(tagCOINITEE) ()
>>    from /dotnet/runtime.release.20160222.2/libcoreclr.so
>> #12 0x76857998 in CorHost2::Start() ()
>>    from /dotnet/runtime.release.20160222.2/libcoreclr.so
>> #13 0x76842eac in coreclr_initialize ()
>>    from /dotnet/runtime.release.20160222.2/libcoreclr.so
>> #14 0x0000a52e in ExecuteManagedAssembly(char const*, char const*, char const*, int, char const**) ()
>> #15 0x00009852 in corerun(int, char const**) ()
>> #16 0x76d38632 in __libc_start_main () from /lib/arm-linux-gnueabihf/libc.so.6
>> #17 0x00009590 in _start ()
>> Backtrace stopped: previous frame identical to this frame (corrupt stack?)
>> (gdb)
>> (gdb)
>> (gdb)
>> (gdb)
>>
>>
>>
>>
>> At that time, I defined the "global-dynamic" TLS model by default  with
>>  Clang/LLVM compiler as following:
>> SET (CMAKE_C_FLAGS_INIT                "-Wall -std=c11
>> -ftls-model=global-dynamic")
>>
>> We can build and run hello!!! console application with clang/LLVM with
>> -O1 optimization level for Ubuntu/ARM 14.04 32bit as following.
>> src/pal/tools/clang-compiler-override.txt | 13 +++++++++++--
>>
>>  1 file changed, 11 insertions(+), 2 deletions(-)
>>
>> diff --git a/src/pal/tools/clang-compiler-override.txt b/src/pal/tools/clang-compiler-override.txt
>> index 8bba4a9..ba86d15 100644
>> --- a/src/pal/tools/clang-compiler-override.txt
>> +++ b/src/pal/tools/clang-compiler-override.txt
>> @@ -1,13 +1,22 @@
>>  SET (CMAKE_C_FLAGS_INIT                "-Wall -std=c11")
>>  SET (CMAKE_C_FLAGS_DEBUG_INIT          "-g -O0")
>>  SET (CLR_C_FLAGS_CHECKED_INIT          "-g -O2")
>> -SET (CMAKE_C_FLAGS_RELEASE_INIT        "-g -O3")
>> +if(CMAKE_SYSTEM_NAME STREQUAL Linux AND CMAKE_SYSTEM_PROCESSOR STREQUAL armv7l)
>> +    SET (CMAKE_C_FLAGS_RELEASE_INIT        "-g0 -O1")
>> +else()
>> +    SET (CMAKE_C_FLAGS_RELEASE_INIT        "-g0 -O3")
>> +    message(FATAL_ERROR "RELEASE... c flag no-arm")
>> +endif()
>>  SET (CMAKE_C_FLAGS_RELWITHDEBINFO_INIT "-g -O2")
>>
>>  SET (CMAKE_CXX_FLAGS_INIT                "-Wall -Wno-null-conversion -std=c++11")
>>  SET (CMAKE_CXX_FLAGS_DEBUG_INIT          "-g -O0")
>>  SET (CLR_CXX_FLAGS_CHECKED_INIT          "-g -O2")
>> -SET (CMAKE_CXX_FLAGS_RELEASE_INIT        "-g -O3")
>> +if(CMAKE_SYSTEM_NAME STREQUAL Linux AND CMAKE_SYSTEM_PROCESSOR STREQUAL armv7l)
>> +    SET (CMAKE_CXX_FLAGS_RELEASE_INIT        "-g0 -O1")
>> +else()
>> +    SET (CMAKE_CXX_FLAGS_RELEASE_INIT        "-g0 -O3")
>> +endif()
>>  SET (CMAKE_CXX_FLAGS_RELWITHDEBINFO_INIT "-g -O2")
>>
>>
>> In the summary, We can build and run clang/LLVM source code including
>> __thread variable with -O1 /-O2 flags.
>>
>> However, We cannot run (build is okay.) normally hello!!! console
>> application with clang/LLVM using -O2 and -O3 optimization level.
>>
>> Anyone who has similar experience that gets the __thread variable issue
>> with -O2/-O3 flags like me?
>>
>>
>>
>>
>>
>>
>>
>> --
>> http://leemgs.fedorapeople.org
>> Don't try to avoid pain if you fail.
>> If you decided to face the challenges in life,
>> you can gain a lot by giving your best.
>> Cheolsang Jeong's Book & life
>> --
>>
>
>
>
> --
> http://leemgs.fedorapeople.org
> Don't try to avoid pain if you fail.
> If you decided to face the challenges in life,
> you can gain a lot by giving your best.
> Cheolsang Jeong's Book & life
> --
>



-- 
http://leemgs.fedorapeople.org
Don't try to avoid pain if you fail.
If you decided to face the challenges in life,
you can gain a lot by giving your best.
Cheolsang Jeong's Book & life
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160503/a2477966/attachment.html>


More information about the llvm-dev mailing list