[lldb-dev] run tests with ASAN (dotest.py)

Fri Aug 3 03:48:23 PDT 2018

Hi Gábor,

That error is kind of correct. As far as c++ standard is concerned,
this is an ODR violation, as both lldb and liblldb link in a copy of
LLVMSupport.a. However, all LLVM symbols in liblldb have "hidden"
visibility (and we make sure we don't pass around llvm objects on the
SO boundary), so this is not a problem in practice.

 I am not sure why ASAN_OPTIONS is not respected here, but you should
be able to get around this error by building with
LLVM_LINK_LLVM_DYLIB=On. This will cause both liblldb and lldb to link
llvm dynamically, so you should have only one llvm copy floating
around.
On Fri, 3 Aug 2018 at 11:36, Gábor Márton <martongabesz at gmail.com> wrote:
>
> Hi Vedant and Pavel,
>
> Thanks for your reply.
> I agree that asan may report false positives in case of leak
> detection: E.g. library A (instrumented) allocates memory, while
> library B (not instrumented by asan) is responsible for deallocation.
> However, my goal is to discover memory corruption errors around the
> ASTImporter. For that it is enough if all llvm and clang libs are
> sanitized, plus liblldb.so too; I don't think that we need to sanitize
> python as well.
>
> So what I have discovered so far is that the lldb binary itself
> statically links the asan library:
> ```
> nm -C bin/lldb | ag __asan_op
> 000000000098cdf0 B __asan_option_detect_stack_use_after_return
> ```
>
> In the meanwhile the liblldb.so  is not:
> ```
> nm -C ./lib/liblldb.so | ag __asan_op
>                  U __asan_option_detect_stack_use_after_return
> ```
>
> liblldb.so is then loaded by SWIG from dotest.py and that causes the error.
> With LD_PRELOAD yes I can preload one libasan.so but it is not trivial
> to find the compatible ASAN runtime.
>  When I could find the compatible ASAN runtime then I couldn't get rid
> of an annoying error:
> ```
>  python dotest.py --executable ~/WORK/llvm2/build/debug_san/bin/lldb
> ../packages/Python/lldbsuite/test/ -v --excluded ~/lldb_test_exclude
> --skip-category dataformatters --skip-category flakey --skip-category
> lldb-mi
> ['dotest.py', '--executable',
> '/home/egbomrt/WORK/llvm2/build/debug_san/bin/lldb',
> '../packages/Python/lldbsuite/test/', '-v', '--excluded',
> '/home/egbomrt/lldb_test_exclude', '--skip-category',
> 'dataformatters', '--skip-category', 'flakey', '--skip-category',
> 'lldb-mi']
> LLDB library dir: /home/egbomrt/WORK/llvm2/build/debug_san/bin
> LLDB import library dir: /home/egbomrt/WORK/llvm2/build/debug_san/bin
> ==25952==The following global variable is not properly aligned.
> ==25952==This may happen if another global with the same name
> ==25952==resides in another non-instrumented module.
> ==25952==Or the global comes from a C file built w/o -fno-common.
> ==25952==In either case this is likely an ODR violation bug,
> ==25952==but AddressSanitizer can not provide more details.
> =================================================================
> ==25952==ERROR: AddressSanitizer: odr-violation (0x7f28e415f713):
>   [1] size=1 'llvm::ErrorInfoBase::ID'
> ../../git/llvm/lib/Support/Error.cpp:52:21
>   [2] size=1 'llvm::ErrorInfoBase::ID'
> ../../git/llvm/lib/Support/Error.cpp:52:21
> These globals were registered at these points:
>   [1]:
>     #0 0x4345f0  (/home/egbomrt/WORK/llvm2/build/debug_san/bin/lldb+0x4345f0)
>     #1 0x7f28e0a523bb
> (/home/egbomrt/WORK/llvm2/build/debug_san/bin/../lib/libLLVMSupport.so.6+0x4fe3bb)
>
>   [2]:
>     #0 0x4345f0  (/home/egbomrt/WORK/llvm2/build/debug_san/bin/lldb+0x4345f0)
>     #1 0x7f28e0a523bb
> (/home/egbomrt/WORK/llvm2/build/debug_san/bin/../lib/libLLVMSupport.so.6+0x4fe3bb)
>
> ==25952==HINT: if you don't care about these errors you may set
> ASAN_OPTIONS=detect_odr_violation=0
> SUMMARY: AddressSanitizer: odr-violation: global
> 'llvm::ErrorInfoBase::ID' at
> ../../git/llvm/lib/Support/Error.cpp:52:21
> ```
> Even if I exported ASAN_OPTIONS=detect_odr_violation=0 the error remained.
>
> So, this is where I gave up.
> Maybe one other workaround could be to link statically the whole LLVM
> project... ,but that would take too much time in our CI. Probably we
> will run only the lldb unittests with asan turned on.
>
> GaborOn Fri, Aug 3, 2018 at 11:31 AM Pavel Labath <labath at google.com> wrote:
> >
> > When I looked into this in the past (two years ago), the problem was
> > that the sanitizer runtimes were expected to be linked into the main
> > executable. For the dotest tests, the main executable is "python", so
> > unless you have built an asanified python, you will not have them.
> >
> > You might be able to get them loaded via some LD_PRELOAD tricks, but I
> > am not sure if the overall result will be worth the trouble. In
> > general, the sanitizers expect that your whole process is sanitized,
> > and they tend to report a lot of false positives if that is not the
> > case.
> > On Fri, 3 Aug 2018 at 00:31, Vedant Kumar via lldb-dev
> > <lldb-dev at lists.llvm.org> wrote:
> > >
> > > Hi Gábor,
> > >
> > > That symbol appears to be defined in compiler-rt/lib/asan/asan_rtl.cc, so it should be provided by the asan runtime. Are you sure that the runtime expected by the host compiler is being dynamically loaded here? You can check this using 'ldd' (IIRC) on Linux or 'otool -l' on Darwin. Also, did you take the extra step needed to preload the runtime (LD_PRELOAD/DYLD_INSERT_LIBRARIES)?
> > >
> > > best,
> > > vedant
> > >
> > >
> > > On Aug 2, 2018, at 12:24 PM, Gábor Márton via lldb-dev <lldb-dev at lists.llvm.org> wrote:
> > >
> > > Hi,
> > >
> > > I'd like to run the tests, when LLVM, Clang and LLDB are all built with ASAN.
> > > I am using release_60 version of LLDB.
> > > The unittests just run fine, but with `dotest.py`, there is an error.
> > > Maybe the LLDB build does not respect the global -DLLVM_USE_SANITIZER
> > > flag of cmake ?
> > >
> > > Any help would be appreciated,
> > > Thanks,
> > > Gabor
> > >
> > >
> > > ```
> > > /var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/test/dotest.py
> > > --executable /var/jenkins_home/workspace/ctu_pipeline/build/bin/lldb
> > > /var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/packages/Python/lldbsuite/test/
> > > -v --excluded ./lldb_test_exclude
> > > lldb version 6.0.0 (https://github.com/llvm-mirror/lldb.git revision
> > > b6df24ff1b258b18041161b8f32ac316a3b5d8d9)
> > >  clang revision 64eed461cdd3705e7bc1ccc95df9858f7fe216a8
> > >  llvm revision 089d4c0c490687db6c75f1d074e99c4d42936a50
> > > ['/var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/test/dotest.py',
> > > '--executable',
> > > '/var/jenkins_home/workspace/ctu_pipeline/build/bin/lldb',
> > > '/var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/packages/Python/lldbsuite/test/',
> > > '-v', '--excluded', './lldb_test_exclude']
> > > LLDB library dir: /var/jenkins_home/workspace/ctu_pipeline/build/bin
> > > LLDB import library dir: /var/jenkins_home/workspace/ctu_pipeline/build/bin
> > > The 'lldb-mi' executable cannot be located.  The lldb-mi tests can not
> > > be run as a result.
> > > Traceback (most recent call last):
> > >  File "/var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/test/dotest.py",
> > > line 7, in <module>
> > >    lldbsuite.test.run_suite()
> > >  File "/var/jenkins_home/workspace/ctu_pipeline/llvm/tools/lldb/packages/Python/lldbsuite/test/dotest.py",
> > > line 1129, in run_suite
> > >    import lldb
> > >  File "/var/jenkins_home/workspace/ctu_pipeline/build/lib/python2.7/site-packages/lldb/__init__.py",
> > > line 53, in <module>
> > >    _lldb = swig_import_helper()
> > >  File "/var/jenkins_home/workspace/ctu_pipeline/build/lib/python2.7/site-packages/lldb/__init__.py",
> > > line 49, in swig_import_helper
> > >    _mod = imp.load_module('_lldb', fp, pathname, description)
> > > ImportError: /var/jenkins_home/workspace/ctu_pipeline/build/lib/libLLVMDemangle.so.6:
> > > undefined symbol: __asan_option_detect_stack_use_after_return
> > >
> > > ```
> > > _______________________________________________
> > > lldb-dev mailing list
> > > lldb-dev at lists.llvm.org
> > > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> > >
> > >
> > > _______________________________________________
> > > lldb-dev mailing list
> > > lldb-dev at lists.llvm.org
> > > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev