[LLVMdev] compiler-rt tests in cmake?
Greg Fitzgerald
garious at gmail.com
Wed May 29 16:40:35 PDT 2013
> Cool, can you use clang 3.3 then? :)
I can, but digging deeper I see that the compiler-rt sanitizer tests depend
on just-built-clang for its object instrumentation. The next time the
instrumentation changes, I'd expect those tests to break. If the lit tests
that require -fsanitize were moved to the clang repo, then I think it'd be
safe to build compiler-rt with clang 3.3 or gcc 4.6.3.
Back to Android, I built the ASan shared object and successfully ran this
example:
https://code.google.com/p/address-sanitizer/source/browse/wiki/example_UseAfterFree.cc?r=1580
But unlike these instructions:
http://www.chromium.org/developers/testing/addresssanitizer
the Android instructions don't mention asan_symbolize.py
https://code.google.com/p/address-sanitizer/wiki/Android
When I use asan_symbolize.py (from Linux), I see no symbols in the stack
trace and the following error messages:
addr2line: '/data/example_UseAfterFree': No such file
addr2line: '/data/libclang_rt.asan-arm-android.so': No such file
addr2line: '/system/lib/libstdc++.so': No such file
How can I decode those addresses in the stack trace? Is there a way to
configure asan_symbolize.py to find my binaries and
arm-linux-androideabi-addr2line?
Thanks,
Greg
P.S. Thanks for the colorful output. You make address sanitizing feel like
Christmas.
On Wed, May 29, 2013 at 7:28 AM, Alexey Samsonov <samsonov at google.com>wrote:
> On Wed, May 29, 2013 at 5:40 PM, Greg Fitzgerald <garious at gmail.com>wrote:
>
>> For me, UBsan fails with clang 3.2 and passes with clang 3.3.
>>
>
> Cool, can you use clang 3.3 then? :) I think that the reason selected
> UBSan tests fail under clang 3.2 is a bug in Clang, which was fixed
> (Richard may correct me if I'm wrong).
> I don't really want to mark these tests as "failing on a certain version
> of host compiler".
>
>>
>> Using a fixed version allows you to build all clang/llvm/compiler-rt with
>> one compiler. It simplifies the build process quite a bit. Also better
>> for isolating regressions in compiler-rt, especially if you use git-bisect.
>>
>
> I think you may fix a host compiler (it may be clang 3.3 or gcc 4.6.3),
> build w/o -Werror and stay happy most of the time.
> Bootstrap process (use system compiler to build Clang, use this Clang to
> build LLVM+Clang+compiler-rt) isn't really scary, it adds just a few lines
> to your build script.
> AFAIK some developers actually do this every day. But, yes, this is not
> user-friendly, and we probably should document it better (or even provide
> the scripts) somewhere...
>
>
>>
>> Greg
>>
>>
>> On May 29, 2013, at 12:30 AM, Alexey Samsonov <samsonov at google.com>
>> wrote:
>>
>> UBsan tests work for me when I run "check-ubsan" in both build trees (the
>> one with gcc 4.6.3 as a host compiler, and the one with fresh Clang).
>> It's pretty convenient for us to use fresh Clang to configure LLVM and
>> compiler-rt. One major reason is that autoconf/make build system always
>> builds compiler-rt with just-built Clang.
>> There are other benefits, like keeping sanitizers code "-Werror"-clean
>> under the fresh Clang, ability to catch regressions in compiler etc. Why
>> would you need a fixed version?
>>
>>
>> On Tue, May 28, 2013 at 10:26 PM, Greg Fitzgerald <garious at gmail.com>wrote:
>>
>>> Okay, dropping gcc 4.4.3 makes sense. How do you feel about using clang
>>> 3.2 (and the upcoming 3.3) instead of tip-of-the-trunk clang? It looks
>>> like everything works great, but that you just need to make those UB tests
>>> 'unsupported' since they fail with "libclang_rt.ubsan was built without
>>> __int128 support".
>>>
>>> Thanks,
>>> Greg
>>>
>>>
>>>
>>> On Mon, May 27, 2013 at 12:36 AM, Alexey Samsonov <samsonov at google.com>wrote:
>>>
>>>>
>>>> On Sun, May 26, 2013 at 12:17 AM, Evgeniy Stepanov <
>>>> eugeni.stepanov at gmail.com> wrote:
>>>>
>>>>> On Sat, May 25, 2013 at 4:12 AM, Greg Fitzgerald <garious at gmail.com>
>>>>> wrote:
>>>>> > When I build compiler-rt with clang 3.2, all lsan tests pass. The
>>>>> only
>>>>> > failing tests I see are in ubsan:
>>>>> >
>>>>> > Failing Tests (6):
>>>>> > UndefinedBehaviorSanitizer :: Float/cast-overflow.cpp
>>>>> > UndefinedBehaviorSanitizer :: Integer/add-overflow.cpp
>>>>> > UndefinedBehaviorSanitizer :: Integer/div-zero.cpp
>>>>> > UndefinedBehaviorSanitizer :: Integer/sub-overflow.cpp
>>>>> > UndefinedBehaviorSanitizer :: Integer/uadd-overflow.cpp
>>>>> > UndefinedBehaviorSanitizer :: Integer/usub-overflow.cpp
>>>>> >
>>>>> >
>>>>> > When I build with gcc 4.4.3, I need the changes below to get the
>>>>> source to
>>>>> > compile (and then the lsan tests fails). What are you all using to
>>>>> build
>>>>> > compiler-rt? are you developing on linux, osx or windows? Using
>>>>> gcc or
>>>>> > llvm? Which should I expect to work? Any buildbots running?
>>>>>
>>>>>
>>>> As Evgeniy said, we build compiler-rt on Linux and on Mac OS X, and run
>>>> tests for various sanitizers on our buildbots.
>>>> Note that CMake build system of compiler-rt uses host compiler to build
>>>> it. On Linux we use:
>>>> 1) gcc 4.6.3
>>>> 2) tip-of-the-trunk Clang,
>>>> and tests pass under both of these.
>>>>
>>>> gcc 4.4.3 seems way too old, so in some sense we've dropped support for
>>>> it (you have to make changes, as old gcc is not
>>>> smart enough to make some POD variables thread-local).
>>>>
>>>>
>>>>> I think we build compiler-rt with the host compiler, except for
>>>>> Android. Which is usually a recently-built clang. n Linux, but we
>>>>> exercise osx build on our buildbots.
>>>>> We never build with compiler-rt not in projects/.
>>>>> Our bot scripts are here:
>>>>>
>>>>> https://code.google.com/p/address-sanitizer/source/browse/#svn%2Ftrunk%2Fbuild%2Fscripts%2Fslave
>>>>> The bots itself are not publicly visible, we hope to change that soon.
>>>>>
>>>>> >
>>>>> > And lastly, are there build instructions to build the Android shared
>>>>> object?
>>>>> > Is there any way to build an android static lib? How about for
>>>>> arm-linux?
>>>>>
>>>>> Android runtime is special, we build it in a separate build tree
>>>>> configured with
>>>>> -DCMAKE_TOOLCHAIN_FILE=$LLVM_CHECKOUT/cmake/platforms/Android.cmake
>>>>> See buildbot_cmake.sh for details.
>>>>>
>>>>> >
>>>>> > Thanks,
>>>>> > Greg
>>>>> >
>>>>> >
>>>>> > diff --git a/lib/interception/interception.h
>>>>> > b/lib/interception/interception.h
>>>>> > index d50af35..1771d4e 100644
>>>>> > --- a/lib/interception/interception.h
>>>>> > +++ b/lib/interception/interception.h
>>>>> > @@ -27,8 +27,8 @@ typedef __sanitizer::uptr SIZE_T;
>>>>> > typedef __sanitizer::sptr SSIZE_T;
>>>>> > typedef __sanitizer::sptr PTRDIFF_T;
>>>>> > typedef __sanitizer::s64 INTMAX_T;
>>>>> > -typedef __sanitizer::OFF_T OFF_T;
>>>>> > -typedef __sanitizer::OFF64_T OFF64_T;
>>>>> > +//typedef __sanitizer::OFF_T OFF_T;
>>>>> > +//typedef __sanitizer::OFF64_T OFF64_T;
>>>>> >
>>>>> > // How to add an interceptor:
>>>>> > // Suppose you need to wrap/replace system function (generally,
>>>>> from libc):
>>>>> > diff --git a/lib/lsan/lsan_allocator.cc b/lib/lsan/lsan_allocator.cc
>>>>> > index 9bf27b1..190dce8 100644
>>>>> > --- a/lib/lsan/lsan_allocator.cc
>>>>> > +++ b/lib/lsan/lsan_allocator.cc
>>>>> > @@ -43,7 +43,7 @@ typedef CombinedAllocator<PrimaryAllocator,
>>>>> > AllocatorCache,
>>>>> > SecondaryAllocator> Allocator;
>>>>> >
>>>>> > static Allocator allocator;
>>>>> > -static THREADLOCAL AllocatorCache cache;
>>>>> > +static /*THREADLOCAL*/ AllocatorCache cache;
>>>>> >
>>>>> > void InitializeAllocator() {
>>>>> > allocator.Init();
>>>>> > diff --git a/lib/msan/msan_allocator.cc b/lib/msan/msan_allocator.cc
>>>>> > index 7435843..3e6adb6 100644
>>>>> > --- a/lib/msan/msan_allocator.cc
>>>>> > +++ b/lib/msan/msan_allocator.cc
>>>>> > @@ -33,7 +33,7 @@ typedef LargeMmapAllocator<> SecondaryAllocator;
>>>>> > typedef CombinedAllocator<PrimaryAllocator, AllocatorCache,
>>>>> > SecondaryAllocator> Allocator;
>>>>> >
>>>>> > -static THREADLOCAL AllocatorCache cache;
>>>>> > +static /*THREADLOCAL*/ AllocatorCache cache;
>>>>> > static Allocator allocator;
>>>>> >
>>>>> > static int inited = 0;
>>>>> >
>>>>> >
>>>>> >
>>>>> > On Fri, May 24, 2013 at 6:10 AM, Sergey Matveev <earthdok at google.com>
>>>>> wrote:
>>>>> >>
>>>>> >> I blame this line in lsan/lit_tests/lit.cfg:
>>>>> >>
>>>>> >> # Setup attributes common for all compiler-rt projects.
>>>>> >> compiler_rt_lit_cfg = os.path.join(llvm_src_root, "projects",
>>>>> >> "compiler-rt",
>>>>> >> "lib", "lit.common.cfg")
>>>>> >>
>>>>> >>
>>>>> >> On Fri, May 24, 2013 at 2:53 PM, Alexey Samsonov <
>>>>> samsonov at google.com>
>>>>> >> wrote:
>>>>> >>>
>>>>> >>>
>>>>> >>> On Fri, May 24, 2013 at 3:37 AM, Greg Fitzgerald <
>>>>> garious at gmail.com>
>>>>> >>> wrote:
>>>>> >>>>
>>>>> >>>> > it assumes that compiler-rt is checked out to
>>>>> >>>> > llvm/projects/compiler-rt. Apparently, this is a problem.
>>>>> >>>>
>>>>> >>>> I have a patch for this ready. I'll send it to you and
>>>>> llvm-commits.
>>>>> >>>> Most of the tests pass with "make check-all" but the
>>>>> recently-added lsan
>>>>> >>>> tests are all failing. Do those fail for you as well? If so,
>>>>> can we XFAIL
>>>>> >>>> them for now and try to keep the "make check-all" build clean?
>>>>> >>>
>>>>> >>>
>>>>> >>> Thanks! I'll take a look at the patch today. LSan tests work fine
>>>>> for me,
>>>>> >>> and we have them on our buildbot as well. What are the failures
>>>>> you see?
>>>>> >>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> Thanks,
>>>>> >>>> Greg
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> On Wed, May 22, 2013 at 9:39 PM, Alexey Samsonov <
>>>>> samsonov at google.com>
>>>>> >>>> wrote:
>>>>> >>>>>
>>>>> >>>>> Hi!
>>>>> >>>>>
>>>>> >>>>> The docs look strange to me - I don't indeed see any CMake
>>>>> support for
>>>>> >>>>> running compiler-rt tests.
>>>>> >>>>> Probably compiler-rt folks can comment on this...
>>>>> >>>>> I think you should run compilert-rt tests manually by smth. like
>>>>> >>>>> compiler-rt/test/Unit/test.
>>>>> >>>>>
>>>>> >>>>> CMake build system is able of running a bunch of sanitizer tests
>>>>> >>>>> (AddressSanitizer, ThreadSanitizer etc.), and it assumes that
>>>>> >>>>> compiler-rt is checked out to llvm/projects/compiler-rt.
>>>>> Apparently,
>>>>> >>>>> this is a problem. There was a patch that tried to address this,
>>>>> but
>>>>> >>>>> it never got committed.
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> On Wed, May 22, 2013 at 11:38 PM, Greg Fitzgerald <
>>>>> garious at gmail.com>
>>>>> >>>>> wrote:
>>>>> >>>>>>
>>>>> >>>>>> Anybody working on porting the compiler-rt tests to cmake?
>>>>> >>>>>>
>>>>> >>>>>> The online documentation shows a preference for cmake and
>>>>> ctest, but
>>>>> >>>>>> the CMakeLists file has the comment "Currently the tests have
>>>>> not been
>>>>> >>>>>> ported to CMake, so disable this directory." How should we be
>>>>> running the
>>>>> >>>>>> test suite?
>>>>> >>>>>>
>>>>> >>>>>> http://compiler-rt.llvm.org/
>>>>> >>>>>>
>>>>> >>>>>> My immediate issue is that "make check-all" fails in the cmake
>>>>> build
>>>>> >>>>>> when compiler-rt is outside the projects directory. When I
>>>>> point to
>>>>> >>>>>> compiler-rt with LLVM_EXTERNAL_COMPILER_RT_SOURCE_DIR, lit
>>>>> still looks for
>>>>> >>>>>> lit.common.cfg within "projects/compiler-rt" instead of the
>>>>> external
>>>>> >>>>>> directory. I use similar CMake variables for Clang and Polly
>>>>> and have had
>>>>> >>>>>> no trouble. Any recommendations for how to fix?
>>>>> >>>>>>
>>>>> >>>>>> Thanks,
>>>>> >>>>>> Greg
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> _______________________________________________
>>>>> >>>>>> LLVM Developers mailing list
>>>>> >>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>> >>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>> >>>>>>
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> --
>>>>> >>>>> Alexey Samsonov, MSK
>>>>> >>>>
>>>>> >>>>
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>> --
>>>>> >>> Alexey Samsonov, MSK
>>>>> >>
>>>>> >>
>>>>> >
>>>>> >
>>>>> > _______________________________________________
>>>>> > LLVM Developers mailing list
>>>>> > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>> >
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Alexey Samsonov, MSK
>>>>
>>>
>>>
>>
>>
>> --
>> Alexey Samsonov, MSK
>>
>>
>
>
> --
> Alexey Samsonov, MSK
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130529/77315176/attachment.html>
More information about the llvm-dev
mailing list