[LLVMdev] compiler-rt tests in cmake?
Greg Fitzgerald
garious at gmail.com
Thu May 30 11:05:20 PDT 2013
The sanitizer common and asan that mention 'thread' are failing for me
this morning. How are your bots looking? Last good commit here was
512c616cacf70ca029a2bf719a482b902f3687cd.
> You could try preprocessing your report with perl or sed to fix paths
> to your binaries. It would be great to have an option for that in
> asan_symbolize.py.
>
> As for addr2line, we just install binutils-multiarch ubuntu package.
Cool, that gets the job done, thanks. Looks like there's some effort
going into embedding the addr2line functionality into compiler-rt. Is
that something folks are actively working on?
> Tests are different: certainly tests that depend on instrumentation
> are built with clang from the same build tree (that is tests "depend" on Clang).
The compiler-rt library definitions have no dependency on the clang
instrumentation, correct? Only the other way around?
Thanks,
Greg
On Thu, May 30, 2013 at 1:03 AM, Alexey Samsonov <samsonov at google.com> wrote:
>
> On Thu, May 30, 2013 at 3:40 AM, Greg Fitzgerald <garious at gmail.com> wrote:
>>
>> > Cool, can you use clang 3.3 then? :)
>>
>> I can, but digging deeper I see that the compiler-rt sanitizer tests
>> depend on just-built-clang for its object instrumentation. The next time
>> the instrumentation changes, I'd expect those tests to break. If the lit
>> tests that require -fsanitize were moved to the clang repo, then I think
>> it'd be safe to build compiler-rt with clang 3.3 or gcc 4.6.3.
>
>
> Tests are different: certainly tests that depend on instrumentation are
> built with clang from the same build tree (that is tests "depend" on Clang).
> * when you run "make" in your build tree, you build Clang and runtimes with
> a host compiler
> * when you run "make check-all" in your build tree, you use this Clang and
> runtimes to build/run tests.
>
>>
>>
>>
>> Back to Android, I built the ASan shared object and successfully ran this
>> example:
>>
>> https://code.google.com/p/address-sanitizer/source/browse/wiki/example_UseAfterFree.cc?r=1580
>>
>> But unlike these instructions:
>> http://www.chromium.org/developers/testing/addresssanitizer
>>
>> the Android instructions don't mention asan_symbolize.py
>> https://code.google.com/p/address-sanitizer/wiki/Android
>>
>> When I use asan_symbolize.py (from Linux), I see no symbols in the stack
>> trace and the following error messages:
>>
>> addr2line: '/data/example_UseAfterFree': No such file
>> addr2line: '/data/libclang_rt.asan-arm-android.so': No such file
>> addr2line: '/system/lib/libstdc++.so': No such file
>>
>> How can I decode those addresses in the stack trace? Is there a way to
>> configure asan_symbolize.py to find my binaries and
>> arm-linux-androideabi-addr2line?
>>
>> Thanks,
>> Greg
>>
>> P.S. Thanks for the colorful output. You make address sanitizing feel
>> like Christmas.
>>
>>
>> On Wed, May 29, 2013 at 7:28 AM, Alexey Samsonov <samsonov at google.com>
>> wrote:
>>>
>>> On Wed, May 29, 2013 at 5:40 PM, Greg Fitzgerald <garious at gmail.com>
>>> wrote:
>>>>
>>>> For me, UBsan fails with clang 3.2 and passes with clang 3.3.
>>>
>>>
>>> Cool, can you use clang 3.3 then? :) I think that the reason selected
>>> UBSan tests fail under clang 3.2 is a bug in Clang, which was fixed (Richard
>>> may correct me if I'm wrong).
>>> I don't really want to mark these tests as "failing on a certain version
>>> of host compiler".
>>>>
>>>>
>>>> Using a fixed version allows you to build all clang/llvm/compiler-rt
>>>> with one compiler. It simplifies the build process quite a bit. Also
>>>> better for isolating regressions in compiler-rt, especially if you use
>>>> git-bisect.
>>>
>>>
>>> I think you may fix a host compiler (it may be clang 3.3 or gcc 4.6.3),
>>> build w/o -Werror and stay happy most of the time.
>>> Bootstrap process (use system compiler to build Clang, use this Clang to
>>> build LLVM+Clang+compiler-rt) isn't really scary, it adds just a few lines
>>> to your build script.
>>> AFAIK some developers actually do this every day. But, yes, this is not
>>> user-friendly, and we probably should document it better (or even provide
>>> the scripts) somewhere...
>>>
>>>>
>>>>
>>>> Greg
>>>>
>>>>
>>>> On May 29, 2013, at 12:30 AM, Alexey Samsonov <samsonov at google.com>
>>>> wrote:
>>>>
>>>> UBsan tests work for me when I run "check-ubsan" in both build trees
>>>> (the one with gcc 4.6.3 as a host compiler, and the one with fresh Clang).
>>>> It's pretty convenient for us to use fresh Clang to configure LLVM and
>>>> compiler-rt. One major reason is that autoconf/make build system always
>>>> builds compiler-rt with just-built Clang.
>>>> There are other benefits, like keeping sanitizers code "-Werror"-clean
>>>> under the fresh Clang, ability to catch regressions in compiler etc. Why
>>>> would you need a fixed version?
>>>>
>>>>
>>>> On Tue, May 28, 2013 at 10:26 PM, Greg Fitzgerald <garious at gmail.com>
>>>> wrote:
>>>>>
>>>>> Okay, dropping gcc 4.4.3 makes sense. How do you feel about using
>>>>> clang 3.2 (and the upcoming 3.3) instead of tip-of-the-trunk clang? It
>>>>> looks like everything works great, but that you just need to make those UB
>>>>> tests 'unsupported' since they fail with "libclang_rt.ubsan was built
>>>>> without __int128 support".
>>>>>
>>>>> Thanks,
>>>>> Greg
>>>>>
>>>>>
>>>>>
>>>>> On Mon, May 27, 2013 at 12:36 AM, Alexey Samsonov <samsonov at google.com>
>>>>> wrote:
>>>>>>
>>>>>>
>>>>>> On Sun, May 26, 2013 at 12:17 AM, Evgeniy Stepanov
>>>>>> <eugeni.stepanov at gmail.com> wrote:
>>>>>>>
>>>>>>> On Sat, May 25, 2013 at 4:12 AM, Greg Fitzgerald <garious at gmail.com>
>>>>>>> wrote:
>>>>>>> > When I build compiler-rt with clang 3.2, all lsan tests pass. The
>>>>>>> > only
>>>>>>> > failing tests I see are in ubsan:
>>>>>>> >
>>>>>>> > Failing Tests (6):
>>>>>>> > UndefinedBehaviorSanitizer :: Float/cast-overflow.cpp
>>>>>>> > UndefinedBehaviorSanitizer :: Integer/add-overflow.cpp
>>>>>>> > UndefinedBehaviorSanitizer :: Integer/div-zero.cpp
>>>>>>> > UndefinedBehaviorSanitizer :: Integer/sub-overflow.cpp
>>>>>>> > UndefinedBehaviorSanitizer :: Integer/uadd-overflow.cpp
>>>>>>> > UndefinedBehaviorSanitizer :: Integer/usub-overflow.cpp
>>>>>>> >
>>>>>>> >
>>>>>>> > When I build with gcc 4.4.3, I need the changes below to get the
>>>>>>> > source to
>>>>>>> > compile (and then the lsan tests fails). What are you all using to
>>>>>>> > build
>>>>>>> > compiler-rt? are you developing on linux, osx or windows? Using
>>>>>>> > gcc or
>>>>>>> > llvm? Which should I expect to work? Any buildbots running?
>>>>>>>
>>>>>>
>>>>>> As Evgeniy said, we build compiler-rt on Linux and on Mac OS X, and
>>>>>> run tests for various sanitizers on our buildbots.
>>>>>> Note that CMake build system of compiler-rt uses host compiler to
>>>>>> build it. On Linux we use:
>>>>>> 1) gcc 4.6.3
>>>>>> 2) tip-of-the-trunk Clang,
>>>>>> and tests pass under both of these.
>>>>>>
>>>>>> gcc 4.4.3 seems way too old, so in some sense we've dropped support
>>>>>> for it (you have to make changes, as old gcc is not
>>>>>> smart enough to make some POD variables thread-local).
>>>>>>
>>>>>>>
>>>>>>> I think we build compiler-rt with the host compiler, except for
>>>>>>> Android. Which is usually a recently-built clang. n Linux, but we
>>>>>>> exercise osx build on our buildbots.
>>>>>>> We never build with compiler-rt not in projects/.
>>>>>>> Our bot scripts are here:
>>>>>>>
>>>>>>> https://code.google.com/p/address-sanitizer/source/browse/#svn%2Ftrunk%2Fbuild%2Fscripts%2Fslave
>>>>>>> The bots itself are not publicly visible, we hope to change that
>>>>>>> soon.
>>>>>>>
>>>>>>> >
>>>>>>> > And lastly, are there build instructions to build the Android
>>>>>>> > shared object?
>>>>>>> > Is there any way to build an android static lib? How about for
>>>>>>> > arm-linux?
>>>>>>>
>>>>>>> Android runtime is special, we build it in a separate build tree
>>>>>>> configured with
>>>>>>> -DCMAKE_TOOLCHAIN_FILE=$LLVM_CHECKOUT/cmake/platforms/Android.cmake
>>>>>>> See buildbot_cmake.sh for details.
>>>>>>>
>>>>>>> >
>>>>>>> > Thanks,
>>>>>>> > Greg
>>>>>>> >
>>>>>>> >
>>>>>>> > diff --git a/lib/interception/interception.h
>>>>>>> > b/lib/interception/interception.h
>>>>>>> > index d50af35..1771d4e 100644
>>>>>>> > --- a/lib/interception/interception.h
>>>>>>> > +++ b/lib/interception/interception.h
>>>>>>> > @@ -27,8 +27,8 @@ typedef __sanitizer::uptr SIZE_T;
>>>>>>> > typedef __sanitizer::sptr SSIZE_T;
>>>>>>> > typedef __sanitizer::sptr PTRDIFF_T;
>>>>>>> > typedef __sanitizer::s64 INTMAX_T;
>>>>>>> > -typedef __sanitizer::OFF_T OFF_T;
>>>>>>> > -typedef __sanitizer::OFF64_T OFF64_T;
>>>>>>> > +//typedef __sanitizer::OFF_T OFF_T;
>>>>>>> > +//typedef __sanitizer::OFF64_T OFF64_T;
>>>>>>> >
>>>>>>> > // How to add an interceptor:
>>>>>>> > // Suppose you need to wrap/replace system function (generally,
>>>>>>> > from libc):
>>>>>>> > diff --git a/lib/lsan/lsan_allocator.cc
>>>>>>> > b/lib/lsan/lsan_allocator.cc
>>>>>>> > index 9bf27b1..190dce8 100644
>>>>>>> > --- a/lib/lsan/lsan_allocator.cc
>>>>>>> > +++ b/lib/lsan/lsan_allocator.cc
>>>>>>> > @@ -43,7 +43,7 @@ typedef CombinedAllocator<PrimaryAllocator,
>>>>>>> > AllocatorCache,
>>>>>>> > SecondaryAllocator> Allocator;
>>>>>>> >
>>>>>>> > static Allocator allocator;
>>>>>>> > -static THREADLOCAL AllocatorCache cache;
>>>>>>> > +static /*THREADLOCAL*/ AllocatorCache cache;
>>>>>>> >
>>>>>>> > void InitializeAllocator() {
>>>>>>> > allocator.Init();
>>>>>>> > diff --git a/lib/msan/msan_allocator.cc
>>>>>>> > b/lib/msan/msan_allocator.cc
>>>>>>> > index 7435843..3e6adb6 100644
>>>>>>> > --- a/lib/msan/msan_allocator.cc
>>>>>>> > +++ b/lib/msan/msan_allocator.cc
>>>>>>> > @@ -33,7 +33,7 @@ typedef LargeMmapAllocator<> SecondaryAllocator;
>>>>>>> > typedef CombinedAllocator<PrimaryAllocator, AllocatorCache,
>>>>>>> > SecondaryAllocator> Allocator;
>>>>>>> >
>>>>>>> > -static THREADLOCAL AllocatorCache cache;
>>>>>>> > +static /*THREADLOCAL*/ AllocatorCache cache;
>>>>>>> > static Allocator allocator;
>>>>>>> >
>>>>>>> > static int inited = 0;
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> > On Fri, May 24, 2013 at 6:10 AM, Sergey Matveev
>>>>>>> > <earthdok at google.com> wrote:
>>>>>>> >>
>>>>>>> >> I blame this line in lsan/lit_tests/lit.cfg:
>>>>>>> >>
>>>>>>> >> # Setup attributes common for all compiler-rt projects.
>>>>>>> >> compiler_rt_lit_cfg = os.path.join(llvm_src_root, "projects",
>>>>>>> >> "compiler-rt",
>>>>>>> >> "lib", "lit.common.cfg")
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> On Fri, May 24, 2013 at 2:53 PM, Alexey Samsonov
>>>>>>> >> <samsonov at google.com>
>>>>>>> >> wrote:
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>> On Fri, May 24, 2013 at 3:37 AM, Greg Fitzgerald
>>>>>>> >>> <garious at gmail.com>
>>>>>>> >>> wrote:
>>>>>>> >>>>
>>>>>>> >>>> > it assumes that compiler-rt is checked out to
>>>>>>> >>>> > llvm/projects/compiler-rt. Apparently, this is a problem.
>>>>>>> >>>>
>>>>>>> >>>> I have a patch for this ready. I'll send it to you and
>>>>>>> >>>> llvm-commits.
>>>>>>> >>>> Most of the tests pass with "make check-all" but the
>>>>>>> >>>> recently-added lsan
>>>>>>> >>>> tests are all failing. Do those fail for you as well? If so,
>>>>>>> >>>> can we XFAIL
>>>>>>> >>>> them for now and try to keep the "make check-all" build clean?
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>> Thanks! I'll take a look at the patch today. LSan tests work fine
>>>>>>> >>> for me,
>>>>>>> >>> and we have them on our buildbot as well. What are the failures
>>>>>>> >>> you see?
>>>>>>> >>>
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>> Thanks,
>>>>>>> >>>> Greg
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>> On Wed, May 22, 2013 at 9:39 PM, Alexey Samsonov
>>>>>>> >>>> <samsonov at google.com>
>>>>>>> >>>> wrote:
>>>>>>> >>>>>
>>>>>>> >>>>> Hi!
>>>>>>> >>>>>
>>>>>>> >>>>> The docs look strange to me - I don't indeed see any CMake
>>>>>>> >>>>> support for
>>>>>>> >>>>> running compiler-rt tests.
>>>>>>> >>>>> Probably compiler-rt folks can comment on this...
>>>>>>> >>>>> I think you should run compilert-rt tests manually by smth.
>>>>>>> >>>>> like
>>>>>>> >>>>> compiler-rt/test/Unit/test.
>>>>>>> >>>>>
>>>>>>> >>>>> CMake build system is able of running a bunch of sanitizer
>>>>>>> >>>>> tests
>>>>>>> >>>>> (AddressSanitizer, ThreadSanitizer etc.), and it assumes that
>>>>>>> >>>>> compiler-rt is checked out to llvm/projects/compiler-rt.
>>>>>>> >>>>> Apparently,
>>>>>>> >>>>> this is a problem. There was a patch that tried to address
>>>>>>> >>>>> this, but
>>>>>>> >>>>> it never got committed.
>>>>>>> >>>>>
>>>>>>> >>>>>
>>>>>>> >>>>>
>>>>>>> >>>>> On Wed, May 22, 2013 at 11:38 PM, Greg Fitzgerald
>>>>>>> >>>>> <garious at gmail.com>
>>>>>>> >>>>> wrote:
>>>>>>> >>>>>>
>>>>>>> >>>>>> Anybody working on porting the compiler-rt tests to cmake?
>>>>>>> >>>>>>
>>>>>>> >>>>>> The online documentation shows a preference for cmake and
>>>>>>> >>>>>> ctest, but
>>>>>>> >>>>>> the CMakeLists file has the comment "Currently the tests have
>>>>>>> >>>>>> not been
>>>>>>> >>>>>> ported to CMake, so disable this directory." How should we be
>>>>>>> >>>>>> running the
>>>>>>> >>>>>> test suite?
>>>>>>> >>>>>>
>>>>>>> >>>>>> http://compiler-rt.llvm.org/
>>>>>>> >>>>>>
>>>>>>> >>>>>> My immediate issue is that "make check-all" fails in the cmake
>>>>>>> >>>>>> build
>>>>>>> >>>>>> when compiler-rt is outside the projects directory. When I
>>>>>>> >>>>>> point to
>>>>>>> >>>>>> compiler-rt with LLVM_EXTERNAL_COMPILER_RT_SOURCE_DIR, lit
>>>>>>> >>>>>> still looks for
>>>>>>> >>>>>> lit.common.cfg within "projects/compiler-rt" instead of the
>>>>>>> >>>>>> external
>>>>>>> >>>>>> directory. I use similar CMake variables for Clang and Polly
>>>>>>> >>>>>> and have had
>>>>>>> >>>>>> no trouble. Any recommendations for how to fix?
>>>>>>> >>>>>>
>>>>>>> >>>>>> Thanks,
>>>>>>> >>>>>> Greg
>>>>>>> >>>>>>
>>>>>>> >>>>>>
>>>>>>> >>>>>>
>>>>>>> >>>>>> _______________________________________________
>>>>>>> >>>>>> LLVM Developers mailing list
>>>>>>> >>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>>>> >>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>> >>>>>>
>>>>>>> >>>>>
>>>>>>> >>>>>
>>>>>>> >>>>>
>>>>>>> >>>>> --
>>>>>>> >>>>> Alexey Samsonov, MSK
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>> --
>>>>>>> >>> Alexey Samsonov, MSK
>>>>>>> >>
>>>>>>> >>
>>>>>>> >
>>>>>>> >
>>>>>>> > _______________________________________________
>>>>>>> > LLVM Developers mailing list
>>>>>>> > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>>>> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>> >
>>>>>>> _______________________________________________
>>>>>>> LLVM Developers mailing list
>>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Alexey Samsonov, MSK
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Alexey Samsonov, MSK
>>>
>>>
>>>
>>>
>>> --
>>> Alexey Samsonov, MSK
>>
>>
>
>
>
> --
> Alexey Samsonov, MSK
More information about the llvm-dev
mailing list