[LLVMdev] compiler-rt tests in cmake?
Kostya Serebryany
kcc at google.com
Thu May 30 00:18:08 PDT 2013
+eugenis@, our Android expert.
On Thu, May 30, 2013 at 3:40 AM, Greg Fitzgerald <garious at gmail.com> wrote:
> > Cool, can you use clang 3.3 then? :)
>
> I can, but digging deeper I see that the compiler-rt sanitizer tests
> depend on just-built-clang for its object instrumentation. The next time
> the instrumentation changes, I'd expect those tests to break. If the lit
> tests that require -fsanitize were moved to the clang repo, then I think
> it'd be safe to build compiler-rt with clang 3.3 or gcc 4.6.3.
>
>
> Back to Android, I built the ASan shared object and successfully ran this
> example:
>
> https://code.google.com/p/address-sanitizer/source/browse/wiki/example_UseAfterFree.cc?r=1580
>
> But unlike these instructions:
> http://www.chromium.org/developers/testing/addresssanitizer
>
> the Android instructions don't mention asan_symbolize.py
> https://code.google.com/p/address-sanitizer/wiki/Android
>
> When I use asan_symbolize.py (from Linux), I see no symbols in the stack
> trace and the following error messages:
>
> addr2line: '/data/example_UseAfterFree': No such file
> addr2line: '/data/libclang_rt.asan-arm-android.so': No such file
> addr2line: '/system/lib/libstdc++.so': No such file
>
> How can I decode those addresses in the stack trace? Is there a way to
> configure asan_symbolize.py to find my binaries and
> arm-linux-androideabi-addr2line?
>
> Thanks,
> Greg
>
> P.S. Thanks for the colorful output. You make address sanitizing feel
> like Christmas.
>
>
> On Wed, May 29, 2013 at 7:28 AM, Alexey Samsonov <samsonov at google.com>wrote:
>
>> On Wed, May 29, 2013 at 5:40 PM, Greg Fitzgerald <garious at gmail.com>wrote:
>>
>>> For me, UBsan fails with clang 3.2 and passes with clang 3.3.
>>>
>>
>> Cool, can you use clang 3.3 then? :) I think that the reason selected
>> UBSan tests fail under clang 3.2 is a bug in Clang, which was fixed
>> (Richard may correct me if I'm wrong).
>> I don't really want to mark these tests as "failing on a certain version
>> of host compiler".
>>
>>>
>>> Using a fixed version allows you to build all clang/llvm/compiler-rt
>>> with one compiler. It simplifies the build process quite a bit. Also
>>> better for isolating regressions in compiler-rt, especially if you use
>>> git-bisect.
>>>
>>
>> I think you may fix a host compiler (it may be clang 3.3 or gcc 4.6.3),
>> build w/o -Werror and stay happy most of the time.
>> Bootstrap process (use system compiler to build Clang, use this Clang to
>> build LLVM+Clang+compiler-rt) isn't really scary, it adds just a few lines
>> to your build script.
>> AFAIK some developers actually do this every day. But, yes, this is not
>> user-friendly, and we probably should document it better (or even provide
>> the scripts) somewhere...
>>
>>
>>>
>>> Greg
>>>
>>>
>>> On May 29, 2013, at 12:30 AM, Alexey Samsonov <samsonov at google.com>
>>> wrote:
>>>
>>> UBsan tests work for me when I run "check-ubsan" in both build trees
>>> (the one with gcc 4.6.3 as a host compiler, and the one with fresh Clang).
>>> It's pretty convenient for us to use fresh Clang to configure LLVM and
>>> compiler-rt. One major reason is that autoconf/make build system always
>>> builds compiler-rt with just-built Clang.
>>> There are other benefits, like keeping sanitizers code "-Werror"-clean
>>> under the fresh Clang, ability to catch regressions in compiler etc. Why
>>> would you need a fixed version?
>>>
>>>
>>> On Tue, May 28, 2013 at 10:26 PM, Greg Fitzgerald <garious at gmail.com>wrote:
>>>
>>>> Okay, dropping gcc 4.4.3 makes sense. How do you feel about using
>>>> clang 3.2 (and the upcoming 3.3) instead of tip-of-the-trunk clang? It
>>>> looks like everything works great, but that you just need to make those UB
>>>> tests 'unsupported' since they fail with "libclang_rt.ubsan was built
>>>> without __int128 support".
>>>>
>>>> Thanks,
>>>> Greg
>>>>
>>>>
>>>>
>>>> On Mon, May 27, 2013 at 12:36 AM, Alexey Samsonov <samsonov at google.com>wrote:
>>>>
>>>>>
>>>>> On Sun, May 26, 2013 at 12:17 AM, Evgeniy Stepanov <
>>>>> eugeni.stepanov at gmail.com> wrote:
>>>>>
>>>>>> On Sat, May 25, 2013 at 4:12 AM, Greg Fitzgerald <garious at gmail.com>
>>>>>> wrote:
>>>>>> > When I build compiler-rt with clang 3.2, all lsan tests pass. The
>>>>>> only
>>>>>> > failing tests I see are in ubsan:
>>>>>> >
>>>>>> > Failing Tests (6):
>>>>>> > UndefinedBehaviorSanitizer :: Float/cast-overflow.cpp
>>>>>> > UndefinedBehaviorSanitizer :: Integer/add-overflow.cpp
>>>>>> > UndefinedBehaviorSanitizer :: Integer/div-zero.cpp
>>>>>> > UndefinedBehaviorSanitizer :: Integer/sub-overflow.cpp
>>>>>> > UndefinedBehaviorSanitizer :: Integer/uadd-overflow.cpp
>>>>>> > UndefinedBehaviorSanitizer :: Integer/usub-overflow.cpp
>>>>>> >
>>>>>> >
>>>>>> > When I build with gcc 4.4.3, I need the changes below to get the
>>>>>> source to
>>>>>> > compile (and then the lsan tests fails). What are you all using to
>>>>>> build
>>>>>> > compiler-rt? are you developing on linux, osx or windows? Using
>>>>>> gcc or
>>>>>> > llvm? Which should I expect to work? Any buildbots running?
>>>>>>
>>>>>>
>>>>> As Evgeniy said, we build compiler-rt on Linux and on Mac OS X, and
>>>>> run tests for various sanitizers on our buildbots.
>>>>> Note that CMake build system of compiler-rt uses host compiler to
>>>>> build it. On Linux we use:
>>>>> 1) gcc 4.6.3
>>>>> 2) tip-of-the-trunk Clang,
>>>>> and tests pass under both of these.
>>>>>
>>>>> gcc 4.4.3 seems way too old, so in some sense we've dropped support
>>>>> for it (you have to make changes, as old gcc is not
>>>>> smart enough to make some POD variables thread-local).
>>>>>
>>>>>
>>>>>> I think we build compiler-rt with the host compiler, except for
>>>>>> Android. Which is usually a recently-built clang. n Linux, but we
>>>>>> exercise osx build on our buildbots.
>>>>>> We never build with compiler-rt not in projects/.
>>>>>> Our bot scripts are here:
>>>>>>
>>>>>> https://code.google.com/p/address-sanitizer/source/browse/#svn%2Ftrunk%2Fbuild%2Fscripts%2Fslave
>>>>>> The bots itself are not publicly visible, we hope to change that soon.
>>>>>>
>>>>>> >
>>>>>> > And lastly, are there build instructions to build the Android
>>>>>> shared object?
>>>>>> > Is there any way to build an android static lib? How about for
>>>>>> arm-linux?
>>>>>>
>>>>>> Android runtime is special, we build it in a separate build tree
>>>>>> configured with
>>>>>> -DCMAKE_TOOLCHAIN_FILE=$LLVM_CHECKOUT/cmake/platforms/Android.cmake
>>>>>> See buildbot_cmake.sh for details.
>>>>>>
>>>>>> >
>>>>>> > Thanks,
>>>>>> > Greg
>>>>>> >
>>>>>> >
>>>>>> > diff --git a/lib/interception/interception.h
>>>>>> > b/lib/interception/interception.h
>>>>>> > index d50af35..1771d4e 100644
>>>>>> > --- a/lib/interception/interception.h
>>>>>> > +++ b/lib/interception/interception.h
>>>>>> > @@ -27,8 +27,8 @@ typedef __sanitizer::uptr SIZE_T;
>>>>>> > typedef __sanitizer::sptr SSIZE_T;
>>>>>> > typedef __sanitizer::sptr PTRDIFF_T;
>>>>>> > typedef __sanitizer::s64 INTMAX_T;
>>>>>> > -typedef __sanitizer::OFF_T OFF_T;
>>>>>> > -typedef __sanitizer::OFF64_T OFF64_T;
>>>>>> > +//typedef __sanitizer::OFF_T OFF_T;
>>>>>> > +//typedef __sanitizer::OFF64_T OFF64_T;
>>>>>> >
>>>>>> > // How to add an interceptor:
>>>>>> > // Suppose you need to wrap/replace system function (generally,
>>>>>> from libc):
>>>>>> > diff --git a/lib/lsan/lsan_allocator.cc b/lib/lsan/lsan_allocator.cc
>>>>>> > index 9bf27b1..190dce8 100644
>>>>>> > --- a/lib/lsan/lsan_allocator.cc
>>>>>> > +++ b/lib/lsan/lsan_allocator.cc
>>>>>> > @@ -43,7 +43,7 @@ typedef CombinedAllocator<PrimaryAllocator,
>>>>>> > AllocatorCache,
>>>>>> > SecondaryAllocator> Allocator;
>>>>>> >
>>>>>> > static Allocator allocator;
>>>>>> > -static THREADLOCAL AllocatorCache cache;
>>>>>> > +static /*THREADLOCAL*/ AllocatorCache cache;
>>>>>> >
>>>>>> > void InitializeAllocator() {
>>>>>> > allocator.Init();
>>>>>> > diff --git a/lib/msan/msan_allocator.cc b/lib/msan/msan_allocator.cc
>>>>>> > index 7435843..3e6adb6 100644
>>>>>> > --- a/lib/msan/msan_allocator.cc
>>>>>> > +++ b/lib/msan/msan_allocator.cc
>>>>>> > @@ -33,7 +33,7 @@ typedef LargeMmapAllocator<> SecondaryAllocator;
>>>>>> > typedef CombinedAllocator<PrimaryAllocator, AllocatorCache,
>>>>>> > SecondaryAllocator> Allocator;
>>>>>> >
>>>>>> > -static THREADLOCAL AllocatorCache cache;
>>>>>> > +static /*THREADLOCAL*/ AllocatorCache cache;
>>>>>> > static Allocator allocator;
>>>>>> >
>>>>>> > static int inited = 0;
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > On Fri, May 24, 2013 at 6:10 AM, Sergey Matveev <
>>>>>> earthdok at google.com> wrote:
>>>>>> >>
>>>>>> >> I blame this line in lsan/lit_tests/lit.cfg:
>>>>>> >>
>>>>>> >> # Setup attributes common for all compiler-rt projects.
>>>>>> >> compiler_rt_lit_cfg = os.path.join(llvm_src_root, "projects",
>>>>>> >> "compiler-rt",
>>>>>> >> "lib", "lit.common.cfg")
>>>>>> >>
>>>>>> >>
>>>>>> >> On Fri, May 24, 2013 at 2:53 PM, Alexey Samsonov <
>>>>>> samsonov at google.com>
>>>>>> >> wrote:
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> On Fri, May 24, 2013 at 3:37 AM, Greg Fitzgerald <
>>>>>> garious at gmail.com>
>>>>>> >>> wrote:
>>>>>> >>>>
>>>>>> >>>> > it assumes that compiler-rt is checked out to
>>>>>> >>>> > llvm/projects/compiler-rt. Apparently, this is a problem.
>>>>>> >>>>
>>>>>> >>>> I have a patch for this ready. I'll send it to you and
>>>>>> llvm-commits.
>>>>>> >>>> Most of the tests pass with "make check-all" but the
>>>>>> recently-added lsan
>>>>>> >>>> tests are all failing. Do those fail for you as well? If so,
>>>>>> can we XFAIL
>>>>>> >>>> them for now and try to keep the "make check-all" build clean?
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> Thanks! I'll take a look at the patch today. LSan tests work fine
>>>>>> for me,
>>>>>> >>> and we have them on our buildbot as well. What are the failures
>>>>>> you see?
>>>>>> >>>
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> Thanks,
>>>>>> >>>> Greg
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> On Wed, May 22, 2013 at 9:39 PM, Alexey Samsonov <
>>>>>> samsonov at google.com>
>>>>>> >>>> wrote:
>>>>>> >>>>>
>>>>>> >>>>> Hi!
>>>>>> >>>>>
>>>>>> >>>>> The docs look strange to me - I don't indeed see any CMake
>>>>>> support for
>>>>>> >>>>> running compiler-rt tests.
>>>>>> >>>>> Probably compiler-rt folks can comment on this...
>>>>>> >>>>> I think you should run compilert-rt tests manually by smth. like
>>>>>> >>>>> compiler-rt/test/Unit/test.
>>>>>> >>>>>
>>>>>> >>>>> CMake build system is able of running a bunch of sanitizer tests
>>>>>> >>>>> (AddressSanitizer, ThreadSanitizer etc.), and it assumes that
>>>>>> >>>>> compiler-rt is checked out to llvm/projects/compiler-rt.
>>>>>> Apparently,
>>>>>> >>>>> this is a problem. There was a patch that tried to address
>>>>>> this, but
>>>>>> >>>>> it never got committed.
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>> On Wed, May 22, 2013 at 11:38 PM, Greg Fitzgerald <
>>>>>> garious at gmail.com>
>>>>>> >>>>> wrote:
>>>>>> >>>>>>
>>>>>> >>>>>> Anybody working on porting the compiler-rt tests to cmake?
>>>>>> >>>>>>
>>>>>> >>>>>> The online documentation shows a preference for cmake and
>>>>>> ctest, but
>>>>>> >>>>>> the CMakeLists file has the comment "Currently the tests have
>>>>>> not been
>>>>>> >>>>>> ported to CMake, so disable this directory." How should we be
>>>>>> running the
>>>>>> >>>>>> test suite?
>>>>>> >>>>>>
>>>>>> >>>>>> http://compiler-rt.llvm.org/
>>>>>> >>>>>>
>>>>>> >>>>>> My immediate issue is that "make check-all" fails in the cmake
>>>>>> build
>>>>>> >>>>>> when compiler-rt is outside the projects directory. When I
>>>>>> point to
>>>>>> >>>>>> compiler-rt with LLVM_EXTERNAL_COMPILER_RT_SOURCE_DIR, lit
>>>>>> still looks for
>>>>>> >>>>>> lit.common.cfg within "projects/compiler-rt" instead of the
>>>>>> external
>>>>>> >>>>>> directory. I use similar CMake variables for Clang and Polly
>>>>>> and have had
>>>>>> >>>>>> no trouble. Any recommendations for how to fix?
>>>>>> >>>>>>
>>>>>> >>>>>> Thanks,
>>>>>> >>>>>> Greg
>>>>>> >>>>>>
>>>>>> >>>>>>
>>>>>> >>>>>>
>>>>>> >>>>>> _______________________________________________
>>>>>> >>>>>> LLVM Developers mailing list
>>>>>> >>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>>> >>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>> >>>>>>
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>> --
>>>>>> >>>>> Alexey Samsonov, MSK
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> --
>>>>>> >>> Alexey Samsonov, MSK
>>>>>> >>
>>>>>> >>
>>>>>> >
>>>>>> >
>>>>>> > _______________________________________________
>>>>>> > LLVM Developers mailing list
>>>>>> > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>>> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>> >
>>>>>> _______________________________________________
>>>>>> LLVM Developers mailing list
>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Alexey Samsonov, MSK
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Alexey Samsonov, MSK
>>>
>>>
>>
>>
>> --
>> Alexey Samsonov, MSK
>>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130530/52b2bbc9/attachment.html>
More information about the llvm-dev
mailing list