[cfe-dev] 3.7.1-rc1 has been tagged. Let's begin testing!

Tom Stellard via cfe-dev cfe-dev at lists.llvm.org
Thu Nov 19 07:13:23 PST 2015


On Wed, Nov 18, 2015 at 11:47:28PM +0800, Ben Pope wrote:
> On Tuesday, November 17, 2015 11:32 AM, Tom Stellard wrote:
> >Hi,
> >
> >I have just tagged 3.7.1-rc1, so it is ready for testing.  As a
> >reminder, when doing regression testing, use the 3.7.0 release
> >as your baseline.
> 
> Ubuntu 15.10 x64 uploaded:
> clang+llvm-3.7.1-rc1-x86_64-linux-gnu-ubuntu-15.10.tar.xz
> 
> The regression run seems very good compared to the 3.7.0 run (I have
> a faster SSD now, which may be relevant, I'll rerun the tests
> another time), but PolyBench may have some significant regressions.
> Performance Regressions     18     0
> Performance Improvements     463     0
> Unchanged Tests     1013     1494
> *Total Tests*     *1494*
> 
> 
> Performance Regressions - Execution Time     Δ ▴     Previous
> Current     σ     Δ (B)     σ (B)
> SingleSource/Benchmarks/Polybench/datamining/correlation/correlation
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.193=3>
> 357.01%     0.9829     4.4920     -     0.00%     -
> SingleSource/Benchmarks/Polybench/datamining/covariance/covariance
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.28=3>
> 292.68%     1.1368     4.4640     -     0.00%     -
> SingleSource/Benchmarks/Polybench/linear-algebra/solvers/durbin/durbin
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.284=3>
> 156.05%     0.1406     0.3600     -     0.00%     -
> SingleSource/Benchmarks/Polybench/linear-algebra/kernels/mvt/mvt
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.453=3>
> 79.10%     0.1273     0.2280     -     0.00%     -
> SingleSource/Benchmarks/Polybench/linear-algebra/kernels/gemver/gemver
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.478=3>
> 48.34%     0.1564     0.2320     -     0.00%     -
> SingleSource/Benchmarks/Polybench/linear-algebra/kernels/gemm/gemm
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.367=3>
> 46.46%     4.1840     6.1280     -     0.00%     -
> MultiSource/Benchmarks/tramp3d-v4/tramp3d-v4
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.355=3>
> 44.21%     0.2108     0.3040     -     0.00%     -
> SingleSource/Benchmarks/Polybench/linear-algebra/kernels/3mm/3mm
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.65=3>
> 24.87%     15.0882     18.8400     -     0.00%     -
> SingleSource/Benchmarks/Shootout-C++/hash2
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.46=3>
> 19.16%     1.3796     1.6440     -     0.00%     -
> SingleSource/Benchmarks/Polybench/linear-algebra/kernels/2mm/2mm
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.150=3>
> 14.45%     11.2117     12.8320     -     0.00%     -
> SingleSource/Benchmarks/Polybench/linear-algebra/kernels/bicg/bicg
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.255=3>
> 14.29%     0.0945     0.1080     -     0.00%     -
> MultiSource/Benchmarks/Olden/treeadd/treeadd
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.198=3>
> 13.38%     0.0882     0.1000     -     0.00%     -
> MultiSource/Benchmarks/TSVC/Symbolics-dbl/Symbolics-dbl
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.135=3>
> 11.98%     1.6967     1.9000     -     0.00%     -
> MultiSource/Benchmarks/VersaBench/bmm/bmm
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.410=3>
> 7.16%     1.3662     1.4640     -     0.00%     -
> SingleSource/Benchmarks/Dhrystone/dry
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.153=3>
> 4.23%     0.3684     0.3840     -     0.00%     -
> MultiSource/Benchmarks/Ptrdist/anagram/anagram
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.128=3>
> 3.38%     0.6384     0.6600     -     0.00%     -
> 
> 
> 

Are these tests below new failing tests, or did they fail in 3.7.0 too?

-Tom

> Failing Tests (154):
>     LeakSanitizer-AddressSanitizer ::
> TestCases/cleanup_in_tsd_destructor.cc
>     LeakSanitizer-AddressSanitizer :: TestCases/disabler.cc
>     LeakSanitizer-AddressSanitizer ::
> TestCases/disabler_in_tsd_destructor.cc
>     LeakSanitizer-AddressSanitizer :: TestCases/ignore_object.cc
>     LeakSanitizer-Standalone :: TestCases/cleanup_in_tsd_destructor.cc
>     LeakSanitizer-Standalone :: TestCases/disabler.cc
>     LeakSanitizer-Standalone :: TestCases/disabler_in_tsd_destructor.cc
>     LeakSanitizer-Standalone :: TestCases/ignore_object.cc
>     MemorySanitizer :: Linux/fopencookie.cc
>     MemorySanitizer :: Linux/getresid.cc
>     MemorySanitizer :: Linux/glob.cc
>     MemorySanitizer :: Linux/glob_altdirfunc.cc
>     MemorySanitizer :: Linux/glob_nomatch.cc
>     MemorySanitizer :: Linux/ioctl_sound.cc
>     MemorySanitizer :: Linux/mallinfo.cc
>     MemorySanitizer :: Linux/obstack.cc
>     MemorySanitizer :: Linux/sunrpc.cc
>     MemorySanitizer :: Linux/sunrpc_bytes.cc
>     MemorySanitizer :: Linux/sunrpc_string.cc
>     MemorySanitizer :: Linux/syscalls.cc
>     MemorySanitizer :: Linux/tcgetattr.cc
>     MemorySanitizer :: Linux/xattr.cc
>     MemorySanitizer :: allocator_returns_null.cc
>     MemorySanitizer :: backtrace.cc
>     MemorySanitizer :: c-strdup.c
>     MemorySanitizer :: chained_origin.cc
>     MemorySanitizer :: chained_origin_empty_stack.cc
>     MemorySanitizer :: chained_origin_limits.cc
>     MemorySanitizer :: chained_origin_memcpy.cc
>     MemorySanitizer :: chained_origin_with_signals.cc
>     MemorySanitizer :: check_mem_is_initialized.cc
>     MemorySanitizer :: coverage-levels.cc
>     MemorySanitizer :: cxa_atexit.cc
>     MemorySanitizer :: death-callback.cc
>     MemorySanitizer :: dlerror.cc
>     MemorySanitizer :: dso-origin.cc
>     MemorySanitizer :: dtls_test.c
>     MemorySanitizer :: errno.cc
>     MemorySanitizer :: fork.cc
>     MemorySanitizer :: ftime.cc
>     MemorySanitizer :: getaddrinfo-positive.cc
>     MemorySanitizer :: getaddrinfo.cc
>     MemorySanitizer :: getc_unlocked.c
>     MemorySanitizer :: getline.cc
>     MemorySanitizer :: heap-origin.cc
>     MemorySanitizer :: iconv.cc
>     MemorySanitizer :: if_indextoname.cc
>     MemorySanitizer :: ifaddrs.cc
>     MemorySanitizer :: initgroups.cc
>     MemorySanitizer :: inline.cc
>     MemorySanitizer :: insertvalue_origin.cc
>     MemorySanitizer :: ioctl.cc
>     MemorySanitizer :: ioctl_custom.cc
>     MemorySanitizer :: mktime.cc
>     MemorySanitizer :: mmap.cc
>     MemorySanitizer :: mmap_below_shadow.cc
>     MemorySanitizer :: msan_check_mem_is_initialized.cc
>     MemorySanitizer :: msan_dump_shadow.cc
>     MemorySanitizer :: msan_print_shadow.cc
>     MemorySanitizer :: msan_print_shadow2.cc
>     MemorySanitizer :: msan_print_shadow3.cc
>     MemorySanitizer :: mul_by_const.cc
>     MemorySanitizer :: no_sanitize_memory.cc
>     MemorySanitizer :: no_sanitize_memory_prop.cc
>     MemorySanitizer :: origin-store-long.cc
>     MemorySanitizer :: param_tls_limit.cc
>     MemorySanitizer :: poison_in_free.cc
>     MemorySanitizer :: print_stats.cc
>     MemorySanitizer :: pthread_getattr_np_deadlock.cc
>     MemorySanitizer :: rand_r.cc
>     MemorySanitizer :: readdir64.cc
>     MemorySanitizer :: realloc-large-origin.cc
>     MemorySanitizer :: realloc-origin.cc
>     MemorySanitizer :: report-demangling.cc
>     MemorySanitizer :: scandir.cc
>     MemorySanitizer :: scandir_null.cc
>     MemorySanitizer :: select.cc
>     MemorySanitizer :: select_float_origin.cc
>     MemorySanitizer :: select_origin.cc
>     MemorySanitizer :: setlocale.cc
>     MemorySanitizer :: signal_stress_test.cc
>     MemorySanitizer :: sigwait.cc
>     MemorySanitizer :: sigwaitinfo.cc
>     MemorySanitizer :: stack-origin.cc
>     MemorySanitizer :: stack-origin2.cc
>     MemorySanitizer :: strerror_r-non-gnu.c
>     MemorySanitizer :: strlen_of_shadow.cc
>     MemorySanitizer :: strxfrm.cc
>     MemorySanitizer :: sync_lock_set_and_test.cc
>     MemorySanitizer :: textdomain.cc
>     MemorySanitizer :: times.cc
>     MemorySanitizer :: tls_reuse.cc
>     MemorySanitizer :: tsearch.cc
>     MemorySanitizer :: tzset.cc
>     MemorySanitizer :
> 1 warning(s) in tests.
> : unaligned_read_origin.cc
>     MemorySanitizer :: unpoison_string.cc
>     MemorySanitizer :: use-after-free.cc
>     MemorySanitizer :: vector_cvt.cc
>     SanitizerCommon-msan :: Linux/aligned_alloc.c
>     SanitizerCommon-msan :: Linux/clock_gettime.c
>     SanitizerCommon-msan :: Linux/getpass.cc
>     SanitizerCommon-msan :: Linux/getpwnam_r_invalid_user.cc
>     SanitizerCommon-msan :: Linux/mlock_test.cc
>     SanitizerCommon-msan :: Linux/open_memstream.cc
>     SanitizerCommon-msan :: Linux/ptrace.cc
>     SanitizerCommon-msan :: Linux/sanitizer_set_death_callback_test.cc
>     SanitizerCommon-msan :: Linux/sched_getparam.cc
>     SanitizerCommon-msan :: Linux/signal_segv_handler.cc
>     SanitizerCommon-msan :: Linux/timerfd.cc
>     SanitizerCommon-msan :: Posix/decorate_proc_maps.cc
>     SanitizerCommon-msan :: corelimit.cc
>     SanitizerCommon-msan :: malloc_hook.cc
>     SanitizerCommon-msan :: options-help.cc
>     SanitizerCommon-msan :: options-include.cc
>     SanitizerCommon-msan :: options-invalid.cc
>     SanitizerCommon-msan :: print-stack-trace.cc
>     SanitizerCommon-msan :: pthread_mutexattr_get.cc
>     SanitizerCommon-msan :: strcasestr.c
>     SanitizerCommon-msan :: strcspn.c
>     SanitizerCommon-msan :: strpbrk.c
>     SanitizerCommon-msan :: strspn.c
>     SanitizerCommon-msan :: strstr.c
>     UBSan-MSan-x86_64 :: TestCases/Float/cast-overflow.cpp
>     UBSan-MSan-x86_64 :: TestCases/Integer/add-overflow.cpp
>     UBSan-MSan-x86_64 :: TestCases/Integer/div-overflow.cpp
>     UBSan-MSan-x86_64 :: TestCases/Integer/div-zero.cpp
>     UBSan-MSan-x86_64 :: TestCases/Integer/incdec-overflow.cpp
>     UBSan-MSan-x86_64 :: TestCases/Integer/mul-overflow.cpp
>     UBSan-MSan-x86_64 :: TestCases/Integer/negate-overflow.cpp
>     UBSan-MSan-x86_64 :: TestCases/Integer/no-recover.cpp
>     UBSan-MSan-x86_64 :: TestCases/Integer/shift.cpp
>     UBSan-MSan-x86_64 :: TestCases/Integer/sub-overflow.cpp
>     UBSan-MSan-x86_64 :: TestCases/Integer/uadd-overflow.cpp
>     UBSan-MSan-x86_64 :: TestCases/Integer/uincdec-overflow.cpp
>     UBSan-MSan-x86_64 :: TestCases/Integer/umul-overflow.cpp
>     UBSan-MSan-x86_64 :: TestCases/Integer/usub-overflow.cpp
>     UBSan-MSan-x86_64 :: TestCases/Misc/Linux/coverage-levels.cc
>     UBSan-MSan-x86_64 :: TestCases/Misc/Linux/ubsan_options.cc
>     UBSan-MSan-x86_64 :: TestCases/Misc/bool.cpp
>     UBSan-MSan-x86_64 :: TestCases/Misc/bounds.cpp
>     UBSan-MSan-x86_64 :: TestCases/Misc/deduplication.cpp
>     UBSan-MSan-x86_64 :: TestCases/Misc/enum.cpp
>     UBSan-MSan-x86_64 :: TestCases/Misc/log-path_test.cc
>     UBSan-MSan-x86_64 :: TestCases/Misc/missing_return.cpp
>     UBSan-MSan-x86_64 :: TestCases/Misc/nonnull-arg.cpp
>     UBSan-MSan-x86_64 :: TestCases/Misc/nonnull.cpp
>     UBSan-MSan-x86_64 :: TestCases/Misc/unreachable.cpp
>     UBSan-MSan-x86_64 :: TestCases/Misc/vla.c
>     UBSan-MSan-x86_64 :: TestCases/TypeCheck/Function/function.cpp
>     UBSan-MSan-x86_64 :: TestCases/TypeCheck/misaligned.cpp
>     UBSan-MSan-x86_64 :: TestCases/TypeCheck/null.cpp
>     UBSan-MSan-x86_64 ::
> TestCases/TypeCheck/vptr-virtual-base-construction.cpp
>     UBSan-MSan-x86_64 :: TestCases/TypeCheck/vptr-virtual-base.cpp
>     UBSan-MSan-x86_64 :: TestCases/TypeCheck/vptr.cpp
> 
>   Expected Passes    : 24015
>   Expected Failures  : 138
>   Unsupported Tests  : 498
>   Unexpected Failures: 154
> 
> Ben



More information about the cfe-dev mailing list