[cfe-dev] 3.7.1-rc1 has been tagged. Let's begin testing!
Tom Stellard via cfe-dev
cfe-dev at lists.llvm.org
Thu Nov 19 07:13:23 PST 2015
On Wed, Nov 18, 2015 at 11:47:28PM +0800, Ben Pope wrote:
> On Tuesday, November 17, 2015 11:32 AM, Tom Stellard wrote:
> >Hi,
> >
> >I have just tagged 3.7.1-rc1, so it is ready for testing. As a
> >reminder, when doing regression testing, use the 3.7.0 release
> >as your baseline.
>
> Ubuntu 15.10 x64 uploaded:
> clang+llvm-3.7.1-rc1-x86_64-linux-gnu-ubuntu-15.10.tar.xz
>
> The regression run seems very good compared to the 3.7.0 run (I have
> a faster SSD now, which may be relevant, I'll rerun the tests
> another time), but PolyBench may have some significant regressions.
> Performance Regressions 18 0
> Performance Improvements 463 0
> Unchanged Tests 1013 1494
> *Total Tests* *1494*
>
>
> Performance Regressions - Execution Time Δ ▴ Previous
> Current σ Δ (B) σ (B)
> SingleSource/Benchmarks/Polybench/datamining/correlation/correlation
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.193=3>
> 357.01% 0.9829 4.4920 - 0.00% -
> SingleSource/Benchmarks/Polybench/datamining/covariance/covariance
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.28=3>
> 292.68% 1.1368 4.4640 - 0.00% -
> SingleSource/Benchmarks/Polybench/linear-algebra/solvers/durbin/durbin
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.284=3>
> 156.05% 0.1406 0.3600 - 0.00% -
> SingleSource/Benchmarks/Polybench/linear-algebra/kernels/mvt/mvt
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.453=3>
> 79.10% 0.1273 0.2280 - 0.00% -
> SingleSource/Benchmarks/Polybench/linear-algebra/kernels/gemver/gemver
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.478=3>
> 48.34% 0.1564 0.2320 - 0.00% -
> SingleSource/Benchmarks/Polybench/linear-algebra/kernels/gemm/gemm
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.367=3>
> 46.46% 4.1840 6.1280 - 0.00% -
> MultiSource/Benchmarks/tramp3d-v4/tramp3d-v4
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.355=3>
> 44.21% 0.2108 0.3040 - 0.00% -
> SingleSource/Benchmarks/Polybench/linear-algebra/kernels/3mm/3mm
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.65=3>
> 24.87% 15.0882 18.8400 - 0.00% -
> SingleSource/Benchmarks/Shootout-C++/hash2
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.46=3>
> 19.16% 1.3796 1.6440 - 0.00% -
> SingleSource/Benchmarks/Polybench/linear-algebra/kernels/2mm/2mm
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.150=3>
> 14.45% 11.2117 12.8320 - 0.00% -
> SingleSource/Benchmarks/Polybench/linear-algebra/kernels/bicg/bicg
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.255=3>
> 14.29% 0.0945 0.1080 - 0.00% -
> MultiSource/Benchmarks/Olden/treeadd/treeadd
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.198=3>
> 13.38% 0.0882 0.1000 - 0.00% -
> MultiSource/Benchmarks/TSVC/Symbolics-dbl/Symbolics-dbl
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.135=3>
> 11.98% 1.6967 1.9000 - 0.00% -
> MultiSource/Benchmarks/VersaBench/bmm/bmm
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.410=3>
> 7.16% 1.3662 1.4640 - 0.00% -
> SingleSource/Benchmarks/Dhrystone/dry
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.153=3>
> 4.23% 0.3684 0.3840 - 0.00% -
> MultiSource/Benchmarks/Ptrdist/anagram/anagram
> <http://192.168.18.12:8000/db_default/v4/nts/44/graph?test.128=3>
> 3.38% 0.6384 0.6600 - 0.00% -
>
>
>
Are these tests below new failing tests, or did they fail in 3.7.0 too?
-Tom
> Failing Tests (154):
> LeakSanitizer-AddressSanitizer ::
> TestCases/cleanup_in_tsd_destructor.cc
> LeakSanitizer-AddressSanitizer :: TestCases/disabler.cc
> LeakSanitizer-AddressSanitizer ::
> TestCases/disabler_in_tsd_destructor.cc
> LeakSanitizer-AddressSanitizer :: TestCases/ignore_object.cc
> LeakSanitizer-Standalone :: TestCases/cleanup_in_tsd_destructor.cc
> LeakSanitizer-Standalone :: TestCases/disabler.cc
> LeakSanitizer-Standalone :: TestCases/disabler_in_tsd_destructor.cc
> LeakSanitizer-Standalone :: TestCases/ignore_object.cc
> MemorySanitizer :: Linux/fopencookie.cc
> MemorySanitizer :: Linux/getresid.cc
> MemorySanitizer :: Linux/glob.cc
> MemorySanitizer :: Linux/glob_altdirfunc.cc
> MemorySanitizer :: Linux/glob_nomatch.cc
> MemorySanitizer :: Linux/ioctl_sound.cc
> MemorySanitizer :: Linux/mallinfo.cc
> MemorySanitizer :: Linux/obstack.cc
> MemorySanitizer :: Linux/sunrpc.cc
> MemorySanitizer :: Linux/sunrpc_bytes.cc
> MemorySanitizer :: Linux/sunrpc_string.cc
> MemorySanitizer :: Linux/syscalls.cc
> MemorySanitizer :: Linux/tcgetattr.cc
> MemorySanitizer :: Linux/xattr.cc
> MemorySanitizer :: allocator_returns_null.cc
> MemorySanitizer :: backtrace.cc
> MemorySanitizer :: c-strdup.c
> MemorySanitizer :: chained_origin.cc
> MemorySanitizer :: chained_origin_empty_stack.cc
> MemorySanitizer :: chained_origin_limits.cc
> MemorySanitizer :: chained_origin_memcpy.cc
> MemorySanitizer :: chained_origin_with_signals.cc
> MemorySanitizer :: check_mem_is_initialized.cc
> MemorySanitizer :: coverage-levels.cc
> MemorySanitizer :: cxa_atexit.cc
> MemorySanitizer :: death-callback.cc
> MemorySanitizer :: dlerror.cc
> MemorySanitizer :: dso-origin.cc
> MemorySanitizer :: dtls_test.c
> MemorySanitizer :: errno.cc
> MemorySanitizer :: fork.cc
> MemorySanitizer :: ftime.cc
> MemorySanitizer :: getaddrinfo-positive.cc
> MemorySanitizer :: getaddrinfo.cc
> MemorySanitizer :: getc_unlocked.c
> MemorySanitizer :: getline.cc
> MemorySanitizer :: heap-origin.cc
> MemorySanitizer :: iconv.cc
> MemorySanitizer :: if_indextoname.cc
> MemorySanitizer :: ifaddrs.cc
> MemorySanitizer :: initgroups.cc
> MemorySanitizer :: inline.cc
> MemorySanitizer :: insertvalue_origin.cc
> MemorySanitizer :: ioctl.cc
> MemorySanitizer :: ioctl_custom.cc
> MemorySanitizer :: mktime.cc
> MemorySanitizer :: mmap.cc
> MemorySanitizer :: mmap_below_shadow.cc
> MemorySanitizer :: msan_check_mem_is_initialized.cc
> MemorySanitizer :: msan_dump_shadow.cc
> MemorySanitizer :: msan_print_shadow.cc
> MemorySanitizer :: msan_print_shadow2.cc
> MemorySanitizer :: msan_print_shadow3.cc
> MemorySanitizer :: mul_by_const.cc
> MemorySanitizer :: no_sanitize_memory.cc
> MemorySanitizer :: no_sanitize_memory_prop.cc
> MemorySanitizer :: origin-store-long.cc
> MemorySanitizer :: param_tls_limit.cc
> MemorySanitizer :: poison_in_free.cc
> MemorySanitizer :: print_stats.cc
> MemorySanitizer :: pthread_getattr_np_deadlock.cc
> MemorySanitizer :: rand_r.cc
> MemorySanitizer :: readdir64.cc
> MemorySanitizer :: realloc-large-origin.cc
> MemorySanitizer :: realloc-origin.cc
> MemorySanitizer :: report-demangling.cc
> MemorySanitizer :: scandir.cc
> MemorySanitizer :: scandir_null.cc
> MemorySanitizer :: select.cc
> MemorySanitizer :: select_float_origin.cc
> MemorySanitizer :: select_origin.cc
> MemorySanitizer :: setlocale.cc
> MemorySanitizer :: signal_stress_test.cc
> MemorySanitizer :: sigwait.cc
> MemorySanitizer :: sigwaitinfo.cc
> MemorySanitizer :: stack-origin.cc
> MemorySanitizer :: stack-origin2.cc
> MemorySanitizer :: strerror_r-non-gnu.c
> MemorySanitizer :: strlen_of_shadow.cc
> MemorySanitizer :: strxfrm.cc
> MemorySanitizer :: sync_lock_set_and_test.cc
> MemorySanitizer :: textdomain.cc
> MemorySanitizer :: times.cc
> MemorySanitizer :: tls_reuse.cc
> MemorySanitizer :: tsearch.cc
> MemorySanitizer :: tzset.cc
> MemorySanitizer :
> 1 warning(s) in tests.
> : unaligned_read_origin.cc
> MemorySanitizer :: unpoison_string.cc
> MemorySanitizer :: use-after-free.cc
> MemorySanitizer :: vector_cvt.cc
> SanitizerCommon-msan :: Linux/aligned_alloc.c
> SanitizerCommon-msan :: Linux/clock_gettime.c
> SanitizerCommon-msan :: Linux/getpass.cc
> SanitizerCommon-msan :: Linux/getpwnam_r_invalid_user.cc
> SanitizerCommon-msan :: Linux/mlock_test.cc
> SanitizerCommon-msan :: Linux/open_memstream.cc
> SanitizerCommon-msan :: Linux/ptrace.cc
> SanitizerCommon-msan :: Linux/sanitizer_set_death_callback_test.cc
> SanitizerCommon-msan :: Linux/sched_getparam.cc
> SanitizerCommon-msan :: Linux/signal_segv_handler.cc
> SanitizerCommon-msan :: Linux/timerfd.cc
> SanitizerCommon-msan :: Posix/decorate_proc_maps.cc
> SanitizerCommon-msan :: corelimit.cc
> SanitizerCommon-msan :: malloc_hook.cc
> SanitizerCommon-msan :: options-help.cc
> SanitizerCommon-msan :: options-include.cc
> SanitizerCommon-msan :: options-invalid.cc
> SanitizerCommon-msan :: print-stack-trace.cc
> SanitizerCommon-msan :: pthread_mutexattr_get.cc
> SanitizerCommon-msan :: strcasestr.c
> SanitizerCommon-msan :: strcspn.c
> SanitizerCommon-msan :: strpbrk.c
> SanitizerCommon-msan :: strspn.c
> SanitizerCommon-msan :: strstr.c
> UBSan-MSan-x86_64 :: TestCases/Float/cast-overflow.cpp
> UBSan-MSan-x86_64 :: TestCases/Integer/add-overflow.cpp
> UBSan-MSan-x86_64 :: TestCases/Integer/div-overflow.cpp
> UBSan-MSan-x86_64 :: TestCases/Integer/div-zero.cpp
> UBSan-MSan-x86_64 :: TestCases/Integer/incdec-overflow.cpp
> UBSan-MSan-x86_64 :: TestCases/Integer/mul-overflow.cpp
> UBSan-MSan-x86_64 :: TestCases/Integer/negate-overflow.cpp
> UBSan-MSan-x86_64 :: TestCases/Integer/no-recover.cpp
> UBSan-MSan-x86_64 :: TestCases/Integer/shift.cpp
> UBSan-MSan-x86_64 :: TestCases/Integer/sub-overflow.cpp
> UBSan-MSan-x86_64 :: TestCases/Integer/uadd-overflow.cpp
> UBSan-MSan-x86_64 :: TestCases/Integer/uincdec-overflow.cpp
> UBSan-MSan-x86_64 :: TestCases/Integer/umul-overflow.cpp
> UBSan-MSan-x86_64 :: TestCases/Integer/usub-overflow.cpp
> UBSan-MSan-x86_64 :: TestCases/Misc/Linux/coverage-levels.cc
> UBSan-MSan-x86_64 :: TestCases/Misc/Linux/ubsan_options.cc
> UBSan-MSan-x86_64 :: TestCases/Misc/bool.cpp
> UBSan-MSan-x86_64 :: TestCases/Misc/bounds.cpp
> UBSan-MSan-x86_64 :: TestCases/Misc/deduplication.cpp
> UBSan-MSan-x86_64 :: TestCases/Misc/enum.cpp
> UBSan-MSan-x86_64 :: TestCases/Misc/log-path_test.cc
> UBSan-MSan-x86_64 :: TestCases/Misc/missing_return.cpp
> UBSan-MSan-x86_64 :: TestCases/Misc/nonnull-arg.cpp
> UBSan-MSan-x86_64 :: TestCases/Misc/nonnull.cpp
> UBSan-MSan-x86_64 :: TestCases/Misc/unreachable.cpp
> UBSan-MSan-x86_64 :: TestCases/Misc/vla.c
> UBSan-MSan-x86_64 :: TestCases/TypeCheck/Function/function.cpp
> UBSan-MSan-x86_64 :: TestCases/TypeCheck/misaligned.cpp
> UBSan-MSan-x86_64 :: TestCases/TypeCheck/null.cpp
> UBSan-MSan-x86_64 ::
> TestCases/TypeCheck/vptr-virtual-base-construction.cpp
> UBSan-MSan-x86_64 :: TestCases/TypeCheck/vptr-virtual-base.cpp
> UBSan-MSan-x86_64 :: TestCases/TypeCheck/vptr.cpp
>
> Expected Passes : 24015
> Expected Failures : 138
> Unsupported Tests : 498
> Unexpected Failures: 154
>
> Ben
More information about the cfe-dev
mailing list