[llvm-dev] [Release-testers] 6.0.1-rc2 has been tagged

[Jonas Hahnfeld via llvm-dev]

On 2018-06-07 19:16, Dimitry Andric via llvm-dev wrote:
> On 7 Jun 2018, at 17:12, Tom Stellard <tstellar at redhat.com> wrote:
>> 
>> On 06/05/2018 10:47 AM, Dimitry Andric wrote:
>>> On 4 Jun 2018, at 18:01, Tom Stellard via Release-testers 
>>> <release-testers at lists.llvm.org> wrote:
>>>> 
>>>> The 6.0.1-rc2 release has been tagged.  Testers may begin testing 
>>>> and
>>>> reporting results.
>>> 
>>> Built for FreeBSD 10, tested and uploaded:
>>> 
>>> SHA256 (clang+llvm-6.0.1-rc2-amd64-unknown-freebsd10.tar.xz) = 
>>> b2858b1bf254f9aed977638229d90e3a8aeb245e9f12156bda5597cd82e6ae20
>>> SHA256 (clang+llvm-6.0.1-rc2-i386-unknown-freebsd10.tar.xz) = 
>>> 5d42eb7022a1974edcb74091b93a6c6eaa30cac12ea6f1f93324577b5e2a4261
>>> 
>>> ======================================================================
>>> Test results on amd64-freebsd10:
>>> 
>>> Testing Time: 1396.97s
>>> ********************
>>> Unexpected Passing Tests (1):
>>>    lldb :: Expr/TestCallStdStringFunction.test
>>> 
>>> ********************
>>> Failing Tests (523):
>> 
>> Are these regressions?
> 
> For i386, one failure (Bindings/Go/go.test) was fixed, and no new
> failures were introduced:
> 
> rc1> Failing Tests (247):
> rc2> Failing Tests (246):
> rc1>     LLVM :: Bindings/Go/go.test
> rc1>   Expected Passes    : 44251
> rc2>   Expected Passes    : 44268
> rc1>   Unexpected Failures: 247
> rc2>   Unexpected Failures: 246
> 
> For amd64, one failure (Bindings/Go/go.test) was fixed, but a new
> failure (api/omp_get_wtime.c) was introduced:
> 
> rc1>     LLVM :: Bindings/Go/go.test
> rc2>     libomp :: api/omp_get_wtime.c
> rc1>   Expected Passes    : 45407
> rc2>   Expected Passes    : 45423
> 
> It's not immediately clear why the omp_get_wtime test fails:
> 
> FAIL: libomp :: api/omp_get_wtime.c (47166 of 49069)
> ******************** TEST 'libomp :: api/omp_get_wtime.c' FAILED
> ********************
> Script:
> --
> /home/dim/llvm-6.0.1/rc2/Phase3/Release/llvmCore-6.0.1-rc2.obj/./bin/clang
> -fopenmp -fno-experimental-isel  -I
> /home/dim/llvm-6.0.1/rc2/llvm.src/projects/openmp/runtime/test -I
> /home/dim/llvm-6.0.1/rc2/Phase3/Release/llvmCore-6.0.1-rc2.obj/projects/openmp/runtime/src
> -L /home/dim/llvm-6.0.1/rc2/Phase3/Release/llvmCore-6.0.1-rc2.obj/lib
> -I /home/dim/llvm-6.0.1/rc2/llvm.src/projects/openmp/runtime/test/ompt
> /home/dim/llvm-6.0.1/rc2/llvm.src/projects/openmp/runtime/test/api/omp_get_wtime.c
> -o
> /home/dim/llvm-6.0.1/rc2/Phase3/Release/llvmCore-6.0.1-rc2.obj/projects/openmp/runtime/test/api/Output/omp_get_wtime.c.tmp
> -lm &&
> /home/dim/llvm-6.0.1/rc2/Phase3/Release/llvmCore-6.0.1-rc2.obj/projects/openmp/runtime/test/api/Output/omp_get_wtime.c.tmp
> --
> Exit Code: 1
> 
> Command Output (stdout):
> --
> $
> "/home/dim/llvm-6.0.1/rc2/Phase3/Release/llvmCore-6.0.1-rc2.obj/./bin/clang"
> "-fopenmp" "-fno-experimental-isel" "-I"
> "/home/dim/llvm-6.0.1/rc2/llvm.src/projects/openmp/runtime/test" "-I"
> "/home/dim/llvm-6.0.1/rc2/Phase3/Release/llvmCore-6.0.1-rc2.obj/projects/openmp/runtime/src"
> "-L"
> "/home/dim/llvm-6.0.1/rc2/Phase3/Release/llvmCore-6.0.1-rc2.obj/lib"
> "-I"
> "/home/dim/llvm-6.0.1/rc2/llvm.src/projects/openmp/runtime/test/ompt"
> "/home/dim/llvm-6.0.1/rc2/llvm.src/projects/openmp/runtime/test/api/omp_get_wtime.c"
> "-o"
> "/home/dim/llvm-6.0.1/rc2/Phase3/Release/llvmCore-6.0.1-rc2.obj/projects/openmp/runtime/test/api/Output/omp_get_wtime.c.tmp"
> "-lm"
> $
> "/home/dim/llvm-6.0.1/rc2/Phase3/Release/llvmCore-6.0.1-rc2.obj/projects/openmp/runtime/test/api/Output/omp_get_wtime.c.tmp"
> note: command had no output on stdout or stderr
> error: command failed with exit status: 1

This test fails in rare cases when you heavily overload your system 
because it compares measured timings. Because of that the check allows 
quite some deviation that is usually enough. Maybe we should think about 
a better way to test this...