[LLVMdev] [cfe-dev] [3.7 Release] RC1 has been tagged, Testing Phase I begins

[Hans Wennborg]

On Thu, Jul 23, 2015 at 3:36 AM, Daniel Sanders
<Daniel.Sanders at imgtec.com> wrote:
>> -----Original Message-----
>> From: hwennborg at google.com [mailto:hwennborg at google.com] On Behalf
>> Of Hans Wennborg
>> Sent: 22 July 2015 17:42
>> To: Daniel Sanders
>> Cc: llvmdev; cfe-dev; lldb-dev at cs.uiuc.edu; Ben Pope; Ed Maste; Sebastian
>> Dreßler
>> Subject: Re: [cfe-dev] [3.7 Release] RC1 has been tagged, Testing Phase I
>> begins
>>
>> On Wed, Jul 22, 2015 at 7:03 AM, Daniel Sanders
>> <Daniel.Sanders at imgtec.com> wrote:
>> >> Ben reports that he didn't get a tarball even after the rpath fix and
>> >> had to disable -o pipefail to get one. Still not sure what caused
>> >> this.
>> >
>> > 'make check' failures can cause this. I had to revert r241599 to get a
>> package.
>>
>> Oh, I'm silly. Some part of my brain assumed "make -k" would not just
>> "keep going" but also ignore errors. That's not the case of course.
>>
>> I suppose we could disable -o pipefail for the test step. On the other
>> hand, we should make these tests pass :-) Let me know what you think.
>
> Generally speaking, I agree that we should fix these tests although I do have one failure in the mipsel-linux-gnu package which isn't fixable without either replacing the machine or enabling atime on an NFS partition (there's no local storage). In that particular case, I think it's fine to accept the expected failure. It would be nice to automatically XFAIL that test on noatime filesystems though.
>
> I think the release script shouldn't give up at the first sign of problems. We're going to get the occasional failure during release testing and we should always get a package to enable further testing beyond what the release script does. The ideal would be to have the release script detect whether there were failures and print something like 'Errors occurred, please check the logs for details' to highlight the problem but otherwise behave as if everything was fine.
>
> This is completely untested and may be bash/Linux specific but we could have something like:
>         defer_fail() {
>           local exit_code
>           set +e
>           "$@"
>           exit_code=$?
>           set -e
>           if [ "$exit_code" != 0 ]; then
>            SOMETHING_FAILED=1
>           fi
>         }
>
>         set -e
>         set -o pipefail
>         command1            # The script will stop if command1 fails
>         command2 | command3 # The script will stop if either command2/command3 fails
>         defer_fail command4 # The script will report an error later if command4 fails but will proceed
>         defer_fail command5 | defer_fail command6 # The script will report an error later if either command5/command6 fails but will proceed
>
>         if [ "$SOMETHING_FAILED" = 1 ]; then
>           echo "Something failed, please check the logs"
>         fi

I uploaded a patch that does something like this: http://reviews.llvm.org/D11478

Let me know what you think.

Thanks,
Hans