[llvm-dev] Determine reason for failure at -O1

[Jeffrey Walton via llvm-dev]

On Sat, Jun 30, 2018 at 3:49 AM, Jeffrey Walton <noloader at gmail.com> wrote:
> Hi Everyone,
>
> We caught a report for a failed self test when using Clang 5.0 and 6.0
> with -DDEBUG and -O1 (i.e., a "debug build"). The code is question is
> located at https://github.com/weidai11/cryptopp/blob/master/cham-simd.cpp
> . It is the SSSE3 code path for CHAM64.
>
> ...

We cleared the issue at
https://github.com/weidai11/cryptopp/commit/aa80c7d4acb6 . We did not
have a methodology. We teased-out the solution through knob turning.

The aa80c7d4acb6 change modifies a loop to perform 4
encryption/decryption rounds per iteration rather than 8. 8 rounds per
iteration decryption was bad. Dropping from 8 to 4 relieved some
register pressure by requiring 4 fewer loads of a mask used to shuffle
a key.

Thanks for the help and suggestions.

Jeff