[cfe-users] Clang9 UBSan and GMP

[Hans Wennborg via cfe-users]

I'm a fan of GMP, so I'll jump :-) Regardless of whether this is a bug
in LLVM or GMP, it would be satisfying to get to the root of the
problem.

Bisecting over LLVM shows that GMP's mpn test t-sqrlo starts failing
on my x86_64 Linux machine after this commit

at 8ee477a2ab6  "[InstSimplify] SimplifyICmpInst - icmp eq/ne %X,
undef -> undef"

(My bisection command was
ninja -C build.release -j900 clang && ( cd /tmp/gmp-6.1.2/build &&
make clean && CC=/work/llvm.monorepo/build.release/bin/clang
../configure && make -j50 && make check )
)

Comparing the object files of GMP builds using that LLVM version and
the one right before, shows a diff in ./tests/mpn/t-sqrlo.o

Looking at LLVM's -print-after-all shows a diff after GVN which seems
to come from this if-statement in tests/mpn/t-sqrlo.c:

      if (pp[-1] != p_before || pp[n] != p_after
  || scratch[-1] != s_before || scratch[itch] != s_after
  || mpn_cmp (refp, pp, n) != 0)
{

It looks like the "scratch[-1] != s_before" expression was previously
folded to false, but now it's folded to undef (and the branch ends up
going the other way). That matches the commit message from the
bisection.

It seems the memory pointed to by scratch is never initialized, so the
code is reading uninitialized memory. I'm not a language lawyer, but
it smells like Undefined Behaviour.

The source has a commented out block that I guess would initialize
scratch. Maybe the fix is to reinstate that code, or also comment out
or remove the code that accesses scratch?

Thanks,
Hans

On Sat, Oct 26, 2019 at 12:14 AM David Blaikie via cfe-users
<cfe-users at lists.llvm.org> wrote:
>
> Yeah, coming across compiler bugs does happen - but more often it's bugs in input programs. (one of the reasons compiler engineers aren't likely to jump on reproducing and reducing misbehaving programs, because on the odds, it's not a bug in the compiler)
>
> On Fri, Oct 25, 2019 at 3:12 PM Hans Åberg <haberg-1 at telia.com> wrote:
>>
>> The sources are available at [1]; it is written in C, not C++. I was was hoping that that something like UBSan would shed light on it, but the original question is answered: it changes optimization. The GMP developers say that they have caught some compiler bugs, but that is hard to do and time consuming.
>>
>> 1. https://gmplib.org
>>
>>
>> > On 25 Oct 2019, at 23:38, David Blaikie <dblaikie at gmail.com> wrote:
>> >
>> > It's hard to know if it's the compiler's fault without a test case - due to the nature of undefined behavior and other things (implementation defined behavior and unspecified behavior) in C++, that the program behaves as expected with another compiler or another set of flags doesn't give a strong indication as to where the problem is (in the code, in one of the compilers, etc).