[llvm-dev] Inexplicable ASAN report. Code generation bug?

[Greg Stark via llvm-dev]

I'm struggling to explain an ASAN report I'm now getting that I didn't
get previously on the same code. In fact the report only happens with
-O2 and not when I remove the -O flags which makes it hard to debug
and makes me suspect it's dependent on exactly which instructions the
code generation decides to access the bytes involved. Afaict the C
code shouldn't be accessing the poisoned bytes.

The report looks like:

init_var_from_num: NUMERIC (short) w=0 d=1 POS 6 bytes: 18 00 00 00 80 80
=================================================================
==28714==ERROR: AddressSanitizer: unknown-crash on address
0x62900003d7ac at pc 0x00000144a6d6 bp 0x7ffed6680db0 sp
0x7ffed6680da8
READ of size 4 at 0x62900003d7ac thread T0
    #0 0x144a6d5 in init_var_from_num
/home/stark/src/pg/postgresql-master/src/backend/utils/adt/numeric.c:4764:17
...
SUMMARY: AddressSanitizer: unknown-crash
/home/stark/src/pg/postgresql-master/src/backend/utils/adt/numeric.c:4764:17
in init_var_from_num
Shadow bytes around the buggy address:
...
=>0x0c527ffffaf0: f7 f7 00 00 f7[06]00 00 f7 00 00 00 00 f7 00 00


The "18 00 00 00 80 80" is a hex dump of the bytes in the 6-byte
object. The preceding and following bytes were poisoned explicitly by
my code which agrees with the shadow bytes listed.

It looks like asan is complaining that the code is doing a 4-byte read
of the last 2 bytes of the object. But in fact the code is accessing
those last two bytes through a 2-byte short which an expression that
looks like (n)->choice.n_short.n_header with the structure and union
looking like these:

struct NumericData
{
    int32 vl_len_; /* varlena header (do not touch directly!) */
    union NumericChoice choice; /* choice of format */
};

union NumericChoice
{
    uint16 n_header; /* Header word */
    struct NumericLong n_long; /* Long form (4-byte header) */
    struct NumericShort n_short; /* Short form (2-byte header) */
};

struct NumericShort
{
    uint16 n_header; /* Sign + display scale + weight */
    NumericDigit n_data[FLEXIBLE_ARRAY_MEMBER]; /* Digits */
};

struct NumericLong
{
    uint16 n_sign_dscale; /* Sign + display scale */
    int16 n_weight; /* Weight of 1st digit */
    NumericDigit n_data[FLEXIBLE_ARRAY_MEMBER]; /* Digits */
};


So it looks to me like -O2 is causing the optimizer to turn the 2-byte
read into a 4-byte read and overrun the allocated object. But I
haven't tried looking at the assembly yet.


-- 
greg