[cfe-dev] Security fail (memset being optimized away)

[via cfe-dev]

> > The compiler knows that the buffer has to be all zeros as it
> > knows, it just cleared it before. This is basically a more
> > complicated version of:
> > {
> >      int a = 0;
> >      if( a != 0 )
> >          abort();
> > }
> > This can never call abort and will therefore be removed
> > completely.
> 
> Compilers are not static analysers, they don't know when ram addresses
> were touched as far as I am aware. Do you have a source for this
> information?

For the simple 'int a = 0;' case, it's a well understood constant-value
propagation optimization. The value of 'a' is known to be zero, there
are no other assignments to 'a', therefore we can replace uses of 'a' 
with '0', which makes 'if (a != 0)' become 'if (0 != 0)' which is 
unconditionally false, so the code under it is unreachable, so poof.

A static analyzer would report the always-false condition; the compiler
just optimizes it away.

It requires slightly more smarts to do the same thing for pointed-to
values, but only slightly.

FTR, I was a security guy fighting this same battle 20 years ago, and
compilers have only gotten more clever in the meantime.  We generally
used 'volatile' and bought the performance hit.
--paulr