[PATCH] -fstack-protector-strong part 3: data layout

[Magee, Joshua]

Hi Evan,

> 
> On Feb 11, 2013, at 7:01 PM, "Magee, Josh" <Joshua.Magee at am.sony.com>
> wrote:
> 
> > Hi,
> >
> > This is part 3 in the stack-protector-strong series.
> > This patch adds support for data layout rules.
> >
> > The rules for ssp are:
> > 1) Large arrays and structures containing large arrays should be closest to
> >    the stack protector.  "Large" is defined as >= ssp-buffer-size.
> > 2) The remaining stack objects are arranged normally.
> >
> > These rules for sspstrong and sspreq are the same:
> > 1) Large arrays and structures containing large arrays should be closest to
> >    the stack protector.  "Large" is defined as >= ssp-buffer-size.
> > 2) Small arrays and structures containing small arrays should be 2nd closet
> to
> >    the protector.  "Small" is defined as < ssp-buffer-size.
> > 3) Variables that have had their address taken should be 3rd closest to the
> >    protector.
> > 4) The remaining stack objects are arranged normally.
> 
> This is probably a dumb question. Could you explain a bit more why these
> allocation rules would improve safety?

The idea is that if stack smashing occurs then the damage is isolated/limited
to a portion of the stack (specifically, between the exploited stack object and
the stack protector.)  The specific ordering of "large", "small", and "address
taken" corresponds to likelihood of the object being exploited (i.e., a large
array is considered more likely to be attacked than a small array which in turn
is more likely to be attacked than a variable that has its address taken.)
The general idea is to limit the amount of stack that can be trashed
(before it is detected by the canary upon function exit) by grouping objects
vulnerable to stack smashing close to the protector.

> 
> I am also wondering if these changes would have performance impact. For
> example, would this push certain stack objects further away from sp and
> then can make their references less efficient for some targets.
> 
These changes could certainly have a performance impact.  For example, an
architecture could have instructions with special forms that can fold/encode
loads from stack offsets directly into the opcode, but only for offsets under a
certain threshold.  The compiler may re-arrange the stack to make use of this
fact; an optimization that would be undermined by the SSP stack layout rules.
This is just one example, I'm sure there are others.  Re-ordering the data can
also have cache effects.
In any event, the performance impact will be context sensitive (depending on
the code) and I wouldn't expect it to be significant in general - or rather - I
would expect that it less than the overhead of having a stack protector in the
first place.  That said, I haven't done any benchmarking specifically targeting
the effect of the data layout rules on runtime performance.

Ultimately SSP is trade-off of some measure of performance for some measure of
security.  (Probably the attractiveness of SSP is that the impact to
performance is relatively small.  Similarly, one the common criticisms is that
the improvement to security is also relatively small.)   SSP can always be
disabled if the impact to performance is too great.

- Josh

> Thanks,
> 
> Evan
> 
> >
> > LLVM already allocates SSP-triggering objects near the stack protector, but:
> > a) The heuristic used is very basic and doesn't necessarily match what the
> >    StackProtector pass does.
> > b) It doesn't handle the finer grained rules for strong/req listed above.
> >
> > Rather than determining if a stack object may need to be placed near
> > the protector in FunctionLoweringInfo.cpp, I thought it is better to
> > make this decision in the StackProtector pass (where we already do the
> > needed analysis
> > anyway.)
> >
> > The changes I made are:
> > 1) Add an enum type to represent the SSP Layout.
> >   * SSPLayoutKind
> > 2) Add a field to AllocaInst to keep track of the SSPLayoutKind.
> > 3) Update the StackProtector pass analysis to set the SSPLayoutKind for an
> >    AllocaInst.  This requires two unfortunate changes:
> >   * Since all variables vulnerable to stack smashing need to be re-arranged
> >     (and not just the first one encountered), the analyze can't exit early
> >     after finding the first one.
> >   * The analysis needs to be run for SSPReq.  Before it wasn't necessary for
> >     SSPReq because all SSPReq functions get a protector regardless of their
> >     variables. Now the analysis is needed to mark the vulnerable variables.
> > 4) Teach the PrologEpilogInserter and LocalStackAllocation passes the new
> SSP
> >    stack re-arrangement rules.
> >
> > Thanks!
> > -Josh
> > <part3-sspstrong-
> datalayout.diff>_____________________________________
> > __________
> > llvm-commits mailing list
> > llvm-commits at cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>