[PATCH] [stack protector] Improved data layout rules, WIP, part 1

[Josh Magee]

jmagee added you to the CC list for the revision "[stack protector] Improved data layout rules, WIP, part 1".

Hi,

This is the first part in a series of patches that will update the stack layout
rules for SSP, specifically to support the ssp-strong rules.

I had previously submitted different variations of this, the history can be
found here:
http://comments.gmane.org/gmane.comp.compilers.llvm.cvs/136136

A summary of the end goal of the patch series:
 Currenly, "large stack objects" (i.e., those that triggered a protector) are
placed closest to the protector on the stack.  The goal of this patch series is
to support stricter rules for sspstrong and sspreq.
Specifically, these rules are:
 1) Large arrays and structures containing large arrays should be closest to
    the stack protector.  "Large" is defined as >= ssp-buffer-size.
 2) Small arrays and structures containing small arrays should be 2nd closest to
    the protector.  "Small" is defined as < ssp-buffer-size.
 3) Variables that have had their address taken should be 3rd closest to the
    protector.
 4) The remaining stack objects are arranged normally.

The rules for standard ssp mode would not change, although the way the
layout is determined would.  Currently, "large stack objects" are identified
through a check in FunctionLoweringInfo.cpp (grep for MayNeedSP).  After this
patch series the large stack objects will be identified by the StackProtector
Pass itself.


I think the initial patches were too large to effectively review (plus, I let
too much time slip between my updates+pings.)

To make it easier to review, I decided to split the patch into several smaller
and easier to review chunks.

The breakdown is:

Patch 1 (this one): Split the StackProtector pass from a single cpp file,
StackProtector.cpp, into StackProtector.h and StackProtector.cpp.
  - This is necessary because analysis will be added to the pass that is
    re-used in other passes.
  - This patch introduces no functional change, only refactoring.
  - Note, no regression test is provided since there is no change in
    functionality.

Patch 2: Update the StackProtector pass to do the datalayout analysis.
  - This modifies the pass to classify every SSP-triggering AllocaInst according
    to an SSPLayoutKind (LargeArray, SmallArray, AddrOf).
  - The analysis is collected by the pass and made available for use, but at
    this point no other pass will actually use it.

Patch 3: Use the new layout analysis to replicate the old layout behaviour. 
 - Essentially, this is changing the MachineFrameInfo API to use the new
   SSPLayoutKind information (instead of a boolean flag) and updating a few
   pass dependencies (to preserve the SSP analysis).
 - The actual layout would stay roughly the same - i.e., only LargeArray stack
   objects will be placed near the canary.  Everything else would be laid out
   normally.  The stack layout would not be identical in all cases because the
   current heuristic used for layout (MayNeedSP) misses structures that contain
   large arrays.

Patch 4: A patch that implements the improved layout rules.
 - This patch would change the PrologueEpilogInserter and
   LocalStackSlotAllocation passes to follow the improved stack layout rules.
 - This patch would also include the documentation updates and final tests.

Thanks,
Josh

http://llvm-reviews.chandlerc.com/D1521

Files:
  include/llvm/CodeGen/StackProtector.h
  lib/CodeGen/StackProtector.cpp
-------------- next part --------------
A non-text attachment was scrubbed...
Name: D1521.1.patch
Type: text/x-patch
Size: 7426 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20130826/0880d091/attachment.bin>