<div dir="ltr">Cool. I think this deserves to be mentioned in <a href="http://clang.llvm.org/docs/ControlFlowIntegrityDesign.html">http://clang.llvm.org/docs/ControlFlowIntegrityDesign.html</a></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Feb 24, 2015 at 3:17 PM, Peter Collingbourne <span dir="ltr"><<a href="mailto:peter@pcc.me.uk" target="_blank">peter@pcc.me.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: pcc<br>
Date: Tue Feb 24 17:17:02 2015<br>
New Revision: 230394<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=230394&view=rev" target="_blank">http://llvm.org/viewvc/llvm-project?rev=230394&view=rev</a><br>
Log:<br>
LowerBitSets: Introduce global layout builder.<br>
<br>
The builder is based on a layout algorithm that tries to keep members of<br>
small bit sets together. The new layout compresses Chromium's bit sets to<br>
around 15% of their original size.<br>
<br>
Differential Revision: <a href="http://reviews.llvm.org/D7796" target="_blank">http://reviews.llvm.org/D7796</a><br>
<br>
Added:<br>
llvm/trunk/test/Transforms/LowerBitSets/layout.ll<br>
Modified:<br>
llvm/trunk/docs/BitSets.rst<br>
llvm/trunk/include/llvm/Transforms/IPO/LowerBitSets.h<br>
llvm/trunk/lib/Transforms/IPO/LowerBitSets.cpp<br>
llvm/trunk/unittests/Transforms/IPO/LowerBitSets.cpp<br>
<br>
Modified: llvm/trunk/docs/BitSets.rst<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/BitSets.rst?rev=230394&r1=230393&r2=230394&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/BitSets.rst?rev=230394&r1=230393&r2=230394&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/docs/BitSets.rst (original)<br>
+++ llvm/trunk/docs/BitSets.rst Tue Feb 24 17:17:02 2015<br>
@@ -17,8 +17,10 @@ global variable.<br>
This will cause a link-time optimization pass to generate bitsets from the<br>
memory addresses referenced from the elements of the bitset metadata. The pass<br>
will lay out the referenced globals consecutively, so their definitions must<br>
-be available at LTO time. An intrinsic, :ref:`llvm.bitset.test <bitset.test>`,<br>
-generates code to test whether a given pointer is a member of a bitset.<br>
+be available at LTO time. The `GlobalLayoutBuilder`_ class is responsible for<br>
+laying out the globals efficiently to minimize the sizes of the underlying<br>
+bitsets. An intrinsic, :ref:`llvm.bitset.test <bitset.test>`, generates code<br>
+to test whether a given pointer is a member of a bitset.<br>
<br>
:Example:<br>
<br>
@@ -64,3 +66,5 @@ generates code to test whether a given p<br>
%d12 = call i1 @bar(i32* getelementptr ([2 x i32]* @d, i32 0, i32 1)) ; returns 1<br>
ret void<br>
}<br>
+<br>
+.. _GlobalLayoutBuilder: <a href="http://llvm.org/klaus/llvm/blob/master/include/llvm/Transforms/IPO/LowerBitSets.h" target="_blank">http://llvm.org/klaus/llvm/blob/master/include/llvm/Transforms/IPO/LowerBitSets.h</a><br>
<br>
Modified: llvm/trunk/include/llvm/Transforms/IPO/LowerBitSets.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Transforms/IPO/LowerBitSets.h?rev=230394&r1=230393&r2=230394&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Transforms/IPO/LowerBitSets.h?rev=230394&r1=230393&r2=230394&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/include/llvm/Transforms/IPO/LowerBitSets.h (original)<br>
+++ llvm/trunk/include/llvm/Transforms/IPO/LowerBitSets.h Tue Feb 24 17:17:02 2015<br>
@@ -20,6 +20,7 @@<br>
<br>
#include <stdint.h><br>
#include <limits><br>
+#include <set><br>
#include <vector><br>
<br>
namespace llvm {<br>
@@ -73,6 +74,69 @@ struct BitSetBuilder {<br>
BitSetInfo build();<br>
};<br>
<br>
+/// This class implements a layout algorithm for globals referenced by bit sets<br>
+/// that tries to keep members of small bit sets together. This can<br>
+/// significantly reduce bit set sizes in many cases.<br>
+///<br>
+/// It works by assembling fragments of layout from sets of referenced globals.<br>
+/// Each set of referenced globals causes the algorithm to create a new<br>
+/// fragment, which is assembled by appending each referenced global in the set<br>
+/// into the fragment. If a referenced global has already been referenced by an<br>
+/// fragment created earlier, we instead delete that fragment and append its<br>
+/// contents into the fragment we are assembling.<br>
+///<br>
+/// By starting with the smallest fragments, we minimize the size of the<br>
+/// fragments that are copied into larger fragments. This is most intuitively<br>
+/// thought about when considering the case where the globals are virtual tables<br>
+/// and the bit sets represent their derived classes: in a single inheritance<br>
+/// hierarchy, the optimum layout would involve a depth-first search of the<br>
+/// class hierarchy (and in fact the computed layout ends up looking a lot like<br>
+/// a DFS), but a naive DFS would not work well in the presence of multiple<br>
+/// inheritance. This aspect of the algorithm ends up fitting smaller<br>
+/// hierarchies inside larger ones where that would be beneficial.<br>
+///<br>
+/// For example, consider this class hierarchy:<br>
+///<br>
+/// A B<br>
+/// \ / | \<br>
+/// C D E<br>
+///<br>
+/// We have five bit sets: bsA (A, C), bsB (B, C, D, E), bsC (C), bsD (D) and<br>
+/// bsE (E). If we laid out our objects by DFS traversing B followed by A, our<br>
+/// layout would be {B, C, D, E, A}. This is optimal for bsB as it needs to<br>
+/// cover the only 4 objects in its hierarchy, but not for bsA as it needs to<br>
+/// cover 5 objects, i.e. the entire layout. Our algorithm proceeds as follows:<br>
+///<br>
+/// Add bsC, fragments {{C}}<br>
+/// Add bsD, fragments {{C}, {D}}<br>
+/// Add bsE, fragments {{C}, {D}, {E}}<br>
+/// Add bsA, fragments {{A, C}, {D}, {E}}<br>
+/// Add bsB, fragments {{B, A, C, D, E}}<br>
+///<br>
+/// This layout is optimal for bsA, as it now only needs to cover two (i.e. 3<br>
+/// fewer) objects, at the cost of bsB needing to cover 1 more object.<br>
+///<br>
+/// The bit set lowering pass assigns an object index to each object that needs<br>
+/// to be laid out, and calls addFragment for each bit set passing the object<br>
+/// indices of its referenced globals. It then assembles a layout from the<br>
+/// computed layout in the Fragments field.<br>
+struct GlobalLayoutBuilder {<br>
+ /// The computed layout. Each element of this vector contains a fragment of<br>
+ /// layout (which may be empty) consisting of object indices.<br>
+ std::vector<std::vector<uint64_t>> Fragments;<br>
+<br>
+ /// Mapping from object index to fragment index.<br>
+ std::vector<uint64_t> FragmentMap;<br>
+<br>
+ GlobalLayoutBuilder(uint64_t NumObjects)<br>
+ : Fragments(1), FragmentMap(NumObjects) {}<br>
+<br>
+ /// Add \param F to the layout while trying to keep its indices contiguous.<br>
+ /// If a previously seen fragment uses any of \param F's indices, that<br>
+ /// fragment will be laid out inside \param F.<br>
+ void addFragment(const std::set<uint64_t> &F);<br>
+};<br>
+<br>
} // namespace llvm<br>
<br>
#endif<br>
<br>
Modified: llvm/trunk/lib/Transforms/IPO/LowerBitSets.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/IPO/LowerBitSets.cpp?rev=230394&r1=230393&r2=230394&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/IPO/LowerBitSets.cpp?rev=230394&r1=230393&r2=230394&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Transforms/IPO/LowerBitSets.cpp (original)<br>
+++ llvm/trunk/lib/Transforms/IPO/LowerBitSets.cpp Tue Feb 24 17:17:02 2015<br>
@@ -118,6 +118,35 @@ BitSetInfo BitSetBuilder::build() {<br>
return BSI;<br>
}<br>
<br>
+void GlobalLayoutBuilder::addFragment(const std::set<uint64_t> &F) {<br>
+ // Create a new fragment to hold the layout for F.<br>
+ Fragments.emplace_back();<br>
+ std::vector<uint64_t> &Fragment = Fragments.back();<br>
+ uint64_t FragmentIndex = Fragments.size() - 1;<br>
+<br>
+ for (auto ObjIndex : F) {<br>
+ uint64_t OldFragmentIndex = FragmentMap[ObjIndex];<br>
+ if (OldFragmentIndex == 0) {<br>
+ // We haven't seen this object index before, so just add it to the current<br>
+ // fragment.<br>
+ Fragment.push_back(ObjIndex);<br>
+ } else {<br>
+ // This index belongs to an existing fragment. Copy the elements of the<br>
+ // old fragment into this one and clear the old fragment. We don't update<br>
+ // the fragment map just yet, this ensures that any further references to<br>
+ // indices from the old fragment in this fragment do not insert any more<br>
+ // indices.<br>
+ std::vector<uint64_t> &OldFragment = Fragments[OldFragmentIndex];<br>
+ Fragment.insert(Fragment.end(), OldFragment.begin(), OldFragment.end());<br>
+ OldFragment.clear();<br>
+ }<br>
+ }<br>
+<br>
+ // Update the fragment map to point our object indices to this fragment.<br>
+ for (uint64_t ObjIndex : Fragment)<br>
+ FragmentMap[ObjIndex] = FragmentIndex;<br>
+}<br>
+<br>
namespace {<br>
<br>
struct LowerBitSets : public ModulePass {<br>
@@ -485,27 +514,66 @@ bool LowerBitSets::buildBitSets(Module &<br>
// Build the list of bitsets and referenced globals in this disjoint set.<br>
std::vector<MDString *> BitSets;<br>
std::vector<GlobalVariable *> Globals;<br>
+ llvm::DenseMap<MDString *, uint64_t> BitSetIndices;<br>
+ llvm::DenseMap<GlobalVariable *, uint64_t> GlobalIndices;<br>
for (GlobalClassesTy::member_iterator MI = GlobalClasses.member_begin(I);<br>
MI != GlobalClasses.member_end(); ++MI) {<br>
- if ((*MI).is<MDString *>())<br>
+ if ((*MI).is<MDString *>()) {<br>
+ BitSetIndices[MI->get<MDString *>()] = BitSets.size();<br>
BitSets.push_back(MI->get<MDString *>());<br>
- else<br>
+ } else {<br>
+ GlobalIndices[MI->get<GlobalVariable *>()] = Globals.size();<br>
Globals.push_back(MI->get<GlobalVariable *>());<br>
+ }<br>
+ }<br>
+<br>
+ // For each bitset, build a set of indices that refer to globals referenced<br>
+ // by the bitset.<br>
+ std::vector<std::set<uint64_t>> BitSetMembers(BitSets.size());<br>
+ if (BitSetNM) {<br>
+ for (MDNode *Op : BitSetNM->operands()) {<br>
+ // Op = { bitset name, global, offset }<br>
+ if (!Op->getOperand(1))<br>
+ continue;<br>
+ auto I = BitSetIndices.find(cast<MDString>(Op->getOperand(0)));<br>
+ if (I == BitSetIndices.end())<br>
+ continue;<br>
+<br>
+ auto OpGlobal = cast<GlobalVariable>(<br>
+ cast<ConstantAsMetadata>(Op->getOperand(1))->getValue());<br>
+ BitSetMembers[I->second].insert(GlobalIndices[OpGlobal]);<br>
+ }<br>
}<br>
<br>
- // Order bitsets and globals by name for determinism. TODO: We may later<br>
- // want to use a more sophisticated ordering that lays out globals so as to<br>
- // minimize the sizes of the bitsets.<br>
+ // Order the sets of indices by size. The GlobalLayoutBuilder works best<br>
+ // when given small index sets first.<br>
+ std::stable_sort(<br>
+ BitSetMembers.begin(), BitSetMembers.end(),<br>
+ [](const std::set<uint64_t> &O1, const std::set<uint64_t> &O2) {<br>
+ return O1.size() < O2.size();<br>
+ });<br>
+<br>
+ // Create a GlobalLayoutBuilder and provide it with index sets as layout<br>
+ // fragments. The GlobalLayoutBuilder tries to lay out members of fragments<br>
+ // as close together as possible.<br>
+ GlobalLayoutBuilder GLB(Globals.size());<br>
+ for (auto &&MemSet : BitSetMembers)<br>
+ GLB.addFragment(MemSet);<br>
+<br>
+ // Build a vector of globals with the computed layout.<br>
+ std::vector<GlobalVariable *> OrderedGlobals(Globals.size());<br>
+ auto OGI = OrderedGlobals.begin();<br>
+ for (auto &&F : GLB.Fragments)<br>
+ for (auto &&Offset : F)<br>
+ *OGI++ = Globals[Offset];<br>
+<br>
+ // Order bitsets by name for determinism.<br>
std::sort(BitSets.begin(), BitSets.end(), [](MDString *S1, MDString *S2) {<br>
return S1->getString() < S2->getString();<br>
});<br>
- std::sort(Globals.begin(), Globals.end(),<br>
- [](GlobalVariable *GV1, GlobalVariable *GV2) {<br>
- return GV1->getName() < GV2->getName();<br>
- });<br>
<br>
// Build the bitsets from this disjoint set.<br>
- buildBitSetsFromGlobals(M, BitSets, Globals);<br>
+ buildBitSetsFromGlobals(M, BitSets, OrderedGlobals);<br>
}<br>
<br>
return true;<br>
<br>
Added: llvm/trunk/test/Transforms/LowerBitSets/layout.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/LowerBitSets/layout.ll?rev=230394&view=auto" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/LowerBitSets/layout.ll?rev=230394&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/test/Transforms/LowerBitSets/layout.ll (added)<br>
+++ llvm/trunk/test/Transforms/LowerBitSets/layout.ll Tue Feb 24 17:17:02 2015<br>
@@ -0,0 +1,35 @@<br>
+; RUN: opt -S -lowerbitsets < %s | FileCheck %s<br>
+<br>
+target datalayout = "e-p:32:32"<br>
+<br>
+; Tests that this set of globals is laid out according to our layout algorithm<br>
+; (see GlobalLayoutBuilder in include/llvm/Transforms/IPO/LowerBitSets.h).<br>
+; The chosen layout in this case is a, e, b, d, c.<br>
+<br>
+; CHECK: private constant { i32, i32, i32, i32, i32 } { i32 1, i32 5, i32 2, i32 4, i32 3 }<br>
+@a = constant i32 1<br>
+@b = constant i32 2<br>
+@c = constant i32 3<br>
+@d = constant i32 4<br>
+@e = constant i32 5<br>
+<br>
+!0 = !{!"bitset1", i32* @a, i32 0}<br>
+!1 = !{!"bitset1", i32* @b, i32 0}<br>
+!2 = !{!"bitset1", i32* @c, i32 0}<br>
+<br>
+!3 = !{!"bitset2", i32* @b, i32 0}<br>
+!4 = !{!"bitset2", i32* @d, i32 0}<br>
+<br>
+!5 = !{!"bitset3", i32* @a, i32 0}<br>
+!6 = !{!"bitset3", i32* @e, i32 0}<br>
+<br>
+!llvm.bitsets = !{ !0, !1, !2, !3, !4, !5, !6 }<br>
+<br>
+declare i1 @llvm.bitset.test(i8* %ptr, metadata %bitset) nounwind readnone<br>
+<br>
+define void @foo() {<br>
+ %x = call i1 @llvm.bitset.test(i8* undef, metadata !"bitset1")<br>
+ %y = call i1 @llvm.bitset.test(i8* undef, metadata !"bitset2")<br>
+ %z = call i1 @llvm.bitset.test(i8* undef, metadata !"bitset3")<br>
+ ret void<br>
+}<br>
<br>
Modified: llvm/trunk/unittests/Transforms/IPO/LowerBitSets.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/Transforms/IPO/LowerBitSets.cpp?rev=230394&r1=230393&r2=230394&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/Transforms/IPO/LowerBitSets.cpp?rev=230394&r1=230393&r2=230394&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/unittests/Transforms/IPO/LowerBitSets.cpp (original)<br>
+++ llvm/trunk/unittests/Transforms/IPO/LowerBitSets.cpp Tue Feb 24 17:17:02 2015<br>
@@ -62,3 +62,30 @@ TEST(LowerBitSets, BitSetBuilder) {<br>
}<br>
}<br>
}<br>
+<br>
+TEST(LowerBitSets, GlobalLayoutBuilder) {<br>
+ struct {<br>
+ uint64_t NumObjects;<br>
+ std::vector<std::set<uint64_t>> Fragments;<br>
+ std::vector<uint64_t> WantLayout;<br>
+ } GLBTests[] = {<br>
+ {0, {}, {}},<br>
+ {4, {{0, 1}, {2, 3}}, {0, 1, 2, 3}},<br>
+ {3, {{0, 1}, {1, 2}}, {0, 1, 2}},<br>
+ {4, {{0, 1}, {1, 2}, {2, 3}}, {0, 1, 2, 3}},<br>
+ {4, {{0, 1}, {2, 3}, {1, 2}}, {0, 1, 2, 3}},<br>
+ {6, {{2, 5}, {0, 1, 2, 3, 4, 5}}, {0, 1, 2, 5, 3, 4}},<br>
+ };<br>
+<br>
+ for (auto &&T : GLBTests) {<br>
+ GlobalLayoutBuilder GLB(T.NumObjects);<br>
+ for (auto &&F : T.Fragments)<br>
+ GLB.addFragment(F);<br>
+<br>
+ std::vector<uint64_t> ComputedLayout;<br>
+ for (auto &&F : GLB.Fragments)<br>
+ ComputedLayout.insert(ComputedLayout.end(), F.begin(), F.end());<br>
+<br>
+ EXPECT_EQ(T.WantLayout, ComputedLayout);<br>
+ }<br>
+}<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@cs.uiuc.edu">llvm-commits@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a><br>
</blockquote></div><br></div>