Lowering switch statements with hashing
Jasper Neumann
jn at sirrida.de
Thu Jan 16 12:58:41 PST 2014
Hello all,
here is the first public version of my patch, however a lot should still
be done before the patch is actually committed.
The patch modifies lib/CodeGen/SelectionDAG/SelectionDAGBuilder.*.
The patch for http://llvm.org/bugs/show_bug.cgi?id=18347 is already
embedded but can be turned off with the llvm switch
-tree-split-classic=true (see TreeSplitClassic).
Place gen_hash.inc and hashlib.* in lib/CodeGen/SelectionDAG/.
gen_hash.inc currently is a separate file to ease editing for me but
shall be integrated into SelectionDAGBuilder.cpp later.
You will find some documentation in the *.txt files.
To test the patch you may generate a c++ file which tests a randomly
generated switch statement you can use switchgen.cpp.
To test the hashing library hashlib there is hashtest.cpp.
Currently the default behavior is that the classical jump table method
is disabled; you can change this with the llvm switch
-switch-perfect-hash=... (search SelectionDAGBuilder.cpp and
gen_hash.inc for SwitchPerfectHashMethods).
Please observe how e.g. the following routines are treated using my
patch compared to the release version of clang.
int reversible(int x) {
switch (x) {
case 0: return 0;
case 100: return 1;
case 200: return 2;
case 300: return 3;
case 400: return 4;
case 500: return 5;
case 600: return 6;
case 700: return 7;
case 800: return 8;
case 900: return 9;
default: return -1;
}
}
int reversible_simple(unsigned int x) {
switch (x) {
case 0:
case 10:
case 20:
case 30:
case 40:
case 50:
case 60:
case 70:
case 80:
case 90:
case 100:
return 1;
default: return 0;
}
}
bool is_power_2(unsigned int x) {
switch (x) {
case 0x00000001:
case 0x00000002:
case 0x00000004:
case 0x00000008:
case 0x00000010:
case 0x00000020:
case 0x00000040:
case 0x00000080:
case 0x00000100:
case 0x00000200:
case 0x00000400:
case 0x00000800:
case 0x00001000:
case 0x00002000:
case 0x00004000:
case 0x00008000:
case 0x00010000:
case 0x00020000:
case 0x00040000:
case 0x00080000:
case 0x00100000:
case 0x00200000:
case 0x00400000:
case 0x00800000:
case 0x01000000:
case 0x02000000:
case 0x04000000:
case 0x08000000:
case 0x10000000:
case 0x20000000:
case 0x40000000:
case 0x80000000: return true;
default: return false;
}
}
int map_power_2(unsigned int x) {
switch (x) {
case 0x00000001: return 0x00;
case 0x00000002: return 0x01;
case 0x00000004: return 0x02;
case 0x00000008: return 0x03;
case 0x00000010: return 0x04;
case 0x00000020: return 0x05;
case 0x00000040: return 0x06;
case 0x00000080: return 0x07;
case 0x00000100: return 0x08;
case 0x00000200: return 0x09;
case 0x00000400: return 0x0a;
case 0x00000800: return 0x0b;
case 0x00001000: return 0x0c;
case 0x00002000: return 0x0d;
case 0x00004000: return 0x0e;
case 0x00008000: return 0x0f;
case 0x00010000: return 0x10;
case 0x00020000: return 0x11;
case 0x00040000: return 0x12;
case 0x00080000: return 0x13;
case 0x00100000: return 0x14;
case 0x00200000: return 0x15;
case 0x00400000: return 0x16;
case 0x00800000: return 0x17;
case 0x01000000: return 0x18;
case 0x02000000: return 0x19;
case 0x04000000: return 0x1a;
case 0x08000000: return 0x1b;
case 0x10000000: return 0x1c;
case 0x20000000: return 0x1d;
case 0x40000000: return 0x1e;
case 0x80000000: return 0x1f;
default: return -1;
}
}
As an example for big sparse switches in real-life you might e.g. look
at conversions.c from Debian's msort package downloadable at
http://packages.debian.org/source/sid/msort.
Waiting for your comments and advice. Happy testing
Jasper
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hash.patch
Type: text/x-patch
Size: 25495 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140116/328d7376/attachment.bin>
-------------- next part --------------
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
2014-01-15
Comments on the current implementation of switch via hashing in LLVM
The patch modifies lib/CodeGen/SelectionDAG/SelectionDAGBuilder.*.
The patch for http://llvm.org/bugs/show_bug.cgi?id=18347 is already embedded.
Place gen_hash.inc and hashlib.* in lib/CodeGen/SelectionDAG/.
gen_hash.inc currently is a separate file to ease editing for me but shall be integrated into SelectionDAGBuilder.cpp later.
hashlib.* contains the code where all magic things concerning the generation of a hash function happen; motivation, parts of the implementation and some speed tests are described in http://programming.sirrida.de/hashsuper.pdf. In contrast to that paper I dropped imperfect hashing but added perfect hashing as implemented by Bob Jenkins (see hashlib.*) instead.
I have provided some switches at the top of SelectionDAGBuilder.cpp to fine tune the behavior such as the selection of hash methods. These can be triggered with clang -mllvm -X=Y.
The LLVM code first tests the applicability of hashing such as the availability of the needed operations or jump tables, whether the switch value is at most 32 bit, whether the label set is big enough, whether large ranges kill the approach, and whether hashing succeeds. The limits and limitations can be discussed.
Some tables are set up as needed: tabb/BTable (for perfect hashing a la Jenkins), scramble/STable (for large tabb), ValTable for value comparison, DTable for double dispatching (to keep the jump table small; yet producing an error and thus disabled), and the jump table. All integer value tables use the smallest usable size, i.e. 8, 16, or 32 bit and the values can be signed or unsigned.
A jump table with only one element is replaced by a simple jump. This can happen if minimal perfect hashing is achieved and all (non-default) labels address the same jump target.
The code generator callback functions are plain functions but might as well become static member functions of SelectionDAGBuilder; then the transfer structure type HashContextType could become a local type.
I have not yet done comprehensive tests on the generated code in all combinations and for platforms different from x86-64 on Ubuntu but at least I have tested the hash engine quite well.
The code should be tidied up; all references to "jane" (JAsper NEumann) will be eliminated (these currently ease locating the patches); all lines beginning with /*@/// or /*@\\\ will be removed (these are artifacts from my folding editor).
For the final version double dispatching should work and large ranges should be treated in a decision tree after the hashing, also http://llvm.org/bugs/show_bug.cgi?id=1255 should have been fixed in order to allow for arbitrary large ranges.
It might be that the place where I lower switch statements is too low level; a better place might be just before it is tested whether a switch statement can be lowered to a table lookup; if the hashing approach succeeds a table lookup might follow naturally replacing the jump table which is currently generated.
Future plans include preparing a patch for GCC too using the same hashlib. For Free Pascal the Pascal version of hashlib could be used (I already have a working version for Pascal).
The coding style of hashlib is quite different from the usual LLVM style. However since it should be treated as a black box library this should not matter much. Also, it is much more C than C++ for speed and portability reasons.
Please tell me whatever quirks you find and whatever you want to be changed.
For testing don't be shy to use very large sparse label sets; in the code up to 700_000
labels are accepted but this is a more or less arbitrary value.
The TODO and FIXME comments give hints about what are open questions to me. Please help me on this!
As of documentation and testing: I am an LLVM newbie and not at all fit at reading or writing LL code which seems to be needed for the test programs.
Have fun
Jasper
(c) 2013..2014 by Jasper L. Neumann
www.sirrida.de / programming.sirrida.de
E-Mail: info at sirrida.de
-------------- next part --------------
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
2014-01-15
Comments on hashlib
The purpose of hashlib is to provide (optionally minimal) perfect hashing for 32 bit integers (signed and unsigned; called "label" here) to be used for a static set of constants and is meant to be used e.g. in a compiler. It is usable up to some 100000 labels.
Perfect hashing is a technique which makes the advantages of a jump table or a value table lookup also available for arbitrarily sparse label sets which often occur in switch statements.
The goal is to generate special code for the given specific label set.
This is done by trying several methods; each of the methods can be activated or left unused.
Here are the methods in detail.
x is the given label value, a and b are auxiliary values, all other variables are determined by hashlib.
ror_32 is the rotate right function for 32 bit integers where the first argument is rotated right by the second.
rol_32 is the rotate left function for 32 bit integers where the first argument is rotated left by the second.
All other operators (+, -, *, &, ^, <<, >>) are as defined in C for unsigned 32 bit values; overflows have to be discarded.
gen_reversible:
This method catches the case when the constants can be described as x_i = d * i + c .
The charm of this method is that there is a simple function h which can invert the formula such that h(x_i)=i and that this function is invertible; thus this is called reversible hashing. This means that after applying the hash function h a simple range check is necessary to verify that x_i is a member of the label set. All other methods of hashlib need to check the value after applying the hash function.
h(x) = ror_32(x-salt, a_shr) * a_mask
The next methods are usually only applicable for a small label set.
gen_simple_and:
This hash function simply masks out the lowest bits.
h(x) = x & a_mask
gen_simple_shr:
This hash function simply masks out the highest bits.
h(x) = x >> a_shr
gen_simple_rol:
This hash function masks out a contiguous row of bits.
h(x) = rol_32(x, salt) & a_mask
gen_simple_rol_xor:
h(x) = (rol_32(x, salt) ^ x) & a_mask
gen_simple_rol_add:
h(x) = (rol_32(x, salt) + x) & a_mask
gen_simple_rol_sub:
h(x) = (rol_32(x, salt) - x) & a_mask
gen_simple_mul:
h(x) = (x * salt) >> a_shr
The next methods can be used also on large label sets. They calculate the auxiliary values a and b and need up to 2 tables.
The scramble table is used to keep the memory needed for p_tabb small because in this case p_tabb only contains bytes; to see p_scramble in action it takes at least 8000 labels.
These methods as well as the technique to generate the tables were developed by Bob Jenkins, see http://burtleburtle.net/bob/hash/perfect.html
There is a common auxiliary function calc_tab(a,b):
if tabb_len == 0:
return a
if tabb_len == 1:
return a ^ p_tabb[0]
if tabb_len > 1 && scramble_len == 0:
return a ^ p_tabb[b]
if tabb_len > 1 && scramble_len > 0:
return a ^ p_scramble[p_tabb[b]]
gen_ab_1:
a = (x << a_shl) >> a_shr
b = (x >> b_shr) & b_mask
h(x) = calc_tab(a,b)
gen_ab_2:
a = (x >> a_shl) & a_mask
b = (x << b_shr) >> b_shr
h(x) = calc_tab(a,b)
gen_ab_3:
x = x + salt
if do_xor_shr_16:
x = x ^ (x >> 16)
if do_add_shl_8:
x = x + (x << 8)
x = x ^ (x >> 4)
if a_shl == 0:
a = x >> a_shr
if a_shl != 0:
a = ((x << a_shl) + x) >> a_shr
b = (x >> b_shr) & b_mask
h(x) = calc_tab(a,b)
Obviously there is room for improvement while generating the code such as detecting masking with 0 or with -1, shifting by 0 or even the omission of the calculation of b when tabb_len <= 1.
It might make sense to think of a function which can effectively replace gen_ab_3 because this function can become quite big. All what is needed is a function which calculates a and b giving disjunct a/b pairs. Ideas include extracting two bit strings from x*c where c is a suitable constant. On most modern CPUs this can be faster than gen_ab_3.
For simplicity I have left out some stuff of Bob Jenkin's code such as the special treatment of very small label sets and strings. Most of the identifier names are still the original ones.
The interface might not be the best one can think of; I am working on it.
Options: ...
Other generated info: ...
Caveats: ...
...to be continued...
Have fun
Jasper
(c) 2013..2014 by Jasper L. Neumann
www.sirrida.de / programming.sirrida.de
E-Mail: info at sirrida.de
-------------- next part --------------
//===-- gen_hash.inc (SelectionDAGBuilder.cpp) - Selection-DAG building ---===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
// This implements switch lowering by hashing using the library hashlib.
//
//===----------------------------------------------------------------------===//
// TODO: Double dispatch raises error (MVT::Other needed), see visitJumpTable
// TODO: Replace double dispatch jump table by bit test for #dest=2 ?
// TODO: Double dispatch using 1/2/4 bit table (?)
// TODO: Lots of parameters steering double dispatch and table value sizes (?)
// TODO: Replacement of jmp [] after double dispatch by switch
// TODO: Comments on the table accesses in *.s files
// TODO: Pre-hash for 64 bit input
// TODO: Treat ranges (pushback) => BigRanges
// I don't know where to put the code
// (they currently land after jmp [] and are optimized away)
// TODO: Re-issue as new switch statement, see also rev.199025, no endless loop
// Could this be a job for SelectionDAGBuilder::Clusterify?
// See also lib/Transforms/Utils/SimplifyCFG.cpp, class SwitchLookupTable
// TODO: Compare # large ranges with # cases matched by hashing
//
// What can be done to prevent AVs in case of heap overflow?
/*@/// static void gen_rev(const tr_gen_opt &Opt, t_pointer GenStuff) */
static void gen_rev(const tr_gen_opt &Opt, t_pointer GenStuff) {
//DEBUG(dbgs() << "gen_rev: "
// << "ror_32(x+" << (t_32u)(-Opt.salt) << ", " << Opt.a_shr
// << ") * " << Opt.a_mask
// << "\n");
HashContextType *HCtx = (HashContextType *)(GenStuff);
EVT VT = MVT::i32; // HCtx->Source->getValueType();
const SelectionDAGBuilder *SDB = HCtx->SDB;
SDValue V = *HCtx->Source;
// ror_32(x - gen_opt.salt, gen_opt.a_shr) * gen_opt.a_mask;
if (Opt.salt != 0) {
V = SDB->DAG.getNode(
ISD::SUB, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.salt, VT));
}
if (Opt.a_shr != 0) {
V = SDB->DAG.getNode(
ISD::ROTR, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.a_shr, VT));
}
if (Opt.a_mask != 1) {
V = SDB->DAG.getNode(
ISD::MUL, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.a_mask, VT));
}
HCtx->Target = V;
}
/*@\\\0000000717+9504*/
/*@/// static void gen_and(const tr_gen_opt &Opt, t_pointer GenStuff) */
static void gen_and(const tr_gen_opt &Opt, t_pointer GenStuff) {
//DEBUG(dbgs() << "gen_and: "
// << "x & " << Opt.a_mask
// << "\n");
HashContextType *HCtx = (HashContextType *)(GenStuff);
EVT VT = MVT::i32; // HCtx->Source->getValueType();
const SelectionDAGBuilder *SDB = HCtx->SDB;
SDValue V = *HCtx->Source;
// x & gen_opt.a_mask;
if (Opt.a_mask != (t_32u)(-1)) {
V = SDB->DAG.getNode(
ISD::AND, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.a_mask, VT));
}
HCtx->Target = V;
}
/*@\\\+F1D2*/
/*@/// static void gen_shr(const tr_gen_opt &Opt, t_pointer GenStuff) */
static void gen_shr(const tr_gen_opt &Opt, t_pointer GenStuff) {
//DEBUG(dbgs() << "gen_shr: "
// << "x >> " << Opt.a_shr
// << "\n");
HashContextType *HCtx = (HashContextType *)(GenStuff);
EVT VT = MVT::i32; // HCtx->Source->getValueType();
const SelectionDAGBuilder *SDB = HCtx->SDB;
SDValue V = *HCtx->Source;
// x >> gen_opt.a_shr;
if (Opt.a_shr != 0) {
V = SDB->DAG.getNode(
ISD::SRL, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.a_shr, VT));
}
HCtx->Target = V;
}
/*@\\\+E1CC*/
/*@/// static void gen_rol(const tr_gen_opt &Opt, t_pointer GenStuff) */
static void gen_rol(const tr_gen_opt &Opt, t_pointer GenStuff) {
//DEBUG(dbgs() << "gen_rol: "
// << "rol_32(x, " << Opt.salt << ") & " << Opt.a_mask
// << "\n");
HashContextType *HCtx = (HashContextType *)(GenStuff);
EVT VT = MVT::i32; // HCtx->Source->getValueType();
const SelectionDAGBuilder *SDB = HCtx->SDB;
SDValue V = *HCtx->Source;
// rol_32(x, Opt.salt) & Opt.a_mask;
if (Opt.salt != 0) {
V = SDB->DAG.getNode(
ISD::ROTL, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.salt, VT));
}
if (Opt.a_mask != (t_32u)(-1)) {
V = SDB->DAG.getNode(
ISD::AND, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.a_mask, VT));
}
HCtx->Target = V;
}
/*@\\\+0614*/
/*@/// static void gen_rol_xor(const tr_gen_opt &Opt, t_pointer GenStuff) */
static void gen_rol_xor(const tr_gen_opt &Opt, t_pointer GenStuff) {
//DEBUG(dbgs() << "gen_rol: "
// << "(rol_32(x, " << Opt.salt << ") ^ x) & " << Opt.a_mask
// << "\n");
HashContextType *HCtx = (HashContextType *)(GenStuff);
EVT VT = MVT::i32; // HCtx->Source->getValueType();
const SelectionDAGBuilder *SDB = HCtx->SDB;
SDValue V = *HCtx->Source;
// (rol_32(x, Opt.salt) ^ x) & Opt.a_mask;
if (Opt.salt != 0) {
V = SDB->DAG.getNode(
ISD::ROTL, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.salt, VT));
}
V = SDB->DAG.getNode(
ISD::XOR, SDB->getCurSDLoc(), VT,
V,
*HCtx->Source);
if (Opt.a_mask != (t_32u)(-1)) {
V = SDB->DAG.getNode(
ISD::AND, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.a_mask, VT));
}
HCtx->Target = V;
}
/*@\\\0000000307+3CAC*/
/*@/// static void gen_rol_add(const tr_gen_opt &Opt, t_pointer GenStuff) */
static void gen_rol_add(const tr_gen_opt &Opt, t_pointer GenStuff) {
//DEBUG(dbgs() << "gen_rol: "
// << "(rol_32(x, " << Opt.salt << ") + x) & " << Opt.a_mask
// << "\n");
HashContextType *HCtx = (HashContextType *)(GenStuff);
EVT VT = MVT::i32; // HCtx->Source->getValueType();
const SelectionDAGBuilder *SDB = HCtx->SDB;
SDValue V = *HCtx->Source;
// (rol_32(x, Opt.salt) + x) & Opt.a_mask;
if (Opt.salt != 0) {
V = SDB->DAG.getNode(
ISD::ROTL, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.salt, VT));
}
V = SDB->DAG.getNode(
ISD::ADD, SDB->getCurSDLoc(), VT,
V,
*HCtx->Source);
if (Opt.a_mask != (t_32u)(-1)) {
V = SDB->DAG.getNode(
ISD::AND, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.a_mask, VT));
}
HCtx->Target = V;
}
/*@\\\0000000307+BC94*/
/*@/// static void gen_rol_sub(const tr_gen_opt &Opt, t_pointer GenStuff) */
static void gen_rol_sub(const tr_gen_opt &Opt, t_pointer GenStuff) {
//DEBUG(dbgs() << "gen_rol: "
// << "(rol_32(x, " << Opt.salt << ") - x) & " << Opt.a_mask
// << "\n");
HashContextType *HCtx = (HashContextType *)(GenStuff);
EVT VT = MVT::i32; // HCtx->Source->getValueType();
const SelectionDAGBuilder *SDB = HCtx->SDB;
SDValue V = *HCtx->Source;
// (rol_32(x, Opt.salt) - x) & Opt.a_mask;
if (Opt.salt != 0) {
V = SDB->DAG.getNode(
ISD::ROTL, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.salt, VT));
}
V = SDB->DAG.getNode(
ISD::SUB, SDB->getCurSDLoc(), VT,
V,
*HCtx->Source);
if (Opt.a_mask != (t_32u)(-1)) {
V = SDB->DAG.getNode(
ISD::AND, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.a_mask, VT));
}
HCtx->Target = V;
}
/*@\\\0000000307+76B5*/
/*@/// static void gen_mul(const tr_gen_opt &Opt, t_pointer GenStuff) */
static void gen_mul(const tr_gen_opt &Opt, t_pointer GenStuff) {
//DEBUG(dbgs() << "gen_mul: "
// << "(t_32u)(x * " << Opt.salt << ") >> " << Opt.a_shr
// << "\n");
HashContextType *HCtx = (HashContextType *)(GenStuff);
EVT VT = MVT::i32; // HCtx->Source->getValueType();
const SelectionDAGBuilder *SDB = HCtx->SDB;
SDValue V = *HCtx->Source;
// (t_32u)(x * gen_opt.salt) >> gen_opt.a_shr;
if (Opt.salt != 0) {
V = SDB->DAG.getNode(
ISD::MUL, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.salt, VT));
}
if (Opt.a_shr != 0) {
V = SDB->DAG.getNode(
ISD::SRL, SDB->getCurSDLoc(), VT,
V,
SDB->DAG.getConstant(Opt.a_shr, VT));
}
HCtx->Target = V;
}
/*@\\\0000000307+E7AD*/
/*@/// static void calc_tab(const tr_gen_opt &Opt, t_pointer GenStuff, SDValue &a, SDValue &b) */
static void calc_tab(const tr_gen_opt &Opt, t_pointer GenStuff,
SDValue &a, SDValue &b) {
//DEBUG(dbgs() << "calc_tab"
// << "\n");
HashContextType *HCtx = (HashContextType *)(GenStuff);
EVT VT = MVT::i32; // HCtx->Source->getValueType();
const SelectionDAGBuilder *SDB = HCtx->SDB;
if (Opt.tabb_len == 0) {
// a
// do not calculate b
HCtx->Target = a;
} else if (Opt.tabb_len == 1) {
/*@/// a ^ Opt.p_tabb[0] */
// do not calculate b, tabb[0]=const!
HCtx->Target = SDB->DAG.getNode(
ISD::XOR, SDB->getCurSDLoc(), VT,
a,
SDB->DAG.getConstant(Opt.p_tabb[0], VT));
/*@\\\+F470*/
} else {
/*@/// a ^ Opt.p_tabb[b] / a ^ Opt.p_scramble[Opt.p_tabb[b]] */
// ConvVal = Opt.p_tabb[b]
const TargetLowering *TLI = SDB->TM.getTargetLowering();
EVT PTy = TLI->getPointerTy();
SDValue HashPtr = SDB->DAG.getZExtOrTrunc(b, SDB->getCurSDLoc(), PTy);
IntegerType *ValType = IntegerType::get(*SDB->Context,HCtx->BBits);
// DEBUG(dbgs() << "ValBType=" << *ValType << "\n");
SDValue VTab = SDB->DAG.getConstantPool(HCtx->BTable, PTy,
SDB->TD->getPrefTypeAlignment(ValType));
unsigned Alignment = cast<ConstantPoolSDNode>(VTab)->getAlignment();
unsigned EltSize = (unsigned)SDB->TD->getTypeAllocSize(ValType);
SDValue VIdx = SDB->DAG.getNode(ISD::MUL, SDB->getCurSDLoc(), PTy,
HashPtr,
SDB->DAG.getConstant(EltSize, PTy));
SDValue VOfs = SDB->DAG.getNode(ISD::ADD, SDB->getCurSDLoc(), PTy,
VIdx,
VTab);
SDValue CmpVal = SDB->DAG.getLoad(
MVT::getIntegerVT(HCtx->BBits), SDB->getCurSDLoc(),
SDB->DAG.getEntryNode(),
VOfs,
MachinePointerInfo::getConstantPool(), false,
false, false, Alignment);
SDValue ConvVal =
SDB->DAG.getZExtOrTrunc(CmpVal, SDB->getCurSDLoc(), MVT::i32);
if (Opt.scramble_len != 0) {
// ConvVal = Opt.p_scramble[ConvVal]
IntegerType *ValType = IntegerType::get(*SDB->Context,HCtx->SBits);
// DEBUG(dbgs() << "ValSType=" << *ValType << "\n");
VTab = SDB->DAG.getConstantPool(HCtx->STable, PTy,
SDB->TD->getPrefTypeAlignment(ValType));
Alignment = cast<ConstantPoolSDNode>(VTab)->getAlignment();
EltSize = (unsigned)SDB->TD->getTypeAllocSize(ValType);
ConvVal = SDB->DAG.getZExtOrTrunc(ConvVal, SDB->getCurSDLoc(), PTy);
VIdx = SDB->DAG.getNode(ISD::MUL, SDB->getCurSDLoc(), PTy,
ConvVal,
SDB->DAG.getConstant(EltSize, PTy));
VOfs = SDB->DAG.getNode(ISD::ADD, SDB->getCurSDLoc(), PTy,
VIdx,
VTab);
CmpVal = SDB->DAG.getLoad(
MVT::getIntegerVT(HCtx->SBits), SDB->getCurSDLoc(),
SDB->DAG.getEntryNode(),
VOfs,
MachinePointerInfo::getConstantPool(), false,
false, false, Alignment);
ConvVal = SDB->DAG.getZExtOrTrunc(CmpVal, SDB->getCurSDLoc(), MVT::i32);
}
// Target = a ^ ConvVal
HCtx->Target = SDB->DAG.getNode(
ISD::XOR, SDB->getCurSDLoc(), VT,
a,
ConvVal);
/*@\\\0000001C36+A3AC*/
}
}
/*@\\\0000001005+1706*/
/*@/// static void gen_ab_1(const tr_gen_opt &Opt, t_pointer GenStuff) */
static void gen_ab_1(const tr_gen_opt &Opt, t_pointer GenStuff) {
//DEBUG(dbgs() << "gen_ab_1"
// << "\n");
// a = (x << Opt.a_shl) >> Opt.a_shr;
// b = (x >> Opt.b_shr) & Opt.b_mask;
// tab(a,b);
HashContextType *HCtx = (HashContextType *)(GenStuff);
EVT VT = MVT::i32; // HCtx->Source->getValueType();
const SelectionDAGBuilder *SDB = HCtx->SDB;
SDValue a = *HCtx->Source;
if (Opt.a_shl != 0) {
a = SDB->DAG.getNode(
ISD::SHL, SDB->getCurSDLoc(), VT,
a,
SDB->DAG.getConstant(Opt.a_shl, VT));
}
if (Opt.a_shr != 0) {
a = SDB->DAG.getNode(
ISD::SRL, SDB->getCurSDLoc(), VT,
a,
SDB->DAG.getConstant(Opt.a_shr, VT));
}
SDValue b = *HCtx->Source;
if (Opt.b_shr != 0) {
b = SDB->DAG.getNode(
ISD::SRL, SDB->getCurSDLoc(), VT,
b,
SDB->DAG.getConstant(Opt.b_shr, VT));
}
if (Opt.b_mask != (t_32u)(-1)) {
b = SDB->DAG.getNode(
ISD::AND, SDB->getCurSDLoc(), VT,
b,
SDB->DAG.getConstant(Opt.b_mask, VT));
}
calc_tab(Opt, GenStuff, a, b);
}
/*@\\\0000000307+BE1A*/
/*@/// static void gen_ab_2(const tr_gen_opt &Opt, t_pointer GenStuff) */
static void gen_ab_2(const tr_gen_opt &Opt, t_pointer GenStuff) {
//DEBUG(dbgs() << "gen_ab_2"
// << "\n");
// a = (x >> Opt.a_shl) & Opt.a_mask;
// b = (x << Opt.b_shr) >> Opt.b_shr;
// tab(a,b);
HashContextType *HCtx = (HashContextType *)(GenStuff);
EVT VT = MVT::i32; // HCtx->Source->getValueType();
const SelectionDAGBuilder *SDB = HCtx->SDB;
SDValue a = *HCtx->Source;
if (Opt.a_shr != 0) {
a = SDB->DAG.getNode(
ISD::SRL, SDB->getCurSDLoc(), VT,
a,
SDB->DAG.getConstant(Opt.a_shr, VT));
}
if (Opt.a_mask != (t_32u)(-1)) {
a = SDB->DAG.getNode(
ISD::AND, SDB->getCurSDLoc(), VT,
a,
SDB->DAG.getConstant(Opt.a_mask, VT));
}
SDValue b = *HCtx->Source;
if (Opt.b_shl != 0) {
b = SDB->DAG.getNode(
ISD::SHL, SDB->getCurSDLoc(), VT,
b,
SDB->DAG.getConstant(Opt.b_shl, VT));
}
if (Opt.b_shr != 0) {
b = SDB->DAG.getNode(
ISD::SRL, SDB->getCurSDLoc(), VT,
b,
SDB->DAG.getConstant(Opt.b_shr, VT));
}
calc_tab(Opt, GenStuff, a, b);
}
/*@\\\000000030F+9893*/
/*@/// static void gen_ab_3(const tr_gen_opt &Opt, t_pointer GenStuff) */
static void gen_ab_3(const tr_gen_opt &Opt, t_pointer GenStuff) {
//DEBUG(dbgs() << "gen_ab_3"
// << "\n");
// x = x + Opt.salt;
// if (Opt.do_xor_shr_16)
// x = x ^ (x >> 16);
// if (Opt.do_add_shl_8)
// x = x + (x << 8);
// x = x ^ (x >> 4);
// if (Opt.a_shl == 0)
// a = x >> Opt.a_shr;
// else
// a = ((x << Opt.a_shl) + x) >> Opt.a_shr;
// b = (x >> Opt.b_shr) & Opt.b_mask;
// tab(a,b);
HashContextType *HCtx = (HashContextType *)(GenStuff);
EVT VT = MVT::i32; // HCtx->Source->getValueType();
const SelectionDAGBuilder *SDB = HCtx->SDB;
SDValue a,b,t;
SDValue x = *HCtx->Source;
// x = x + Opt.salt;
if (Opt.salt != 0) {
x = SDB->DAG.getNode(
ISD::ADD, SDB->getCurSDLoc(), VT,
x,
SDB->DAG.getConstant(Opt.salt, VT));
}
if (Opt.do_xor_shr_16) {
// x = x ^ (x >> 16);
t = SDB->DAG.getNode(
ISD::SRL, SDB->getCurSDLoc(), VT,
x,
SDB->DAG.getConstant(16, VT));
x = SDB->DAG.getNode(
ISD::XOR, SDB->getCurSDLoc(), VT,
t,
x);
}
if (Opt.do_add_shl_8) {
// x = x + (x << 8);
t = SDB->DAG.getNode(
ISD::SHL, SDB->getCurSDLoc(), VT,
x,
SDB->DAG.getConstant(8, VT));
x = SDB->DAG.getNode(
ISD::ADD, SDB->getCurSDLoc(), VT,
t,
x);
}
// x = x ^ (x >> 4);
t = SDB->DAG.getNode(
ISD::SRL, SDB->getCurSDLoc(), VT,
x,
SDB->DAG.getConstant(4, VT));
x = SDB->DAG.getNode(
ISD::XOR, SDB->getCurSDLoc(), VT,
t,
x);
if (Opt.a_shl == 0) {
// a = x >> Opt.a_shr;
a = SDB->DAG.getNode(
ISD::SRL, SDB->getCurSDLoc(), VT,
x,
SDB->DAG.getConstant(Opt.a_shr, VT));
} else {
// a = ((x << Opt.a_shl) + x) >> Opt.a_shr;
a = SDB->DAG.getNode(
ISD::SHL, SDB->getCurSDLoc(), VT,
x,
SDB->DAG.getConstant(Opt.a_shl, VT));
a = SDB->DAG.getNode(
ISD::ADD, SDB->getCurSDLoc(), VT,
a,
x);
a = SDB->DAG.getNode(
ISD::SRL, SDB->getCurSDLoc(), VT,
a,
SDB->DAG.getConstant(Opt.a_shr, VT));
}
// b = (x >> Opt.b_shr) & Opt.b_mask;
b = SDB->DAG.getNode(
ISD::SRL, SDB->getCurSDLoc(), VT,
x,
SDB->DAG.getConstant(Opt.b_shr, VT));
b = SDB->DAG.getNode(
ISD::AND, SDB->getCurSDLoc(), VT,
b,
SDB->DAG.getConstant(Opt.b_mask, VT));
calc_tab(Opt, GenStuff, a, b);
}
/*@\\\0000006602+730C*/
/*@/// bool SelectionDAGBuilder::handleHashSwitchCase(CaseRec &CR, ...) */
bool SelectionDAGBuilder::handleHashSwitchCase(CaseRec &CR,
CaseRecVector &WorkList,
const Value *SV,
MachineBasicBlock *Default,
MachineBasicBlock *SwitchBB) {
/*@/// first checks */
const TargetLowering *TLI = TM.getTargetLowering();
EVT PTy = TLI->getPointerTy();
if (! areJTsAllowed(*TLI))
return false;
if (!TLI->isOperationLegal(ISD::ROTL, PTy))
return false;
if (!TLI->isOperationLegal(ISD::ROTR, PTy))
return false;
if (!TLI->isOperationLegal(ISD::SHL, PTy))
return false;
if (!TLI->isOperationLegal(ISD::SRL, PTy))
return false;
if (!TLI->isOperationLegal(ISD::MUL, PTy))
return false;
Case& FrontCase = *CR.Range.first;
Case& BackCase = *(CR.Range.second-1);
const APInt &First = cast<ConstantInt>(FrontCase.Low)->getValue();
const APInt &Last = cast<ConstantInt>(BackCase.High)->getValue();
if (First.getBitWidth() > 32) {
DEBUG(dbgs()
<< "handleHashSwitchCase: Too many bits, NeededBits="
<< First.getBitWidth()
<< "\n");
return false;
}
APInt TSize(First.getBitWidth(), 0); // sum of all ranges
typedef std::map<const MachineBasicBlock *, uint64_t> MBBMap;
MBBMap DestMap; // map of all used jump targets
std::vector<MachineBasicBlock *> DestVector;
std::pair<MBBMap::iterator,bool> ret;
uint64_t NrIfs = 0; // # if needed for a decision tree
uint64_t NrRanges = 0; // # of ranges of single values
uint64_t MaxRange = 0; // largest range
uint64_t DestIndex = 0; // # different jump targets
for (CaseItr I = CR.Range.first, E = CR.Range.second; I != E; ++I) {
uint64_t Size = I->size().getLimitedValue(UINT64_MAX);
if (Size <= SwitchPerfHashMaxRange) {
ret = DestMap.insert(MBBMap::value_type(I->BB,DestIndex));
if (ret.second) {
++DestIndex;
DestVector.push_back(I->BB);
}
TSize += I->size();
++NrRanges;
if (MaxRange < Size)
MaxRange = Size;
if (Size <= 1)
NrIfs += 1;
else
NrIfs += 2;
}
}
DEBUG(dbgs()
// << "# SwitchHashMinIf=" << SwitchHashMinIf
<< "# ranges=" << NrRanges
<< ", MaxRange=" << MaxRange
<< ", TSize=" << TSize
<< ", # ifs=" << NrIfs
<< ", # DestMap=" << DestMap.size() // =DestIndex
<< "\n");
if (TSize.ult(TLI->getMinimumJumpTableEntries())) {
DEBUG(dbgs()
<< "handleHashSwitchCase: Not enough entries\n");
return false;
}
uint64_t IntTSize = TSize.getLimitedValue(UINT64_MAX/100);
if (IntTSize > SwitchPerfHashMaxCases) {
DEBUG(dbgs()
<< "Too many cases, IntTSize=" << IntTSize
<< "\n");
return false;
}
if (IntTSize * SwitchHashUsage > NrIfs * 100) {
DEBUG(dbgs()
<< "Hash not dense enough, IntTSize=" << IntTSize << ", NrIfs=" << NrIfs
<< "\n");
return false;
}
if (NrIfs < SwitchHashMinIf) {
DEBUG(dbgs()
<< "Hash not enough ifs, NrIfs=" << NrIfs
<< "\n");
return false;
}
/*@\\\+4021*/
to_perfect_hash *PerfHash = new to_perfect_hash;
/*@/// PerfHash->calc_opt.* = */
if (SwitchPerfHashMulTries != 0)
PerfHash->calc_opt.nr_mul_checks = SwitchPerfHashMulTries;
// listlen_limit
// retry_initkey
// retry_perfect
PerfHash->calc_opt.use_scramble = 512;
// count_limit;
// work_limit
// min_load_factor
// near_minimal_factor
// keyspace_factor
// minimal
// fast
/*@\\\+C196*/
/*@/// PerfHash->cb.* = */
// 0x1: allow jump table, see visitSwitch
if (SwitchPerfectHashMethods & 0x2)
PerfHash->cb.a_gen[HM_REVERSIBLE] = gen_rev;
if (SwitchPerfectHashMethods & 0x4)
PerfHash->cb.a_gen[HM_SIMPLE_AND] = gen_and;
if (SwitchPerfectHashMethods & 0x8)
PerfHash->cb.a_gen[HM_SIMPLE_SHR] = gen_shr;
if (SwitchPerfectHashMethods & 0x10)
PerfHash->cb.a_gen[HM_SIMPLE_ROL] = gen_rol;
if (SwitchPerfectHashMethods & 0x20)
PerfHash->cb.a_gen[HM_SIMPLE_ROL_XOR] = gen_rol_xor;
if (SwitchPerfectHashMethods & 0x40)
PerfHash->cb.a_gen[HM_SIMPLE_ROL_ADD] = gen_rol_add;
if (SwitchPerfectHashMethods & 0x80)
PerfHash->cb.a_gen[HM_SIMPLE_ROL_SUB] = gen_rol_sub;
if (SwitchPerfectHashMethods & 0x100)
PerfHash->cb.a_gen[HM_SIMPLE_MUL] = gen_mul;
if (SwitchPerfectHashMethods & 0x200)
PerfHash->cb.a_gen[HM_AB_1] = gen_ab_1;
if (SwitchPerfectHashMethods & 0x400)
PerfHash->cb.a_gen[HM_AB_2] = gen_ab_2;
if (SwitchPerfectHashMethods & 0x800)
PerfHash->cb.a_gen[HM_AB_3] = gen_ab_3;
/*@\\\003200030D000318000101+371E*/
/*@/// PerfHash->add_key(...) */
bool OK = true;
assert(BigRanges.empty());
// BigRanges.clear();
for (CaseItr I = CR.Range.first, E = CR.Range.second; I != E; ++I) {
uint64_t Size = I->size().getLimitedValue(UINT64_MAX);
if (Size > SwitchPerfHashMaxRange) {
// FIXME: activate line below when big range handling works
// BigRanges.push_back(*I);
OK = false;
} else {
APInt N = cast<ConstantInt>(I->Low)->getValue(),
N2 = cast<ConstantInt>(I->High)->getValue();
for (; N.ule(N2); ++N)
PerfHash->add_key((uint32_t)(*N.getRawData()), (t_pointer)this);
}
}
/*@\\\0000000B0D+AE40*/
/*@/// other checks */
if (! OK) {
DEBUG(dbgs()
<< "handleHashSwitchCase: Too large ranges detected\n");
delete(PerfHash);
BigRanges.clear();
return false;
}
/*@\\\000000033A+2003*/
if (! PerfHash->generate_hash()) {
DEBUG(dbgs()
<< "handleHashSwitchCase: No perfect hash found\n");
delete(PerfHash);
BigRanges.clear();
return false;
}
DEBUG(dbgs()
<< "handleHashSwitchCase: Perfect hash found\n");
/*@/// generate code */
// Get the MachineFunction which holds the current MBB. This is used when
// inserting any additional MBBs necessary to represent the switch.
MachineFunction *CurMF = FuncInfo.MF;
// Figure out which block is immediately after the current one.
MachineFunction::iterator BBI = CR.CaseBB;
++BBI;
const BasicBlock *LLVMBB = CR.CaseBB->getBasicBlock();
// Create a new basic block to hold the code for loading the address
// of the jump table, and jumping to it. Update successor information;
// we will either branch to the default case for the switch, or the jump
// table.
MachineBasicBlock *JumpTableBB = CurMF->CreateMachineBasicBlock(LLVMBB);
CurMF->insert(BBI, JumpTableBB);
addSuccessorWithWeight(CR.CaseBB, Default);
addSuccessorWithWeight(CR.CaseBB, JumpTableBB);
// Build a vector of destination BBs, corresponding to each target
// of the jump table. If the value of the jump table slot corresponds to
// a case statement, push the case's BB onto the vector, otherwise, push
// the default BB.
if (! PerfHash->gen_opt.is_minimal) {
ret = DestMap.insert(MBBMap::value_type(Default,DestIndex));
if (ret.second) {
++DestIndex;
DestVector.push_back(Default);
}
}
uint32_t HashRange;
if (PerfHash->gen_opt.check_range)
HashRange = PerfHash->gen_opt.hash_used_max+1;
else
HashRange = PerfHash->gen_opt.hash_gen_max+1;
// dbgs() << "HashRange=" << HashRange << "\n";
bool DoubleDispatch =
SwitchDoubleDispatch &&
(DestMap.size() <= 65536) && // use bytes/words
(DestMap.size() > 1) && // to get this special case running
(HashRange >= 8) && // TODO: 8 => parameter
(HashRange > DestMap.size()*3); // TODO: 3 => parameter
uint32_t BB_range;
int DBits = 0;
std::vector<MachineBasicBlock*> DestBBs;
Constant *DTable = 0;
if (DoubleDispatch) {
/*@/// double dispatch */
if (DestMap.size() <= 256)
DBits = 8;
else
DBits = 16;
BB_range = DestMap.size();
{for (uint32_t H = 0; H < BB_range; ++H) {
DestBBs.push_back(DestVector[H]);
}}
IntegerType *ValType = IntegerType::get(*Context, DBits);
Constant **DestIndex = new Constant *[HashRange];
// Fill with dummys
{
uint32_t index = DestMap.find(Default)->second;
for (uint32_t H = 0; H < HashRange; ++H)
DestIndex[H] = ConstantInt::get(ValType, index, false);
}
// And now the real entries
{for (CaseItr I = CR.Range.first, E = CR.Range.second; I != E; ++I) {
uint32_t index = DestMap.find(I->BB)->second;
for (APInt N = cast<ConstantInt>(I->Low)->getValue(),
N2 = cast<ConstantInt>(I->High)->getValue();
N.ule(N2); ++N) {
uint32_t H = PerfHash->calc_hash((uint32_t)(*N.getRawData()));
assert((H < HashRange) && "hash out of range");
DestIndex[H] = ConstantInt::get(ValType, index, false);
}
}}
// Create a ConstantArray of the constants.
DTable = ConstantArray::get(
ArrayType::get(ValType, HashRange),
ArrayRef<Constant *>(DestIndex, HashRange));
/*@\\\0000002233+BF13*/
} else {
/*@/// normal dispatch */
BB_range = HashRange;
// Fill with dummys
{for (uint32_t H = 0; H < BB_range; ++H) {
DestBBs.push_back(Default);
}}
// And now the real entries
{for (CaseItr I = CR.Range.first, E = CR.Range.second; I != E; ++I) {
for (APInt N = cast<ConstantInt>(I->Low)->getValue(),
N2 = cast<ConstantInt>(I->High)->getValue();
N.ule(N2); ++N) {
uint32_t H = PerfHash->calc_hash((uint32_t)(*N.getRawData()));
assert((H < HashRange) && "hash out of range");
DestBBs[H] = I->BB;
// dbgs() << I->BB << "\n";
}
}}
/*@\\\+8D71*/
}
// Calculate weight for each unique destination in CR.
DenseMap<MachineBasicBlock*, uint32_t> DestWeights;
if (FuncInfo.BPI) {
for (CaseItr I = CR.Range.first, E = CR.Range.second; I != E; ++I) {
DenseMap<MachineBasicBlock*, uint32_t>::iterator Itr =
DestWeights.find(I->BB);
if (Itr != DestWeights.end())
Itr->second += I->ExtraWeight;
else
DestWeights[I->BB] = I->ExtraWeight;
}
}
// Update successor info. Add one edge to each unique successor.
BitVector SuccsHandled(CR.CaseBB->getParent()->getNumBlockIDs());
for (std::vector<MachineBasicBlock*>::iterator I = DestBBs.begin(),
E = DestBBs.end(); I != E; ++I) {
if (!SuccsHandled[(*I)->getNumber()]) {
SuccsHandled[(*I)->getNumber()] = true;
DenseMap<MachineBasicBlock*, uint32_t>::iterator Itr =
DestWeights.find(*I);
addSuccessorWithWeight(JumpTableBB, *I,
Itr != DestWeights.end() ? Itr->second : 0);
}
}
// Create a jump table index for this jump table.
unsigned JTEncoding = TLI->getJumpTableEncoding();
unsigned JTI = CurMF->getOrCreateJumpTableInfo(JTEncoding)
->createJumpTableIndex(DestBBs);
// Set the jump table information so that we can codegen it as a second
// MachineBasicBlock
JumpTable JT(-1U, JTI, JumpTableBB, Default, DTable, DBits);
JT.JSize = DestMap.size(); // not necessarily correct but sufficient
JT.SingleBB = FrontCase.BB;
PerfHash->ref_count = 1; // prepare and allow for copying JTH struct
JumpTableHeader JTH(First, Last, SV, CR.CaseBB, (CR.CaseBB == SwitchBB),
PerfHash);
if (CR.CaseBB == SwitchBB)
visitJumpTableHeader(JT, JTH, SwitchBB);
JTCases.push_back(JumpTableBlock(JTH, JT));
// FIXME: This does not put the code into the right location (after ind. jump)
if (! BigRanges.empty()) {
BigRange = CaseRange(BigRanges.begin(), BigRanges.end());
MachineFunction *CurMF = FuncInfo.MF;
// const BasicBlock *LLVMBB = CR.CaseBB->getBasicBlock();
const BasicBlock *LLVMBB = Default->getBasicBlock();
// const BasicBlock *LLVMBB = SwitchBB->getBasicBlock();
MachineBasicBlock *BigBB = CurMF->CreateMachineBasicBlock(LLVMBB);
CurMF->insert(BBI, BigBB);
WorkList.push_back(CaseRec(BigBB, CR.LT, CR.GE, BigRange));
}
// delete(PerfHash) not here since JTH contains the reference
SplitMiddle = true;
return true;
/*@\\\0000007506+761E*/
}
/*@\\\0000000D01+6587*/
/*@\\\0001000011002F01*/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hashlib.cpp
Type: text/x-c++src
Size: 62828 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140116/328d7376/attachment.cpp>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hashlib.hpp
Type: text/x-c++hdr
Size: 15775 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140116/328d7376/attachment.hpp>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: switchgen.cpp
Type: text/x-c++src
Size: 2575 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140116/328d7376/attachment-0001.cpp>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hashtest.cpp
Type: text/x-c++src
Size: 9258 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140116/328d7376/attachment-0002.cpp>
More information about the llvm-commits
mailing list