[PATCH] D41719: [InlineCost] Prevent infinite recursion on function pointers

[Easwaran Raman via Phabricator via llvm-commits]

eraman added a comment.

There are two problems this patch is trying to address:
a. A call chain I1->I2->I3...Ik, where each of the I_i ->I_(i+1) are indirect calls that become direct (because we know the target through inline analysis).  No recursion in the callgraph of the module being compiled.
b. The callgraph has recursion through a sequence of indirect calls.  This is similar to the example in this patch description but could be a more broader cycle (not just self recursion).

One consequence of this patch is that in the second example, we will end up applying a huge bonus to the original callee that is being analyzed. Note that the intent of recursively calling analyzeCall is to apply a bonus (negative cost) by subtracting the cost of indirect call from the indirect threshold parameter. Let's say that you have a self recursive call where the original cost of the body is 25. IndirectCallThreshold is 150. At the maximum depth R, we don't analyze any further and return a cost of 25. In the previous level (R-1), we give a bonus by subtracting 125 (IndirectCallThreshold - call cost (25)). So the cost becomes -125. So at the topmost level  the bonus will be something like -125R.  While the intent of the heuristic is to give some small bonus to each indirect call promoted to a direct call, this will end up applying the bonus multiple times to a call instruction.

One way to fix this is by keeping  track of the callsites visited in this process and not revisit a callsite. A simpler alternative is to limit the recursion depth to just 1 (or some very small number). I don't think we will lose out much doing so.


https://reviews.llvm.org/D41719