On Mon, Sep 3, 2012 at 9:41 AM, Andy Gibbs <span dir="ltr"><<a href="mailto:andyg1001@hotmail.co.uk" target="_blank">andyg1001@hotmail.co.uk</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On Saturday, August 25, 2012 3:51 AM, Richard Smith wrote:<br>
</div><div class="im">> Have you measured the impact of this cache on "well-written" constexpr<br>
> functions (those not requiring memoization for efficiency)? What's the<br>
> memory usage and performance impact on, say,<br>
> test/SemaCXX/constexpr-turing.cpp, with a more complex Turing machine?<br>
> Can you provide some figures for typical and worst-case slowdowns?<br>
<br>
</div>Ok, I've done some performance testing and here are some rough values.<br>
<br>
There are two clang test-cases which are interesting with regard to the<br>
memoisation feature, namely constexpr-turing and constexpr-nqueens. I<br>
imagine that you would consider these to be "well written" -- it seems,<br>
at any rate, that you are the author, so I am guessing so! :o)<br></blockquote><div><br></div><div>They have the property of not needing memoization in order to be efficient, which is the one I was looking for.</div><div>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
With constexpr-turing, we have a test-case that is unable to make use<br>
of the cached evaluations. In this case, I see a 5% slow-down in<br>
compilation, but also a moderate additional ~200Kb memory consumption.<br></blockquote><div><br></div><div>That's about 2KB per step, which seems rather a lot. Almost every function call in this testcase is passing references to temporaries as arguments. Perhaps that would be a reasonable criterion under which to disable the caching -- would we realistically ever get cache hits with temporaries as arguments? In practice, I think this would mean dropping your local cache.</div>
<div><br></div><div>Some part of the problem here is that APValue is a terrible format for long-term storage of constant values. I think we could get around a 5x reduction in the memory usage of APValues by replacing the representation with something denser. (I had a patch for this out for review ~9 months ago, but it'll take quite a bit of work to get it to apply cleanly to trunk...)</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
With constexpr-nqueens, we have a much more positive result in terms<br>
of a 19% speed-up in compilation, but at a cost of ~13Mb of additional<br>
memory.<br></blockquote><div><br></div><div>That's a worryingly high memory overhead here; again, we have lots of temporaries being passed as arguments in this case.</div><div><br></div><div>Do you know where the speedup comes from? There should be very few repeated constexpr function evaluations in this testcase. Are we evaluating some of the top-level expressions (such as q8's initializer or the static_assert condition) multiple times? I've observed in the past some pretty pathological behavior where we evaluate non-type template arguments repeatedly.</div>
<div><br></div><div>(In short, I wonder whether there's another bug here which this caching is working around.)</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
In my own code, I have seen anything in the range of 2% slow-down up to<br>
25% speed-up -- this is, of course, only for "well written" constexpr<br>
functions since I get a 2000% to 5000% speed-up for the rest, at least<br>
for those I was willing to test without caching (the others being too<br>
slow)!<br>
<br>
In terms of additional compiler memory, I've seen up to 100Mb increase<br>
but this is really down to constexpr algorithms in use.<br></blockquote><div>[...]</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
We could still take the route of an attribute/f-switch pair to<br>
override the behaviour of the cache. My suggestion, if you feel the<br>
performance changes are too costly, would be a -fconstexpr-cache on/<br>
off switch for global use and a __attribute__((constexpr-cache on/<br>
off)) for local use.</blockquote><div><br></div><div>I would strongly prefer to not add user-visible knobs here. If we can add a few simple rules for when to cache and when not to, I think that would be much better.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
> There is a nasty corner case where the value of a global changes<br>
> between successive calls to a constexpr function (this happens if a<br>
> variable is read after it's zero-initialized but before it's<br>
> dynamically initialized, from a constexpr function called within its<br>
> own initializer). Your patch doesn't seem to handle that (though<br>
> perhaps I just missed it).<br>
<br>
</div>I fixed the code to support your corner case. I'm still a little bit<br>
surprised that it isn't outlawed, but I've seen your other posts on the<br>
net talking about what's permitted and not, so I'm more than willing to<br>
accept your better understanding. For myself, I'll class it as "don't<br>
try this at home"!</blockquote><div><br></div><div>It's definitely in that category, and the standard itself isn't perfectly clear. Independent of what the standard says, this is important for cases which aren't constant expressions, but which we constant-fold anyway, where we must correctly model the zero-initialization-precedes-dynamic-initialization semantics.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
> Have you considered using llvm::FoldingSet for the cache? That would<br>
> be particularly useful if we ever want to allow arbitrary literal<br>
> types as non-type template arguments, or other such cases which might<br>
> require reducing an APValue to a FoldingSetNodeID.<br>
<br>
</div>I believe this means supporting FoldingSetTrait for APValues, which is<br>
not currently done -- I don't have the time to do this at present. Can<br>
it be left to another day?</blockquote><div><br></div><div>I think this would significantly simplify your implementation (you wouldn't need both a hash function and an equality comparison for APValue). You are performing some unnecessary APValue copies still, due to using a DenseMap as your cache, with APValues in your key and value types. It would be better to use the ASTContext's BumpPtrAllocator for allocating your cache entries, which is easy to do if you can switch to a FoldingSet.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
> There seems to be a fair amount of APValue copying going on in this<br>
> change. For large APValues, that's not cheap. Perhaps you could<br>
> allocate the cache entry up-front, and evaluate the function arguments<br>
> and return value directly into its APValue slots, or swap them in?<br>
<br>
</div>Done -- this took the constexpr-nqueens test-case from ~11% to ~19%<br>
speed-up, for example.<br>
<div class="im"><br>
> Does this have an impact on the constexpr recursion limit? For instance,<br>
> if f(500) recurses to depth 500, and g(500) recurses to depth 500 then<br>
> calls f(500), will the result of calling g(500) depend on whether I've<br>
> called f(500) already in this translation unit? That seems like a highly<br>
> unfortunate property, but it's consistent with how template<br>
> instantiations behave at least.<br>
<br>
</div>Yes, there is an impact on recursion limit, but not in a negative way<br>
(in my opinion!), meaning that if f(500) is already cached, then calling<br>
g(500) doesn't run to a recursion depth of 1000, only 501, since f(500)<br>
is picked out of the cache and not recursively evaluated.</blockquote><div><br></div><div>This still seems like it could lead to a flaky compilation experience (we hit the limit if we include a.h then b.h, but not if we include them in the other order). This problem already exists for templates, but I suspect it'll be worse for constexpr, since I would expect deep recursion to occur more often. Essentially, I would like the caching to be *entirely* undetectable (other than in performance).</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
> Thanks for looking into this!<br>
<br>
</div>No problems. I'll try and split out the revised patches and post them<br>
shortly... Any further things before I do so?</blockquote><div><br></div><div>Well, some more comments on the new patches:</div><div><br></div><div>Firstly, there's not really any point in splitting up the patches along file boundaries as you have done. There's value in splitting up patches when they can be decomposed into pieces which can stand alone (in particular, the pieces should have their own tests and should be suitable for committing separately) -- that makes reviewing easier.</div>
<div><br></div><div>Please don't use a static data member for your global cache. The global cache should live on the ASTContext.</div><div><br></div><div><div>+ APValue FillerA, FillerB;</div><div>+ if (A.hasArrayFiller()) FillerA = A.getArrayFiller();</div>
<div>+ if (B.hasArrayFiller()) FillerB = B.getArrayFiller();</div></div><div><br></div><div>Use APValue *FillerA; etc. here, to avoid copying the APValue.</div><div><br></div><div><div>+ case APValue::Struct: {</div>
<div>+ unsigned bases = A.getStructNumBases();</div></div><div><br></div><div>This variable (and others) should be capitalized, per the coding standard.</div><div><br></div><div><div>+ if (SA->getLength() != SB->getLength() ||</div>
<div>+ SA->getCharByteWidth() != SB->getCharByteWidth() ||</div><div>+ SA->getBytes() != SB->getBytes())</div><div>+ return false;</div></div><div><br></div><div>
You can't use the contents of a string literal to determine equality for APValue::LValue, since different string literals with the same contents do not always produce the same result (the function could test the address).</div>
<div><br></div><div>I don't like changing the diagnostic consumer in order to implement your cache tests. I would prefer you found another way to do this -- perhaps adding a #pragma to produce a diagnostic containing this information?</div>
<div><br></div><div>Thanks!</div><div>Richard</div></div>