<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div>I have actually added a few benchmark tests recently. We admittedly are not that good with running those tests ever (because they're not run by default most likely - and I do wonder if some of them would take a long time to run.. I don't think I have ever run the full set, just my own as I increment on performance work).</div><div id="AppleMailSignature"><br></div><div id="AppleMailSignature">Maybe we could try flipping the default to be "run the benchmarks", see if test suite run times explode and take it from there in terms of feasibility as well as whether they all still make sense.</div><div id="AppleMailSignature"><br></div><div id="AppleMailSignature">The other problem with the tests as they stand is that they mark themselves as PASS or FAIL purely on the basis of whether they encounter command or API errors, and do nothing to track performance regressions. That is admittedly a harder problem to tackle given heterogeneous hardware and workload - but maybe we could have them fail if the timings go wildly crazy over some threshold?</div><div id="AppleMailSignature"><br>Sent from my iPhone</div><div><br>On Dec 9, 2015, at 1:22 PM, Todd Fiala via lldb-dev <<a href="mailto:lldb-dev@lists.llvm.org">lldb-dev@lists.llvm.org</a>> wrote:<br><br></div><blockquote type="cite"><div><div dir="ltr">Hey Jason,<div><br></div><div>Are you the benchmark user?</div><div><br></div><div>-Todd</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec 9, 2015 at 12:32 PM, Zachary Turner via lldb-dev <span dir="ltr"><<a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Is anyone using the benchmark tests? None of the <a href="https://docs.google.com/spreadsheets/d/1wkxAY7l0_cJOHhhsSlh3aKKlQShlX1D7X1Dn8kpqxy4/edit?pli=1#gid=0" target="_blank">command line options</a> related to the benchmark tests were claimed as being used by anyone. Which makes me wonder if the tests are even being used by anyone. <div><br></div><div>What I really want to know is: Is it really ok to delete the -x and -y command line options? And what is the status of these tests? Does anyone use them?</div></div>
<br>_______________________________________________<br>
lldb-dev mailing list<br>
<a href="mailto:lldb-dev@lists.llvm.org">lldb-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr">-Todd</div></div>
</div>
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>lldb-dev mailing list</span><br><span><a href="mailto:lldb-dev@lists.llvm.org">lldb-dev@lists.llvm.org</a></span><br><span><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev</a></span><br></div></blockquote></body></html>