<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div>Historically I would do</div><div id="AppleMailSignature"><br></div><div id="AppleMailSignature">$ ./dotest.py +b <path to my benchmark test></div><div id="AppleMailSignature"><br></div><div id="AppleMailSignature">but I am not strongly attached to that workflow - it's just what I learnt the first time I needed to run one<br><br>Sent from my iPhone</div><div><br>On Dec 9, 2015, at 2:08 PM, Zachary Turner <<a href="mailto:zturner@google.com">zturner@google.com</a>> wrote:<br><br></div><blockquote type="cite"><div><div dir="ltr">When you do run the benchmark tests, what command line options do you use? At the moment I'm mostly just trying to remove dead options from the test suite. I removed one already that allowed you to specify the benchmark executable, but then when I started looking at the rest and seeing how tightly integrated they are with the benchamrk tests in general, I started to wonder.<div><br></div><div>The three benchmark related command line options are:</div><div><br></div><div>1. An option to specify the benchmark executable (defaults to lldb.exe)</div><div>2. An option to specify the breakpoint spec (defaults to -n main)</div><div>3. An option to specify the breakpoint iteration count (defaults to 30 I think)</div><div>4. An option to specify that you only want to run benchmark tests and no other tests.</div><div><br></div><div>I deleted #4 because you can use the category system for that. I deleted #1 because nobody said they needed it on the spreadsheet. Nobody said they needed #2 or #3 either, but I just want to double check that deleting them is fine.</div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, Dec 9, 2015 at 2:01 PM Enrico Granata <<a href="mailto:egranata@apple.com">egranata@apple.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div>I have actually added a few benchmark tests recently. We admittedly are not that good with running those tests ever (because they're not run by default most likely - and I do wonder if some of them would take a long time to run.. I don't think I have ever run the full set, just my own as I increment on performance work).</div><div><br></div><div>Maybe we could try flipping the default to be "run the benchmarks", see if test suite run times explode and take it from there in terms of feasibility as well as whether they all still make sense.</div><div><br></div><div>The other problem with the tests as they stand is that they mark themselves as PASS or FAIL purely on the basis of whether they encounter command or API errors, and do nothing to track performance regressions. That is admittedly a harder problem to tackle given heterogeneous hardware and workload - but maybe we could have them fail if the timings go wildly crazy over some threshold?</div><div><br>Sent from my iPhone</div></div><div dir="auto"><div><br>On Dec 9, 2015, at 1:22 PM, Todd Fiala via lldb-dev <<a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a>> wrote:<br><br></div><blockquote type="cite"><div><div dir="ltr">Hey Jason,<div><br></div><div>Are you the benchmark user?</div><div><br></div><div>-Todd</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec 9, 2015 at 12:32 PM, Zachary Turner via lldb-dev <span dir="ltr"><<a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Is anyone using the benchmark tests? None of the <a href="https://docs.google.com/spreadsheets/d/1wkxAY7l0_cJOHhhsSlh3aKKlQShlX1D7X1Dn8kpqxy4/edit?pli=1#gid=0" target="_blank">command line options</a> related to the benchmark tests were claimed as being used by anyone. Which makes me wonder if the tests are even being used by anyone. <div><br></div><div>What I really want to know is: Is it really ok to delete the -x and -y command line options? And what is the status of these tests? Does anyone use them?</div></div>
<br>_______________________________________________<br>
lldb-dev mailing list<br>
<a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div><div dir="ltr">-Todd</div></div>
</div>
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>lldb-dev mailing list</span><br><span><a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a></span><br><span><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev</a></span><br></div></blockquote></div></blockquote></div>
</div></blockquote></body></html>