<div dir="ltr">As of r212660, lldb tests run cleanly on MacOSX 10.9.4 for x86_64 (i.e. -A x86_64). I've marked as XFAIL all tests I found that consistently fail, and marked the two tests I found intermittent there as skip on MacOSX. I filed bugs for all tests that were marked XFAIL or skip.<div>
<br></div><div>Tests also run reasonably fast on MacOSX. So - if you are on OSX, please run the tests, and if any of them fail, you can be reasonably assured that there is a decent chance that the pending change may be introducing it. </div>
<div><br></div><div>I'm about to make one more pass on the Linux side for the same goal: so that test failures mean new breakages. (Note we still need to go back and address all the bugs filed against XFAIL/skipped tests, but at least new breakages can be detected much more readily --- any test failure should raise suspicion).</div>
<div><br></div><div>Happy days!</div><div><br></div><div>-Todd</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jul 8, 2014 at 1:50 PM, Todd Fiala <span dir="ltr"><<a href="mailto:tfiala@google.com" target="_blank">tfiala@google.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">FWIW I got the same set of 11 errors on MacOSX 10.10 (14A283o) - Yosemite Preview Update 3 - and Xcode Version 6.0 (6A254o) - Xcode 6 Beta 3.<div>
<br></div><div>The first set was from OS X 10.9.3 with Xcode 6 Beta 2.</div>
</div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jul 8, 2014 at 10:42 AM, Todd Fiala <span dir="ltr"><<a href="mailto:todd.fiala@gmail.com" target="_blank">todd.fiala@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Here's how I run the tests:<div><br></div><div><div>Make sure your Xcode project "locations" setting is set to "relative", so that the DerivedData is stored relative to your lldb source directory. (Above, that will be /my/workspace-root/lldb/DerivedData).</div>
</div><div><br></div><div>I have the canonical MacOSX Xcode source layout:</div><div><br></div><div><font face="courier new, monospace">/my/workspace-root/</font></div><div><font face="courier new, monospace"> |- lldb/</font></div>
<div><font face="courier new, monospace"> |- llvm/</font></div><div><font face="courier new, monospace"> |- tools/clang/</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">$ cd /my/workspace-root</font></div>
<div><font face="courier new, monospace">$ mkdir test-run</font></div><div><font face="courier new, monospace">$ cd test-run</font></div><div><font face="courier new, monospace"># {save attached do-tests.sh file here}</font></div>
<div><font face="courier new, monospace">$ chmod a+x do-tests.sh</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># Now run it - this will run the tests using one parallel test per core. Tests are chunked up in units of directories, so all tests in a given directory are serialized.</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">$ ./do-tests.sh</font></div><div><br></div><div>The script currently only runs x86_64 - just adjust the -A value if you want to run more/different architectures in the tests. It also assumes that it will put test output into the test-run/lldb-test-results directory.</div>
<div><br></div><div>The behavior is no different from the previous multithreaded behavior on Linux/FreeBSD test runs.</div><div><br></div><div>Let me know if you run into any issues with that.</div></div><div class="gmail_extra">
<br><br><div class="gmail_quote"><div><div>On Tue, Jul 8, 2014 at 9:50 AM, Todd Fiala <span dir="ltr"><<a href="mailto:tfiala@google.com" target="_blank">tfiala@google.com</a>></span> wrote:<br></div></div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>
<div dir="ltr">For the record, these are the set of tests that currently fail for me on MacOSX 10.9.3 as of svn r212548 on MacOSX:<div><br></div><div><div>Ran 297 tests.</div><div>Failing Tests (11)</div><div>FAIL: LLDB (suite) :: TestDataFormatterObjC.py (Darwin <a href="http://tfiala-macpro.mtv.corp.google.com" target="_blank">tfiala-macpro.mtv.corp.google.com</a> 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64 i386)</div>
<div>FAIL: LLDB (suite) :: TestDataFormatterStdMap.py (Darwin <a href="http://tfiala-macpro.mtv.corp.google.com" target="_blank">tfiala-macpro.mtv.corp.google.com</a> 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64 i386)</div>
<div>FAIL: LLDB (suite) :: TestDataFormatterStdVector.py (Darwin <a href="http://tfiala-macpro.mtv.corp.google.com" target="_blank">tfiala-macpro.mtv.corp.google.com</a> 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64 i386)</div>
<div>FAIL: LLDB (suite) :: TestProcessLaunch.py (Darwin <a href="http://tfiala-macpro.mtv.corp.google.com" target="_blank">tfiala-macpro.mtv.corp.google.com</a> 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64 i386)</div>
<div>FAIL: LLDB (suite) :: TestRegisterVariables.py (Darwin <a href="http://tfiala-macpro.mtv.corp.google.com" target="_blank">tfiala-macpro.mtv.corp.google.com</a> 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64 i386)</div>
<div>FAIL: LLDB (suite) :: TestObjCMethods.py (Darwin <a href="http://tfiala-macpro.mtv.corp.google.com" target="_blank">tfiala-macpro.mtv.corp.google.com</a> 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64 i386)</div>
<div>FAIL: LLDB (suite) :: TestHiddenIvars.py (Darwin <a href="http://tfiala-macpro.mtv.corp.google.com" target="_blank">tfiala-macpro.mtv.corp.google.com</a> 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64 i386)</div>
<div>FAIL: LLDB (suite) :: TestObjCDynamicSBType.py (Darwin <a href="http://tfiala-macpro.mtv.corp.google.com" target="_blank">tfiala-macpro.mtv.corp.google.com</a> 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64 i386)</div>
<div>FAIL: LLDB (suite) :: TestObjCDynamicValue.py (Darwin <a href="http://tfiala-macpro.mtv.corp.google.com" target="_blank">tfiala-macpro.mtv.corp.google.com</a> 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64 i386)</div>
<div>FAIL: LLDB (suite) :: TestRealDefinition.py (Darwin <a href="http://tfiala-macpro.mtv.corp.google.com" target="_blank">tfiala-macpro.mtv.corp.google.com</a> 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64 i386)</div>
<div>FAIL: LLDB (suite) :: TestTargetAPI.py (Darwin <a href="http://tfiala-macpro.mtv.corp.google.com" target="_blank">tfiala-macpro.mtv.corp.google.com</a> 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64 i386)</div>
<div><br></div><div><br></div><div>Also, there were no performance regressions detected on Linux multithreaded test run for the change in implementation of the multi-core testing. What I changed was implementing the worker threads via Python's multiprocessing.Pool class instead of using the threading module's threads. All the MacOSX threads were locked on the Python global interpreter lock. Not entirely sure what the diff was that prevented that in Linux and FreeBSD, but no matter. The Pool implementation is slightly cleaner anyway.</div>
<span><font color="#888888">
<div><br></div><div>-Todd</div></font></span></div></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jul 8, 2014 at 9:46 AM, Todd Fiala <span dir="ltr"><<a href="mailto:tfiala@google.com" target="_blank">tfiala@google.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi all,<div><br></div><div>I've updated the test runner for lldb so it now runs multithreaded on MacOSX with similar performance gains to the Linux/FreeBSD test speedups.</div>
<div><br></div><div>On my MacBookPro (mid-2012 Retina), test runs when from ~28 minutes to 7.5 minutes.</div>
<div>On my MacPro (late-2013), test runs went from ~25 minutes to 3.5 minutes.</div><div><br></div><div>It's totally dependent on the number of cores, so YMMV, but it is faster.</div><div><br></div><div>Also, I've been discussing ways of mitigating tests that are load-sensitive (which multithreaded test running exposes). I'm looking at adding a new test declaration that lists a test as load sensitive (@load_sensitive_test), and somehow allowing that to fail in the multithreaded pass without failing the test run. Then, run all the load-sensitive tests in a follow-up single-threaded pass if they failed under load. Then, only mark them as failed if they fail under the single-threaded pass. I'll have more to say on that when I get some time behind it. If we do something like that, it should eliminate the tests that are hard to simplify and work under load, without forcing us to run all tests in a *much longer* single-threaded manner.<span><font color="#888888"><br clear="all">
<div><br></div>-- <br><div dir="ltr"><table cellspacing="0" cellpadding="0" style="color:rgb(136,136,136);font-family:'Times New Roman'"><tbody><tr style="color:rgb(85,85,85);font-family:sans-serif;font-size:small">
<td nowrap style="border-top-style:solid;border-top-color:rgb(213,15,37);border-top-width:2px">Todd Fiala |</td><td nowrap style="border-top-style:solid;border-top-color:rgb(51,105,232);border-top-width:2px"> Software Engineer |</td>
<td nowrap style="border-top-style:solid;border-top-color:rgb(0,153,57);border-top-width:2px"> <a href="mailto:tfiala@google.com" style="color:rgb(17,85,204)" target="_blank"><span style="background-color:rgb(255,255,204);color:rgb(34,34,34);background-repeat:initial initial">tfiala@google.com</span></a> |</td>
<td nowrap style="border-top-style:solid;border-top-color:rgb(238,178,17);border-top-width:2px"><font color="#1155cc"> <a>650-943-3180</a></font></td></tr></tbody></table><br></div>
</font></span></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><table cellspacing="0" cellpadding="0" style="color:rgb(136,136,136);font-family:'Times New Roman'"><tbody><tr style="color:rgb(85,85,85);font-family:sans-serif;font-size:small">
<td nowrap style="border-top-style:solid;border-top-color:rgb(213,15,37);border-top-width:2px">Todd Fiala |</td><td nowrap style="border-top-style:solid;border-top-color:rgb(51,105,232);border-top-width:2px"> Software Engineer |</td>
<td nowrap style="border-top-style:solid;border-top-color:rgb(0,153,57);border-top-width:2px"> <a href="mailto:tfiala@google.com" style="color:rgb(17,85,204)" target="_blank"><span style="background-color:rgb(255,255,204);color:rgb(34,34,34);background-repeat:initial initial">tfiala@google.com</span></a> |</td>
<td nowrap style="border-top-style:solid;border-top-color:rgb(238,178,17);border-top-width:2px"><font color="#1155cc"> <a>650-943-3180</a></font></td></tr></tbody></table><br></div>
</div>
</div></div><br></div></div>_______________________________________________<br>
lldb-dev mailing list<br>
<a href="mailto:lldb-dev@cs.uiuc.edu" target="_blank">lldb-dev@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev</a><br>
<br></blockquote></div><span><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div dir="ltr">-Todd</div>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><table cellspacing="0" cellpadding="0" style="color:rgb(136,136,136);font-family:'Times New Roman'"><tbody><tr style="color:rgb(85,85,85);font-family:sans-serif;font-size:small">
<td nowrap style="border-top-style:solid;border-top-color:rgb(213,15,37);border-top-width:2px">Todd Fiala |</td><td nowrap style="border-top-style:solid;border-top-color:rgb(51,105,232);border-top-width:2px"> Software Engineer |</td>
<td nowrap style="border-top-style:solid;border-top-color:rgb(0,153,57);border-top-width:2px"> <a href="mailto:tfiala@google.com" style="color:rgb(17,85,204)" target="_blank"><span style="background-color:rgb(255,255,204);color:rgb(34,34,34);background-repeat:initial initial">tfiala@google.com</span></a> |</td>
<td nowrap style="border-top-style:solid;border-top-color:rgb(238,178,17);border-top-width:2px"><font color="#1155cc"> <a>650-943-3180</a></font></td></tr></tbody></table><br></div>
</div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr">-Todd</div>
</div>