<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Jul 20, 2015, at 12:36 PM, Sean Silva <<a href="mailto:chisophugis@gmail.com" class="">chisophugis@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Mon, Jul 20, 2015 at 9:40 AM, Pete Cooper <span dir="ltr" class=""><<a href="mailto:peter_cooper@apple.com" target="_blank" class="">peter_cooper@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><blockquote type="cite" class=""><span class=""><div class="">On Jul 18, 2015, at 11:27 AM, Hal Finkel <<a href="mailto:hfinkel@anl.gov" target="_blank" class="">hfinkel@anl.gov</a>> wrote:</div><br class=""></span><span class="">2. We don't have a good set of tests for it, nor do we have a good set of tutorials/documentation for it. Our tutorials, specifically, are in C++, not in C. We could break the C API and we'd likely remain unaware for quite awhile.</span></blockquote>I think this is the most important point, that we lack testing for it.</div><div class=""><br class=""></div><div class="">IMO, the language doesn’t matter too much. I’m happy with C or C++, but whichever (or both) or those are exposed in a stable way, we need the *users* of those APIs to help test it.</div><div class=""><br class=""></div><div class="">How about we add a StableAPI directory in unittests? Then have a test written in C/C++ for each of the users of it. So a WebKit.c test, Go.c, SomeProject.cpp, etc.</div><div class=""><br class=""></div><div class="">Then adding anything to the stable API must have a corresponding test, and changing the API shows us exactly which test broke, who cares about that test, and who to talk to about updating that API if we need to. If the only test which breaks is WebKit then talk to WebKit, if its Go too then add them in, and so on.</div><div class=""><br class=""></div><div class="">Sure the tests will get large, but thats the point. It would show us exactly what API users care about. And the tests don’t need to actually run anything, just ensure that methods signatures are compatible with what they are using.</div><div class=""><br class=""></div></div></blockquote><div class=""><br class=""></div><div class="">Part (most?) of the point of having a stable API is as a way of decoupling the development processes of two separate projects (modulo well-documented release-to-release updating). Requiring our users to add tests in our tree doesn't really achieve much decoupling. </div></div></div></div></div></blockquote><div><br class=""></div><div>I’m not sure there is much “coupling” here. The point is that we expose a C API that is supposed to be stable but is not well tested. And some part of the C API is just a wrapper around the C++ and hasn’t really been designed to be “stable” in time.</div><div>It seems also that we don’t really know what part of the C API really needs to be stable and is important for the users, so I read Pete’s proposal as “let’s collect the current use-cases and make them tests in LLVM, so that we define what is part of the stable C API and so that we won’t (inadvertently) break valid use cases".</div><div><br class=""></div><div>Now if you see “coupling" because of the naming of the test, they can be made in a more abstract way by naming the test “compile_multiple_module_with_mcjit.test” instead of “webkit.test”. </div><div><br class=""></div><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">As a hyperbolic analogy: imagine if libjpeg required every user to add tests into its tree for their usage of the API.</div></div></div></div></div></blockquote><div><br class=""></div><div>I believe this is already what any reasonable API would do: they would define use cases and write test. The expectation being that the client would mimic the use-case present in the various tests. As I understand it, in our case we start in a situation where the coverage of the use-cases in tests is low and we would gather our clients use-cases to narrow down what needs to be supported.</div><div><br class=""></div><div>As of having this in-tree or out-of-tree, I’m not sure about that and there is a trade-off.</div><div><br class=""></div><div>— </div><div>Mehdi</div><div><br class=""></div></div></body></html>