<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Jul 20, 2015, at 1:03 PM, Mehdi Amini <<a href="mailto:mehdi.amini@apple.com" class="">mehdi.amini@apple.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Jul 20, 2015, at 12:36 PM, Sean Silva <<a href="mailto:chisophugis@gmail.com" class="">chisophugis@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Mon, Jul 20, 2015 at 9:40 AM, Pete Cooper <span dir="ltr" class=""><<a href="mailto:peter_cooper@apple.com" target="_blank" class="">peter_cooper@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><blockquote type="cite" class=""><span class=""><div class="">On Jul 18, 2015, at 11:27 AM, Hal Finkel <<a href="mailto:hfinkel@anl.gov" target="_blank" class="">hfinkel@anl.gov</a>> wrote:</div><br class=""></span><span class="">2. We don't have a good set of tests for it, nor do we have a good set of tutorials/documentation for it. Our tutorials, specifically, are in C++, not in C. We could break the C API and we'd likely remain unaware for quite awhile.</span></blockquote>I think this is the most important point, that we lack testing for it.</div><div class=""><br class=""></div><div class="">IMO, the language doesn’t matter too much. I’m happy with C or C++, but whichever (or both) or those are exposed in a stable way, we need the *users* of those APIs to help test it.</div><div class=""><br class=""></div><div class="">How about we add a StableAPI directory in unittests? Then have a test written in C/C++ for each of the users of it. So a WebKit.c test, Go.c, SomeProject.cpp, etc.</div><div class=""><br class=""></div><div class="">Then adding anything to the stable API must have a corresponding test, and changing the API shows us exactly which test broke, who cares about that test, and who to talk to about updating that API if we need to. If the only test which breaks is WebKit then talk to WebKit, if its Go too then add them in, and so on.</div><div class=""><br class=""></div><div class="">Sure the tests will get large, but thats the point. It would show us exactly what API users care about. And the tests don’t need to actually run anything, just ensure that methods signatures are compatible with what they are using.</div><div class=""><br class=""></div></div></blockquote><div class=""><br class=""></div><div class="">Part (most?) of the point of having a stable API is as a way of decoupling the development processes of two separate projects (modulo well-documented release-to-release updating). Requiring our users to add tests in our tree doesn't really achieve much decoupling. </div></div></div></div></div></blockquote><div class=""><br class=""></div><div class="">I’m not sure there is much “coupling” here. The point is that we expose a C API that is supposed to be stable but is not well tested. And some part of the C API is just a wrapper around the C++ and hasn’t really been designed to be “stable” in time.</div><div class="">It seems also that we don’t really know what part of the C API really needs to be stable and is important for the users, so I read Pete’s proposal as “let’s collect the current use-cases and make them tests in LLVM, so that we define what is part of the stable C API and so that we won’t (inadvertently) break valid use cases”.</div></div></div></div></blockquote>Yes, exactly. We (I don’t mind who TBH) should make tests which reflect what people are doing with the API.<br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><div class=""><br class=""></div><div class="">Now if you see “coupling" because of the naming of the test, they can be made in a more abstract way by naming the test “compile_multiple_module_with_mcjit.test” instead of “webkit.test”. </div></div></div></div></blockquote>My initial thinking for actually naming the tests is that it helps to know who the main users of an API are when we do want to change it. That way they can be involved in the conversation. If you name it webkit.c, go.c, whatever and the test breaks because of something you are trying to commit, then you can see who maintains that test and ask them if they are ok with the change.</div><div><br class=""></div><div>Its like choosing who to ask for commit review on a pass for example. You look at the blame list for the pass and see who is actively developing it and ask for a review. I see the API users as being like the maintainers of the API they use, in that they should be on the emails asking for review when something changes.</div><div><br class=""></div><div>Cheers,</div><div>Pete<br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><div class=""><br class=""></div><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">As a hyperbolic analogy: imagine if libjpeg required every user to add tests into its tree for their usage of the API.</div></div></div></div></div></blockquote><div class=""><br class=""></div><div class="">I believe this is already what any reasonable API would do: they would define use cases and write test. The expectation being that the client would mimic the use-case present in the various tests. As I understand it, in our case we start in a situation where the coverage of the use-cases in tests is low and we would gather our clients use-cases to narrow down what needs to be supported.</div><div class=""><br class=""></div><div class="">As of having this in-tree or out-of-tree, I’m not sure about that and there is a trade-off.</div><div class=""><br class=""></div><div class="">— </div><div class="">Mehdi</div><div class=""><br class=""></div></div></div></div></blockquote></div><br class=""></body></html>