[LLVMdev] [RFC] Developer Policy for LLVM C API

[Pete Cooper]

> On Jul 20, 2015, at 1:03 PM, Mehdi Amini <mehdi.amini at apple.com> wrote:
> 
> 
>> On Jul 20, 2015, at 12:36 PM, Sean Silva <chisophugis at gmail.com <mailto:chisophugis at gmail.com>> wrote:
>> 
>> 
>> 
>> On Mon, Jul 20, 2015 at 9:40 AM, Pete Cooper <peter_cooper at apple.com <mailto:peter_cooper at apple.com>> wrote:
>> 
>>> On Jul 18, 2015, at 11:27 AM, Hal Finkel <hfinkel at anl.gov <mailto:hfinkel at anl.gov>> wrote:
>>> 
>>> 2. We don't have a good set of tests for it, nor do we have a good set of tutorials/documentation for it. Our tutorials, specifically, are in C++, not in C. We could break the C API and we'd likely remain unaware for quite awhile.
>> I think this is the most important point, that we lack testing for it.
>> 
>> IMO, the language doesn’t matter too much.  I’m happy with C or C++, but whichever (or both) or those are exposed in a stable way, we need the *users* of those APIs to help test it.
>> 
>> How about we add a StableAPI directory in unittests?  Then have a test written in C/C++ for each of the users of it.  So a WebKit.c test, Go.c, SomeProject.cpp, etc.
>> 
>> Then adding anything to the stable API must have a corresponding test, and changing the API shows us exactly which test broke, who cares about that test, and who to talk to about updating that API if we need to.  If the only test which breaks is WebKit then talk to WebKit, if its Go too then add them in, and so on.
>> 
>> Sure the tests will get large, but thats the point.  It would show us exactly what API users care about.  And the tests don’t need to actually run anything, just ensure that methods signatures are compatible with what they are using.
>> 
>> 
>> Part (most?) of the point of having a stable API is as a way of decoupling the development processes of two separate projects (modulo well-documented release-to-release updating). Requiring our users to add tests in our tree doesn't really achieve much decoupling.
> 
> I’m not sure there is much “coupling” here. The point is that we expose a C API that is supposed to be stable but is not well tested. And some part of the C API is just a wrapper around the C++ and hasn’t really been designed to be “stable” in time.
> It seems also that we don’t really know what part of the C API really needs to be stable and is important for the users, so I read Pete’s proposal as “let’s collect the current use-cases and make them tests in LLVM, so that we define what is part of the stable C API and so that we won’t (inadvertently) break valid use cases”.
Yes, exactly.  We (I don’t mind who TBH) should make tests which reflect what people are doing with the API.
> 
> Now if you see “coupling" because of the naming of the test, they can be made in a more abstract way by naming the test “compile_multiple_module_with_mcjit.test” instead of “webkit.test”. 
My initial thinking for actually naming the tests is that it helps to know who the main users of an API are when we do want to change it.  That way they can be involved in the conversation.  If you name it webkit.c, go.c, whatever and the test breaks because of something you are trying to commit, then you can see who maintains that test and ask them if they are ok with the change.

Its like choosing who to ask for commit review on a pass for example.  You look at the blame list for the pass and see who is actively developing it and ask for a review.  I see the API users as being like the maintainers of the API they use, in that they should be on the emails asking for review when something changes.

Cheers,
Pete
> 
>> As a hyperbolic analogy: imagine if libjpeg required every user to add tests into its tree for their usage of the API.
> 
> I believe this is already what any reasonable API would do: they would define use cases and write test. The expectation being that the client would mimic the use-case present in the various tests. As I understand it, in our case we start in a situation where the coverage of the use-cases in tests is low and we would gather our clients use-cases to narrow down what needs to be supported.
> 
> As of having this in-tree or out-of-tree, I’m not sure about that and there is a trade-off.
> 
> — 
> Mehdi
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150720/ae6a8c56/attachment.html>