<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Feb 4, 2016 at 7:39 PM, Joseph Tremoulet <span dir="ltr"><<a href="mailto:jotrem@microsoft.com" target="_blank">jotrem@microsoft.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I replied to this in Phabricator via the web interface (<a href="http://reviews.llvm.org/D16513#inline-139715" rel="noreferrer" target="_blank">http://reviews.llvm.org/D16513#inline-139715</a>), but it doesn't seem to have generated e-mail, so here it is:<br>
---<br>
<span class=""><br>
> it might be simpler to a separate function, attributed with LLVM_UNUSED (so no warnings fire on it) rather than the volatile approach<br>
<br>
</span>OK, I can do that. The comment at the definition of LLVM_ATTRIBUTE_UNUSED mentions "Prefer cast-to-void wherever it is sufficient" because it's more portable, so I wonder: would it be better still to wrap this in a lambda and then just cast the closure to void?<br></blockquote><div><br></div><div>That's probably overkill/not a great improvement over what you've already got (& that's for the case where the function is called/executed - which you don't intend/want here)</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">> (also, I worry about API testing like this - if we have a formal layer concept we want to implement - we should be able to (& we should) check that independently of any particular implementation (& it should check the semantics as well - as we are testing above, presumably). To keep the tests isolated, to avoid slipping API boundaries, narrow tests, etc<br>
<br>
</span>Yeah, it felt hokey adding a "just make sure this compiles" test. I went forward with it because it is an improvement over the current state of things and I verified that it would have caught the regression that hit us downstream. I'd be happy to be pointed in the direction of something more proper (e.g. is there some way to capture this as a static_assert over type traits or something?),</blockquote><div><br></div><div>There would be, with all the usual ridiculous template metaprogramming, etc... </div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> but I don't see what that would be. You predicated the suggestion on "if we have a formal layer concept", and ISTM the issue here is that we have only an informal layer concept, because the formalism we want is precisely a `concept` describing object layers, but `concept`s haven't made it into the language yet.</blockquote><div><br></div><div>A language level construct isn't a necessary feature of the formality I was speaking of. I mean the same formality as, say, the C++ container libraries.<br><br>No one tests that std::copy works with std::list specifically, you just test that std::list meets its interface (which is a narrower interface than what std::copy accepts) and std::copy accepts generic iterators.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> I think that our informal "concept" of an object layer here is "something that has the same signature as the ObjectLinkingLayer", which is why this test conjoins the ObjectTransformLayer being tested specifically to an ObjectLinkingLayer instance, and why there are no other similar tests (we don't, to my knowledge, have any other object layers in-tree).<br>
<br>
To make it more concrete, ObjectLinkingLayer and the informal "object layer" concept have a template method with this signature:<br>
<br>
template <typename ObjSetT,<br>
typename MemoryManagerPtrT,<br>
typename SymbolResolverPtrT><br>
ObjSetHandleT addObjectSet(ObjSetT Objects,<br>
MemoryManagerPtrT MemMgr,<br>
SymbolResolverPtrT Resolver)<br>
<br>
<br>
This test was added with rL258630, which fixed a regression introduced by rL258185 adjusting the signature of the ObjectLinkingLayer (and the general/informal "object layer" "concept"), without updating the ObjectTransformLayer. The signature change actually looks benign, as the change was of the Objects parameter from `const ObjSetT &Objects` to `ObjSetT Objects`, but the issue is that the ObjectTransformLayer's attempt to forward Objects then changed to involve a copy, and ObjSetT can be instantiated to a non-copyable type. This was the best way I thought of to write a test that would catch that sort of change, but I'm happy for suggestions on a better way.<br></blockquote><div><br></div><div>Yeah, more or less I think this should be tested in two ways:<br><br>Generic tests against each API, which probably would not have caught this regression unless someone was smart enough to think about testing ObjectLinkingLayer with a non-copyable type as that parameter. That's the sort of level of unit testing the API probably should have, but even when we aspire to it we don't always get there.<br><br>The other part is that we should have some end-to-end tests (which, conveniently, are pretty cheap/fast and still "unit"-y in the sense of LLVM's regression suite, so they can probably live there without too much philosophical discord about test architecture/layering/etc)</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Lastly, I'm afraid I don't entirely understand what you mean regarding testing semantics when you say "check that independently of any particular implementation (& it should check the semantics as well - as we are testing above, presumably" -- the object layer concept, as I understand it, is essentially just a signature, so I'm not following what semantics we'd check in a way that's independent of any particular implementation. </blockquote><div><br></div><div>Test it two fold: the implementation should be tested given a wide variety of inputs that conform to its (rather broad) contract. The users should be tester with a stub/mock that fails fast on anything outside that contract.</div><div><br></div><div>Sorry, perhaps I'm still being too vague... if/when I get time I can try to work up a prototype/some patches.<br><br>- Dave</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> The ObjectTransformLayer in particular has the semantics that it applies its transformation to objects passed in and forwards everything else to the base layer; yes that is what the code above is testing, but I'm not following how you're suggesting it be restructured.<br>
<br>
<br>
Thanks<br>
<span class="HOEnZb"><font color="#888888">-Joseph<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
<br>
-----Original Message-----<br>
From: David Blaikie [mailto:<a href="mailto:dblaikie@gmail.com">dblaikie@gmail.com</a>]<br>
Sent: Wednesday, February 3, 2016 1:01 PM<br>
To: Joseph Tremoulet <<a href="mailto:jotrem@microsoft.com">jotrem@microsoft.com</a>>; <a href="mailto:lhames@gmail.com">lhames@gmail.com</a><br>
Cc: <a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
Subject: Re: [PATCH] D16513: [Unittest] Clean up formatting, NFC<br>
<br>
dblaikie added inline comments.<br>
<br>
================<br>
Comment at: unittests/ExecutionEngine/Orc/ObjectTransformLayerTest.cpp:282<br>
@@ -281,56 +281,3 @@<br>
volatile bool RunStaticChecks = false;<br>
- if (RunStaticChecks) {<br>
- // Make sure that ObjectTransformLayer implements the object layer concept<br>
- // correctly by sandwitching one between an ObjectLinkingLayer and an<br>
- // IRCompileLayer, verifying that it compiles if we have a call to the<br>
- // IRComileLayer's addModuleSet that should call the transform layer's<br>
- // addObjectSet, and also calling the other public transform layer methods<br>
- // directly to make sure the methods they intend to forward to exist on<br>
- // the ObjectLinkingLayer.<br>
-<br>
- // We'll need a concrete MemoryManager class.<br>
- class NullManager : public llvm::RuntimeDyld::MemoryManager {<br>
- public:<br>
- uint8_t *allocateCodeSection(uintptr_t, unsigned, unsigned,<br>
- llvm::StringRef) override {<br>
- return nullptr;<br>
- }<br>
- uint8_t *allocateDataSection(uintptr_t, unsigned, unsigned,<br>
- llvm::StringRef, bool) override {<br>
- return nullptr;<br>
- }<br>
- void registerEHFrames(uint8_t *, uint64_t, size_t) override {}<br>
- void deregisterEHFrames(uint8_t *, uint64_t, size_t) override {}<br>
- bool finalizeMemory(std::string *) override { return false; }<br>
- };<br>
-<br>
- // Construct the jit layers.<br>
- ObjectLinkingLayer<> BaseLayer;<br>
- auto IdentityTransform = [](<br>
- std::unique_ptr<llvm::object::OwningBinary<llvm::object::ObjectFile>><br>
- Obj) { return Obj; };<br>
- ObjectTransformLayer<decltype(BaseLayer), decltype(IdentityTransform)><br>
- TransformLayer(BaseLayer, IdentityTransform);<br>
- auto NullCompiler = [](llvm::Module &) {<br>
- return llvm::object::OwningBinary<llvm::object::ObjectFile>();<br>
- };<br>
- IRCompileLayer<decltype(TransformLayer)> CompileLayer(TransformLayer,<br>
- NullCompiler);<br>
- std::vector<llvm::Module *> Modules;<br>
-<br>
- // Make sure that the calls from IRCompileLayer to ObjectTransformLayer<br>
- // compile.<br>
- NullResolver Resolver;<br>
- NullManager Manager;<br>
- CompileLayer.addModuleSet(std::vector<llvm::Module *>(), &Manager,<br>
- &Resolver);<br>
-<br>
- // Make sure that the calls from ObjectTransformLayer to ObjectLinkingLayer<br>
- // compile.<br>
- decltype(TransformLayer)::ObjSetHandleT ObjSet;<br>
- TransformLayer.emitAndFinalize(ObjSet);<br>
- TransformLayer.findSymbolIn(ObjSet, Name, false);<br>
- TransformLayer.findSymbol(Name, true);<br>
- TransformLayer.mapSectionAddress(ObjSet, nullptr, 0);<br>
- TransformLayer.removeObjectSet(ObjSet);<br>
- }<br>
+ if (!RunStaticChecks)<br>
+ return;<br>
----------------<br>
Also, it might be simpler to a separate function, attributed with LLVM_UNUSED (so no warnings fire on it) rather than the volatile approach.<br>
<br>
(also, I worry about API testing like this - if we have a formal layer concept we want to implement - we should be able to (& we should) check that independently of any particular implementation (& it should check the semantics as well - as we are testing above, presumably). To keep the tests isolated, to avoid slipping API boundaries, narrow tests, etc)<br>
<br>
<br>
<br>
<br>
</div></div></blockquote></div><br></div></div>