<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Mar 6, 2015 at 8:09 AM, Ahmed Bougacha <span dir="ltr"><<a href="mailto:ahmed.bougacha@gmail.com" target="_blank">ahmed.bougacha@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Fri, Mar 6, 2015 at 7:04 AM, Hal Finkel <<a href="mailto:hfinkel@anl.gov">hfinkel@anl.gov</a>> wrote:<br>
</span><span class="">> Hi everyone,<br>
><br>
> In response to yet-another fix in CodeGen affecting only an out-of-tree target (r231186), our lack of the ability to properly unit test CodeGen components has been highlighted. It was suggested that improving this situation might be a good GSoC project, and I agree, provided that we can settle on the scope and basic design ahead of time.<br>
><br>
> I'd like to add that I feel this is a serious problem even for in-tree targets. We currently construct IR-level tests for CodeGen components, but<br>
> this is very fragile. Many of the IR-level CodeGen tests, especially "bug-triggering" regression tests, don't currently test the logic they were originally designed to cover.<br>
<br>
</span>Yes, yes! Doing something - anything - in the area would be great =)<br>
<span class=""><br>
> Now, for a design:<br>
><br>
> One idea that I've had for some time is to develop a 'mock' target for testing. For this target, all of the various type/operation legality settings would be determined by some input configuration file. It would contain instructions, mostly in 1:1 correspondence to our SelectionDAG node types, and many different register classes of different sizes, different calling-conventions, etc. (again, some input configuration file would determine which were active). We could then use this mock target to right regression tests for CodeGen components. We could also use it write units tests, especially at the MI level.<br>
<br>
</span>This is arguably a different and bigger issue ("Target" vs "CodeGen"),<br>
but that only helps for generic (lib/CodeGen) bugs, not for<br>
target-specific ones (I'm thinking ISel, or stuff like AnalyzeBranch),<br>
right?<br>
<br>
For the latter, serializing MI (or more importantly the SelectionDAG)<br>
has been floating around for a while.<br>
I think that would help a lot, but once you start serializing an<br>
internal representation, you don't know if it's still possible to get<br>
to it from IR, so you have the same staleness problem we currently<br>
have.<br>
That's solved by adding a companion tests, that's written like we do<br>
now (from IR), checking the serialized representation right before the<br>
to-be-tested component. When someone makes a change that affects this<br>
companion, they have an opportunity to re-evaluate the unit test as<br>
well. A bit verbose, yes, but does it sound sensible?<br></blockquote><div><br>Arguably that state already exists with middle-end optimizations - we might make a singular IR test for a particular optimization only to, later on, change canonicalization upstream such that that particular IR construct is never seen.<br><br>If we were to continue to have the full IR tests as we do today, I don't think a lot would be gained by having the more precise tests - we'd still have much of the flaky issues we do today (I guess we'd lose the false negatives, I described - the narrow test would ensure the issue was always covered - but we'd still have the false positives of the broad IR test).<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Don't get me wrong, a mock target would be pretty simple and very<br>
useful - enough for lib/CodeGen, so +1 for that by itself.<br>
<span class="HOEnZb"><font color="#888888"><br>
-Ahmed<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
> Thoughts?<br>
><br>
> -Hal<br>
><br>
> --<br>
> Hal Finkel<br>
> Assistant Computational Scientist<br>
> Leadership Computing Facility<br>
> Argonne National Laboratory<br>
> _______________________________________________<br>
> LLVM Developers mailing list<br>
> <a href="mailto:LLVMdev@cs.uiuc.edu">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
> <a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
<br>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
</div></div></blockquote></div><br></div></div>