[LLVMdev] [RFC] Project for GSoC: Unit/Regression testing for CodeGen

[Quentin Colombet]

Hi,

Ultimately, I would love llc (or another tool) to work the same way as opt but for target specific IR to IR, IR to MIR, and MIR to MIR passes.

Even with such a tool, we would still need a mock target to exercise the tricky corner cases.

The bottom line is +1 for the mock target Hal suggest.

Thanks,
-Quentin

> On Mar 6, 2015, at 8:39 AM, Tom Stellard <tom at stellard.net> wrote:
> 
> On Fri, Mar 06, 2015 at 09:04:50AM -0600, Hal Finkel wrote:
>> Hi everyone,
>> 
>> In response to yet-another fix in CodeGen affecting only an out-of-tree target (r231186), our lack of the ability to properly unit test CodeGen components has been highlighted. It was suggested that improving this situation might be a good GSoC project, and I agree, provided that we can settle on the scope and basic design ahead of time.
>> 
>> I'd like to add that I feel this is a serious problem even for in-tree targets. We currently construct IR-level tests for CodeGen components, but
>> this is very fragile. Many of the IR-level CodeGen tests, especially "bug-triggering" regression tests, don't currently test the logic they were originally designed to cover.
>> 
>> Now, for a design:
>> 
>> One idea that I've had for some time is to develop a 'mock' target for testing. For this target, all of the various type/operation legality settings would be determined by some input configuration file. It would contain instructions, mostly in 1:1 correspondence to our SelectionDAG node types, and many different register classes of different sizes, different calling-conventions, etc. (again, some input configuration file would determine which were active). We could then use this mock target to right regression tests for CodeGen components. We could also use it write units tests, especially at the MI level.
>> 
> 
> One advantage of creating a mock target with a 1:1 correspondence of
> SelectionDAG nodes to MachineInstrs, is it would give us a generic
> MachineInstr instruction set, which is a requirement of Jakob's global
> isel proposal.  Maybe it would help kick start work on this.
> 
> Another idea I've had for a 'mock' target would be to create a skeleton
> target that you can do 's/YourTarget/MockTarget/g' and instantly get a
> target that builds.
> 
> This would have to be separate from any mock target used for testing,
> but I think creating something like this as a side-effect of your
> proposal would be nice.
> 
> -Tom
> 
>> Thoughts?
>> 
>> -Hal
>> 
>> -- 
>> Hal Finkel
>> Assistant Computational Scientist
>> Leadership Computing Facility
>> Argonne National Laboratory
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu>         http://llvm.cs.uiuc.edu <http://llvm.cs.uiuc.edu/>
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu>         http://llvm.cs.uiuc.edu <http://llvm.cs.uiuc.edu/>
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150306/b9043739/attachment.html>