[LLVMdev] Newbee question: LLVM backend regression tests for thumb1 targets on simulator possible?

[Kristof Beyls]

I've been wondering too about how to get better ARM v6m compile-and-execute
testing going.

As you say Jon, the non-execution-based regression tests are surprisingly
good at catching issues; but they're no full substitute for executing the
code produced by the backend for a reasonably-sized test suite.

If somehow it would be possible to compile and run the LLVM test-suite
for v6m, I think that would be a good step forward. It would also allow
to get a buildbot going without too much effort continuously checking
basic correctness of v6m code generation.

My guess is that the biggest hurdle would be to get linux or a similar
operating system going on a v6m/thumb1 simulator. Does anyone have an idea
if this is feasible or completely impossible?

Thanks,

Kristof

> -----Original Message-----
> From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu]
> On Behalf Of Jonathan Roelofs
> Sent: 15 December 2014 18:31
> To: Bjoern Haase; llvmdev at cs.uiuc.edu
> Subject: Re: [LLVMdev] Newbee question: LLVM backend regression tests
> for thumb1 targets on simulator possible?
> 
> 
> 
> On 12/15/14 4:29 AM, Bjoern Haase wrote:
> >
> > Hello,
> >
> > as a newbee, I'd appreciate some support on regression test setup.
> >
> > Specifically, I am interrested in the feature of tail call
> > optimizations for the ARM v6m targets. This feature currently seems to
> > be completely deactivated at the moment (v6m being based on thumb1
> > ?!). According to my code-reading, this feature will involve some
> > modifications in epilogue generation.
> >
> > My work on a gcc backend did show me that for a beginner like me, it
> > is rather likely that the first attempts will break something.
> :)
> >
> > Thus, being completely new to the llvm project I think that it's
> > essential to first establish a suitable regression test setup. My plan
> > is to run the tests for the v4t platform in thumb-only mode where free
> > simulators are available and which behaves (at least with respect to
> > the
> > epilogue) very much like v6m.
> Why not test your v6m changes on a Cortex-m0 QEMU? Semihosted
> applications are pretty easy to get going, something like:
> 
> $ qemu-system-arm -semihosting -M integratorcp -cpu cortex-m0 -kernel
> a.out
> 
> Thumv4t support is a bit spotty in llvm as it's not very well tested.
> The problem I run into most (and fixed several times), is that thumbv4t
> doesn't have a lo->lo mov instruction that doesn't clobber cpsr. That
> being said, it does work for the most part... just something to watch
> out for.
> >
> >  From former work on gcc I'm used to the possibilty to run regression
> > tests in a simulator for the target platform (there with the expect
> > script mechanism). In the documentation on the web, I did, however,
> > not yet find information on a comparable feature for llvm.
> >
> > Therefore the specific questions:
> >
> > - Is there a mechanism to run backend testsuite runs in a simulation
> > framework and if yes, could you give me a hint on how to find
> > information
> Not that I know of, no.
> 
> I routinely run the libc++ & libc++abi test suites in QEMU, but that's
> more of a whole-toolchain test, rather that just a backend test. You can
> get surprisingly far without simulator testing, so for most things, LLVM
> doesn't use it for testing. This makes it easier for any random
> developer to check out the llvm sources, build them, and run the
> testsuite, and not have to have simulators installed for all the
> different backends.
> > - If not, how is regression testing for the thumb1 targets currently
> > implemented?
> We use the LIT framework http://llvm.org/docs/CommandGuide/lit.html to
> test the compiler. Mostly this means feeding in llvm IR, and using
> FileCheck http://llvm.org/docs/CommandGuide/FileCheck.html (which is
> basically a glorified
> grep) to verify that the assembly produced is the same as what the test
> expects to be generated.
> 
> To run these tests, use the 'make check-all' target from the build
> directory.
> 
> 
> Cheers,
> 
> Jon
> >
> > Yours,
> >
> > Björn Haase
> >
> > P.S.: Of course, any hint with respect to sibling call optimization
> > would also be appreciated in a second step. I actually already found
> > some helpful information in the sources.
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
> --
> Jon Roelofs
> jonathan at codesourcery.com
> CodeSourcery / Mentor Embedded
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev