[LLVMdev] Newbee question: LLVM backend regression tests for thumb1 targets on simulator possible?

[Bjoern Haase]

Am 19.12.2014 11:00, schrieb Renato Golin:
> On 18 December 2014 at 04:02, Tim Northover <t.p.northover at gmail.com> wrote:
>>> You have some special version of qemu? My default distribution's qemu
>>> package did not have a cortex-M0 nor has qemu head freshly taken from git:
>> The trouble with Cortex-M0 (from a virtualised testing perspective) is
>> that it doesn't have an MMU, so any "real" operating system
>> environment will be far more artificial than we'd like, and probably
>> very low down on qemu's list of priorities to boot.
>>
>>
>> Why not test on real hardware?
>>
>> https://developer.mbed.org/platforms/DipCortex-M0/
>>
>> cheers,
>> --renato
>>
Of course, it would not be bad to run also some limited tests on 
real-world hardware. However, the answer why "why not"... is in my opinion:

Because you will have big difficulties matching existing regression test 
code (designed for bigger targets) to a system being constrained to say 
16k Flash and 4k RAM. My experience with the testsuite of gcc being run 
on microcontrollers (AVR in this case) is that you encounter so many 
pseudo failures just because the test cases don't account for the 
limitations of the DUT that it's a tedious work to get any useful 
information out of the results. Imagine, e.g. a test that uses alloca 
and tests for correct stack pointer re-adjustment from frame pointers 
and just allocate two buffers of 2k RAM.

In my opinion there is more need for a regression test framework in a 
simulation environment in a first step. Of course an optional, second 
step with tests on real-world HW would not hurt. Actually qemu does not 
seem to be this bad for this purpose.

Björn.