[LLVMdev] QEMU testing for LIT execution tests

[Jonathan Roelofs]

Dan/Daniel/Eric,

I'm testing a bare-metal ARM toolchain, and I've hacked up my local copy of lit 
to get it to run libcxx tests on qemu. I wanted to pick your collective brains 
to see if there was a better way of doing this.

What I have implemented is here (don't take this as a formal review request, we 
can do that later once we figure out the best direction to go):
lit part:    http://reviews.llvm.org/D5014
libcxx part: http://reviews.llvm.org/D5015



Talking with Dan Albert earlier, he pointed out an existing review 
(http://reviews.llvm.org/D4594) where he has implemented something vaguely 
similar for Android. Looking at that, I've realized that maybe it makes sense to 
make an interface for the different actions of:
   *  Compiling the test
      o  Maybe the compiler isn't on the machine that runs the test?
      o  Maybe the compile jobs could be distributed to different machines?
   *  Running the test
      o  Perhaps the test needs to be sent out to a remote device?
      o  Perhaps the test can be scheduled on more than one remote device?

Then that interface could be implemented in a couple of generic ways:
   *  scp & ssh for remote hosts (or QEMU with an OS image)
   *  Wrapper around Android's adb
   *  Thin wrapper around "run it in this machine's shell" (what we currently have)

What are your thoughts on this? Is this worth pursuing? What other use-cases can 
we support by structuring LIT this way? I don't yet have a concrete proposal for 
what this would look like, but I can throw one together if there's interest.


Cheers,

Jon

-- 
Jon Roelofs
jonathan at codesourcery.com
CodeSourcery / Mentor Embedded