[LLVMdev] test-suite RunSafely.sh remote execution with test copy

[Chris Matthews]

So I think this is a great idea, and something we really need. NFS is bad for this sort of testing because a NFS glitch can cause a test to hang.  I think any transparent pretend filesystem solution is complicating things by interleaving the benchmark with the network file access.  Explicit rsync or scp are the way to go here.  A few questions:

1. How much slower is it?
2. How do you deal with tests where there are a bunch of tests in the same location? Shootout for example.
3. Why did you choose to use a device native timeit?
4. Do all the test currently pass?


> On Aug 18, 2014, at 8:58 AM, Brian M. Rzycki <brzycki at gmail.com> wrote:
> 
> Hi James,
> Thanks for the feedback. Yes I did look into sshfs and It's a great choice for devices running Linux distributions. Unfortunately some of our devices are Android based and have no plans for Linux support. I started examining cross-compiling libfuse for Android but ran into complications there due to the very limited nature of the device's root filesystem libraries, binaries, and compiled kernel options.
> 
> It was much simpler cross-compiling dropbear + scp for Android which lead to the relatively simple scp patch in RunSafely.sh.
> 
> I will take another stab at sshfs to see if there may be a way to get it working on Android with minimal pain.
> 
> I'd be curious to know if others doing similar work rely mostly on NFS, SMB, sshfs or some other network filesystem to accomplish remote execution testing.
> 
> -Brian
> 
> 
> On Sat, Aug 16, 2014 at 1:29 AM, James Courtier-Dutton <james.dutton at gmail.com <mailto:james.dutton at gmail.com>> wrote:
> On Aug 15, 2014 10:37 PM, "Brian M. Rzycki" <brzycki at gmail.com <mailto:brzycki at gmail.com>> wrote:
> >
> > Hello everyone,
> >
> > I am working to run the LLVM Nightly Testsuite on embedded arm devices. Not all of these devices can mount NFS to share a common directory tree. To mitigate this I have patched test-suite/RunSafely.sh (attached) to create a temporary directory, run the test remotely, collect the output, and cleanup after itself. The patch breaks compatibility with the current remote device model in the following ways:
> >
> > 1. uses scp/rcp to copy files from the host running test-suite instead of assuming an identical tree on the remote device.
> > 2. expects the remote device to have the mktemp command to create the temporary workspace (usually under /tmp/*)
> > 3. expects the remote device to have timeit installed in its path. If timeit were compiled statically I could scp the one passed into RunSafely.sh to the destination. Hosts with a different libc than the tested compiler (Android, embedded uClibc devices) cannot run the dynamically-linked timeit.
> > 4. It's a bit slower due to all the network copies for the entire test suite.
> >
> > I'd appreciate feedback and suggestions as to how I can rework this model for inclusion upstream. I'm sure there are others out there that would find this kind of testing useful. I'm fine with a new remote-copy mode that can be passed in via lnt nt command line invocation.
> >
> > I hope others find this useful.
> > -Brian
> 
> Have you tried sshfs.
> I does much the same as nfs mount but uses ssh instead.
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140818/22519470/attachment.html>