I see, thanks. How about something like: %try 10 not %run %t | FileCheck %s Meaning, try $* up to 10 times, capturing stdout/stderr each time. When $* returns zero, write its output to stdout/stderr. http://reviews.llvm.org/D3913