[cfe-dev] Running tests on OS X 10.10 vs "Killed: 9"

[Sean Silva via cfe-dev]

On Fri, Sep 4, 2015 at 12:46 PM, Sean Silva <chisophugis at gmail.com> wrote:

>
>
> On Fri, Sep 4, 2015 at 10:27 AM, Nico Weber via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
>> Hi,
>>
>> building 'check-all' on any of my machines running OS X 10.10 usually
>> fails because a few tests fail due to some processes being killed by the
>> kernel (there's always "Killed: 9" somewhere in lit's error output).
>> Everything's fine on 10.9.
>>
>> How do folks deal with this? Don't use 10.10 for building llvm? Is there
>> some tweakable to tell the kernel "please don't kill my processes"?
>>
>> Here's an example from just now:
>>
>> FAIL: LLVM :: MC/X86/shuffle-comments.s (18589 of 28139)
>>
>
> Btw, for the sake of having this in writing somewhere: one other thing
> I've run into (I was generating a huge number of lit tests for an
> artificial reason; wasn't during normal development) is that once you go
> past ~32K tests (2^15) lit's current way it queues up processes will cause
> it to completely hang on Mac. Essentially the problem is that it has a for
> loop like:
>
> for t in tests:
>     push_onto_threadsafe_queue(t)
> ... kick off worker threads that consume from the queue ...
>

Maybe clearer:

for t in tests:
    push_onto_threadsafe_queue(t)
...
kick off worker threads that consume from the queue
...

i.e. It pushes all tests onto the queue before it starts kicking off jobs.

-- Sean Silva


>
> unfortunately, push_onto_threadsafe_queue ups an OS semaphore and on mac
> there is an upper limit of ~32K. Once you hit that
> the push_onto_threadsafe_queue operation will block and therefore lit will
> be completely blocked.
>
> Someday we will need to e.g. spawn off a feeder thread to do the
> push_onto_threadsafe_queue loop.
>
> Filipe, did you say that you had actually run into the 32K limit at some
> point during regular development? (what combination of LLVM repos were
> checked out and being tested simultaneously?)
>
> -- Sean Silva
>
>
>> ******************** TEST 'LLVM :: MC/X86/shuffle-comments.s' FAILED
>> ********************
>> Script:
>> --
>> /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/llvm-mc
>> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s
>> -triple=x86_64-unknown-unknown |
>> /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/FileCheck
>> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s
>> --
>> Exit Code: 137
>>
>> Command Output (stderr):
>> --
>> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm-bootstrap/test/MC/X86/Output/shuffle-comments.s.script:
>> line 1: 47457 Killed: 9
>> /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/llvm-mc
>> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s
>> -triple=x86_64-unknown-unknown
>>      47459                       |
>> /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/FileCheck
>> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s
>>
>>
>> Thanks,
>> Nico
>>
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20150904/a83456ac/attachment.html>