[cfe-dev] Running tests on OS X 10.10 vs "Killed: 9"

[Filipe Cabecinhas via cfe-dev]

If you have:
llvm, clang, libcxx libcxxabi, compiler-rt, lld
Then you get around 31K tests. IIRC, those problems were arising before
Chris Bieneman fixed compiler-rt tests to not test iOS arches on a Mac.

Right now, even enabling compiler-rt for iOS will still generate ~31K
tests. But it's very close to the limit.

Regards,
  Filipe

On Fri, Sep 4, 2015 at 12:49 PM, Sean Silva <chisophugis at gmail.com> wrote:

>
>
> On Fri, Sep 4, 2015 at 12:46 PM, Sean Silva <chisophugis at gmail.com> wrote:
>
>>
>>
>> On Fri, Sep 4, 2015 at 10:27 AM, Nico Weber via cfe-dev <
>> cfe-dev at lists.llvm.org> wrote:
>>
>>> Hi,
>>>
>>> building 'check-all' on any of my machines running OS X 10.10 usually
>>> fails because a few tests fail due to some processes being killed by the
>>> kernel (there's always "Killed: 9" somewhere in lit's error output).
>>> Everything's fine on 10.9.
>>>
>>> How do folks deal with this? Don't use 10.10 for building llvm? Is there
>>> some tweakable to tell the kernel "please don't kill my processes"?
>>>
>>> Here's an example from just now:
>>>
>>> FAIL: LLVM :: MC/X86/shuffle-comments.s (18589 of 28139)
>>>
>>
>> Btw, for the sake of having this in writing somewhere: one other thing
>> I've run into (I was generating a huge number of lit tests for an
>> artificial reason; wasn't during normal development) is that once you go
>> past ~32K tests (2^15) lit's current way it queues up processes will cause
>> it to completely hang on Mac. Essentially the problem is that it has a for
>> loop like:
>>
>> for t in tests:
>>     push_onto_threadsafe_queue(t)
>> ... kick off worker threads that consume from the queue ...
>>
>
> Maybe clearer:
>
> for t in tests:
>     push_onto_threadsafe_queue(t)
> ...
> kick off worker threads that consume from the queue
> ...
>
> i.e. It pushes all tests onto the queue before it starts kicking off jobs.
>
> -- Sean Silva
>
>
>>
>> unfortunately, push_onto_threadsafe_queue ups an OS semaphore and on mac
>> there is an upper limit of ~32K. Once you hit that
>> the push_onto_threadsafe_queue operation will block and therefore lit will
>> be completely blocked.
>>
>> Someday we will need to e.g. spawn off a feeder thread to do the
>> push_onto_threadsafe_queue loop.
>>
>> Filipe, did you say that you had actually run into the 32K limit at some
>> point during regular development? (what combination of LLVM repos were
>> checked out and being tested simultaneously?)
>>
>> -- Sean Silva
>>
>>
>>> ******************** TEST 'LLVM :: MC/X86/shuffle-comments.s' FAILED
>>> ********************
>>> Script:
>>> --
>>> /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/llvm-mc
>>> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s
>>> -triple=x86_64-unknown-unknown |
>>> /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/FileCheck
>>> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s
>>> --
>>> Exit Code: 137
>>>
>>> Command Output (stderr):
>>> --
>>> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm-bootstrap/test/MC/X86/Output/shuffle-comments.s.script:
>>> line 1: 47457 Killed: 9
>>> /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/llvm-mc
>>> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s
>>> -triple=x86_64-unknown-unknown
>>>      47459                       |
>>> /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/FileCheck
>>> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s
>>>
>>>
>>> Thanks,
>>> Nico
>>>
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20150904/d6e9d026/attachment.html>