[llvm] r193279 - [lit] Use multiprocessing based parallelism by default, on Unix.

[Rafael Espíndola]

ok, 193413 might fix it.

On 25 October 2013 10:39, Daniel Sanders <Daniel.Sanders at imgtec.com> wrote:
> I don't have a system to test on but GuessOS() in http://v8.googlecode.com/svn/trunk/tools/testrunner/local/utils.py says it should be 'OpenBSD'.
>
>> -----Original Message-----
>> From: Rafael Espíndola [mailto:rafael.espindola at gmail.com]
>> Sent: 25 October 2013 14:10
>> To: Daniel Sanders
>> Cc: Daniel Dunbar; llvm-commits
>> Subject: Re: [llvm] r193279 - [lit] Use multiprocessing based parallelism by
>> default, on Unix.
>>
>> What does platform.system() return on openbsd?
>>
>> On 25 October 2013 06:35, Daniel Sanders <Daniel.Sanders at imgtec.com>
>> wrote:
>> > Hi Daniel and Rafael,
>> >
>> >
>> >
>> > I've noticed that after the switch to multiprocessing,
>> > clang-amd64-openbsd
>> > (http://lab.llvm.org:8011/builders/clang-amd64-openbsd/builds/2019)
>> > has stopped running tests. Buildbot probably didn't send out an email
>> > because the previous build failed for different reasons (4 unexpected
>> > failures in
>> > check-all) so I thought I ought to let you know.
>> >
>> >
>> >
>> > From the stdio for check-all:
>> >
>> > Making lit.site.cfg for Clang extra tools...
>> >
>> > make[2]: Leaving directory
>> > `/buildbot-llvm/clang-openbsd/llvm/tools/clang/tools/extra/test'
>> >
>> > ( ulimit -t 600 ; ulimit -d 512000 ; ulimit -m 512000 ; ulimit -s 8192
>> > ; \
>> >
>> >   /usr/local/bin/python2.7
>> > /buildbot-llvm/clang-openbsd/llvm/utils/lit/lit.py -v -j 2 .
>> > /buildbot-llvm/clang-openbsd/llvm/test/../tools/clang/test
>> > /buildbot-llvm/clang-openbsd/llvm/test/../tools/clang/tools/extra/test
>> > )
>> >
>> > lit.py: <string>:187: note: using clang:
>> > '/buildbot-llvm/clang-openbsd/llvm/Release+Asserts/bin/clang'
>> >
>> > lit.py: <string>:180: note: using clang:
>> > '/buildbot-llvm/clang-openbsd/llvm/Release+Asserts/bin/clang'
>> >
>> > lit.py: discovery.py:190: warning: test suite 'Extra Tools Unit Tests'
>> > contained no tests
>> >
>> > -- Testing: 15823 tests, 2 threads --
>> >
>> > Traceback (most recent call last):
>> >
>> >   File "/buildbot-llvm/clang-openbsd/llvm/utils/lit/lit.py", line 5,
>> > in <module>
>> >
>> >     lit.main()
>> >
>> >   File "/buildbot-llvm/clang-openbsd/llvm/utils/lit/lit/main.py", line
>> > 327, in main
>> >
>> >     opts.useProcesses)
>> >
>> >   File "/buildbot-llvm/clang-openbsd/llvm/utils/lit/lit/run.py", line
>> > 220, in execute_tests
>> >
>> >     canceled_flag =  multiprocessing.Value('i', 0)
>> >
>> >   File "/usr/local/lib/python2.7/multiprocessing/__init__.py", line
>> > 253, in Value
>> >
>> >     return Value(typecode_or_type, *args, **kwds)
>> >
>> >   File "/usr/local/lib/python2.7/multiprocessing/sharedctypes.py",
>> > line 103, in Value
>> >
>> >     lock = RLock()
>> >
>> >   File "/usr/local/lib/python2.7/multiprocessing/__init__.py", line
>> > 182, in RLock
>> >
>> >     from multiprocessing.synchronize import RLock
>> >
>> >   File "/usr/local/lib/python2.7/multiprocessing/synchronize.py", line
>> > 59, in <module>
>> >
>> >     " function, see issue 3770.")
>> >
>> > ImportError: This platform lacks a functioning sem_open
>> > implementation, therefore, the required synchronization primitives
>> > needed will not function, see issue 3770.
>> >
>> > make[1]: *** [check-local-all] Error 1
>> >
>> > make[1]: Leaving directory `/buildbot-llvm/clang-openbsd/llvm/test'
>> >
>> > make: *** [check-all] Error 2
>> >
>> >
>> >
>> > From: llvm-commits-bounces at cs.uiuc.edu
>> > [mailto:llvm-commits-bounces at cs.uiuc.edu] On Behalf Of Daniel Dunbar
>> > Sent: 24 October 2013 00:53
>> > To: Rafael Espíndola
>> > Cc: llvm-commits
>> > Subject: Re: [llvm] r193279 - [lit] Use multiprocessing based
>> > parallelism by default, on Unix.
>> >
>> >
>> >
>> > Oh, thanks!
>> >
>> >
>> > On Wednesday, October 23, 2013, Rafael Espíndola wrote:
>> >
>> >>      group.add_option("", "--use-processes", dest="useProcesses",
>> >>                        help="Run tests in parallel with processes
>> >> (not threads)",
>> >> -                      action="store_true", default=False)
>> >> +                      action="store_true",
>> >> + default=useProcessesIsDefault)
>> >>      group.add_option("", "--use-threads", dest="useProcesses",
>> >>                        help="Run tests in parallel with threads (not
>> >> processes)",
>> >> -                      action="store_false", default=False)
>> >> +                      action="store_false", default=not
>> >> useProcessesIsDefault)
>> >>      parser.add_option_group(group)
>> >
>> > Having two options pointing to the same variable with different
>> > defaults is odd. I was still getting multitherading on linux. I fixed
>> > this in r193282 and multiprocessing is now being used by default.
>> >
>> > Cheers,
>> > Rafael
>> >
>> >
>> >
>> > --
>> > - Daniel