[llvm] r193279 - [lit] Use multiprocessing based parallelism by default, on Unix.

[Daniel Sanders]

I don't have a system to test on but GuessOS() in http://v8.googlecode.com/svn/trunk/tools/testrunner/local/utils.py says it should be 'OpenBSD'.

> -----Original Message-----
> From: Rafael Espíndola [mailto:rafael.espindola at gmail.com]
> Sent: 25 October 2013 14:10
> To: Daniel Sanders
> Cc: Daniel Dunbar; llvm-commits
> Subject: Re: [llvm] r193279 - [lit] Use multiprocessing based parallelism by
> default, on Unix.
> 
> What does platform.system() return on openbsd?
> 
> On 25 October 2013 06:35, Daniel Sanders <Daniel.Sanders at imgtec.com>
> wrote:
> > Hi Daniel and Rafael,
> >
> >
> >
> > I've noticed that after the switch to multiprocessing,
> > clang-amd64-openbsd
> > (http://lab.llvm.org:8011/builders/clang-amd64-openbsd/builds/2019)
> > has stopped running tests. Buildbot probably didn't send out an email
> > because the previous build failed for different reasons (4 unexpected
> > failures in
> > check-all) so I thought I ought to let you know.
> >
> >
> >
> > From the stdio for check-all:
> >
> > Making lit.site.cfg for Clang extra tools...
> >
> > make[2]: Leaving directory
> > `/buildbot-llvm/clang-openbsd/llvm/tools/clang/tools/extra/test'
> >
> > ( ulimit -t 600 ; ulimit -d 512000 ; ulimit -m 512000 ; ulimit -s 8192
> > ; \
> >
> >   /usr/local/bin/python2.7
> > /buildbot-llvm/clang-openbsd/llvm/utils/lit/lit.py -v -j 2 .
> > /buildbot-llvm/clang-openbsd/llvm/test/../tools/clang/test
> > /buildbot-llvm/clang-openbsd/llvm/test/../tools/clang/tools/extra/test
> > )
> >
> > lit.py: <string>:187: note: using clang:
> > '/buildbot-llvm/clang-openbsd/llvm/Release+Asserts/bin/clang'
> >
> > lit.py: <string>:180: note: using clang:
> > '/buildbot-llvm/clang-openbsd/llvm/Release+Asserts/bin/clang'
> >
> > lit.py: discovery.py:190: warning: test suite 'Extra Tools Unit Tests'
> > contained no tests
> >
> > -- Testing: 15823 tests, 2 threads --
> >
> > Traceback (most recent call last):
> >
> >   File "/buildbot-llvm/clang-openbsd/llvm/utils/lit/lit.py", line 5,
> > in <module>
> >
> >     lit.main()
> >
> >   File "/buildbot-llvm/clang-openbsd/llvm/utils/lit/lit/main.py", line
> > 327, in main
> >
> >     opts.useProcesses)
> >
> >   File "/buildbot-llvm/clang-openbsd/llvm/utils/lit/lit/run.py", line
> > 220, in execute_tests
> >
> >     canceled_flag =  multiprocessing.Value('i', 0)
> >
> >   File "/usr/local/lib/python2.7/multiprocessing/__init__.py", line
> > 253, in Value
> >
> >     return Value(typecode_or_type, *args, **kwds)
> >
> >   File "/usr/local/lib/python2.7/multiprocessing/sharedctypes.py",
> > line 103, in Value
> >
> >     lock = RLock()
> >
> >   File "/usr/local/lib/python2.7/multiprocessing/__init__.py", line
> > 182, in RLock
> >
> >     from multiprocessing.synchronize import RLock
> >
> >   File "/usr/local/lib/python2.7/multiprocessing/synchronize.py", line
> > 59, in <module>
> >
> >     " function, see issue 3770.")
> >
> > ImportError: This platform lacks a functioning sem_open
> > implementation, therefore, the required synchronization primitives
> > needed will not function, see issue 3770.
> >
> > make[1]: *** [check-local-all] Error 1
> >
> > make[1]: Leaving directory `/buildbot-llvm/clang-openbsd/llvm/test'
> >
> > make: *** [check-all] Error 2
> >
> >
> >
> > From: llvm-commits-bounces at cs.uiuc.edu
> > [mailto:llvm-commits-bounces at cs.uiuc.edu] On Behalf Of Daniel Dunbar
> > Sent: 24 October 2013 00:53
> > To: Rafael Espíndola
> > Cc: llvm-commits
> > Subject: Re: [llvm] r193279 - [lit] Use multiprocessing based
> > parallelism by default, on Unix.
> >
> >
> >
> > Oh, thanks!
> >
> >
> > On Wednesday, October 23, 2013, Rafael Espíndola wrote:
> >
> >>      group.add_option("", "--use-processes", dest="useProcesses",
> >>                        help="Run tests in parallel with processes
> >> (not threads)",
> >> -                      action="store_true", default=False)
> >> +                      action="store_true",
> >> + default=useProcessesIsDefault)
> >>      group.add_option("", "--use-threads", dest="useProcesses",
> >>                        help="Run tests in parallel with threads (not
> >> processes)",
> >> -                      action="store_false", default=False)
> >> +                      action="store_false", default=not
> >> useProcessesIsDefault)
> >>      parser.add_option_group(group)
> >
> > Having two options pointing to the same variable with different
> > defaults is odd. I was still getting multitherading on linux. I fixed
> > this in r193282 and multiprocessing is now being used by default.
> >
> > Cheers,
> > Rafael
> >
> >
> >
> > --
> > - Daniel