[llvm] r193279 - [lit] Use multiprocessing based parallelism by default, on Unix.

[Daniel Sanders]

Hi Daniel and Rafael,

I've noticed that after the switch to multiprocessing, clang-amd64-openbsd (http://lab.llvm.org:8011/builders/clang-amd64-openbsd/builds/2019) has stopped running tests. Buildbot probably didn't send out an email because the previous build failed for different reasons (4 unexpected failures in check-all) so I thought I ought to let you know.

>From the stdio for check-all:
Making lit.site.cfg for Clang extra tools...
make[2]: Leaving directory `/buildbot-llvm/clang-openbsd/llvm/tools/clang/tools/extra/test'
( ulimit -t 600 ; ulimit -d 512000 ; ulimit -m 512000 ; ulimit -s 8192 ; \
  /usr/local/bin/python2.7 /buildbot-llvm/clang-openbsd/llvm/utils/lit/lit.py -v -j 2 . /buildbot-llvm/clang-openbsd/llvm/test/../tools/clang/test /buildbot-llvm/clang-openbsd/llvm/test/../tools/clang/tools/extra/test )
lit.py: <string>:187: note: using clang: '/buildbot-llvm/clang-openbsd/llvm/Release+Asserts/bin/clang'
lit.py: <string>:180: note: using clang: '/buildbot-llvm/clang-openbsd/llvm/Release+Asserts/bin/clang'
lit.py: discovery.py:190: warning: test suite 'Extra Tools Unit Tests' contained no tests
-- Testing: 15823 tests, 2 threads --
Traceback (most recent call last):
  File "/buildbot-llvm/clang-openbsd/llvm/utils/lit/lit.py", line 5, in <module>
    lit.main()
  File "/buildbot-llvm/clang-openbsd/llvm/utils/lit/lit/main.py", line 327, in main
    opts.useProcesses)
  File "/buildbot-llvm/clang-openbsd/llvm/utils/lit/lit/run.py", line 220, in execute_tests
    canceled_flag =  multiprocessing.Value('i', 0)
  File "/usr/local/lib/python2.7/multiprocessing/__init__.py", line 253, in Value
    return Value(typecode_or_type, *args, **kwds)
  File "/usr/local/lib/python2.7/multiprocessing/sharedctypes.py", line 103, in Value
    lock = RLock()
  File "/usr/local/lib/python2.7/multiprocessing/__init__.py", line 182, in RLock
    from multiprocessing.synchronize import RLock
  File "/usr/local/lib/python2.7/multiprocessing/synchronize.py", line 59, in <module>
    " function, see issue 3770.")
ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770.
make[1]: *** [check-local-all] Error 1
make[1]: Leaving directory `/buildbot-llvm/clang-openbsd/llvm/test'
make: *** [check-all] Error 2

From: llvm-commits-bounces at cs.uiuc.edu [mailto:llvm-commits-bounces at cs.uiuc.edu] On Behalf Of Daniel Dunbar
Sent: 24 October 2013 00:53
To: Rafael Espíndola
Cc: llvm-commits
Subject: Re: [llvm] r193279 - [lit] Use multiprocessing based parallelism by default, on Unix.

Oh, thanks!

On Wednesday, October 23, 2013, Rafael Espíndola wrote:
>      group.add_option("", "--use-processes", dest="useProcesses",
>                        help="Run tests in parallel with processes (not threads)",
> -                      action="store_true", default=False)
> +                      action="store_true", default=useProcessesIsDefault)
>      group.add_option("", "--use-threads", dest="useProcesses",
>                        help="Run tests in parallel with threads (not processes)",
> -                      action="store_false", default=False)
> +                      action="store_false", default=not useProcessesIsDefault)
>      parser.add_option_group(group)

Having two options pointing to the same variable with different
defaults is odd. I was still getting multitherading on linux. I fixed
this in r193282 and multiprocessing is now being used by default.

Cheers,
Rafael


--
- Daniel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20131025/3bc006f7/attachment.html>