[compiler-rt] r355018 - [Darwin][NFC] Refactor throttling of 64bit sanitizer tests on Darwin
Julian Lettner via llvm-commits
llvm-commits at lists.llvm.org
Wed Feb 27 11:06:20 PST 2019
Author: yln
Date: Wed Feb 27 11:06:20 2019
New Revision: 355018
URL: http://llvm.org/viewvc/llvm-project?rev=355018&view=rev
Log:
[Darwin][NFC] Refactor throttling of 64bit sanitizer tests on Darwin
Underlying condition for throttling is "has large mmap'd regions" (i.e.,
shadow memory) and not sanitizers in general (e.g., UBSan does not need
to be throttled).
Rename parallelism group `darwin-64bit-sanitizer` to `shadow-memory` and
apply it unconditionally to all tests which require it. We can then have
all the Darwin throttling logic in one place in the commen lit config.
Throttle sanitizer_common unit tests. Configuration was previously
missing from sanitizer_common/Unit/lit.site.cfg.
Reviewed by: kubamracek
Differential Revision: https://reviews.llvm.org/D58677
Modified:
compiler-rt/trunk/test/asan/lit.cfg
compiler-rt/trunk/test/fuzzer/lit.cfg
compiler-rt/trunk/test/lit.common.cfg
compiler-rt/trunk/test/sanitizer_common/Unit/lit.site.cfg.in
compiler-rt/trunk/test/sanitizer_common/lit.common.cfg
compiler-rt/trunk/test/tsan/lit.cfg
compiler-rt/trunk/unittests/lit.common.unit.cfg
compiler-rt/trunk/unittests/lit_unittest_cfg_utils.py
Modified: compiler-rt/trunk/test/asan/lit.cfg
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/asan/lit.cfg?rev=355018&r1=355017&r2=355018&view=diff
==============================================================================
--- compiler-rt/trunk/test/asan/lit.cfg (original)
+++ compiler-rt/trunk/test/asan/lit.cfg Wed Feb 27 11:06:20 2019
@@ -220,6 +220,4 @@ else:
if config.host_os not in ['Linux', 'Darwin', 'FreeBSD', 'SunOS', 'Windows', 'NetBSD']:
config.unsupported = True
-if config.host_os == 'Darwin':
- if config.target_arch in ["x86_64", "x86_64h"]:
- config.parallelism_group = "darwin-64bit-sanitizer"
+config.parallelism_group = 'shadow-memory'
Modified: compiler-rt/trunk/test/fuzzer/lit.cfg
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/fuzzer/lit.cfg?rev=355018&r1=355017&r2=355018&view=diff
==============================================================================
--- compiler-rt/trunk/test/fuzzer/lit.cfg (original)
+++ compiler-rt/trunk/test/fuzzer/lit.cfg Wed Feb 27 11:06:20 2019
@@ -114,6 +114,4 @@ if default_asan_opts_str:
config.substitutions.append(('%env_asan_opts=',
'env ASAN_OPTIONS=' + default_asan_opts_str))
-if config.host_os == 'Darwin':
- if config.target_arch in ["x86_64", "x86_64h"]:
- config.parallelism_group = "darwin-64bit-sanitizer"
+config.parallelism_group = 'shadow-memory'
Modified: compiler-rt/trunk/test/lit.common.cfg
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/lit.common.cfg?rev=355018&r1=355017&r2=355018&view=diff
==============================================================================
--- compiler-rt/trunk/test/lit.common.cfg (original)
+++ compiler-rt/trunk/test/lit.common.cfg Wed Feb 27 11:06:20 2019
@@ -402,17 +402,23 @@ llvm_config_cmd.wait()
if platform.system() == 'Windows':
config.test_retry_attempts = 2
-# Only run up to 3 64-bit sanitized processes simultaneously on Darwin.
-# Using more scales badly and hogs the system due to inefficient handling
-# of large mmap'd regions (terabytes) by the kernel.
+# No throttling on non-Darwin platforms.
+lit_config.parallelism_groups['shadow-memory'] = None
+
if platform.system() == 'Darwin':
- lit_config.parallelism_groups["darwin-64bit-sanitizer"] = 3
+ ios_device = config.apple_platform != 'osx' and not config.apple_platform.endswith('sim')
+ # Force sequential execution when running tests on iOS devices.
+ if ios_device:
+ lit_config.warning('Forcing sequential execution for iOS device tests')
+ lit_config.parallelism_groups['ios-device'] = 1
+ config.parallelism_group = 'ios-device'
-# Force sequential execution when running tests on iOS devices.
-if config.host_os == 'Darwin' and config.apple_platform != "osx" and not config.apple_platform.endswith("sim"):
- lit_config.warning("iOS device test cases being run sequentially")
- lit_config.parallelism_groups["ios-device"] = 1
- config.parallelism_group = "ios-device"
+ # Only run up to 3 processes that require shadow memory simultaneously on
+ # 64-bit Darwin. Using more scales badly and hogs the system due to
+ # inefficient handling of large mmap'd regions (terabytes) by the kernel.
+ elif config.target_arch in ['x86_64', 'x86_64h']:
+ lit_config.warning('Throttling sanitizer tests that require shadow memory on Darwin 64bit')
+ lit_config.parallelism_groups['shadow-memory'] = 3
# Multiple substitutions are necessary to support multiple shared objects used
# at once.
Modified: compiler-rt/trunk/test/sanitizer_common/Unit/lit.site.cfg.in
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/sanitizer_common/Unit/lit.site.cfg.in?rev=355018&r1=355017&r2=355018&view=diff
==============================================================================
--- compiler-rt/trunk/test/sanitizer_common/Unit/lit.site.cfg.in (original)
+++ compiler-rt/trunk/test/sanitizer_common/Unit/lit.site.cfg.in Wed Feb 27 11:06:20 2019
@@ -12,3 +12,6 @@ config.name = 'SanitizerCommon-Unit'
config.test_exec_root = os.path.join("@COMPILER_RT_BINARY_DIR@", "lib",
"sanitizer_common", "tests")
config.test_source_root = config.test_exec_root
+
+if config.host_os == 'Darwin':
+ config.parallelism_group = config.darwin_sanitizer_parallelism_group_func
Modified: compiler-rt/trunk/test/sanitizer_common/lit.common.cfg
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/sanitizer_common/lit.common.cfg?rev=355018&r1=355017&r2=355018&view=diff
==============================================================================
--- compiler-rt/trunk/test/sanitizer_common/lit.common.cfg (original)
+++ compiler-rt/trunk/test/sanitizer_common/lit.common.cfg Wed Feb 27 11:06:20 2019
@@ -71,6 +71,4 @@ config.suffixes = ['.c', '.cc', '.cpp']
if config.host_os not in ['Linux', 'Darwin', 'NetBSD', 'FreeBSD']:
config.unsupported = True
-if config.host_os == 'Darwin':
- if config.target_arch in ["x86_64", "x86_64h"]:
- config.parallelism_group = "darwin-64bit-sanitizer"
+config.parallelism_group = 'shadow-memory'
Modified: compiler-rt/trunk/test/tsan/lit.cfg
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/tsan/lit.cfg?rev=355018&r1=355017&r2=355018&view=diff
==============================================================================
--- compiler-rt/trunk/test/tsan/lit.cfg (original)
+++ compiler-rt/trunk/test/tsan/lit.cfg Wed Feb 27 11:06:20 2019
@@ -86,6 +86,4 @@ if config.host_os not in ['FreeBSD', 'Li
if config.android:
config.unsupported = True
-if config.host_os == 'Darwin':
- if config.target_arch in ["x86_64", "x86_64h"]:
- config.parallelism_group = "darwin-64bit-sanitizer"
+config.parallelism_group = 'shadow-memory'
Modified: compiler-rt/trunk/unittests/lit.common.unit.cfg
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/unittests/lit.common.unit.cfg?rev=355018&r1=355017&r2=355018&view=diff
==============================================================================
--- compiler-rt/trunk/unittests/lit.common.unit.cfg (original)
+++ compiler-rt/trunk/unittests/lit.common.unit.cfg Wed Feb 27 11:06:20 2019
@@ -30,10 +30,10 @@ if 'TEMP' in os.environ:
config.environment['TEMP'] = os.environ['TEMP']
if config.host_os == 'Darwin':
- # Only run up to 3 64-bit sanitized processes simultaneously on Darwin.
- # Using more scales badly and hogs the system due to inefficient handling
- # of large mmap'd regions (terabytes) by the kernel.
- lit_config.parallelism_groups["darwin-64bit-sanitizer"] = 3
+ # Only run up to 3 processes that require shadow memory simultaneously on
+ # 64-bit Darwin. Using more scales badly and hogs the system due to
+ # inefficient handling of large mmap'd regions (terabytes) by the kernel.
+ lit_config.parallelism_groups["shadow-memory"] = 3
# The test config gets pickled and sent to multiprocessing workers, and that
# only works for code if it is stored at the top level of some module.
Modified: compiler-rt/trunk/unittests/lit_unittest_cfg_utils.py
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/unittests/lit_unittest_cfg_utils.py?rev=355018&r1=355017&r2=355018&view=diff
==============================================================================
--- compiler-rt/trunk/unittests/lit_unittest_cfg_utils.py (original)
+++ compiler-rt/trunk/unittests/lit_unittest_cfg_utils.py Wed Feb 27 11:06:20 2019
@@ -1,4 +1,4 @@
-# Put all 64-bit sanitizer tests in the darwin-64bit-sanitizer parallelism
-# group. This will only run three of them concurrently.
+# Put all 64-bit tests in the shadow-memory parallelism group. We throttle those
+# in our common lit config (lit.common.unit.cfg).
def darwin_sanitizer_parallelism_group_func(test):
- return "darwin-64bit-sanitizer" if "x86_64" in test.file_path else ""
+ return "shadow-memory" if "x86_64" in test.file_path else None
More information about the llvm-commits
mailing list