<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class="">Yes, I did, and the difference appears to be in the noise. On my machine:</div><div class=""><br class=""></div><div class="">Without temporary directories</div><div class="">-----------------------------</div><div class=""><br class=""></div><div class=""> $ time ninja -C build check-cxx </div><div class=""><br class=""></div><div class=""> Testing Time: 291.68s</div><div class=""> Expected Passes : 6034</div><div class=""> Expected Failures : 34</div><div class=""> Unsupported Tests : 360</div><div class=""><br class=""></div><div class=""> 3980.19s user 703.50s system 1591% cpu 4:54.21 total</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">With temporary directories</div><div class="">--------------------------</div><div class=""><br class=""></div><div class=""> $ time ninja -C build check-cxx</div><div class=""><br class=""></div><div class=""> Testing Time: 293.28s</div><div class=""> Expected Passes : 6034</div><div class=""> Expected Failures : 34</div><div class=""> Unsupported Tests : 360</div><div class=""><br class=""></div><div class=""> 3986.36s user 708.69s system 1588% cpu 4:55.62 total</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">This is weird, as creating 6000 temporary directories has a lot more than a 1 second overhead (tried!), but I guess that overhead is somehow buried in the context of the whole test suite.</div><div class=""><br class=""></div><div class="">Note that I would love to explore ways to speed up the execution of the test suite, but for now I'm focusing on making it run correctly everywhere -- so long as there's no big speed regression of course.</div><div class=""><br class=""></div><div class="">Louis</div><div><br class=""><blockquote type="cite" class=""><div class="">On Apr 1, 2020, at 22:19, Nico Weber <<a href="mailto:thakis@chromium.org" class="">thakis@chromium.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Did you check if this slows down test execution at all? Creating a temp dir per test sounds potentially slow.</div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Apr 1, 2020 at 10:17 PM Louis Dionne via libcxx-commits <<a href="mailto:libcxx-commits@lists.llvm.org" class="">libcxx-commits@lists.llvm.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br class="">
Author: Louis Dionne<br class="">
Date: 2020-04-01T22:17:03-04:00<br class="">
New Revision: ff09135fc2b7a9696f87a8a8e89be2ef777895d3<br class="">
<br class="">
URL: <a href="https://github.com/llvm/llvm-project/commit/ff09135fc2b7a9696f87a8a8e89be2ef777895d3" rel="noreferrer" target="_blank" class="">https://github.com/llvm/llvm-project/commit/ff09135fc2b7a9696f87a8a8e89be2ef777895d3</a><br class="">
DIFF: <a href="https://github.com/llvm/llvm-project/commit/ff09135fc2b7a9696f87a8a8e89be2ef777895d3.diff" rel="noreferrer" target="_blank" class="">https://github.com/llvm/llvm-project/commit/ff09135fc2b7a9696f87a8a8e89be2ef777895d3.diff</a><br class="">
<br class="">
LOG: [libc++] Execute tests from the Lit execution root instead of the test tree<br class="">
<br class="">
Instead of executing tests from within the libc++ test suite, we execute<br class="">
them from the Lit execution directory. However, since some tests have<br class="">
file dependencies, we must copy those dependencies to the execution<br class="">
directory where they are executed.<br class="">
<br class="">
This has the major benefit that if a test modifies a file (whether it<br class="">
is wanted or not), other tests will not see those modifications. This<br class="">
is good because current tests assume that input data is never modified,<br class="">
however this could be an incorrect assumption if some test does not<br class="">
behave properly.<br class="">
<br class="">
Added: <br class="">
<br class="">
<br class="">
Modified: <br class="">
libcxx/utils/libcxx/test/config.py<br class="">
libcxx/utils/libcxx/test/executor.py<br class="">
libcxx/utils/libcxx/test/format.py<br class="">
libcxx/utils/run.py<br class="">
<br class="">
Removed: <br class="">
<br class="">
<br class="">
<br class="">
################################################################################<br class="">
diff --git a/libcxx/utils/libcxx/test/config.py b/libcxx/utils/libcxx/test/config.py<br class="">
index 3e983e1b4d4a..20d0a796a3af 100644<br class="">
--- a/libcxx/utils/libcxx/test/config.py<br class="">
+++ b/libcxx/utils/libcxx/test/config.py<br class="">
@@ -1050,7 +1050,6 @@ def configure_substitutions(self):<br class="">
exec_args.append('--host {}'.format(self.executor.user_prefix + self.executor.host))<br class="">
executor = os.path.join(self.libcxx_src_root, 'utils', 'ssh.py')<br class="">
else:<br class="">
- exec_args.append('--working_directory "%S"')<br class="">
executor = os.path.join(self.libcxx_src_root, 'utils', 'run.py')<br class="">
sub.append(('%{exec}', '{} {} {} -- '.format(pipes.quote(sys.executable),<br class="">
pipes.quote(executor),<br class="">
<br class="">
diff --git a/libcxx/utils/libcxx/test/executor.py b/libcxx/utils/libcxx/test/executor.py<br class="">
index b555b1f03df9..c34310cdd2e2 100644<br class="">
--- a/libcxx/utils/libcxx/test/executor.py<br class="">
+++ b/libcxx/utils/libcxx/test/executor.py<br class="">
@@ -10,6 +10,7 @@<br class="">
import os<br class="">
import posixpath<br class="">
import ntpath<br class="">
+import shutil<br class="">
<br class="">
from libcxx.test import tracing<br class="">
from libcxx.util import executeCommand<br class="">
@@ -61,6 +62,12 @@ def run(self, exe_path, cmd=None, work_dir='.', file_deps=None, env=None):<br class="">
if env:<br class="">
env = self.merge_environments(os.environ, env)<br class="">
<br class="">
+ for dep in file_deps:<br class="">
+ if os.path.isdir(dep):<br class="">
+ shutil.copytree(dep, os.path.join(work_dir, os.path.basename(dep)), symlinks=True)<br class="">
+ else:<br class="">
+ shutil.copy2(dep, work_dir)<br class="">
+<br class="">
out, err, rc = executeCommand(cmd, cwd=work_dir, env=env)<br class="">
return (cmd, out, err, rc)<br class="">
<br class="">
<br class="">
diff --git a/libcxx/utils/libcxx/test/format.py b/libcxx/utils/libcxx/test/format.py<br class="">
index a026c4f1ba02..1bc85f24976a 100644<br class="">
--- a/libcxx/utils/libcxx/test/format.py<br class="">
+++ b/libcxx/utils/libcxx/test/format.py<br class="">
@@ -9,6 +9,8 @@<br class="">
import copy<br class="">
import errno<br class="">
import os<br class="">
+import shutil<br class="">
+import tempfile<br class="">
import time<br class="">
import random<br class="">
<br class="">
@@ -209,16 +211,21 @@ def _evaluate_pass_test(self, test, tmpBase, lit_config,<br class="">
report += "Compilation failed unexpectedly!"<br class="">
return lit.Test.Result(lit.Test.FAIL, report)<br class="">
# Run the test<br class="">
- local_cwd = os.path.dirname(source_path)<br class="">
env = None<br class="">
if self.exec_env:<br class="">
env = self.exec_env<br class="">
<br class="">
max_retry = test.allowed_retries + 1<br class="">
for retry_count in range(max_retry):<br class="">
- cmd, out, err, rc = self.executor.run(exec_path, [exec_path],<br class="">
- local_cwd, data_files,<br class="">
- env)<br class="">
+ # Create a temporary directory just for that test and run the<br class="">
+ # test in that directory<br class="">
+ try:<br class="">
+ execDirTmp = tempfile.mkdtemp(dir=execDir)<br class="">
+ cmd, out, err, rc = self.executor.run(exec_path, [exec_path],<br class="">
+ execDirTmp, data_files,<br class="">
+ env)<br class="">
+ finally:<br class="">
+ shutil.rmtree(execDirTmp)<br class="">
report = "Compiled With: '%s'\n" % ' '.join(compile_cmd)<br class="">
report += libcxx.util.makeReport(cmd, out, err, rc)<br class="">
if rc == 0:<br class="">
<br class="">
diff --git a/libcxx/utils/run.py b/libcxx/utils/run.py<br class="">
index 6a89a2b9388a..7de82c78dbfa 100644<br class="">
--- a/libcxx/utils/run.py<br class="">
+++ b/libcxx/utils/run.py<br class="">
@@ -14,14 +14,15 @@<br class="">
<br class="">
import argparse<br class="">
import os<br class="">
+import shutil<br class="">
import subprocess<br class="">
import sys<br class="">
+import tempfile<br class="">
<br class="">
<br class="">
def main():<br class="">
parser = argparse.ArgumentParser()<br class="">
parser.add_argument('--codesign_identity', type=str, required=False)<br class="">
- parser.add_argument('--working_directory', type=str, required=True)<br class="">
parser.add_argument('--dependencies', type=str, nargs='*', required=True)<br class="">
parser.add_argument('--env', type=str, nargs='*', required=True)<br class="">
(args, remaining) = parser.parse_known_args(sys.argv[1:])<br class="">
@@ -42,14 +43,23 @@ def main():<br class="">
# Extract environment variables into a dictionary<br class="">
env = {k : v for (k, v) in map(lambda s: s.split('=', 1), args.env)}<br class="">
<br class="">
- # Ensure the file dependencies exist<br class="">
- for file in args.dependencies:<br class="">
- if not os.path.exists(file):<br class="">
- sys.stderr.write('Missing file {} marked as a dependency of a test'.format(file))<br class="">
- exit(1)<br class="">
+ try:<br class="">
+ tmpDir = tempfile.mkdtemp()<br class="">
<br class="">
- # Run the executable with the given environment in the given working directory<br class="">
- return subprocess.call(' '.join(remaining), cwd=args.working_directory, env=env, shell=True)<br class="">
+ # Ensure the file dependencies exist and copy them to a temporary directory.<br class="">
+ for dep in args.dependencies:<br class="">
+ if not os.path.exists(dep):<br class="">
+ sys.stderr.write('Missing file or directory "{}" marked as a dependency of a test'.format(dep))<br class="">
+ exit(1)<br class="">
+ if os.path.isdir(dep):<br class="">
+ shutil.copytree(dep, os.path.join(tmpDir, os.path.basename(dep)), symlinks=True)<br class="">
+ else:<br class="">
+ shutil.copy2(dep, tmpDir)<br class="">
+<br class="">
+ # Run the executable with the given environment in the temporary directory.<br class="">
+ return subprocess.call(' '.join(remaining), cwd=tmpDir, env=env, shell=True)<br class="">
+ finally:<br class="">
+ shutil.rmtree(tmpDir)<br class="">
<br class="">
if __name__ == '__main__':<br class="">
exit(main())<br class="">
<br class="">
<br class="">
<br class="">
_______________________________________________<br class="">
libcxx-commits mailing list<br class="">
<a href="mailto:libcxx-commits@lists.llvm.org" target="_blank" class="">libcxx-commits@lists.llvm.org</a><br class="">
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/libcxx-commits" rel="noreferrer" target="_blank" class="">https://lists.llvm.org/cgi-bin/mailman/listinfo/libcxx-commits</a><br class="">
</blockquote></div>
</div></blockquote></div><br class=""></body></html>