<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">The Makefiles still use it and I don't plan to change the Makefiles. I leave the discussion when to switch everyone from the Makefiles to cmake/lit in the testsuite is for another day :)</div><div class=""><br class=""></div><div class="">- Matthias</div><div class=""><br class=""></div><div><blockquote type="cite" class=""><div class="">On Mar 14, 2016, at 10:23 PM, David Blaikie via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" class="">llvm-commits@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_extra"><br class="Apple-interchange-newline"><br class=""><div class="gmail_quote">On Mon, Mar 14, 2016 at 10:12 PM, Matthias Braun via llvm-commits<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:llvm-commits@lists.llvm.org" target="_blank" class="">llvm-commits@lists.llvm.org</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;">Author: matze<br class="">Date: Tue Mar 15 00:12:28 2016<br class="">New Revision: 263534<br class=""><br class="">URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project?rev=263534&view=rev" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-project?rev=263534&view=rev</a><br class="">Log:<br class="">lit: Skip RunSafely.sh and invoke timeit directly<br class=""></blockquote><div class=""><br class=""></div><div class="">At what point can we remove RunSafely? (last time I played with the test-suite there were a bunch of intermediate layers - would be great to remove some of them)</div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><br class="">This also allows us to split run_under and remove device handling into<br class="">separate modules.<br class=""><br class="">Added:<br class=""> <span class="Apple-converted-space"> </span>test-suite/trunk/litsupport/remote.py<br class=""> <span class="Apple-converted-space"> </span>test-suite/trunk/litsupport/run_under.py<br class="">Removed:<br class=""> <span class="Apple-converted-space"> </span>test-suite/trunk/litsupport/runsafely.py<br class="">Modified:<br class=""> <span class="Apple-converted-space"> </span>test-suite/trunk/lit.cfg<br class=""> <span class="Apple-converted-space"> </span>test-suite/trunk/litsupport/test.py<br class=""> <span class="Apple-converted-space"> </span>test-suite/trunk/litsupport/timeit.py<br class=""><br class="">Modified: test-suite/trunk/lit.cfg<br class="">URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/test-suite/trunk/lit.cfg?rev=263534&r1=263533&r2=263534&view=diff" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-project/test-suite/trunk/lit.cfg?rev=263534&r1=263533&r2=263534&view=diff</a><br class="">==============================================================================<br class="">--- test-suite/trunk/lit.cfg (original)<br class="">+++ test-suite/trunk/lit.cfg Tue Mar 15 00:12:28 2016<br class="">@@ -8,6 +8,7 @@<span class="Apple-converted-space"> </span><a href="http://config.name/" rel="noreferrer" target="_blank" class="">config.name</a><span class="Apple-converted-space"> </span>= 'test-suite'<br class=""> config.test_format = test.TestSuiteTest()<br class=""> config.suffixes = ['.test']<br class=""> config.excludes = ['ABI-Testsuite']<br class="">+config.remote_flags = ""<br class=""> config.traditional_output = True<br class=""> if 'SSH_AUTH_SOCK' in os.environ:<br class=""> config.environment['SSH_AUTH_SOCK'] = os.environ['SSH_AUTH_SOCK']<br class=""><br class="">Added: test-suite/trunk/litsupport/remote.py<br class="">URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/remote.py?rev=263534&view=auto" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/remote.py?rev=263534&view=auto</a><br class="">==============================================================================<br class="">--- test-suite/trunk/litsupport/remote.py (added)<br class="">+++ test-suite/trunk/litsupport/remote.py Tue Mar 15 00:12:28 2016<br class="">@@ -0,0 +1,30 @@<br class="">+import logging<br class="">+import testplan<br class="">+<br class="">+<br class="">+def mutateCommandline(context, commandline):<br class="">+ shfilename = context.tmpBase + ".sh"<br class="">+ shfile = open(shfilename, "w")<br class="">+ shfile.write(commandline + "\n")<br class="">+ <span class="Apple-converted-space"> </span><a href="http://logging.info/" rel="noreferrer" target="_blank" class="">logging.info</a>("Created shfile '%s'", shfilename)<br class="">+ shfile.close()<br class="">+<br class="">+ config = context.config<br class="">+ remote_commandline = config.remote_client<br class="">+ if config.remote_user:<br class="">+ remote_commandline += " -l %s" % config.remote_user<br class="">+ if config.remote_port:<br class="">+ remote_commandline += " -rp %s" % config.remote_port<br class="">+ if config.remote_flags:<br class="">+ remote_commandline += config.remote_flags<br class="">+ remote_commandline += " %s" % config.remote_host<br class="">+ remote_commandline += " /bin/sh %s" % shfilename<br class="">+ return remote_commandline<br class="">+<br class="">+<br class="">+def mutateScript(context, script):<br class="">+ return testplan.mutateScript(context, script, mutateCommandline)<br class="">+<br class="">+<br class="">+def mutatePlan(context, plan):<br class="">+ plan.runscript = mutateScript(context, plan.runscript)<br class=""><br class="">Added: test-suite/trunk/litsupport/run_under.py<br class="">URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/run_under.py?rev=263534&view=auto" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/run_under.py?rev=263534&view=auto</a><br class="">==============================================================================<br class="">--- test-suite/trunk/litsupport/run_under.py (added)<br class="">+++ test-suite/trunk/litsupport/run_under.py Tue Mar 15 00:12:28 2016<br class="">@@ -0,0 +1,13 @@<br class="">+import shellcommand<br class="">+import testplan<br class="">+<br class="">+<br class="">+def mutateCommandLine(context, commandline):<br class="">+ return context.config.run_under + commandline<br class="">+<br class="">+<br class="">+def mutatePlan(context, plan):<br class="">+ run_under = context.config.run_under<br class="">+ if run_under:<br class="">+ plan.runscript = testplan.mutateScript(context, plan.runscript,<br class="">+ mutateCommandLine)<br class=""><br class="">Removed: test-suite/trunk/litsupport/runsafely.py<br class="">URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/runsafely.py?rev=263533&view=auto" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/runsafely.py?rev=263533&view=auto</a><br class="">==============================================================================<br class="">--- test-suite/trunk/litsupport/runsafely.py (original)<br class="">+++ test-suite/trunk/litsupport/runsafely.py (removed)<br class="">@@ -1,72 +0,0 @@<br class="">-import lit.Test<br class="">-import shellcommand<br class="">-import testplan<br class="">-import timeit<br class="">-<br class="">-<br class="">-def mutateCommandline(context, commandline):<br class="">- outfile = context.tmpBase + ".out"<br class="">- timefile = outfile + ".time"<br class="">- config = context.config<br class="">- cmd = shellcommand.parse(commandline)<br class="">-<br class="">- runsafely = "%s/RunSafely.sh" % config.test_suite_root<br class="">- runsafely_prefix = []<br class="">- if cmd.workdir is not None:<br class="">- runsafely_prefix += ["-d", cmd.workdir]<br class="">- cmd.workdir = None<br class="">- timeit = "%s/tools/timeit" % config.test_source_root<br class="">- if config.remote_host:<br class="">- timeit = "%s/tools/timeit-target" % config.test_source_root<br class="">- runsafely_prefix += ["-r", config.remote_host]<br class="">- if config.remote_user:<br class="">- runsafely_prefix += ["-l", config.remote_user]<br class="">- if config.remote_client:<br class="">- runsafely_prefix += ["-rc", config.remote_client]<br class="">- if config.remote_port:<br class="">- runsafely_prefix += ["-rp", config.remote_port]<br class="">- if config.run_under:<br class="">- runsafely_prefix += ["-u", config.run_under]<br class="">- if not config.traditional_output:<br class="">- runsafely_prefix += ["-n"]<br class="">- if cmd.stdout is not None:<br class="">- runsafely_prefix += ["-o", cmd.stdout]<br class="">- cmd.stdout = None<br class="">- if cmd.stderr is not None:<br class="">- runsafely_prefix += ["-e", cmd.stderr]<br class="">- cmd.stderr = None<br class="">- else:<br class="">- if cmd.stdout is not None or cmd.stderr is not None:<br class="">- raise Exception("Separate stdout/stderr redirection not " +<br class="">- "possible with traditional output")<br class="">- timeout = "7200"<br class="">- if cmd.stdin is not None:<br class="">- stdin = cmd.stdin<br class="">- cmd.stdin = None<br class="">- else:<br class="">- stdin = "/dev/null"<br class="">- runsafely_prefix += ["-t", timeit, timeout, stdin, outfile]<br class="">-<br class="">- context.timefiles.append(outfile + ".time")<br class="">-<br class="">- cmd.wrap(runsafely, runsafely_prefix)<br class="">- return cmd.toCommandline()<br class="">-<br class="">-<br class="">-def mutateScript(context, script):<br class="">- return testplan.mutateScript(context, script, mutateCommandline)<br class="">-<br class="">-<br class="">-def _getTime(context, timefiles, metric_name='exec_time'):<br class="">- time = 0.0<br class="">- for timefile in timefiles:<br class="">- time += timeit.getUserTime(timefile)<br class="">- return {metric_name: lit.Test.toMetricValue(time)}<br class="">-<br class="">-<br class="">-def mutatePlan(context, plan):<br class="">- context.timefiles = []<br class="">- plan.runscript = mutateScript(context, plan.runscript)<br class="">- plan.metric_collectors.append(<br class="">- lambda context: _getTime(context, context.timefiles)<br class="">- )<br class=""><br class="">Modified: test-suite/trunk/litsupport/test.py<br class="">URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/test.py?rev=263534&r1=263533&r2=263534&view=diff" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/test.py?rev=263534&r1=263533&r2=263534&view=diff</a><br class="">==============================================================================<br class="">--- test-suite/trunk/litsupport/test.py (original)<br class="">+++ test-suite/trunk/litsupport/test.py Tue Mar 15 00:12:28 2016<br class="">@@ -13,9 +13,11 @@ import compiletime<br class=""> import hash<br class=""> import perf<br class=""> import profilegen<br class="">-import runsafely<br class="">+import remote<br class="">+import run_under<br class=""> import shellcommand<br class=""> import testplan<br class="">+import timeit<br class=""><br class=""><br class=""> SKIPPED = lit.Test.ResultCode('SKIPPED', False)<br class="">@@ -76,12 +78,15 @@ class TestSuiteTest(FileBasedTest):<br class=""> lit.util.mkdir_p(os.path.dirname(tmpBase))<br class=""><br class=""> # Prepare test plan<br class="">- runsafely.mutatePlan(context, plan)<br class="">+ run_under.mutatePlan(context, plan)<br class="">+ timeit.mutatePlan(context, plan)<br class=""> compiletime.mutatePlan(context, plan)<br class=""> codesize.mutatePlan(context, plan)<br class=""> hash.mutatePlan(context, plan)<br class=""> if config.profile_generate:<br class=""> profilegen.mutatePlan(context, plan)<br class="">+ if config.remote_host:<br class="">+ remote.mutatePlan(context, plan)<br class=""> if litConfig.params.get('profile') == 'perf':<br class=""> perf.mutatePlan(context, plan)<br class=""><br class=""><br class="">Modified: test-suite/trunk/litsupport/timeit.py<br class="">URL:<span class="Apple-converted-space"> </span><a href="http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/timeit.py?rev=263534&r1=263533&r2=263534&view=diff" rel="noreferrer" target="_blank" class="">http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/timeit.py?rev=263534&r1=263533&r2=263534&view=diff</a><br class="">==============================================================================<br class="">--- test-suite/trunk/litsupport/timeit.py (original)<br class="">+++ test-suite/trunk/litsupport/timeit.py Tue Mar 15 00:12:28 2016<br class="">@@ -1,4 +1,71 @@<br class="">+import lit.Test<br class=""> import re<br class="">+import shellcommand<br class="">+import testplan<br class="">+<br class="">+<br class="">+def mutateCommandLine(context, commandline):<br class="">+ outfile = context.tmpBase + ".out"<br class="">+ timefile = context.tmpBase + ".time"<br class="">+ config = context.config<br class="">+ cmd = shellcommand.parse(commandline)<br class="">+<br class="">+ timeit = "%s/tools/timeit" % config.test_source_root<br class="">+ if config.remote_host:<br class="">+ timeit = "%s/tools/timeit-target" % config.test_source_root<br class="">+ args = ["--limit-core", "0"]<br class="">+ args += ["--limit-cpu", "7200"]<br class="">+ args += ["--timeout", "7200"]<br class="">+ args += ["--limit-file-size", "104857600"]<br class="">+ args += ["--limit-rss-size", "838860800"]<br class="">+ if cmd.workdir is not None:<br class="">+ args += ["--chdir", cmd.workdir]<br class="">+ cmd.workdir = None<br class="">+ if not config.traditional_output:<br class="">+ if cmd.stdout is not None:<br class="">+ args += ["--redirect-stdout", cmd.stdout]<br class="">+ cmd.stdout = None<br class="">+ if cmd.stderr is not None:<br class="">+ args += ["--redirect-stderr", cmd.stderr]<br class="">+ cmd.stderr = None<br class="">+ else:<br class="">+ if cmd.stdout is not None or cmd.stderr is not None:<br class="">+ raise Exception("Separate stdout/stderr redirection not " +<br class="">+ "possible with traditional output")<br class="">+ args += ["--append-exitstatus"]<br class="">+ args += ["--redirect-output", outfile]<br class="">+ if cmd.stdin is not None:<br class="">+ args += ["--redirect-input", cmd.stdin]<br class="">+ cmd.stdin = None<br class="">+ else:<br class="">+ args += ["--redirect-input", "/dev/null"]<br class="">+ args += ["--summary", timefile]<br class="">+ # Remember timefilename for later<br class="">+ context.timefiles.append(timefile)<br class="">+<br class="">+ cmd.wrap(timeit, args)<br class="">+ return cmd.toCommandline()<br class="">+<br class="">+<br class="">+def mutateScript(context, script):<br class="">+ if not hasattr(context, "timefiles"):<br class="">+ context.timefiles = []<br class="">+ return testplan.mutateScript(context, script, mutateCommandLine)<br class="">+<br class="">+<br class="">+def _collectTime(context, timefiles, metric_name='exec_time'):<br class="">+ time = 0.0<br class="">+ for timefile in timefiles:<br class="">+ time += getUserTime(timefile)<br class="">+ return { metric_name: lit.Test.toMetricValue(time) }<br class="">+<br class="">+<br class="">+def mutatePlan(context, plan):<br class="">+ context.timefiles = []<br class="">+ plan.runscript = mutateScript(context, plan.runscript)<br class="">+ plan.metric_collectors.append(<br class="">+ lambda context: _collectTime(context, context.timefiles)<br class="">+ )<br class=""><br class=""><br class=""> def getUserTime(filename):<br class=""><br class=""><br class="">_______________________________________________<br class="">llvm-commits mailing list<br class=""><a href="mailto:llvm-commits@lists.llvm.org" class="">llvm-commits@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br class=""></blockquote></div><br class=""></div></div><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">llvm-commits mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="mailto:llvm-commits@lists.llvm.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">llvm-commits@lists.llvm.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a></div></blockquote></div><br class=""></body></html>