<div dir="ltr">OK, thanks for the context/status</div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 14, 2016 at 10:26 PM, Matthias Braun <span dir="ltr"><<a href="mailto:matze@braunis.de" target="_blank">matze@braunis.de</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div>The Makefiles still use it and I don't plan to change the Makefiles. I leave the discussion when to switch everyone from the Makefiles to cmake/lit in the testsuite is for another day :)</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>- Matthias</div></font></span><div><div class="h5"><div><br></div><div><blockquote type="cite"><div>On Mar 14, 2016, at 10:23 PM, David Blaikie via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:</div><br><div><div dir="ltr" style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Mar 14, 2016 at 10:12 PM, Matthias Braun via llvm-commits<span> </span><span dir="ltr"><<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>></span><span> </span>wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Author: matze<br>Date: Tue Mar 15 00:12:28 2016<br>New Revision: 263534<br><br>URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project?rev=263534&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=263534&view=rev</a><br>Log:<br>lit: Skip RunSafely.sh and invoke timeit directly<br></blockquote><div><br></div><div>At what point can we remove RunSafely? (last time I played with the test-suite there were a bunch of intermediate layers - would be great to remove some of them)</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>This also allows us to split run_under and remove device handling into<br>separate modules.<br><br>Added:<br>   <span> </span>test-suite/trunk/litsupport/remote.py<br>   <span> </span>test-suite/trunk/litsupport/run_under.py<br>Removed:<br>   <span> </span>test-suite/trunk/litsupport/runsafely.py<br>Modified:<br>   <span> </span>test-suite/trunk/lit.cfg<br>   <span> </span>test-suite/trunk/litsupport/test.py<br>   <span> </span>test-suite/trunk/litsupport/timeit.py<br><br>Modified: test-suite/trunk/lit.cfg<br>URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project/test-suite/trunk/lit.cfg?rev=263534&r1=263533&r2=263534&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/test-suite/trunk/lit.cfg?rev=263534&r1=263533&r2=263534&view=diff</a><br>==============================================================================<br>--- test-suite/trunk/lit.cfg (original)<br>+++ test-suite/trunk/lit.cfg Tue Mar 15 00:12:28 2016<br>@@ -8,6 +8,7 @@<span> </span><a href="http://config.name/" rel="noreferrer" target="_blank">config.name</a><span> </span>= 'test-suite'<br> config.test_format = test.TestSuiteTest()<br> config.suffixes = ['.test']<br> config.excludes = ['ABI-Testsuite']<br>+config.remote_flags = ""<br> config.traditional_output = True<br> if 'SSH_AUTH_SOCK' in os.environ:<br>     config.environment['SSH_AUTH_SOCK'] = os.environ['SSH_AUTH_SOCK']<br><br>Added: test-suite/trunk/litsupport/remote.py<br>URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/remote.py?rev=263534&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/remote.py?rev=263534&view=auto</a><br>==============================================================================<br>--- test-suite/trunk/litsupport/remote.py (added)<br>+++ test-suite/trunk/litsupport/remote.py Tue Mar 15 00:12:28 2016<br>@@ -0,0 +1,30 @@<br>+import logging<br>+import testplan<br>+<br>+<br>+def mutateCommandline(context, commandline):<br>+    shfilename = context.tmpBase + ".sh"<br>+    shfile = open(shfilename, "w")<br>+    shfile.write(commandline + "\n")<br>+   <span> </span><a href="http://logging.info/" rel="noreferrer" target="_blank">logging.info</a>("Created shfile '%s'", shfilename)<br>+    shfile.close()<br>+<br>+    config = context.config<br>+    remote_commandline = config.remote_client<br>+    if config.remote_user:<br>+        remote_commandline += " -l %s" % config.remote_user<br>+    if config.remote_port:<br>+        remote_commandline += " -rp %s" % config.remote_port<br>+    if config.remote_flags:<br>+        remote_commandline += config.remote_flags<br>+    remote_commandline += " %s" % config.remote_host<br>+    remote_commandline += " /bin/sh %s" % shfilename<br>+    return remote_commandline<br>+<br>+<br>+def mutateScript(context, script):<br>+    return testplan.mutateScript(context, script, mutateCommandline)<br>+<br>+<br>+def mutatePlan(context, plan):<br>+    plan.runscript = mutateScript(context, plan.runscript)<br><br>Added: test-suite/trunk/litsupport/run_under.py<br>URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/run_under.py?rev=263534&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/run_under.py?rev=263534&view=auto</a><br>==============================================================================<br>--- test-suite/trunk/litsupport/run_under.py (added)<br>+++ test-suite/trunk/litsupport/run_under.py Tue Mar 15 00:12:28 2016<br>@@ -0,0 +1,13 @@<br>+import shellcommand<br>+import testplan<br>+<br>+<br>+def mutateCommandLine(context, commandline):<br>+    return context.config.run_under + commandline<br>+<br>+<br>+def mutatePlan(context, plan):<br>+    run_under = context.config.run_under<br>+    if run_under:<br>+        plan.runscript = testplan.mutateScript(context, plan.runscript,<br>+                                               mutateCommandLine)<br><br>Removed: test-suite/trunk/litsupport/runsafely.py<br>URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/runsafely.py?rev=263533&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/runsafely.py?rev=263533&view=auto</a><br>==============================================================================<br>--- test-suite/trunk/litsupport/runsafely.py (original)<br>+++ test-suite/trunk/litsupport/runsafely.py (removed)<br>@@ -1,72 +0,0 @@<br>-import lit.Test<br>-import shellcommand<br>-import testplan<br>-import timeit<br>-<br>-<br>-def mutateCommandline(context, commandline):<br>-    outfile = context.tmpBase + ".out"<br>-    timefile = outfile + ".time"<br>-    config = context.config<br>-    cmd = shellcommand.parse(commandline)<br>-<br>-    runsafely = "%s/RunSafely.sh" % config.test_suite_root<br>-    runsafely_prefix = []<br>-    if cmd.workdir is not None:<br>-        runsafely_prefix += ["-d", cmd.workdir]<br>-        cmd.workdir = None<br>-    timeit = "%s/tools/timeit" % config.test_source_root<br>-    if config.remote_host:<br>-        timeit = "%s/tools/timeit-target" % config.test_source_root<br>-        runsafely_prefix += ["-r", config.remote_host]<br>-        if config.remote_user:<br>-            runsafely_prefix += ["-l", config.remote_user]<br>-        if config.remote_client:<br>-            runsafely_prefix += ["-rc", config.remote_client]<br>-        if config.remote_port:<br>-            runsafely_prefix += ["-rp", config.remote_port]<br>-    if config.run_under:<br>-        runsafely_prefix += ["-u", config.run_under]<br>-    if not config.traditional_output:<br>-        runsafely_prefix += ["-n"]<br>-        if cmd.stdout is not None:<br>-            runsafely_prefix += ["-o", cmd.stdout]<br>-            cmd.stdout = None<br>-        if cmd.stderr is not None:<br>-            runsafely_prefix += ["-e", cmd.stderr]<br>-            cmd.stderr = None<br>-    else:<br>-        if cmd.stdout is not None or cmd.stderr is not None:<br>-            raise Exception("Separate stdout/stderr redirection not " +<br>-                            "possible with traditional output")<br>-    timeout = "7200"<br>-    if cmd.stdin is not None:<br>-        stdin = cmd.stdin<br>-        cmd.stdin = None<br>-    else:<br>-        stdin = "/dev/null"<br>-    runsafely_prefix += ["-t", timeit, timeout, stdin, outfile]<br>-<br>-    context.timefiles.append(outfile + ".time")<br>-<br>-    cmd.wrap(runsafely, runsafely_prefix)<br>-    return cmd.toCommandline()<br>-<br>-<br>-def mutateScript(context, script):<br>-    return testplan.mutateScript(context, script, mutateCommandline)<br>-<br>-<br>-def _getTime(context, timefiles, metric_name='exec_time'):<br>-    time = 0.0<br>-    for timefile in timefiles:<br>-        time += timeit.getUserTime(timefile)<br>-    return {metric_name: lit.Test.toMetricValue(time)}<br>-<br>-<br>-def mutatePlan(context, plan):<br>-    context.timefiles = []<br>-    plan.runscript = mutateScript(context, plan.runscript)<br>-    plan.metric_collectors.append(<br>-        lambda context: _getTime(context, context.timefiles)<br>-    )<br><br>Modified: test-suite/trunk/litsupport/test.py<br>URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/test.py?rev=263534&r1=263533&r2=263534&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/test.py?rev=263534&r1=263533&r2=263534&view=diff</a><br>==============================================================================<br>--- test-suite/trunk/litsupport/test.py (original)<br>+++ test-suite/trunk/litsupport/test.py Tue Mar 15 00:12:28 2016<br>@@ -13,9 +13,11 @@ import compiletime<br> import hash<br> import perf<br> import profilegen<br>-import runsafely<br>+import remote<br>+import run_under<br> import shellcommand<br> import testplan<br>+import timeit<br><br><br> SKIPPED = lit.Test.ResultCode('SKIPPED', False)<br>@@ -76,12 +78,15 @@ class TestSuiteTest(FileBasedTest):<br>         lit.util.mkdir_p(os.path.dirname(tmpBase))<br><br>         # Prepare test plan<br>-        runsafely.mutatePlan(context, plan)<br>+        run_under.mutatePlan(context, plan)<br>+        timeit.mutatePlan(context, plan)<br>         compiletime.mutatePlan(context, plan)<br>         codesize.mutatePlan(context, plan)<br>         hash.mutatePlan(context, plan)<br>         if config.profile_generate:<br>             profilegen.mutatePlan(context, plan)<br>+        if config.remote_host:<br>+            remote.mutatePlan(context, plan)<br>         if litConfig.params.get('profile') == 'perf':<br>             perf.mutatePlan(context, plan)<br><br><br>Modified: test-suite/trunk/litsupport/timeit.py<br>URL:<span> </span><a href="http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/timeit.py?rev=263534&r1=263533&r2=263534&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport/timeit.py?rev=263534&r1=263533&r2=263534&view=diff</a><br>==============================================================================<br>--- test-suite/trunk/litsupport/timeit.py (original)<br>+++ test-suite/trunk/litsupport/timeit.py Tue Mar 15 00:12:28 2016<br>@@ -1,4 +1,71 @@<br>+import lit.Test<br> import re<br>+import shellcommand<br>+import testplan<br>+<br>+<br>+def mutateCommandLine(context, commandline):<br>+    outfile = context.tmpBase + ".out"<br>+    timefile = context.tmpBase + ".time"<br>+    config = context.config<br>+    cmd = shellcommand.parse(commandline)<br>+<br>+    timeit = "%s/tools/timeit" % config.test_source_root<br>+    if config.remote_host:<br>+        timeit = "%s/tools/timeit-target" % config.test_source_root<br>+    args = ["--limit-core", "0"]<br>+    args += ["--limit-cpu", "7200"]<br>+    args += ["--timeout", "7200"]<br>+    args += ["--limit-file-size", "104857600"]<br>+    args += ["--limit-rss-size", "838860800"]<br>+    if cmd.workdir is not None:<br>+        args += ["--chdir", cmd.workdir]<br>+        cmd.workdir = None<br>+    if not config.traditional_output:<br>+        if cmd.stdout is not None:<br>+            args += ["--redirect-stdout", cmd.stdout]<br>+            cmd.stdout = None<br>+        if cmd.stderr is not None:<br>+            args += ["--redirect-stderr", cmd.stderr]<br>+            cmd.stderr = None<br>+    else:<br>+        if cmd.stdout is not None or cmd.stderr is not None:<br>+            raise Exception("Separate stdout/stderr redirection not " +<br>+                            "possible with traditional output")<br>+        args += ["--append-exitstatus"]<br>+        args += ["--redirect-output", outfile]<br>+    if cmd.stdin is not None:<br>+        args += ["--redirect-input", cmd.stdin]<br>+        cmd.stdin = None<br>+    else:<br>+        args += ["--redirect-input", "/dev/null"]<br>+    args += ["--summary", timefile]<br>+    # Remember timefilename for later<br>+    context.timefiles.append(timefile)<br>+<br>+    cmd.wrap(timeit, args)<br>+    return cmd.toCommandline()<br>+<br>+<br>+def mutateScript(context, script):<br>+    if not hasattr(context, "timefiles"):<br>+        context.timefiles = []<br>+    return testplan.mutateScript(context, script, mutateCommandLine)<br>+<br>+<br>+def _collectTime(context, timefiles, metric_name='exec_time'):<br>+    time = 0.0<br>+    for timefile in timefiles:<br>+        time += getUserTime(timefile)<br>+    return { metric_name: lit.Test.toMetricValue(time) }<br>+<br>+<br>+def mutatePlan(context, plan):<br>+    context.timefiles = []<br>+    plan.runscript = mutateScript(context, plan.runscript)<br>+    plan.metric_collectors.append(<br>+        lambda context: _collectTime(context, context.timefiles)<br>+    )<br><br><br> def getUserTime(filename):<br><br><br>_______________________________________________<br>llvm-commits mailing list<br><a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br></blockquote></div><br></div></div><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important">_______________________________________________</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important">llvm-commits mailing list</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><a href="mailto:llvm-commits@lists.llvm.org" style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">llvm-commits@lists.llvm.org</a><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" style="font-family:Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a></div></blockquote></div><br></div></div></div></blockquote></div><br></div>