[llvm] [lit] Truncate process output to 10 kiB (PR #206355)

[via llvm-commits]

llvmorg-github-actions[bot] wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-testing-tools

Author: Alexis Engelke (aengelke)

<details>
<summary>Changes</summary>

The output of processes is transformed multiple times: it is decoded
into utf-8 strings, is pickled, is sent to the main process, is
unpickled, gets things prepended, gets things appended -- and each time,
the entire output is copied into a new memory location. Due to locking,
no other tests can start during this time and as most of this happens in
C code, interrupting also has some delay. Additionally, this can cause
the main lit process to consume a large amount of memory.

We have some tests that produce a large amount of output (llvm-exegesis
in particular has tests that output >50 MiB). This causes the tests to
stall for several seconds, sometimes more than a minute.

Therefore, truncate the output of processes to 10 kiB.

On one of my systems, this reduces the time of check-llvm by 22%:

    Before: 1085.27user 718.78system 1:53.75elapsed 1585%CPU
    After:  1050.81user 702.01system 1:28.42elapsed 1982%CPU

Reproducer of the problem:

    RUN: head -c500000000 /dev/zero


---
Full diff: https://github.com/llvm/llvm-project/pull/206355.diff


1 Files Affected:

- (modified) llvm/utils/lit/lit/TestRunner.py (+15-12) 


``````````diff
diff --git a/llvm/utils/lit/lit/TestRunner.py b/llvm/utils/lit/lit/TestRunner.py
index 1b0f4ad4eebef..77ad44a17d012 100644
--- a/llvm/utils/lit/lit/TestRunner.py
+++ b/llvm/utils/lit/lit/TestRunner.py
@@ -550,20 +550,23 @@ def _executeShCmd(cmd, shenv, results, timeoutHelper):
             res = 1 if res != 0 else 0
 
         # Ensure the resulting output is always of string type.
-        try:
+        # Truncate output as soon as possible so that we don't serialize/process
+        # overly large strings. 10kiB output ought to be enough.
+        def convert_output(out, limit=10 * 1024) -> str:
             if out is None:
-                out = ""
-            else:
+                return ""
+            truncated = len(out) > limit
+            out = out[:limit]
+            try:
                 out = out.decode("utf-8", errors="replace")
-        except:
-            out = str(out)
-        try:
-            if err is None:
-                err = ""
-            else:
-                err = err.decode("utf-8", errors="replace")
-        except:
-            err = str(err)
+            except:
+                out = str(out)
+            if truncated:
+                out += "\n...\ndata was truncated"
+            return out
+
+        out = convert_output(out)
+        err = convert_output(err)
 
         # Gather the redirected output files for failed commands.
         output_files = []

``````````

</details>


https://github.com/llvm/llvm-project/pull/206355