<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/84488>84488</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            post_process_shard_results needs to be more robust
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            llvm-lit
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
            yuanfang-chen
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          rorth
      </td>
    </tr>
</table>

<pre>
    When running a LLVM 18.1.0 build on FreeBSD 14.0/amd64 with `test-release.sh`, `llvm-lit` didn't properly summarize the test results.  The build log shows
```
[...]
PASS: llvm-libc++-shared.cfg.in :: std/utilities/variant/variant.visit/visit_return_type.pass.cpp (97112 of 97112)
Traceback (most recent call last):
  File "/var/llvm/reltest/llvm-18.1.0/final/llvm-project/llvm/utils/lit/lit/formats/googletest.py", line 304, in post_process_shard_results
    testsuites = json.load(f)["testsuites"]
  File "/usr/local/lib/python3.9/json/__init__.py", line 293, in load
 return loads(fp.read(),
  File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads
 return _default_decoder.decode(s)
  File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode
 raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/llvm/reltest/llvm-18.1.0/final/Phase3/Release/llvmCore-18.1.0-final.obj/./bin/llvm-lit", line 65, in <module>
    main(builtin_parameters)
  File "/var/llvm/reltest/llvm-18.1.0/final/llvm-project/llvm/utils/lit/lit/main.py", line 129, in main
    selected_tests, discovered_tests = GoogleTest.post_process_shard_results(
  File "/var/llvm/reltest/llvm-18.1.0/final/llvm-project/llvm/utils/lit/lit/formats/googletest.py", line 306, in post_process_shard_results
    raise RuntimeError(
RuntimeError: Failed to parse json file: /var/llvm/reltest/llvm-18.1.0/final/Phase3/Release/llvmCore-18.1.0-final.obj/runtimes/runtimes-bins/compiler-rt/lib/msan/tests/./Msan-x86_64-Test-MemorySanitizer-Unit-37844-0-9.json

FAILED: CMakeFiles/check-all /var/llvm/reltest/llvm-18.1.0/final/Phase3/Release/llvmCore-18.1.0-final.obj/CMakeFiles/check-all
```
I could trace this to two msan tests producing empty  `*.json` files (Issue #84482), which `googletest.py` chokes on.  This **must** not happen: `llvm-lit` needs to be robust enough to cope with such input, invalid or no.
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJzMVt1u2zoSfhr6ZiBCpuS_C184drTooukWTXf3UqDIscWGIgWSSuI-_QEpOU6C9uIctAcHMCxyNBp-8_dxuPfqZBC3ZHFDGDsP3By5OWWiRUMYI4vDjA-htW7rrAvtrLHyvP1_iwbcYIwyJ-Dw8eP_7mC-pnOaQzMoLcEaqBzizf0B5iXNCat4J5clPKnQAlnmAX3IHGrkHqlvyTInbB9faP3YZVoFssxBKmkIWwXone3R6TP4oeu4U98RQosQjYBDP-jgKcDXFqfTtT2Bb-2TJ_mB5LtoffyN28UNpZQsDuP28-7-nhQ7mE5uBGE3hN1kvuUOJRXHE1UGSLGLSj5IwqohKK2CQk9Y9cid4iZcV_RReZX28Vk7DIMzdTj3SHvuPRV9D4StN6v5nIE9QloQthnRfHVcYMPFQ9TpbHJQoAkguNaguQ9RtdiN2gCV0ggxT-l8wqroBmGVQx3DMwmyMTmEVUdluL5Ie2e_oQjXr6Jj0SmdHBj_j9Z1PETpydqTxmiW9ud05h60MghFXsa1MtBbH-reWYHe1zGCsp4SdAEMKW1-UAE9kOIA37w1VFsuCVsfo3OpDq9KYxH-wN_BJ3-tGB1SDWFVfw6tNQXdEFZFw4RVda2MCnX9DjPbFBPmdPZofkxWkvgIp6cOE7CIi-1_OYiiXL4C4d-iqCUe-aBDLVFYiY6OT8LW_qVc_gqWi7l3UIrVBGU65iVftvkW36CRKV8e9ZE6_lS_hrMHJZ9JcaifLvucsA1F8xK8Xw94sZgAvwIzBZArj_Dv-_98OiTxrXPWJSDs9rlHESJtPXI94GQwIUbn6CTcwNHZDj5ZM5lMRXrB8d5wsYP3ZiOfRJBzEFYPnYF57GfRcpcCMxFR-j8MLn7XciN1XNhjIjfe2EcEfBbYBxXDsAdubGjRXYVghRicQ_lCCH8viXxuuceCsOrLyOST4t46nJSzpEpTCVWUsKpR5mIu0cs1n8tLOkmx76wcNJLi9lqFHY9friPBB2XqnjveYUD3k174TWQYUbyrw3mihgg8QXwB7FGjCCjrRGVRRSov7CO6iyy1078SqX5NpPpz8mTrfxjhL_8E4Y_t-GUwQXXXXkyv3wiLHVRcaZQQLPTceUyXAxyVTh31m8vTjVj8q2XWKBP3wna90ugyF16IqvM8lvKU3VTcd56b7Hm9rJdlFjOa3WFn3fmeGxXUd3TZf40KWbFal2WWZxua-O1V31a7Dx9vD9HV_R1_wJjqdHqL4iGLrfubI_DjU384RH0AYQctIUSqgdAqH5MWnizEuIx3fJzb5CAiqWHXhzNAmvN2o9_LPCXWR4764P0Qq7pYl-WajZctPLVKpGHxbRkucxCtfUAP1qSpT0UTO8J23RBDEVdgbICW9z2aVDhv50qDKBPcBsHZZvAB0Njh1EaZsD2OY6ofRAvK9EMYa_2RayXBOjCWzuS2kJtiw2e4na_yzSZfbPJ81m7FcbNZLFc5nxeS8WIlVoIJuVmtiqJphFjP1JblrMyLfD0vFvMyp4ulEKVkOS_mZb4oJSlz7LjSNEKm1p1mKkZnG0OznmneoPbTsP6aRheHmdsmQTOcPClzrXzwVyNBBY3bn7frm6h01l1CMxuc3rYh9D5eGKyKtKBCOzRU2O5aie_oJUGOVZRQ_xEAAP__6TDsew">