<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/56491>56491</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            Problems with malformed gtest json files
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            new issue
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
            yuanfang-chen
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          rorth
      </td>
    </tr>
</table>

<pre>
    Since the switch to gtest sharding in the testsuite, I'm constantly seeing a `lit` failure that breaks the testsuite.  The `ninja stage2-check-all` run ends with
```
PASS: SanitizerCommon-Unit :: ./Sanitizer-x86_64-Test/17/91 (65005 of 65005)
Traceback (most recent call last):
  File "/var/llvm/dist-amd64-release-stage2-A/tools/clang/stage2-bins/./bin/llvm-lit", line 48, in <module>
    main(builtin_parameters)
  File "/vol/llvm/src/llvm-project/dist/llvm/utils/lit/lit/main.py", line 112, in main
    selected_tests, discovered_tests)
  File "/vol/llvm/src/llvm-project/dist/llvm/utils/lit/lit/formats/googletest.py", line 213, in post_process_shard_results
    testsuites = json.load(f)['testsuites']
  File "/usr/lib/python3.7/json/__init__.py", line 296, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.7/json/decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 32 column 1 (char 598)
```
which unfortunately giving no indication whatsoever which json file is malformed.  I had to add debug code to `llvm/utils/lit/lit/formats/googletest.py` to find what went wrong: the malformed file is `/var/llvm/dist-amd64-release-stage2-A/tools/clang/stage2-bins/tools/polly/unittests/Isl/./IslTests-Polly-Unit-3150-1-20.json` and indeed the file has a superfluous trailing brace.

There are at least two issues here:

- The file shouldn't have ever been created this way in the first place.
- `lit` should give a proper indication of the wrong file, rather than just raising an exception.

I'm attaching the broken file (as .txt although it's .json actually).
[IslTests-Polly-Unit-3150-1-20.txt](https://github.com/llvm/llvm-project/files/9092081/IslTests-Polly-Unit-3150-1-20.txt)

</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJy1VkuP4zYM_jXOhbBhO-9DDrMzO8D20C4w27Mhy3SsGVkKJHky6a8vqTjOTLrtYosWSCxbD_Ij-ZFUbZvT7kkZiRA6BH9UQXYQLOwD-gC-E65RZg_KxHWe9IMKmJT38CUp1z1Ia3wQJugTeETeKyBZ5VoFekIrlB4cCxcBaofixX8UlAF8o2_aa5R5FkCy9limskP5kgqtWYgbDKBpPBC6LskfkvyOpsdf_Px69_SUzO_gSRgV1B_o7m3fW5P-Tp9AC7yWJeXjtJ6-bVbVapF-Ixw0X6zpsS0gKTerZZ4vwbYQX5Jye9bwzQmJtZAvvKe35BuHEk0ASSBBCxazZU1xN8Cj0mRVWZLcV-HoqfVrT0OjfEhF35BuhxqFx3Q0-Y5Wg7Xa0yi1MHsax6VaGZ5lC-h1FJayi1nBPWhlEBYbfqVAJfP73jaDxmT--QIHoBd8clMPSgdlqoNwoseAzk8mfgRt9RW0d_Ki9ODsM8owWnLdMgQVkUdQ45NVZofTe5BFUY4oI54JnSdfyIBNFYnBW0i8tK_ornP_F8zWul6wgse9tXuNrO4GdlnMR9gHCn1F0iV6X8X0qBz6QdP5yZiJ3J5i8QDP3ppMW9GQ-1s2Y_mJMue6iT6S5cN3jBt8JI6q6Xk4hc6aecZEZYE0VJUiNlfVLdbtasQadU6oKOIeq0u6ErCbCTpja3YacUM5X3XWvtCuv87RxqQktt69HL8flH-Hez4RmHG_c6fDMDgDVYOtIEfTKG2DLjuP5NS_4cYPYFzE3KLIRxSj-CsMoTzCL0-__foQVz47Z0kBgS4_vwUnoBFBjKIig6lkTcAiBy4ab2VQcXongb7OSEqqrXroDcS6JIlrsNxuJpE3JfDYKarcgyE2h8GIgFSQ9-qVC7KxZE-jpAjKGjhSKfYWKbXgfIaxQct-U57SUnM-YEOF-Qt0ouFmIJqG3FEPe2DQPMMV_uczimo5nW0JTEQBRy6gR2ep1pHV3Bcm9RMgNvA_rKGX1YPV-sTwiYpjfXn84vVYZemNO4NPv_K22EbSebHM0yIt8ywSiGwRZAfZgoSWsUfEnfDUAP1wQNfqwQ7U7og4msNQcwvJxuCdm0pH9Q0E_wOwIQHCkYLl_UC1gxenjnJ-prFZRkW-s4NuDFcSUvqKECNaIxqQ1GdDBEX-O4rTpXu3lMIBDvoKI33Xqs8CmTMEB6jEkQnveUMtkYXEcEUIzHEnaM5xezfwPHBXpCyJlwBq2W8SD3z0g83nW4MIQciON7LM2tkXHDlIVCcXZuEtgNCUtMO-AybVmiYjU4UMg4jB214ELz_9c8BIGFfYctOFcPDs0_KRyUnXiaHOpO2v3LrpHgyJqbHNt2W-KX5IDdZ0SdBZs5s32_lWzIIKGndfna019udrzDumn69aUxb62eD07qeRnklDL8vVYlvMup1oFytEvsRsxGaTl6IRy_ViLRZyW2zm62amRY3a72I7Kg0ez7zjCrZ8mKldmZdlvi7KIl8UizxboWwW-bJtRTFvi3WRLHKkHq4zxpFZt5-5XYREZcLToqYk9ddF4b3aG8RR3WkQpqX85IueGVWKgcLtdlQRQzeLWHbRlj8Bjhpqog">