<table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Issue</th>
<td>
<a href=https://github.com/llvm/llvm-project/issues/77257>77257</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>
FileCheck should a unified diff output
</td>
</tr>
<tr>
<th>Labels</th>
<td>
testing-tools
</td>
</tr>
<tr>
<th>Assignees</th>
<td>
</td>
</tr>
<tr>
<th>Reporter</th>
<td>
cjdb
</td>
</tr>
</table>
<pre>
I find the way in which FileCheck renders its output very confusing and difficult to read. It would be good if there was a `--diff` option that let users render their mismatches in the form of various diff outputs too. In addition to making the output more readable, this would also make it easier to show multiple mismatches in the same run.
**FileCheck default**
```
--
/tmp/example/test:52:17: error: CHECK_CMAKE: expected string not found in input
// CHECK_CMAKE: -- The C compiler identification is [[COMPILER]] {{\d+(.\d+)*}}
^
<stdin>:6:32: note: scanning from here
-- Running vcpkg install - done
^
<stdin>:6:32: note: with "COMPILER" equal to "GNU"
-- Running vcpkg install - done
^
<stdin>:7:1: note: possible intended match here
-- The C compiler identification is GNU 13.2.1
^
Input file: <stdin>
Check file: /tmp/example/test
-dump-input=help explains the following input dump.
Input was:
<<<<<<
1: -- Bootstrapping vcpkg before install
2: -- Bootstrapping vcpkg before install - done
3: -- Running vcpkg install
4: All requested packages are currently installed.
5: Total install time: 581 ns
6: -- Running vcpkg install - done
check:52'0 X error: no match found
check:52'1 with "COMPILER" equal to "GNU"
7: -- The C compiler identification is GNU 13.2.1
check:52'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:52'2 ? possible intended match
8: -- The CXX compiler identification is GNU 13.2.1
check:52'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
9: -- Detecting C compiler ABI info
check:52'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
10: -- Detecting C compiler ABI info - done
check:52'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
11: -- Check for working C compiler: /usr/bin/gcc - skipped
check:52'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
12: -- Detecting C compile features
check:52'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.
.
.
>>>>>>
```
**Unified diff**
A lot of the information that FileCheck provides should be implicitly available via the change in output structure. The failing check prefix is presented next to the filename.
```
--- /tmp/example/test (CHECK_CMAKE)
+++ <stdin>
@@ -52,2 +7,2 @@
- -- The C compiler identification is GNU {{\d+(.\d+)*}}
+ -- The C compiler identification is GNU 13.2.1
- -- The CXX compiler identification is GNU {{\d+(.\d+)*}}
+ -- The CXX compiler identification is GNU 13.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
```
Substitutions are automatically applied in the diff: perhaps there should be a `--diff=unidiff-no-substitutions` to show `[[COMPILER]]` instead of `GNU`.
**Possible interface**
```
--diff=<value> - Format how output is dumped to stderr
=llvm - Outputs a traditional FileCheck output. `--dump-input-*` are
ignored unless this mode is selected.
=unidiff - Outputs a unified diff
=unidiff-no-substitutions - Outputs a unified diff, without substituting macros in the
diff output
```
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJy8WEtv4zgS_jX0pSBDovw8-ODY8WwwOw_MTgN9W9BiyeKEIjUklXRf9rcvipJtJXbSTrBYgZAViaz6WC9-FeG9OhjEFZvesel2JNpQWbcq_pL70d7K76sHKJWRECqEZ_EdlIHnShUV7JTGTYXFIzg0Ep0HFTzYNjRtgCd036Gwpmy9MgcQRoJUZamKVgcIFhwKOYaHAM-21RL2CAdrJaiS9DjS5EEAm6VJQuvYLAXbBGUNhEoE0Big9aSz002rlINa-VqEokJPMAlxaV0NtoQn4ZRtfQTRY_QQrB3DgwEhpepkW6jFIwGmtf1Wausw4hV7jYxvIFTK97iF9nEJggqAwiuCYsFX9hnqVgfVaLyCyosawbVmzNItS9f9ndM4W1ViKVodutf9lFnaj_hnkhxX7kLdML7Db6JuCOQuoA8sX085y9fZnOVrQOeso4fNP-43P_9788v65_v4_luDRUAJPjjaubEBStsaSWCVadpwUsL47vXqJIE_K4QNFLZulEYHSqIJqlSFiCZVHmJg3W1---X3h3_e_8GmWzbdApvf0ZhuJON3jC_Gp8clbXe-pREVA5ve9xDyjQ9SGZaT7hnL1zntjyAj_fpCGEN7KJ2tgeLoaCf4o-2-PBXN4wGU8UFoDQlIa_pJ8P51M4hnFSpgnJ_2yzng363QFBiM859-_cI4_wiwN1STV7Oh5sZ6r_YaQZlAWSEhht1LQ_zQWz_9-gWyfMzHWa_0pD3eHygioFQ6qhwiip-70D19fiMwB_IS2dZN0sVZvq1QNxSRWijj-wTW2j6TheIcoOnjS0DPwrN8fTLT69EZMusj9s7a4IMTTXO2_B5LyvOjAy5jgt--uPPeFRl5L-O60_v5E5q01hoc_t2ip9RsRPEoDuhBOISidQ5N0N-PC1GOj2untPZPG4Q-SQ2qjr6YLjIw_gqo2bugXuylIO92ZYXP0x9kTLy-nuuOsX08xupyRWB2i0D4UIIN181vrVfnDLi-7f987Loig3clJd-9lbOX6BdD9F-__h_xH3cwRLPs0WwxYBEoagYWXd89gDKl_aT2C21ZepO29yL10zvNjjWjL2zWwbN1jy8x9KWu9Y7x3V4ZxneHooAE_KNqGpT_Ize8iZG_bR8oUYTWof8chkvPj99-dSzLdDq9HNfIS8drvhhVKuzI4QumE-9r0DYQfaOTgJzsanEmgWeq1Dj7pCR6Il49nVR1o1WhqEyKJ6E0kTd4UiKKKiphDiTxyPF8cG1BdhrHBCuF0mTGopeOpfpGmdU49GioIhv8FnlsPKKURiNqfMnmXlO15K3TEBhfDFkVXx4NdNeNy1OWTVI2SSEhL244MH437x7i-17jzbXudipGYD7DIZIPVK7Pobm1Ip540Lu15LZZL1ja6-iO93-1ex9UaAlHd3iLNliK4UJoisym0RT9fVsQk4CIHLpKNL5vhc4xPWyI8m1rFD0lxiZ-qId6pWMPQniusG-aQgc8CknZxWYpHZqz9Eo_8vvwgHKlKPAyTS9ivUfI8s2T0C2y_B4S2MXsBYLVZ53ykc6hjHiDROeO1YXlW62fakjgt75ZExCc6Po0oQe538ka96Y5kcmEMM5SMvpN_H5wqYOxDiW0RqP3XbtXW4mE16OO3dJ4ALR3xPtChxtph0XvQs6FQ2-TwzeRGVmqZqfV5gC1KJw9dp69tkEX_MqDI7nK5TJfihGusnk6mc0mGV-MqhUuZkuZLUWZpov5HEuRTvcSp8tZtl9KnhcjteIpn6RZOufZZDFZjJezyXyKE1nKyXJRZAWbpFgLpcfk2LF1h5HyvsXVfM6n85EWe9Q-_hOCcyqLyhySYK32ROam25Fb0bpk3x48m6Ra-eDPkoIKGlfnoOhz5qWJ-h2PWqdXVQhN7BhiV3tQoWr348LWjO9IaP-TNM7-hUVgfBexesZ3Ee5_AwAA__9d80Zl">