<table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Issue</th>
<td>
<a href=https://github.com/llvm/llvm-project/issues/106249>106249</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>
[llvm-lit] Lit's built-in diff command causing clang test to fail when run with internal shell
</td>
</tr>
<tr>
<th>Labels</th>
<td>
clang
</td>
</tr>
<tr>
<th>Assignees</th>
<td>
</td>
</tr>
<tr>
<th>Reporter</th>
<td>
connieyzhu
</td>
</tr>
</table>
<pre>
Lit's built-in diff command gives the incorrect output in [clang/test/APINotes/yaml-roundtrip.test](https://github.com/llvm/llvm-project/blob/main/clang/test/APINotes/yaml-roundtrip.test), causing the test to fail in lit's internal shell.
Error message:
```
FAIL: Clang :: APINotes/yaml-roundtrip.test (8 of 10)
******************** TEST 'Clang :: APINotes/yaml-roundtrip.test' FAILED ********************
Exit Code: 1
Command Output (stdout):
--
# RUN: at line 1
/usr/local/google/home/connieyzhu/llvm-fork/build/bin/apinotes-test /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes > /usr/local/google/home/connieyzhu/llvm-fork/build/tools/clang/test/APINotes/Output/yaml-roundtrip.test.tmp.result
# executed command: /usr/local/google/home/connieyzhu/llvm-fork/build/bin/apinotes-test /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes
# RUN: at line 2
not diff -b /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes /usr/local/google/home/connieyzhu/llvm-fork/build/tools/clang/test/APINotes/Output/yaml-roundtrip.test.tmp.result | /usr/local/google/home/connieyzhu/llvm-fork/build/bin/FileCheck /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/yaml-roundtrip.test --dump-input-context=1000
# executed command: not diff -b /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes /usr/local/google/home/connieyzhu/llvm-fork/build/tools/clang/test/APINotes/Output/yaml-roundtrip.test.tmp.result
# executed command: /usr/local/google/home/connieyzhu/llvm-fork/build/bin/FileCheck /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/yaml-roundtrip.test --dump-input-context=1000
# .---command stderr------------
# | /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/yaml-roundtrip.test:11:8: error: CHECK: expected string not found in input
# | CHECK: 7c8
# | ^
# | <stdin>:1:1: note: scanning from here
# | *** /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes
# | ^
# | <stdin>:1:11: note: possible intended match here
# | *** /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes
# | ^
# |
# | Input file: <stdin>
# | Check file: /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/yaml-roundtrip.test
# |
# | -dump-input=help explains the following input dump.
# |
# | Input was:
# | <<<<<<
# | 1: *** /usr/local/google/home/connieyzhu/llvm-fork/clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes
# | check:11'0 X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
# | check:11'1 ? possible intended match
# | 2: --- /usr/local/google/home/connieyzhu/llvm-fork/build/tools/clang/test/APINotes/Output/yaml-roundtrip.test.tmp.result
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 3: ***************
# | check:11'0 ~~~~~~~~~~~~~~~~
# | 4: *** 1,28 ****
# | check:11'0 ~~~~~~~~~~~~~~
# | 5: Name: SimpleKit
# | check:11'0 ~~~~~~~~~~~~~~~~~
# | 6: Classes:
# | check:11'0 ~~~~~~~~~~
# | 7: - Name: I
# | check:11'0 ~~~~~~~~~~~
# | 8: Properties:
# | check:11'0 ~~~~~~~~~~~~~
# | 9: - Name: nonnullProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~
# | 10: PropertyKind: Class
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~
# | 11: ! Nullability: N
# | check:11'0 ~~~~~~~~~~~~~~~~~
# | 12: - Name: nonnullNewProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 13: PropertyKind: Class
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~
# | 14: Nullability: Nonnull
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~
# | 15: - Name: optionalProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 16: PropertyKind: Class
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~
# | 17: ! Nullability: O
# | check:11'0 ~~~~~~~~~~~~~~~~~
# | 18: - Name: optionalNewProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 19: PropertyKind: Class
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~
# | 20: Nullability: Optional
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~
# | 21: - Name: unspecifiedProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 22: PropertyKind: Instance
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~
# | 23: ! Nullability: U
# | check:11'0 ~~~~~~~~~~~~~~~~~
# | 24: - Name: unspecifiedNewProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 25: PropertyKind: Instance
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~
# | 26: Nullability: Unspecified
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 27: - Name: scalarProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~
# | 28: PropertyKind: Instance
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~
# | 29: ! Nullability: S
# | check:11'0 ~~~~~~~~~~~~~~~~~
# | 30: - Name: scalarNewProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 31: PropertyKind: Instance
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~
# | 32: ! Nullability: Scalar
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~
# | 33: --- 1,30 ----
# | check:11'0 ~~~~~~~~~~~~~~
# | 34: + ---
# | check:11'0 ~~~~~~
# | 35: Name: SimpleKit
# | check:11'0 ~~~~~~~~~~~~~~~~~
# | 36: Classes:
# | check:11'0 ~~~~~~~~~~
# | 37: - Name: I
# | check:11'0 ~~~~~~~~~~~
# | 38: Properties:
# | check:11'0 ~~~~~~~~~~~~~
# | 39: - Name: nonnullProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~
# | 40: PropertyKind: Class
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~
# | 41: ! Nullability: Nonnull
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~
# | 42: - Name: nonnullNewProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 43: PropertyKind: Class
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~
# | 44: Nullability: Nonnull
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~
# | 45: - Name: optionalProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 46: PropertyKind: Class
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~
# | 47: ! Nullability: Optional
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~
# | 48: - Name: optionalNewProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 49: PropertyKind: Class
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~
# | 50: Nullability: Optional
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~
# | 51: - Name: unspecifiedProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 52: PropertyKind: Instance
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~
# | 53: ! Nullability: Unspecified
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 54: - Name: unspecifiedNewProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 55: PropertyKind: Instance
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~
# | 56: Nullability: Unspecified
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 57: - Name: scalarProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~
# | 58: PropertyKind: Instance
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~
# | 59: ! Nullability: Unspecified
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 60: - Name: scalarNewProperty
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 61: PropertyKind: Instance
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~
# | 62: ! Nullability: Unspecified
# | check:11'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 63: ! ...
# | check:11'0 ~~~~~~
# | >>>>>>
# `-----------------------------
# error: command failed with exit status: 1
```
Steps to reproduce:
1. CMake Config:
```cmake -G Ninja -DLLVM_ENABLE_PROJECTS="clang;llvm;clang-tools-extra;lld;lldb;bolt;mlir;libclc;polly" -DLLVM_ENABLE_RUNTIMES="compiler-rt;libc;libcxx;libunwind;libcxxabi" -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -B build -S .```
2. Use this command to run lit's internal shell on this test.
`LIT_USE_INTERNAL_SHELL=1 LIT_FILTER=yaml-roundtrip.test ninja check-clang`
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJzsWltz4r4V_zTiRWPGljCXBx64mC5dQjIh6WyfMsIWoP_KkseSN0kf-tk7krl6MRuuSaf1ODFY8rn8zk_inPEhSrG5oLQN_C7w-xWS6YVM26EUgtH3fy2yylRG7-0R0wA1FJxmjGuHCRix2QyGMo6JiOCc_aIK6gWFTIQyTWmoocx0kmnIBAR-N-REzAEaaKo0QIPOw3AsNVUADd5JzJ1UZiLSKUuqdoLfB6i50DpRAHcAGgA0mDO9yKbVUMYADTj_tbo4SSr_oqEROuVyCtAgJkwANDhOI2oB1IMhyRQTc-uIuQ21hDPCuHGCLwFgQtNUEA7VgnJeBW4fuJ38f5CmMoUxVYrMqbE8H6u7y9N-HXSGI4A7sGfsg2YW7sDD1kGAmk0oZ9BzjaG5VHT6CZ-CyRMEqHGEDQA1oDE96MNzVC-BemMa9mRkQILeNoS9JaHuc_IA1FQ6kpkN0ApQx1lBgOHj89jIIBpyJuhaFhpkKjUMkSHhhj1SzjkFaLCQsbls2L2i0UymPw2HMsYjc7UcIgkTBhJnGYVTpZZzcSiSTJsPg5TE9FWmP82XCYsTTquz1T2ABt8oiWi6NbiyDQIcnGHaymEtJVcHTc1Dsp8dVR0n1ZSqjOtNbOgbDTNNo9UuYQL1vxSZUpaifERIne-izvSLevCVeAVBo3cx-gwYp70FDX9eBfh9W7jjRFmcOMzExAml0PRNA9z3XNc9vGL-z5Kvt_t8UfpUHcdZZWRKRzRNna1jM--8lXRcYoU7ngdwp2ngpyY_srnPt6D33d55S2hogqR0ahIvQ_aZEWAyLuvsrtXrBxths-CPHxRu4J7SERMAB8aI5Z_RYLMOFRIhjMpZKmO4oCkt4rNKlr7mojve-x33E6kUm3Jq01kR0QjGRIeL_3ogNsfvkOx8s4rhjHELxzZcu4Szq3w97UaL5oDdW_sAwP0F5YlZRJwwkddfM8m5fDXEtnOgmV79MxCvRG0Klg2HimcJ1hB6OTxfmyoFz0MT23xloIYLf_z7U4_N7ijkcjHanfCAzR7cPQAelC3s8sgho9JxnK_0K344UJ8Tn3IE8Q73P1SAH_TPHB_XXttdeR5APdSE52jbne4b-XBMYrsD5ivqO_tDiPa5UJBbt3J7nChFlfXh4wJ3pzasJGdt4_Ao28qRtTnLQyoTmmp2tI0HRbcKJgspRMb5Utn7CeDu1-m5VtFK7neW58EW9RMWWalD3nL_9-A445xMGWf63dwan82TbS1oP2xj-noB5H4HD98IvFq-xArI5b6d6lC5Nr8Aokw0k4JcgnxFAOs3ArBRwr77i7KvWQLcxeh3QHfrNkgidx8V75eOnuxduT6vgGkmVEJDNmM0ujamCO3FdCiUJiKkp-Ug5dpwCUefL8lRVCvH88M0PQjnQe3-GYgeCuTug6i-j6TPG09Pda_csWKKoULCSXo-Q8s1bicf10OyVULKySVJid296F2Cj-U6vZtQcVsjKsPSenuZ33GMV9WayfOxaz465-X3eFlAdOERospRuFa5sK2jvv7tO7Ny2BZ6pSoCN7eJeNkyAt-qjNg6au6-jekDiUiput2naqWVxEfy4aO9uW1Fsa0ZXxnH04uKYz25XUGxrbV-ZfxKa4oPJcPH-_OJ1UWtdV0s_TOqimN98T-xovDRLZI2v7SSOD8DPuDbp1YW_m0qC__mlYV_88rCv0ll4ZdVFlclaf3MSuMPS79cr3cLTOtlFcZ1MV1vN9Vq9SPiiy8Rg-K5Ga-7zqFjM3P9bmzV3jAjjNMIvjK9gPSNaag00ZlNrlc9gLttlxNNEwW1hClNUhll4VaDpv3vVWHvjvyksCfFjM1_b98MYzPq_A2OmfiLQKc_Gv3j7iUYd7qj4OXh8f7vQe9pAnAfIJS_AMNd26eK8-ZXx74dc-ibTokdivL_U4C7U8k1wN2Ys9TcZNOQhwB3E8n5O0CooOvxefw0vAtWumScME5TJ9XLZ5eXt7f8QyZeLSGXN8mU5SJ7d53vwUv3eTjqvzz98yEAuP9IOSWKFvR1JpPg8Wl4PzYa78frR3svvfu7h-EoeAS4b33cDP348dsgQF1bZXZtD3EEnQmsFoKEqvBZUagXTK0jbUKWlbXfQiny2fYF4jpgo-HTy_MkeBmOn4LHcWf0MvkWjEYA9z1ohgbD0ZO1a1-njbDRtcx2crvrbiVq46iFW6RC214D1RqNJkaosmj7zdB3XdykbiukDeT7rSap0RDXm9SrYc-vsDZyUc1toobXcH2Eqv4Uodm0USPY92b1aR3UXBoTxquGK1WZzitMqYy2PbeOaq0KJ1PKle3NXtEKIeD3K2nbvoGdZnMFai5nSquNCM00tw3ddo6Bzu_Dwx3cq9bnPI47zc-vCypsDOxq2w1AJUt5--hObeuiAmiw9PJXG_0nAAD__71yQSM">