<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/115584>115584</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            [lit] Lit's builtin `env` command fails to match environment variables case insensitively on Windows
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          tahonermann
      </td>
    </tr>
</table>

<pre>
    LLVM's lit utility includes a builtin implementation of the `env` command in `llvm/utils/lit/lit/TestRunner.py`; see [here](https://github.com/llvm/llvm-project/blob/26a9f3f5906c62cff7f2245b98affa432b504a87/llvm/utils/lit/lit/TestRunner.py#L734-L754) and [here](https://github.com/llvm/llvm-project/blob/26a9f3f5906c62cff7f2245b98affa432b504a87/llvm/utils/lit/lit/TestRunner.py#L300-L323). The builtin implementation does not match the case-insensitive behavior that would be expected on Windows.

The issue can be demonstrated by adding the following test to an llvm-project workspace and running it as shown. A couple of notes about the test:
1. The test uses `cmd /c set` as a workaround for https://github.com/llvm/llvm-project/issues/115578. Once that issue is fixed, the `cmd /c set` portion of the command line can be removed and the builtin `env` functionality should suffice.
2. The test is specific to Windows. I tried adding `REQUIRES: system-windows` to it, but that didn't work; the test was still skipped as unsupported for me. I'm guessing there is some relevant `lit.cfg` magic that is necessary to make that work right. More work will be necessary for this test to be checked in as a regression test.
3. If this test does end up getting used as a regression test, `llvm/utils/lit/tests/shtest-env-positive.py` should be updated to validate it in the same manner it does for other tests.
4. Setting an environment variable that is already set changes the case of the variable. This differs from the behavior of the `cmd.exe` `set` command and is related to why the unset case doesn't work. The builtin implementation uses a Python dictionary keyed by the (case-sensitive) variable  name. Unsetting by a case-mismatched name fails because the code doesn't attempt to match case; the preset variable therefore remains in the dictionary.

```
> type llvm/utils/lit/tests/Inputs/shtest-env-positive/env-case-insensitive.txt
## Tests env command for preset variables and case sensitive handling.

## Check and make sure preset environment variable were set in lit.cfg.
#
# RUN: env cmd /c set | FileCheck --check-prefix=CHECK-ENV-PRESET %s
#
## Check set of environment variable by lowercase name.
#
# RUN: env foo=changed cmd /c set | FileCheck --check-prefix=CHECK-ENV-SET-LC %s
## Check unset of environment variable by lowercase name.
#
# RUN: env -u foo cmd /c set | FileCheck --check-prefix=CHECK-ENV-UNSET-LC %s

# CHECK-ENV-PRESET: FOO=1
# CHECK-ENV-PRESET: BAR=2

# CHECK-ENV-SET-LC-NOT: FOO=
# CHECK-ENV-SET-LC: foo=changed
# CHECK-ENV-SET-LC: BAR=2

# CHECK-ENV-UNSET-LC-NOT: FOO=
# CHECK-ENV-UNSET-LC: BAR=2

> python llvm-lit.py -a llvm/utils/lit/tests/Inputs/shtest-env-positive/env-case-insensitive.txt
-- Testing: 1 tests, 1 workers --
FAIL: shtest-env :: env-case-insensitive.txt (1 of 1)
******************** TEST 'shtest-env :: env-case-insensitive.txt' FAILED ********************
Exit Code: 1
...
# RUN: at line 9
env foo=changed cmd /c set | FileCheck --check-prefix=CHECK-ENV-SET-LC C:\Users\thonerma\llvm-project\llvm\utils\lit\tests\Inputs\shtest-env-positive\env-case-insensitive.txt
# executed command: env foo=changed cmd /c set
# executed command: FileCheck --check-prefix=CHECK-ENV-SET-LC 'C:\Users\thonerma\llvm-project\llvm\utils\lit\tests\Inputs\shtest-env-positive\env-case-insensitive.txt'
# .---command stderr------------
# | C:\Users\thonerma\llvm-project\llvm\utils\lit\tests\Inputs\shtest-env-positive\env-case-insensitive.txt:17:25: error: CHECK-ENV-SET-LC-NOT: excluded string found in input
# | # CHECK-ENV-SET-LC-NOT: FOO=
# |                         ^
# | <stdin>:13:1: note: found here
# | FOO=1
# | ^~~~
...
# | Input was:
# | <<<<<<
...
# |        13: FOO=1
# | not:17     !~~~ error: no match expected
...
# | >>>>>>
# `-----------------------------
# error: command failed with exit status: 1
...
Failed Tests (1):
  shtest-env :: env-case-insensitive.txt
...
```

Emulating case insensitive matching is important for testing some environment variables. For example, MSVC environments have an environment variable named `VCToolsInstallDir` that Clang consults (see `findVCToolChainViaEnvironment()` [here](https://github.com/llvm/llvm-project/blob/26a9f3f5906c62cff7f2245b98affa432b504a87/llvm/lib/WindowsDriver/MSVCPaths.cpp#L512-L519). Lit's testing configuration allows this environment variable to be observed in tests, but for some reason, it spells the variable name as `VCToolsinstallDir` (note the lowercase "i" in "install"); see [here](https://github.com/llvm/llvm-project/blob/26a9f3f5906c62cff7f2245b98affa432b504a87/llvm/utils/lit/lit/TestingConfig.py#L60). When `lit` loads variables from the environment, it ends up using its own form of the variable name. Attempts to unset it for testing purposes using the name as MSVC sets it (`VCToolsInstallDir` with an uppercase "I") fails to do so. The case-mismatched name in `TestingConfig.py` should be fixed as part of resolving this issue.
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJzMWU1z2zjS_jXwpYssChT1cfDBoaV6Xa8nydpO5gyRTRFrEmABoGRd_Nu3GhAl2ZadTWa2MiqVI0pA99NfD9AdYa1cK8RLln1i2fWF6F2tzaUTtVZoWqHUxUqXu8vb2-9_MD610EgHvZONdDuQqmj6Ei0IWPWycVKBbLsGW1ROOKkV6ApcjcAmCaoNmyRQ6LYVqgSp6Mum2bSML0meZXzZSHf4-4DW3fVKoYm7HZskLP0EFhFY9qlGgyy7ZnxWO9dZll4xvmR8uZau7ldxoUnmXjT9E3VG_xsLkrpq9IrxJZ-IeZVW2TyZFBNeVNW04nycreYzUVVinPJVlozFbHqU82OIPL2dpuPodpqNGZ8DGflPxpomSXSb8pTxeQwPNb4XwVKjBaUdtMIVtY9mISxGUllUVjq5QVhhLTZSG3C1cLDVfVPCCgGfOiwclqAV_ClVqbc2Zsk1S67CX9Iqre1JoqINJbZaWWcEbVrtQJSlVGuvs9JNo7f-Ca0Dp0EoOHUYbLV5tJ0o0Lve9ErRaulAWLC13qoYrqDQfdcgpaXSjhJ3pXvnFZBYio5HNgou8ap6i5ZytWhLYHxZgEVHmSwo7UmpMLpXJVTawE_H2JtPsRqNsmw6i-GLKjC4MXhGWqjkE5aM50MlvQbSaXNaa0OFNVIdHGuw1RssvWfcSayPdVn1qiApwhe2rX0MbV9VssB90PiJU6QF22EhK1lQLIbowg04I0lRiBybJHeLf327uVvcs_QK7M46bKNtWE1qnQZKzxxWPgzCQSlLxfg0xJOKfggObCmQTjYN2EfZdaTFQq9s35EHMESgxRhuGJ-2sO7R2n36GO9Iq1tyRYMboZynH-nioloTkFasyZTgeFBYoLXC7AhgKx5xyGzzCEauaxfDH9pg-GJLkFZ4sqnypSDtIVdXCEWNxSN64vOZY3BtCKBWftXex2kMN9XJXl99qEroO1ijc2RPb4Ppb2SQH98jVfqdnmxNnyJUm6jToXwDvw4xXyH0Xekr0GnYiEbSA9WRVD4WVrQIdDCgoW89QjJYk589jqHKxzHc7zELBag20mhFzAIbYaRYNYdMB9EYFOWOUhqKWqg12gPVDIk9bKI0lBZKWVVoLFRGtyGpBxY6HjpFW8b4hGQemyT7ehkKxB9DlhJisHZb7_zOXnkcpJvMO-bjh0zpiULA152riTdlqCezg0fcBT7zqPjM8-eBPemwOPgDlKAM_kYAvOOIBQPhttJ6CsbSL4JKyMbCCgvRW9xXfnkKWDiHbedCChN3k5ihojqDZONJINBgRTltsBVS2SHcRztecDedyOEdHtMFuF2H8GHy3aiufy8LGV_S4-uzJXZPbq-CzqoU6CCjktgc4kjJ98oc64PrA3g8pWqhykaq9Us7gtScqtNv8tVue3Nw0dm83RKl0K9SwZ5F4qPA4QPcfftMvOfRnrA2sGkOS9lgUBtFnhyizmAln1h6nf_fIv__aPH5e_T1bnG_eADGM_tW_BE4ydTVeairHTR6i8Z7w6fXx0ArrVl6HYqw_EXY94uH6DZ_BfuANpTX34U36gnyLwL99vkt1IOS12EgjcsvX1h6Pfp40aerO5Ze8_flBaXR5y8nMt9fR2teROXjpT_UPhj9Y_3DyvfEpgvoAt35ew1VQreDSPwPaSCKPAVItSZQI9gLzWHkGZpOhCgKS5dXN7f-2nFQA3Q3C3lzVgWx84gyc8T4fHDIr7_hYXFPxTv97xEwPgXCvbiGv6I5YF88SQe5LtG7KnwXx_GbMhIu3Bb3Jv_NLEDZw7L8m0VjWZa7fWfJsvzFZTg8siwPSZPllDRZHuKb5fukyfJzSZPlPzw7AJ-w6Omg358bP-a7j_f-HBFOf68X-PRoTBxF0XB2WleiMdHJ67iO4vx7UadXoylLr3jmY2WMNvThPRrFJz-NIKMMXZ0q35bRPY3Uv7TrZ-iY1r_3YtnileA0t66UiqULgp_SHxJK7WbgcQLlpwIv9r05V7ywbPH8_HymbulH71Rqig5d6wmG1-_zIvYvj_I8AqVDFIKxfPT8_HwMhBoulkOj_44a74uX7-PvkyT66HVSg4Paw71PyAZL2EpHCKQD64Tr7TmuW4al4e5IHE_0PrgNfuKAeG3hq1twYN22b4S_vPsrzOmgxLvLjyUsdQ_aOOpEfbsYzrTQo567F9kYltoAPgnqOujA--P-e3661EItNvhuq0VXqZL8_T1_0LqxN8o60TTX0vhGnBqxvBGEWivbN8FRfuI2SSqpyrAtr4VU36VYHFUwPiN3Uo_1e-ZdjaR9-xnEtZEbNIwvyT1fhattXHQd4-ltNuLRbTaa-4nXLd1Ipvbg90KrSq57E1o50TR6a0Mffr5v9T29Xlk0m9DUH-4hqz5EdD9tEFYr-prys8OmsS962dDKCXsSF_kiLozPiDr8puO1mHEuGed-isr5fgfj3Kf1P3JOKtU69y7ejx8niQ_DnzWq_SSGrG20KO1JF3fo7fE03bwzUZUW-g56G8Z8FvRWkePb1_OCfU99FdphS7ELHYh8WXpdbzpNPXw_jI0O0fGlZtFZ2kP5_k4VeS4SCvquO0bqJgRm37A7DaUGq8Mo4WxnHwZzb7z2Ykbjh4IErRPG91IGrW42ATixi7U9xhflZVrO07m4wMvRNB2N0iRNk4v6MhtPq6KaFeW4nBfZNEknZTIqcFwVs2k6qbILeckTPh6NknmSpXyUxNlsIubztCxHaZlgmrJxgq2QTUyBj7VZX3iVl6NRls3GF41YYWOH_1Uwlz6_Vv3asnHSSOvscZ-TrvH__-AvDdeH0nw7pDxlfnuca5ylyzfk2-xO5tAXvWku_9q0dja-2Fzy_wQAAP__39EC3Q">