<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/110265>110265</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            Buildkite CI builds that include certain projects are testing clang twice
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            clang
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          DavidSpickett
      </td>
    </tr>
</table>

<pre>
    I noticed this looking into https://github.com/llvm/llvm-project/pull/110239, and its build logs https://buildkite.com/llvm-project/github-pull-requests/builds/104650#0192330a-3b1e-45ca-845c-1c442bc2f321.

If you look at the ninja targets used on Linux:
```
ninja -C <...> -k 0 check-all check-clang check-clang-tools
```
It's testing everything and then testing clang and clang-tools for a second time. Or rather, it's reversed due to however ninja orders targets, but the point is that it doesn't know that "check-all" is a superset of "check-anything-else".

We can see it finish one check and start lit again for the next one:
```
GTotal Discovered Tests: 38089^M
G Skipped          :     4 (0.01%)^M
G  Unsupported      :    88 (0.23%)^M
G  Passed           : 37966 (99.68%)^M
G  Expectedly Failed: 31 (0.08%)^M
G^M[8000/8003] Generating GwpAsan-x86_64-Test^[[K^[_bk;t=1727436048237^G^M[8001/8003] Running all regression tests^[[K^[_bk;t=1727436048237^G^M
<...>
Gllvm-lit: /var/lib/buildkite-agent/builds/linux-56-59b8f5d88-f4kc6-1/llvm-project/github-pull-requests/llvm/utils/lit/lit/main.py:72: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 1200 seconds was requested on the command line. Forcing timeout to be 1200 seconds.^M
G-- Testing: 87609 tests, 56 workers --^M
GPASS: Clang :: Driver/clang-offload-bundler.c (1 of 87609)^M
GPASS: Clang :: Driver/darwin-ld.c (2 of 87609)^M
```
And confirm that the same test is run twice:
```
$ cat github-pull-requests_build_104650_linux-linux-x64\ \(1\).log | grep "clang-offload-bundler.c"
clang-offload-bundler.c (10908 of 21210)
clang-offload-bundler.c (1 of 87609)
```

This is about a 24% more tests than it needs to run.

This does not happen for Windows but only by chance:
```
-- Build files have been written to: C:/ws/src/build
+ echo '--- ninja'
--- ninja
+ ninja -C C:/ws/src/build -k 0 check-clang check-clang-tools
```
The `check-all` target is added by `compiler-rt` in `.ci/generate-buildkite-pipeline-premerge`:
```
function check-targets() {
  projects=${@}
  for project in ${projects}; do
 case ${project} in
    clang-tools-extra)
      echo "check-clang-tools"
    ;;
    compiler-rt)
      echo "check-all"
    ;;
```
However compiler-rt testing is disabled for Windows:
```
function exclude-windows() {
  projects=${@}
  for project in ${projects}; do
    case ${project} in
 cross-project-tests) ;; # tests failing
```

I think this pipeline builder script needs to know that `check-all` is a superset of all the enabled `check` targets.
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJy0V0FzozoS_jXypQsXCDDm4EMcx7Op3a19tTNb75iSobH1LEusJOLk32-1BA6ZSebtHF4qRQh0f0Jfd3_dEs7Jo0bcsHLLyt1CDP5k7GYnnmX7tZfNGb1fHEz7unkEbbxssAV_kg6UMWepjyC1N3Dyvncsv2N8z_j-KP1pOCwbc2F8r9Tz9CfprfkDG8_4vh-UYnyfZSnPa8bvQegWpHdwGKRqQZmj-w40vDhLjzPcGWBcMyHcxOJ_B3TeTV50k6XFqkwZz9Os5nmeiiQ_ZJgUZSOSdVE2SdYUBT80vMt5tmTpjqV38frYwasZwn5BePAnBC31HwK8sEf0DgaHLRgN_5B6eKHvjc6rdPwN_0aX5B5Yfr9cLln-AMkZUmhO2JwTodR41yihj_P7xBuj3IeYj57xyoFH5ykS-Iz21Z_oltj0J9S3dxGWHs9AoTMWBDhsDNnLCy7hXxas8Ce0FBQZF7CETJtsBwSKtrnSk5EGY1u0bmKD3A5DpKk3UnuQDvxJeJAeWoNOM155OGtzjY8Z5zcOGOdkLsANPa3owXQzAx13l6ByyDh_F6XfERqhwSHSQp3U0p3AaIxUhp07L6wHJT2Io5A67D5EE188mX4Wui_fjBcKdtI15hkttvAtZFd-B_k6XdesfPjnaAlfz7LvsYXbD1nRTwGMr9NlmjFeMv7OB_6j3dD3xvrJcXRar6MTzz9w-k04N18oOOVVvVqRU10vV-sPvB5eemw8tuoV9kIqbINXNn7cjx7httyu0zRlfL9O05yVO_iCGq0IifXl2t85oZOX9eppVSTEDCsfgphs_x7vng5nlm89y3dZxasiX6XFmucVKx9m8NkM_t-D1iGLlQKLR4vOSRNz2f0qeAzoVHPjroJ2KOlp84zvn4UlQZGHucwk4ojazyVEUX0n5Sop68O6K9v1OumKc7NKsv9bjkYxHLxUEdHfrhch9bJ_Zfldxem7tPGUkfDthGHn4AbpERqjO3kciH2jYYTGFoQGqVv5LNtBqOhABW2GUEPpWOQu1KaYv8p4-vb2KtwMkzg_0ZKXC9WPkhqXsDe2oeBMEN7AAd-hLGf5kyShWKQ-0l7W1SqtxzjyeyhXcDX2TOqRJDOn3-6-fiXz-yBaVJb5HeysfCZV2kcBM12njGiTw6BbhXbZUApntKGwyPss_jlgK-xV6kS1EYR_DPKdKNyRlFIs7CXqGDHlxGUMlnRgBw3-KptPdYXxAhrh4aNceQpZ9xS71lPMvHh9WRWsvAdW3tOGw596qcwRWHUPR4t9EMyPOWKcx6V_xmFap2uigGc8S4mCP_V4R9iHOw3XbzQ2kLwfTEhCXjBewsXYyFloE5rUWyO2jhLLDnr5AwI1EaoOOIm-xyjjv0vdmmvMbqPVKxxeoTkJ_Tn5SQLbMGp0UqGDk3hGOCBquFrpPXVOEzImzh9XqlZnm0kOpgBuAZuTAcarJEliP2S8mlaYntyMb0PAJ7DzkeBXBgHSCLZK3_roKh37ceC7bbElRsjEXHqp0CbWk5HU9HDZSJKsKOqYvElgL3ukqk96ixe0R6QVPyG0G3QTNCl-xW0cWDNeA6u20Qpg1EjH8h3jBb0oUlbtptcUzdEkfF0wuflUO5ZvoTWjdSMcvjdh1Q6knsBgPu0k-OKtuKVobJpj-PiPPE-lElvrln7fUGcs_gwvjjQf43xH39_GmWoGfRveKOmlEweF7Tzb_zQS-NKoocXkOtr_pbEgWn4ejsYa56YemYxdoB5JAcbzUQY6IRU1jM-V5JGOIPocDyJTksajA1pwjZX9TEVms-Z3NfLDrEkTBwk56sj25PBWUG65aDd5W-e1WOAmzBxFnqX14rTpujLLRJ5zxK4om6Krqgq7tipW5QFXTb2QG57yIq15leXZmtfLqhWdyCvR5l3GyzpnRYoXIdWS5oSlsceFdG7ADR2TVuVCiQMqF05qo8RTdpW7hd2E8eMwHB0rUiWdd28QXnqFm-1U1HD_GJmaxnIdkgQatJ6m4im-IEZdfjs-hG62GKza_PJ5L-zDjSe-Vbl43vD_BQAA__-Dc2N1">