<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/66331>66331</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            [fuzzer] RunIndividualFiles may have more runs than the number specified in the argument
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            new issue
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          yingcong-wu
      </td>
    </tr>
</table>

<pre>
    I observe that test `fuzzer-finalstats.test` will fail flakily. After investigation, I found that RunIndividualFiles may runs more than the number given in the argument.

In the https://github.com/llvm/llvm-project/blob/b171849afe46b372b163a725ccb40c754cb918fe/compiler-rt/lib/fuzzer/FuzzerDriver.cpp#L853
```cpp
 for (int Iter = 0; Iter < Runs; Iter++)
        RunOneTest(F, Path.c_str(), Options.MaxLen);
```
We can see that the `Runs` is predetermined, but inside `RunOneTest()`
```cpp
int RunOneTest(Fuzzer *F, const char *InputFilePath, size_t MaxLen) {
  Unit U = FileToVector(InputFilePath);
  if (MaxLen && MaxLen < U.size())
 U.resize(MaxLen);
  F->ExecuteCallback(U.data(), U.size());
  if (Flags.print_full_coverage) {
    // Leak detection is not needed when collecting full coverage data.
    F->TPCUpdateObservedPCs();
  } else {
    F->TryDetectingAMemoryLeak(U.data(), U.size(), true);
  }
 return 0;
}
```
`TryDetectingAMemoryLeak()` will also have the chance to run `ExecuteCallback()`, and also because we call `TryDetectingAMemoryLeak()` with `DuringInitialCorpusExecution=True`, so the check for `TotalNumberOfRuns` inside it will not work.
```cpp
  if (!DuringInitialCorpusExecution &&
      TotalNumberOfRuns >= Options.MaxNumberOfRuns)
    return;
```

First, I come up with this solution:
```cpp
  while (F->getTotalNumberOfRuns() < (size_t)Runs) 
        RunOneTest(F, Path.c_str(), Options.MaxLen);
```
but this is not complete, because in the last round of `RunOneTest()` it is still possible to have 2 runs(one for normal run and one for the leak detection.

Then I think maybe we can disable the leak detection for the last round, but the downside is if we only have 1 run, then we will have no leak detection.

What do you think the proper solution is here?
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJysVlFvozgQ_jXOy6iIGErIQx7aZJEi7V5Xq_b2sTIwgK-OjWzTbPbXn8ZAs03UvTvpqioJNp75ZubzNyOck61G3LDbe3a7W4jBd8ZuTlK3ldHtzXFYlKY-bfZgSof2FcF3woNH54FlcTP8_In2ppFaKOeFdxHtsCyGo1QKGiEVNEq8SHWK4K7xaEHqV3RetsJLoxnfwh4aM-h6NPxt0Htdy1dZD0IVUqGDgziBHbSDg7HBvQbfIejhUKKFVr6iBjmuCdsOB9Q-YvGOxXfj537c67zvHUvuGC8YL1rpu6GMKnNgvFDqdf666a35CyvPeFEqU9LXcrXM07VoMM3KZMXLZZaIFb-tqjKNq9VtWpXrZd4g40VlDr1UaG8snVeSjo8ZYrwowo-dla9oo6rvGU8-57fJhDSLx3_aCCvQGAuM51J72FPeWLKDmCX389OWcuXmBcbvw_96Oj39fRv0g8ZHqgnPC8r2V-G7qHp23jKe0_t8Cw891cJFX8SPz6hpMbm_wDU-fkeohAaHMw06JBYEIFkM0kFvsUaP9iA11mS7HDxI7WQ9v3nGQ45my9cJoMDfww8JBMbvQiCV0c5D1YmwtNf94IkvFB9tO_kTnz28hQRsdT_n5klLD08ho3Tk0fyJlTeUkAsz50QAyIbKMdoDxjPGM5ifki08ReRxzulchqfI4rR-nVyA4oYlnz79wGrwuBVKlaJ6YTx_imrhxbk-l7YvQRVKtC7qrdT-uRmUeq7MK1rR4kXcACP54TOKF6A6VVR4qps2HjRijTUcO9RQGaVoV7dAFmG2CIQsOhsMETx-3T71tfD4MGpE_XXrJrRnqGy1A1QO3wMaz9vTbgSj27sveDD2RAj_MRNb8HbAKzfTT4t-sDrcmYlj89YFq1kWf4xgJOkoZ0I5A50IIohEPV0heEPqROS-LuTEcL4FoevxeImVGBzCka6SUvDvnPuOXtwNVup2r6WXQm2N7Qc3-iQlTXaPlIzRnTMTRKxeRiXJ4kfjhfojqOZD83Zlx6sp_Rgh0eBo7Ev0oSxNlGN8-Ts00wX5VYyu3ANLPtEV_EV93qH7VcvGWn4kS-NnIS3pBDWUyhwQhn5MnO-kA2fUlKa7j0M7dlJhuFDEyhb9dcpCTcKFZzwfNYbx9YQXZjjr_1l3SURDGNNNpUaj0GMQ2IlQUwtUwnmwoaGa5gPJpXJTTjyVvDfOyVIFIgdu89BtGc-NxsAdbexBqMByovG8HLy9E5J3bfeRZGRPsPUL9fBy4ryGWjoRHF4ZONt9i2LuIbRam-NEV0c0PCIYrU4j6iXhC5JAfo848jlsafM7nN-pkdUGTmaYwJKr3poe7RttyGOHFllSLOpNUq-TtVjgZpmt02zFeZotug2u87ISuFyVSZmlaVqXWSPKvMyXgmdilS_khsc8idfLNM6WK55FWMZlUiZ5JXK-TtOapTEehFQRTSKRse1COjfgJsuSZLlQokTlwqDGucYjhE3GOc1tdhOml3JoHUtjJZ13ZyteehUmvGkYud19NGeFfIU5K0xcl8OW67GSjcT6cuBaDFZt_vOEFQJwjBchwL8DAAD__4IgSzc">