<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/66460>66460</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            [fuzzer] A thread-safety problem with RSS thread revealed by flaky failure of symbolize-deadlock.test
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            new issue
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          yingcong-wu
      </td>
    </tr>
</table>

<pre>
    We got flaky failure of symbolize-deadlock.test, investigation revealed there is a thread-safety problem with RSS thread.

So fuzzer has a dedicated thread to monitor the RSS usage, and when OOM, it will call `RssLimitCallback()`.
```cpp
static void RssThread(Fuzzer *F, size_t RssLimitMb) {
  while (true) {
    SleepSeconds(1);
    size_t Peak = GetPeakRSSMb();
    if (Peak > RssLimitMb)
 F->RssLimitCallback();
  }
}
```

And `RssLimitCallback()` will call `DumpCurrentUnit()`, which calls `PrintMutationSequence()`
```cpp
void MutationDispatcher::PrintMutationSequence(bool Verbose) {
  Printf("MS: %zd ", CurrentMutatorSequence.size());
  size_t EntriesToPrint =
 Verbose ? CurrentMutatorSequence.size()
              : std::min(kMaxMutationsToPrint, CurrentMutatorSequence.size());
  for (size_t i = 0; i < EntriesToPrint; i++)
    Printf("%s-", CurrentMutatorSequence[i].Name);
  if (!CurrentDictionaryEntrySequence.empty()) {
    Printf(" DE: ");
 EntriesToPrint = Verbose ? CurrentDictionaryEntrySequence.size()
 : std::min(kMaxMutationsToPrint,
 CurrentDictionaryEntrySequence.size());
    for (size_t i = 0; i < EntriesToPrint; i++) {
      Printf("\"");
 PrintASCII(CurrentDictionaryEntrySequence[i]->GetW(), "\"-");
 }
  }
}
```

But when RSS thread is printing out `CurrentMutatorSequence`, the main fuzzer thread is still able to change the `CurrentMutatorSequence`, which can lead to buffer overflow read of `CurrentMutatorSequence`. In our case, with `-D_GLIBCXX_ASSERTIONS`, an assertion error within the vector header. 
```
stl_vector.h:797: reference std::vector<fuzzer::MutationDispatcher::Mutator>::operator[](size_type) [_Tp = fuzzer::MutationDispatcher::Mutator, _Alloc = std::allocator<fuzzer::MutationDispatcher::Mutator>]: Assertion '__builtin_expect(__n < this->size(), true)' failed.
error: Aborted (core dumped)
```
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJykVlFz2jgQ_jXiZQfGkbGBBx4Ah05mmqYT5659Y2R7jXWRJZ8kh5JffyPZQKBNjutlmITIq2-_3W-9u8wYvpWIcxItSZQMWGsrped7Lre5ktvhrh1kqtjPvyFslYVSsOc9lIyLViOoEsy-zpTgrzgskBVC5c8ji8YSugIuX9BYvmWWKwkaX5AJLMBWqBG4AQa20siKoWEl2j00WmUCa9hxW8FjmvaPRyRISLDofqcKyvb1FTVUzCEUWPCcWQ_rjMEqqJXkVmnnyMO0hm3REWKygF2FEh4e7j1BCzsuBORMCCBx8GjMZ15zu2JCZCx_JnRK6IzEwYFCHHSfvGm6E2OZ5Tm8KF7AozFPngOh03XHkdDF2jky_BU3Fg749xmhMyCTZQcCsKu4QCB0anWLF88AUoHYpJgrWRhCpzeOU_jmeY_-FdkzkDCBT2jd98c0dY6ml-a8dJ5669tzUr3VekjC23eycYIik6TPy_HLIUFvJVvI4sPknmuQtHWzarVGaf-Q3B6tXBp3Fc8rb2mc6VfNpb1vra-vFP9uUeZ4uvCOZF6rw62Em4bZvEJNwgUJF-9BZkoJ-BN1psylPv5K6d3S-5SECyA0ei2AUOo497F4TKUPkCOnWU_1LKe9lrfSao7mSXl0p2pv0HMAEq6vgD5qfvpxBI0tunBrLgmdPt-zH4eYDy5_g3qpXMFP-wi4L8WAhEv_dXURkj8ndOk_b4i-zSahkRl-mEYSLTmJktEXVuM5ma7ICb3pLyY8d-ExvXc89sdgsG7s_hjN-Xv3lgskt5209MzPzzr9SqH3nP-k1PXi9Beud3DRBP6HWOdJupAsWvkUnWfJWyzS1d0dodOPKfeKuv7zCe23A_kVHMGHl-jH7nN1R1q2tpsDpyHj5lHjaHK5BdVa11_eqbmuF7nhUjMuD-PoBGOs62csE-iGUV4xuUVv_m-Qh_YmQfSTLGvLEjWoF9SlUDvwLlT5IdII7iSoVkPOjJ96fpySOBgmm0-f75ar7983izS9fXy6e_iS9q6ZBGYMaj-pUWul_TUuPfEXzN08rZAVqEfwy8waKzad3agi4WIym7hi1liidsROZd0ZkXDVJa47fK8f9-GR8Lb7XzWo_YFfVo4FvG-6rhwtN0-Nr-X_Ak5XsFkIoXJ_80iUuSP2W1yjxAW_OGaU0Mlmk7VcWC43-KPB3A22zUb6l81W3LiCf_u2rqBfBQid-HULD3uQV8fDZ0q7xYfQaa40QtHWDRbHZnLUZ1DMw2IWztgA5zfxbDyJ4tk4GlTz2aQsoyyf3YyLrAgmUUzjKIjjiAZ5HNNZOOBzGtAwmN1EQTSeBdGoQIwpK-g0yqcTZAEZB1gzLkZCvNQjpbcDbkyL8zgex8FAsAyF8YslpRJ34B_6FzkZ6Lm7M8zarSHjQHBjzQnFciv8RtrnPUpgcd2yeFozs_3Vy-qg1WJeWdsYpyRdE7reclu12ShXNaFrx6v_M2y0-svLt_bRGELXPtp_AgAA__9hcH2l">