<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/61685>61685</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            DLL load order of ASan causes false reports on `realloc` calls during DLL initialization
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            bug,
            clang:driver,
            compiler-rt:asan,
            platform:windows
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
            alvinhochun
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          alvinhochun
      </td>
    </tr>
</table>

<pre>
    Short version: If the program tries to `realloc` a block of memory that had been `malloc`-ed before the ASan runtime was initialized, ASan reports an error for "attempting free on address which was not malloc()-ed". Making ASan load earlier can fix this.

Long version:

When building an EXE with ASan enabled, even if it depends on other user library DLLs which have not been rebuilt with ASan, it should still work fine because ASan intercepts allocation functions to capture allocations across all DLLs. However, ASan misses any allocations made before `libclang_rt.asan_dynamic-x86_64.dll` is loaded.

https://reviews.llvm.org/D25946 implemented a change to ignore such allocations made by ucrt when deallocating. In practice, this also ignores any allocations made by other user DLLs. This generally includes C++ dynamic initialization (e.g. global static variables). (This slightly reduces ASan coverage, but at least the program still runs and ASan is able to instrument the rest of the program, which to me is acceptable.) *However*, this only happens for `Deallocate`; `Reallocate` does not contain the same checks. If an uninstrumented DLL allocates a pointer during initialization and this pointer is later reallocated, ASan will report "attempting free on address which was not malloc()-ed".

### Reproducer

[asan_load_test.zip](https://github.com/llvm/llvm-project/files/11062837/asan_load_test.zip)

I tested this with llvm-mingw-20230320-ucrt-x86_64 from https://github.com/mstorsjo/llvm-mingw/releases/tag/20230320 .

To build it, change `Makefile` to point CXX to clang++, then run `mingw32-make` to build. Set `ASAN_OPTIONS=verbosity=1` (optional) and run main.exe to test.

### Proposed Fix 1: Reordering link flags from the Clang driver**

I realized that it is possible to influence the Windows library loader to load the ASan DLL earlier than most other user DLLs, by making it the first entry in the Import Directory Table, earlier than other DLLs. When linking with LLD, the order in which the import libraries are specified affects how the Import Directory Table is arranged. This can be verified with the reproducer with these steps:

1. Run the command to build `main.exe` manually with `--verbose`
2. Run the printed build command manually
3. Edit the link command (ld.lld) to move `libclang_rt.asan_dynamic-x86_64.dll.a` before other input files, then run it manually
4. Run `llvm-readobj --coff-imports main.exe` to verify that the ASan dll is listed first
5. Run `main.exe` should no longer trigger an ASan report.

For the MinGW Clang driver, these link flags are added in:
https://github.com/llvm/llvm-project/blob/8b513407407d00dd8eafa1f6c6f37bf715b4c7e8/clang/lib/Driver/ToolChains/MinGW.cpp#L295-L311
I presume the solution is as simple as moving this block of code further up. (MSVC might also need the same fix, but I haven't checked.)

Downside: This only works when linking through Clang. If LLD is invoked directly, the user needs to be aware of the ordering of the ASan link flags.

### Proposed Fix 2: Teach LLD to always put the ASan DLL in first**

This fixes the issue for both linking through Clang or by the user invoking LLD directly. However, this introduces a special case so I don't really think this is a good idea.

### Proposed Fix 3: Teach ASan `Reallocate` to not report on uninstrumented allocations**

We can probably apply the same thing in https://reviews.llvm.org/D25946 to `Reallocate` and not bother with trying to change the DLL load order. However, this reduces ASan coverage so I think this should be avoided if possible.

---

cc @mstorsjo
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJykWFFv2zgS_jXMy8CGTNmO_ZCHNF7fBUh3F01w7VtBkSOJDUUKJGXH--sPQ0qO3fYWXRxQNLYkDofffPPNJ4sQdGMR79jqA-NcmIO2rZPtYBnnbLW7EUNsnb-7uHFTOXW6e26dj3BAH7SzrLyHxxpii9B713jRQfQaA0QHbF14FMY4ydYFCKiMk6_gauiwc_4EsRURWqGgQrT0dDc9PEO6WDuPKfL9s7DgBxt1h3AUAbTVUQuj_0LF-MN4H3vnYwBhAb13HmrngQ4WI3Z91LaB2iOCsyCU8hgCHFst2xTQugjj7nzD-HZGgfkcPopXWpg2ME4oQOGNRg9SWKj1G8RWhzkrdqy4z_8_OdtcgHN563OLFqpBG0UxhYXfvvwGRx3bHB-tqEw-EB7Qgq5BR1DYo1WB0naxRQ9DQA9GV174E-yenqZjtOKA6RwJTY-0UXwPT2F1hNC6wSgIURsDR-dfodYWoUIphjAirW1EL7EnMAkSEbWzUA9W0odUWin6OHi8uB9ASO9CWpLSmsO_3REP6M8V6nQISAU6Xa3rhMKp2mxdGF1JI2zz1ce5CMJ-VScrOi1nb5v11_VyrowhOumQCoLqCv02xj4Q7HzP-N7jQeMxzI05dHPnG8b3O77aLtegu95ghzaiAgGyFbZBOpduLKURBtn-JMcTDNJHOFIdFU73bTOHRwu9FzJqiXRcYgUIE6aA_-vQp8uaZtBeaGmDFr0w5gTaSjMoDPDA-AfGP8CIxnsL5OowvsF5M4fGuEoYCFFELeEgvCZSBca3c3omRQ9GN200J_CoBokhV0e6A3rRpPyrIYKIYFCEeNXbmTd-oHJbNdIlAO2R4LMh-oFwTas8hkj9fhGBome-RgcdpsWSuEYh5oxvgfH7M3Huz2A6a07Qir5HG3Jnr4vdVAJk64KVH-jap8troBzm3pbORqFtyiSIDkG2KF_DnLRLWBjse-aoqBJTsah00LvUEqAGT537HfQERMpxeoyoKeiDPyfzLlPHBGDSqv9XnS6Jz3iZ_8En7L2juvqr-6sPqZmoZ75GDHH-l-7Zasf45rpnGh3boZpL1zG-p8YZ_8x6776hjIzva50YtV8sijXflLeM738Sm28v938EuoEjUkmVUtRO2-Y44wUvi5IXM-qvsdGh9q6Dv0muC9H58M1NCaZQqeuJtynDKKjnp-hwhdiLy1oMOlJxRg1g6-KjeEU6IhEoulxVePjyJekeKVPuxExNTIMpDS_avuSzTrxOS1P8OTxjpAfun-9___rHny-Pf_z-zMrdAX3lgo4nVu4WtIDxjeuJUcJQHxCtKHQntJ3jW-qvBO7Py_6nd70LqGCv32BBY_kTOq8wMdZo-wq1EU3IqFIbPNBRQHk9ddr9db2IvDRg85jWERLDQ9DnXq_NgFbmEf1ZW-WO4TyYkjZ7ei6NzfMUp9aaZmhsaSg40ohrFUwSdIIuD1-dxaTWPkRAGz2pYrr02KU22mmPMpKjeCEVSQP0coscPctrmsEEB0VONHx62o2lhIQXBR8VqkUaFLRFPhb5GkHjoUepa02zo65RxgCtO_5NRknlvCeCqVHhyT9USEYhB0qpZM2c2vd8LSCEiLkNLkq0mMOnIQMhXdclGZo4ncxU5g1RqxN2SOMkhSSDNcv0S8qZovH3aL3XSQZzqCn2FCM_Xs7hNzVWJrFreozxjVFzYxRxmDTeHX51rM8F5To6gVw0bfshwig4F_2m43f5LHP6tBFpgUehXPUNZjPp6nqWqxjgEpPoMvqjDz0zVBmTFFwnuUqsy1uszltchhkNlSWi24Yo53VDf4W99KVXXbt3Pu33Udt_ff6uER_Gil-0LFFOKIUK9Lul_MeqXRlXMb7fVKtFuSxul8WtKgqlNihqsajXcl2Xt1V9u1hVS3mLG8b3o9jtjaaFuzHB_Ytz5qEV2pLCpiPMZd8zXj7x7Wr2VC4Wk4T0HsPQZYEIzgxpXFIvBAjJgdGnzh2oF9NcOL8hSKcQ6sFnXeiTd_n4_J8H6Mi6ZGtlEdX7QK_12-RcHpMXtozfxjznySRej6OdO9qgFZJOvpz9BfnhkO3dJBGx9W5o2lykZBeennZ0Bm0P7hUVqNTq5jRpSBIxSi355ApBHKl-ows6K_L4Pb9YnEv9K9rOU84oZJIu2kSYozgFoE650lltR_r-oO7pyLV-o_c0ErkQBkyuqnI0mH92dqC7p_cjpvPTU5TEBMKV508V1TZmOSMflXRTGJCCJM3BIyiXy5ScEkUnKPJCWtA4p0ArFL8CTPkOTILgBzcYXXJSo_lyP9i-C3_-I2SfMWl2710lKnMC0ffm9E4_ypycIfzqG0h-P75OkPQzvcFl8cv670-pFu78ltJiqm6aq4lPP6L-U2OfEb-AeJQu4ujB6aQv9XnEXyE-m80uv0oJbFlM_utG3ZVqW27FDd4t1rfb9Xpzuyxu2jsu6xVuhFrhWmG5Vostx6JQArcFR1GXN_putGbLxbLcFsVccHUrcV0ulSpqWS_ZssBOaHPG7yZR9W69WG9WN0ZUaML4-0U1NIxzxh8Y51m4yvtJU6fLruu1QT_zkZX3NITOt3ojYu18x8r7Y3Yy448g_i7JaDU0gS0LGgrvxbyJOhq8uy4GdXaGnV6oA9TCBDz_OOHsd7-KSGFMmF4sctNevlzcDN7c_WOpTyCROiec_hsAAP__ZCACjg">