<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/121176>121176</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            Incorrect Optimization of malloc Calls: Erroneous Assumption of Unconditional Success of malloc
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            new issue
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          jonathan-gruber-jg
      </td>
    </tr>
</table>

<pre>
    Given calls to `malloc` that can potentially fail, Clang incorrectly assumes they will unconditionally succeed. In some cases, Clang even elides such calls to `malloc` and optimizes the surrounding code around the assumption that the call(s) to `malloc` would unconditionally succeed if actually executed.

```c
#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>

#define TOO_LARGE_SIZE ((size_t)PTRDIFF_MAX + 1) /* Request size greater than PTRDIFF_MAX. */

int test_too_big_malloc() {
        int ret;

        void *const p = malloc(TOO_LARGE_SIZE);
        ret = p != NULL;
        free(p);

        return ret;
}
```

On all the (nonzero) optimization levels that I tested, Clang generates x86_64 assembly (Intel syntax) similar to the following, where it not only completely elides the calls to `malloc` and `free` but assumes that the call to `malloc` would have succeeded if actually executed:

```asm
test_too_big_malloc:
        mov     eax, 1
        ret
```

glibc's `malloc` _unconditionally_ fails for requests of sizes greater than `PTRDIFF_MAX`, so Clang compiles the above test case incorrectly for glibc in particular.

Host system: Arch Linux, x86_64.

Clang version: official Arch Linux package of clang, version 18.1.8-5.

Command line to reproduce results: clang -c <file>.c -O<opt-level>, where <opt-level> is g, 1, 2, 3, s, or z.
</pre>
<img width="1" height="1" alt="" src="http://email.email.llvm.org/o/eJx0VU1v4zgP_jXKhYhhy6kTH3xI02beAH23i_kAFnsJZJm2NStLXklOJ_31Cypum85MAcOJRfLh10NKeK86g1ixm1t2c7cQU-itq75bI0IvzLJzU41u-b1b1LY5V5_UCQ1IobWHYIEV6SC0tpIVKYReBJDCwGgDmqCE1mdohdKM72CnhelAGWmdQxn0GYT304AeQo9neFJaw2SkNY0Kyppo6ycpEZsEDga8HRCk8Ojf0JBiQa0a9KTbfxCXMA3YMahBPV_cgZ-cs5NplOlA2gZBxM8oi2GNFMMlITojXMY3nvHyF_QnO-nmo9BBtSBkmOIZ_kA5BWwSlm7pKdLLI-mD58pIPTUILN_50DTYJj3L738nUyZ8KNOqfpVFcYOtMghfHx-PD9vPn-6PXw5_3wPjG0pIPeMxMF7--fXz3WG_P_5_-xcwfgsZZcr4nvEtfMZ_J_QBSBk6hyKgo9IYuLJKgPEtGUS3ygQI6MMxWHusVXecy0VOS2Dr26hWkprDwPLbOdq0PFnVEJS0xgcYgeV38Gr8PgfGyxfL0mGIqiMwntGfP749PLxKW4fI-Ga8spiNJmeuIljfXbflovdoQGgdWcD4xljzjM5SFjOjRGSKxhMS84gwh5g6Nm887dCgEwE9_NgUx2JFHMOh1meCPJiAGvzZBPGDcL0alBaOeEZOW6u1fVKmI7inHh2CCmBsAGv0GaQdRo0BiV6XQXjh6-_ngBVprEaRQj2Fqxm8YvoHFO_FCV94_QGzWb79idvCDyzd_o4Ls2o52BNLS6Tkd5C9dubXTnRa1ZLxtX8f3PGn0TvGleOhtQ7chboebBvp69_zlxXpFYXJEd-Bt3PXqLRKzxUVtT1hbGxcQu8WGXmKwYEyMAoXlJy0cPOY_8_S7Jx9wIHlW9g62cODMlPM98KHWfPi9oTOK2tI17atkkroKyMYhfxHdEgZSdInlNkEsk2SJZvlzQueHQbquab5DxYcjs42k0Rw6CcdPPmIILCUtD5apZHl94mE5SPLd3YMy0hsWiev7PtJAMpDd2kd3wGnVx7LSC_r4DlZNFXelHkpFlhl6_yGF5tVsV70VZ5lZdYUWXmzaXmRljKrM2xXWVvKDV9n-UJVPOWrjPN1mqdZuk5QlLms07LNcN3KTc5WKQ5C6UTr05BY1y2U9xNWGc-ydbHQokbt483GucEniFLGOV10riKjZT11nq1SrXzwbzBBBY3V4aXJ8Hg97LadVxLsaM6oivfOWYN28rB9uz5sC9-uuQlfaHi8fwNYTE5XfQgjgcR1u-9U6Kc6kXZgfE_xzD_L0dnvKAPj-5iFZ3w_p3mq-H8BAAD__-oagzY">