<table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Issue</th>
<td>
<a href=https://github.com/llvm/llvm-project/issues/121219>121219</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>
Optimization of malloc/free Produces Undefined Behavior
</td>
</tr>
<tr>
<th>Labels</th>
<td>
new issue
</td>
</tr>
<tr>
<th>Assignees</th>
<td>
</td>
</tr>
<tr>
<th>Reporter</th>
<td>
jonathan-gruber-jg
</td>
</tr>
</table>
<pre>
Optimization of `malloc` and `free` calls can produce undefined behavior.
Consider this minimal test case:
```c
#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>
#define TOO_LARGE_SIZE ((size_t)PTRDIFF_MAX + 1) /* Request size greater than PTRDIFF_MAX. */
ptrdiff_t test(void) {
char *const p = malloc(TOO_LARGE_SIZE);
if (!p) {
return 0;
}
char *const q = p + TOO_LARGE_SIZE;
const ptrdiff_t ret = q - p;
free(p);
return ret;
}
```
Critically, glibc's `malloc` unconditionally fails for requests larger than `PTRDIFF_MAX`, so, with glibc, the above function `test` would always return `0`. However, on all nonzero optimization levels, Clang assumes that the call to `malloc` will unconditionally succeed, effectively deleting the `if`-statement. As a result, the function invokes undefined behavior by computing a pointer subtraction whose result overflows (since `TOO_LARGE_SIZE` > `PTRDIFF_MAX`). If the `if`-statement were not deleted, the body of the `if`-statement would always be executed, thereby avoiding invoking undefined behavior.
On all nonzero optimization levels, Clang generates (x86_64, Intel syntax) assembly similar to the following:
```asm
test:
movabs rax, -9223372036854775808
ret
```
Host system type: Arch Linux, x86_64 (with glibc, of course).
Clang version: official Arch Linux package clang 18.1.8-5.
</pre>
<img width="1" height="1" alt="" src="http://email.email.llvm.org/o/eJyUVU1v2zgQ_TX0ZWCBpmxZOvjgOPE2QBcpul1gsZeAooYSW4pUSUqu--sXpJXUSZPDAoZtab7e48wbcu9VaxB3ZHNDNrcLPobOut1Xa3jouFm2bqzRLb-2i9o2593DEFSvfvKgrAErgRS051pbQQoK3DTxhXSI8VFwrT0IbmBwthkFwmgalMpgAzV2fFLWZYTuCd0frPGqQQehUx56ZVTPNQT0AQT3SPLoRAp6-Yj4wHJlhB4bBJIffGgalFlH8ru3bMqEd21a1c-2ZL5AhC8PD48f95__uHv86_7fOyCsJKz06ic-BsKqT18-394fj49_7v8Bwm5gRVgFhB0J28Nn_D5G5NEZWoc8JGLcwFVUBoTtY0AqOwTXKCkfQ-JMWDlZ1aSU25vkUYmOuxgirPEBBiD5Lcwnz8qXWAmrSH4z86GVkhfwq-FFQkIrh2F0BuiTd0W2t89hLwt-TwWHxPVVtatSM7hnMg5DivsOSxiuHdOIsHJ4BXUG5DDMby94nvo-z4pTQcXZOhN2gFarWhC29S9HcTTCmkbFMY2eILnSHqR14C7d8aC5a58aQwp61ZtYih3A2_h9UqF7qnKA0CHw2k4IcjQiqYAUNDWtoHCyo26A6xM_e5jJkIJG7Bl8sCec0MUs1gDXGow1P9FZsNei0jih9tHroLlpgXs_9ugjzpDKR-oQ7Eu-J6X1b6T9KARiE3OhlCiCmlCfoUGNQZk2ZSMFVZIUdOkDD9ijCRnsPXBw6Ecdnjg_s1Vmst_QvyFlqM8gbD-MKTeHwSoTJ9-PdXD8En3qrMc5NdgJndT25CEpy4iE5tV0FRRIfvdWg6oM7uU7HOCEDsHYcCF7OYPoGpdY3FvvhV03sEbAHyjGX-EO6zPwqM1IMR1F_PPuWnv4H31u0aDjAdNp_CiLx2IdjfcmoAZ_NoH_iPrl3mNfx-aqXmnu4iSkBlmt7UmZ9tWu5L4ndJ8GNJ9V1tuJ1z7KLaY8wLJiLM-3jOZFuVlvt5uSls96_F1_H2zcbWcfsIdwHuJyhr0THXxUZkwJL-gjj5fisRKEHZ3H2Lyo5ER8QueVNTGNlVIJxfVVPhi4-MZbBJGcV2W2ysrlJls0u7yp8oovcLfa5puc5dtqu-h2slwzRJlXdL0tpShYuZF5XuW1lBu2lfVC7Rhl6xVj21XJ8lWR1XRds5xxRmOAXJE1xZ4rnWk99Zl17UJ5P-JuxVZsVS00r1H7dF8yZvAEyUoYi9en28WgZT22nqypVj74X2mCChp_u0KftvgxLkX4dLkqPfz9PFQ381AtRqd3XQiDj62Md82xVaEb60zYnrBjrDP_LAdnv6IIhB0TOk_YcYY_7dh_AQAA__8AsoaC">