<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/59555>59555</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            Nan boxing misoptimization: missing icmp combine?
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            new issue
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          deadalnix
      </td>
    </tr>
</table>

<pre>
    This code is extracted from a popular javascript engine, but has been reworked to demonstrate the essence of the problem:

```cpp
bool isInteger(uint64_t payload) {
    return (payload & 0xffff000000000000) == 0x0001000000000000;
}

bool isNumber(uint64_t payload) {
    return (payload & 0xffff000000000000) != 0;
}

bool isFloat(uint64_t payload) {
    return isNumber(payload) && !isInteger(payload);
}

bool isFloatExpected(uint64_t payload) {
    return payload >= 0x0002000000000000;
}
```

While `isFloat` and `isFloatExpected` have 100% the same semantic, LLVM greatly complicates `isFloat` to the point it turns into 11 instruction instead of 2 on x64. The crux of the problem seems to be what the get optimized into at the IR level:

```llvm
define i1 @isFloat(i64 %0) {
  %2 = icmp ugt i64 %0, 281474976710655
  %3 = and i64 %0, -281474976710656
  %4 = icmp ne i64 %3, 281474976710656
  %5 = and i1 %2, %4
  ret i1 %5
}

define i1 @isFloatExpected(i64 %0) {
  %2 = icmp ugt i64 %0, 562949953421311
  ret i1 %2
}
```

It is worth noting that any manual inlining of `isInteger` and `isNumber` into `isFloat` produces the expected result, so LLVM confuses itself when optimizing `isInteger` and `isNumber` and is unable to recover after inlining.

This is worth handling properly, as NaN boxing is a very common techniques used in dynamic language's interpreters, so this is probably worth having a look at.
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJy0VVuP2zYT_TX0yyAGSV28evBDko2BAPn24UPQPhYUNZKYUKRKjrx2f31ByRdlkyKbAjUMXzgznMOjc4YqRtM5xD0r3rHicaMm6n3YN6gaZZ05bWrfnPefexNB-wbBRMATBaUJG2iDH0DB6MfJqgBf1FFFHcxIgK4zDpl8D_VE0KsINaKDgM8-fMUGyEODg3eRgiIE6hEwRnQawbfz3zH42uLAsreMPzJ-_Sz58tbjuKzU3lsw8aMj7DAw-TAZR2X-B8GoztarhskK2O7dkg0AEJCm4IDJh0sGMFkCP7Vt2_LVay7MHln2CPzEORffBLPLjmz3uAZ4gfM0DfV_gEaKGc1Pmh-sV_QLvVdw15myTEiYFGt27wmvAfHhNGJSyi-AubGQfbgxL3_C_FUUaxi_98YisJJf-Sg5KNesVm7gSg69OiKIxHIxqy-qASHioBwZnWT86dNv_4MuoCJ7Bu2H0RqtCOOLDuQX8XrjCAxBOlIE48iDEGCS3idNxrv5N6omyV2Cd3Aq8y187hF0mE4vTAARcYhp8xrhuVc0Bzsk8COZwfyFzdLjEvn4f7B4RPtP5rH2OCxLDbbGIRgBLOd35ZgyByYL_uIhMVnIZAkwehhh6gjuie9BPoh8l1e7cid4WRSromwuSuyv8998W1CuCvJ7l4RuKcq-b7KuKe5NxIw0pae9rjkB6RIqfijcH3GxEvC_4qQoZZVXVZHlUmRCfA9FvkbLHynN3WcfqAfnybgOKKlAuTMMyk3KgnHWuBTw7SLJq2XXsr_YvOSLWr6V7hh8M2mMyzC-nBsCxslSOkr0iwm0d-0UMYKhiLaF5x7dVYcJwCu6zw8pwuRUbTHJOqD2RwygWsJwO8t2zcF8Bd1Y6JVrbOo2Bj9isOeEUEV4Uk9Q-1OKmAgKjhhmuw7eAaHunflzwghTnB0DzdmpwWiwynWT6pDJ3exWDGNAwhAvB6dL8-RHVdvzDcUxdVJgvf8KirabZp81VVapDe5FuRNScFnxTb_Pmx2KFnUl6qrO6gfEQissd5nIasUF35i95FIKKUqx43nBtw1_qHQu9K6UJa-aguUcB2XsNpl360O3MTFOuC-qoig2VtVo43yJS-nwGeYgkzLd6WGfat7UUxdZzq2JFO-7kCGL-yflrrwNJl6epkqDimVv01KcKU0S136o092eHTZTsPueaIxp0MgDk4fOUD_VW-0HJg_zmFm-3ozBf0FNTB5mZJHJw4z87wAAAP__NzuODQ">