<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/63895>63895</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            APFloat's `addOrSubtractSignificand` still throws assert in subtler FMA cases.
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          eddyb
      </td>
    </tr>
</table>

<pre>
    ```c
#include <math.h>

float foo() {
    return fmaf(
 -0.000000000000000000000000000000000000014728589,
 0.0000037105144,
 0.000000000000000000000000000000000000000000055
 );
}
```
results in (e.g. [via godbolt](https://godbolt.org/z/dE93eMobW)):
```
llvm/lib/Support/APFloat.cpp:1805:
 llvm::lostFraction llvm::detail::IEEEFloat::addOrSubtractSignificand(const llvm::detail::IEEEFloat&, bool):
    Assertion `!carry' failed.
```

However, that's not how the bug was found. I noticed 8-bit formats (like `Float8E5M2` and `Float8E4M3FN`) were added to `APFloat`, and decided to try brute-forcing all possible inputs for a few common operations (it only takes 8 seconds, and ~98% of that is FMA, because it uniquely has 3 inputs).

So the first example I found was for `Float8E4M3FN`, namely: `FMA(0.0254, 0.0781, -0.00195)` (the encoded byte values being `0x0d`, `0x1a`, `0x81`).

Then @solson helped me turn some formula sketches I made looking at the code, into Z3, which confirmed that the example I found was the only one across *all* possible 8-bit floats, but 16-bit and 32-bit have a lot more cases. (I'm only not including all that here because it's not really set up to generate ready to use examples, it's really finicky as-is)

---

My understanding is that https://github.com/llvm/llvm-project/commit/e62555c129d535d524354351791c5474c9929582 fixed most of the previously problematic cases, but not the ones where both significands are equal before subtraction, which will only work if `lost_fraction == lfExactlyZero`.

But if there is a lost fraction, neither direction will work to subtract equal significands - the code seems to rely on being able to assume that only equal exponents can result in equal significands, but we know that's not true w/ FMA.

(also, the lost fraction is always subtracted, regardless of the subtraction direction, but before https://github.com/llvm/llvm-project/commit/e62555c129d535d524354351791c5474c9929582 the source of the `lost_fraction` was tied to `reverse`, i.e. `a.subtractSignificand(b, lost_fraction != lfExactlyZero)` was always performed with `lost_fraction` coming from `lost_fraction = b.shiftSignificandRight(...);`, regardless of how `a` and `b` mapped to `*this` and `temp_rhs`, respectively - this seems significant as well, but I'm not sure how to tell or test this)
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJy8VkFv67gR_jXMZWBBoixZOvjgl8RoDmmLvgIF9rKgxJHFhhK1HCqOe-hvL4aSE2dfir5eNggMicPhzHzfN0MpInMaEfei-CaKhzs1h975PWp9ae4apy97UabLfyvSB5EehMzN2NpZI4j8flChT3qRP67G-NtZpwJ0zglZCVmD2H1bDAAAHsPsR-gG1bF5Wd-kSfozf9l2J6uiqoW8Xz1Xx3yXpUW23f5g-Lm_oli9hKxFvmYrdg_rwxWC5dUjzTYQmBGErDA5JSCKb69Gwcnpxtkgigchqz6EiUR-EPIo5HE1Jc6fhDz-S8ijfqxzfHbNPzgmhz18Gc3a10HIozWNkMfv8zQ5H4Q8Hv56ZJSTdppEfsiqtHg_AKJLfhD5wToKR6_aYNx4s6wxKGOX56fHx8d41PKqtP6L_z43gb2-m9NoOtOqUQtZtW6k8D9PkaWQ99A4Z2-LYuYPROhjJlyczFrl_UXIHXTKWNTJl-Uvv39yZ3xFzyeHnoPsCEYXoHdnCD1CM5_grAg6N486gSc2mhY1VJvGsBT9oAIxXda8IMePyVaPxbMUZQpq1DeL2-f8-OeYYw1n9AhKa9QQHO9ZgY_m--iosTWrPfgLNH4OuOmcb814AmUtTI7INBbBjNMcOEsPCjo8Q-uGwY3gJvSKkYkpmgButBcI6gUJKiBs3ajpGu7fdSVkAa6LUIAhOD4fIubYqpkQTIB5NL_NaC_QK4J8jStkndyC-t1F7DrjKQC-qWGyCE8Lhiua_ktU7mFUA9qLyA_RzuGrNEllwf3HnberMn6KbZ3VBSuhTLk2Dohj6xiv5hIQXpWdkaBBBkuUafqW6jVIfMvU7VuVLbR8quPvPY4gtik5S26EHu2EGgaEOGjIDRgFMFsF9IKh7ZHgCQalEaxzL5GlEKHgvDiWGYODX3J-PPem7aF1Y2f8wCT3696vAOP1SJ0bEVTrHTGhB2WtkIcPGayiZFgjq80cICvjIhOcy_jYq1cEBdYFGJxHaBUhJQzik5C7YQnETbCM46vYYoI9q_ZDD-_94lFZewHCAPPEgj3hyNJDtugLr7DHWltMbvVePTszmvblAoo2hgV1y8Nms7l9fb7APGr0FNQYszO0Jvd5MJrQz03Sujjl1mFnX4fN5N0_seVZx13CaRyxlEVRtJmsdZEXupDbvNjmRbars7bY7rZtXcu6qCR05o0l4CgsfYIweXw1biZ7gcm7xuKggmkXUK8cMEILhUhwXjB0oQf6GIMEyiPgb7Oy0GDHvNA6LI0bPwRzNtYuDJ2dfwHTsX55Gv_aXcexyB9E_gC2e3xTbbCXX9A7UaafpP1tDuwbYi6GohwoQHcTb0TDZtDG43JwjB3DBvee3Zrzp1I276IHQhyI93uM8l0bUrFcgwNFNHNDMX2xquU0fJvciGMgaNUIy73I1-KPsa4QnxFexji0b6Z48DPCWcgjT7JP9QtZKUtumfv4ufiIhz2rC70XiZp3ejwpry0SXcm_oegDp2tKK41_gCpjKm72LV4T-70oeErGQWLe7xvPFx_hOgVNggmvqoS-vKMb3vQ7ncnsR50tA5lDrRBO6HlIooazCf2XibVuYE103g1fqhmahHrT3Wb0N3Pqg5BVkiTrl9VSxmeK-Bbnmm5u4oafBzVN7zgIeQi9oZs9AYfpV9_T-5k0MbOvrGCWtqFV1h9CDKAIzsjzeCF_GaUsQppZBCxNBwG5ez0EpBAPErK-0_tc13mt7nCflVXNNOfZXb_XVam3KtdNqRrd6W2eV1jku3xXt3WLeXpn9jKVebrLyqzMqiJLmqJSCtt2W9W5Ksut2KY4KGMTVhh_Id4Zohn3ZV7VxZ1VDVq6fqD7fZRhM59IbFNrKNCHWzDB4v76hcLtxbD-ty-6MgUKJl4Z3p2Jmxx97F_WlkXP7bheO3ezt_v_u0ViGSTkMVbynwAAAP__v13qew">