[PATCH] D31074: Fix constant folding of fp2int to large integers

[Simon Pilgrim via Phabricator via llvm-commits]

RKSimon created this revision.

We make the assumption in most of our constant folding code that a fp2int will target an integer of 128-bits or less, calling the APFloat::convertToInteger with only uint64_t[2] of raw bits for the result.

Fuzz testing (PR24662) showed that we don't handle other cases at all, resulting in stack overflows and all sorts of crashes.

This patch uses the APSInt version of APFloat::convertToInteger instead to better handle such cases.

What do people think of making the APFloat::convertToInteger(uint64_t*) version non-public to avoid this issue in future?


Repository:
  rL LLVM

https://reviews.llvm.org/D31074

Files:
  lib/CodeGen/SelectionDAG/FastISel.cpp
  lib/CodeGen/SelectionDAG/SelectionDAG.cpp
  lib/IR/ConstantFold.cpp
  test/CodeGen/Generic/pr24662.ll

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D31074.92130.patch
Type: text/x-patch
Size: 4599 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170317/1c784653/attachment.bin>