[llvm] r271380 - Fix off-by-one error in max integer functions

[Dylan McKay via llvm-commits]

I will be writing the code that calls this. The method bodies are extracted
verbatim from isIntN, which is used exclusively in in-tree MC/backend code.
This would’ve silently overflowed in any of the 34 occurrences of this
function in-tree.

Probably best to return the correct result for N==64 and then (if
appropriate) error out for N<1 and N>64

I agree, I’ll add this in. Let’s hope it doesn’t bring out any latent bugs.
​

On Thu, Jun 2, 2016 at 10:36 PM, John Regehr <regehr at cs.utah.edu> wrote:

> I'm not sure in what circumstances these functions are used.  It is often
> very bad manners to throw assertions from library code.  But if you are
> writing the code that calls this code, then an assertion definitely sounds
> appropriate.
>
> Probably best to return the correct result for N==64 and then (if
> appropriate) error out for N<1 and N>64.
>
> John
>
>
> On 6/2/16 12:32 PM, Dylan McKay wrote:
>
>> How do you think this should be handled? Assertion error?
>>
>> On Thu, Jun 2, 2016 at 10:10 PM, John Regehr <regehr at cs.utah.edu
>> <mailto:regehr at cs.utah.edu>> wrote:
>>
>>          /// Gets the maximum value for a N-bit unsigned integer.
>>         -inline uint64_t maxUIntN(uint64_t N) { return UINT64_C(1) << N; }
>>         +inline uint64_t maxUIntN(uint64_t N) { return (UINT64_C(1) <<
>>         N) - 1; }
>>
>>
>>     This is still wrong unless N is guaranteed to be no greater than 63.
>>
>>     John
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160602/268c0fcc/attachment.html>