[llvm] r258325 - [LibCallSimplifier] don't get fooled by a fake sqrt()

[Sanjay Patel via llvm-commits]

Author: spatel
Date: Wed Jan 20 11:41:14 2016
New Revision: 258325

URL: http://llvm.org/viewvc/llvm-project?rev=258325&view=rev
Log:
[LibCallSimplifier] don't get fooled by a fake sqrt()

The test case will crash without this patch because the subsequent call to
hasUnsafeAlgebra() assumes that the call instruction is an FPMathOperator
(ie, returns an FP type).

This part of the function signature check was omitted for the sqrt() case, 
but seems to be in place for all other transforms.

Before:
http://reviews.llvm.org/rL257400
...we would have needlessly continued execution in optimizeSqrt(), but the
bug was harmless because we'd eventually fail some other check and return
without damage.

This should fix:
https://llvm.org/bugs/show_bug.cgi?id=26211

Differential Revision: http://reviews.llvm.org/D16198


Modified:
    llvm/trunk/lib/Transforms/Utils/SimplifyLibCalls.cpp
    llvm/trunk/test/Transforms/InstCombine/cos-2.ll

Modified: llvm/trunk/lib/Transforms/Utils/SimplifyLibCalls.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Utils/SimplifyLibCalls.cpp?rev=258325&r1=258324&r2=258325&view=diff
==============================================================================
--- llvm/trunk/lib/Transforms/Utils/SimplifyLibCalls.cpp (original)
+++ llvm/trunk/lib/Transforms/Utils/SimplifyLibCalls.cpp Wed Jan 20 11:41:14 2016
@@ -1376,12 +1376,21 @@ Value *LibCallSimplifier::optimizeLog(Ca
 
 Value *LibCallSimplifier::optimizeSqrt(CallInst *CI, IRBuilder<> &B) {
   Function *Callee = CI->getCalledFunction();
-  
+
   Value *Ret = nullptr;
   if (TLI->has(LibFunc::sqrtf) && (Callee->getName() == "sqrt" ||
                                    Callee->getIntrinsicID() == Intrinsic::sqrt))
     Ret = optimizeUnaryDoubleFP(CI, B, true);
 
+  // FIXME: Refactor - this check is repeated all over this file and even in the
+  // preceding call to shrink double -> float.
+
+  // Make sure this has 1 argument of FP type, which matches the result type.
+  FunctionType *FT = Callee->getFunctionType();
+  if (FT->getNumParams() != 1 || FT->getReturnType() != FT->getParamType(0) ||
+      !FT->getParamType(0)->isFloatingPointTy())
+    return Ret;
+
   if (!CI->hasUnsafeAlgebra())
     return Ret;
 

Modified: llvm/trunk/test/Transforms/InstCombine/cos-2.ll
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/InstCombine/cos-2.ll?rev=258325&r1=258324&r2=258325&view=diff
==============================================================================
--- llvm/trunk/test/Transforms/InstCombine/cos-2.ll (original)
+++ llvm/trunk/test/Transforms/InstCombine/cos-2.ll Wed Jan 20 11:41:14 2016
@@ -1,12 +1,11 @@
-; Test that the cos library call simplifier works correctly.
-;
 ; RUN: opt < %s -instcombine -S | FileCheck %s
 
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
 
 declare float @cos(double)
+declare signext i8 @sqrt(...)
 
-; Check that cos functions with the wrong prototype aren't simplified.
+; Check that functions with the wrong prototype aren't simplified.
 
 define float @test_no_simplify1(double %d) {
 ; CHECK-LABEL: @test_no_simplify1(
@@ -15,3 +14,14 @@ define float @test_no_simplify1(double %
 ; CHECK: call float @cos(double %neg)
   ret float %cos
 }
+
+
+define i8 @bogus_sqrt() {
+  %fake_sqrt = call signext i8 (...) @sqrt()
+  ret i8 %fake_sqrt
+
+; CHECK-LABEL: bogus_sqrt(
+; CHECK-NEXT:  %fake_sqrt = call signext i8 (...) @sqrt()
+; CHECK-NEXT:  ret i8 %fake_sqrt
+}
+