[llvm] r333557 - [ValueTracking] Fix endless recursion in isKnownNonZero()

[Karl-Johan Karlsson via llvm-commits]

Author: karka
Date: Wed May 30 08:56:46 2018
New Revision: 333557

URL: http://llvm.org/viewvc/llvm-project?rev=333557&view=rev
Log:
[ValueTracking] Fix endless recursion in isKnownNonZero()

Summary:
The isKnownNonZero() function have checks that abort the recursion when
it reaches the specified max depth. However one of the recursive calls
was placed before the max depth check was done, resulting in a endless
recursion that eventually triggered a segmentation fault.

Fixed the problem by moving the max depth check above the first
recursive call.

Reviewers: Prazek, nlopes, spatel, craig.topper, hfinkel

Reviewed By: hfinkel

Subscribers: hfinkel, bjope, llvm-commits

Differential Revision: https://reviews.llvm.org/D47531

Added:
    llvm/trunk/test/Transforms/CorrelatedValuePropagation/pointer.ll
Modified:
    llvm/trunk/lib/Analysis/ValueTracking.cpp

Modified: llvm/trunk/lib/Analysis/ValueTracking.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Analysis/ValueTracking.cpp?rev=333557&r1=333556&r2=333557&view=diff
==============================================================================
--- llvm/trunk/lib/Analysis/ValueTracking.cpp (original)
+++ llvm/trunk/lib/Analysis/ValueTracking.cpp Wed May 30 08:56:46 2018
@@ -1943,6 +1943,10 @@ bool isKnownNonZero(const Value *V, unsi
     }
   }
 
+  // Some of the tests below are recursive, so bail out if we hit the limit.
+  if (Depth++ >= MaxDepth)
+    return false;
+
   // Check for pointer simplifications.
   if (V->getType()->isPointerTy()) {
     // Alloca never returns null, malloc might.
@@ -1963,13 +1967,10 @@ bool isKnownNonZero(const Value *V, unsi
       if (CS.isReturnNonNull())
         return true;
       if (const auto *RP = getArgumentAliasingToReturnedPointer(CS))
-        return isKnownNonZero(RP, Depth + 1, Q);
+        return isKnownNonZero(RP, Depth, Q);
     }
   }
 
-  // The remaining tests are all recursive, so bail out if we hit the limit.
-  if (Depth++ >= MaxDepth)
-    return false;
 
   // Check for recursive pointer simplifications.
   if (V->getType()->isPointerTy()) {

Added: llvm/trunk/test/Transforms/CorrelatedValuePropagation/pointer.ll
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/CorrelatedValuePropagation/pointer.ll?rev=333557&view=auto
==============================================================================
--- llvm/trunk/test/Transforms/CorrelatedValuePropagation/pointer.ll (added)
+++ llvm/trunk/test/Transforms/CorrelatedValuePropagation/pointer.ll Wed May 30 08:56:46 2018
@@ -0,0 +1,35 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt -correlated-propagation -S -o - %s | FileCheck %s
+
+; Testcase that checks that we don't end in a neverending recursion resulting in
+; a segmentation fault. The checks below verify that nothing is changed.
+
+declare dso_local i16* @f2(i16* readnone returned) local_unnamed_addr
+
+define dso_local void @f3() local_unnamed_addr {
+; CHECK-LABEL: @f3(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    br label [[FOR_COND:%.*]]
+; CHECK:       for.end:
+; CHECK-NEXT:    [[CALL6:%.*]] = call i16* @f2(i16* [[CALL6]])
+; CHECK-NEXT:    br i1 false, label [[FOR_COND]], label [[FOR_COND3:%.*]]
+; CHECK:       for.cond:
+; CHECK-NEXT:    [[C_0:%.*]] = phi i16* [ undef, [[ENTRY:%.*]] ], [ [[CALL6]], [[FOR_END:%.*]] ]
+; CHECK-NEXT:    br label [[FOR_COND3]]
+; CHECK:       for.cond3:
+; CHECK-NEXT:    ret void
+;
+entry:
+  br label %for.cond
+
+for.end:
+  %call6 = call i16* @f2(i16* %call6)
+  br i1 false, label %for.cond, label %for.cond3
+
+for.cond:
+  %c.0 = phi i16* [ undef, %entry ], [ %call6, %for.end ]
+  br label %for.cond3
+
+for.cond3:
+  ret void
+}