r335926 - [analyzer] Fix wrong comparison generation of the ranges generated by the refutation manager

[Mikhail R. Gadelha via cfe-commits]

Author: mramalho
Date: Thu Jun 28 14:26:52 2018
New Revision: 335926

URL: http://llvm.org/viewvc/llvm-project?rev=335926&view=rev
Log:
[analyzer] Fix wrong comparison generation of the ranges generated by the refutation manager

The refutation manager is removing a true bug from the test in this patch.

The problem is that the following constraint:
```
(conj_$1{struct o *}) - (reg_$3<int * r>): [-9223372036854775808, 0]
```
is encoded as:
```
(and (bvuge (bvsub $1 $3) #x8000000000000000)
     (bvule (bvsub $1 $3) #x0000000000000000))
```
The issue is that unsigned comparisons (bvuge and bvule) are being generated instead of signed comparisons (bvsge and bvsle).

When generating the expressions:
```
(conj_$1{p *}) - (reg_$3<int * r>) >= -9223372036854775808
```
and
```
(conj_$1{p *}) - (reg_$3<int * r>) <= 0
```
both -9223372036854775808 and 0 are casted to pointer type and `LTy->isSignedIntegerOrEnumerationType()` in `Z3ConstraintManager::getZ3BinExpr` only checks if the type is signed, not if it's a pointer.

Reviewers: NoQ, george.karpenkov, ddcc

Subscribers: rnkovacs, NoQ, george.karpenkov, ddcc, xazax.hun, szepet, a.sidorin

Differential Revision: https://reviews.llvm.org/D48324

Added:
    cfe/trunk/test/PR37855.c
Modified:
    cfe/trunk/lib/StaticAnalyzer/Core/Z3ConstraintManager.cpp

Modified: cfe/trunk/lib/StaticAnalyzer/Core/Z3ConstraintManager.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/StaticAnalyzer/Core/Z3ConstraintManager.cpp?rev=335926&r1=335925&r2=335926&view=diff
==============================================================================
--- cfe/trunk/lib/StaticAnalyzer/Core/Z3ConstraintManager.cpp (original)
+++ cfe/trunk/lib/StaticAnalyzer/Core/Z3ConstraintManager.cpp Thu Jun 28 14:26:52 2018
@@ -1414,9 +1414,8 @@ Z3Expr Z3ConstraintManager::getZ3BinExpr
 
     // If the two operands are pointers and the operation is a subtraction, the
     // result is of type ptrdiff_t, which is signed
-    if (LTy->isAnyPointerType() && LTy == RTy && Op == BO_Sub) {
-      ASTContext &Ctx = getBasicVals().getContext();
-      *RetTy = Ctx.getIntTypeForBitwidth(Ctx.getTypeSize(LTy), true);
+    if (LTy->isAnyPointerType() && RTy->isAnyPointerType() && Op == BO_Sub) {
+      *RetTy = getBasicVals().getContext().getPointerDiffType();
     }
   }
 

Added: cfe/trunk/test/PR37855.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/PR37855.c?rev=335926&view=auto
==============================================================================
--- cfe/trunk/test/PR37855.c (added)
+++ cfe/trunk/test/PR37855.c Thu Jun 28 14:26:52 2018
@@ -0,0 +1,24 @@
+// RUN: %clang_cc1 -analyze -analyzer-eagerly-assume -analyzer-checker=core -w -DNO_CROSSCHECK -verify %s
+// RUN: %clang_cc1 -analyze -analyzer-eagerly-assume -analyzer-checker=core -w -analyzer-config crosscheck-with-z3=true -verify %s
+// REQUIRES: z3
+
+typedef struct o p;
+struct o {
+  struct {
+  } s;
+};
+
+void q(*r, p2) { r < p2; }
+
+void k(l, node) {
+  struct {
+    p *node;
+  } * n, *nodep, path[sizeof(void)];
+  path->node = l;
+  for (n = path; node != l;) {
+    q(node, n->node);
+    nodep = n;
+  }
+  if (nodep) // expected-warning {{Branch condition evaluates to a garbage value}}
+    n[1].node->s; // expected-warning {{Dereference of undefined pointer value}}
+}