[cfe-commits] unix.Malloc static checker improvement: memory.LeakPtrValChanged

[Anna Zaks]

I'll first summarize what is happening:

Example 1:
>>>>>> {
>>>>>> int * data = malloc(sizeof(int));
>>>>>> data += 1;
>>>>>> free(data);
>>>>>> }

In this case, checkPointerEscape is called, while evaluating a call to 'free'. The Call argument is NULL. Currently, we set Call to NULL unless the pointer is one of the arguments to the call, which is not the case here. The reason why we invalidate the pointer anyway is because we call a function that takes a pointer (data+1), from which our tracked pointer is accessible. This is pessimistic logic, which protects us from false positives. For example, we could have called some foo which would invalidate the pointer and anything accessible from it through pointer arithmetic. In the case at hand, we know that free is not going to do that, but currently this is not special cased. 

The discussion below is about how we could special case it. Ex: check the callback to actually pass the Call parameter and use another argument to specify if the invalidation is direct (a pointer is an argument) or not. I am not sure how generic this solution is and how important it is to handle this. 

>>>>>> {
>>>>>> int * array = malloc(sizeof(int)*5);
>>>>>> array += 1;
>>>>>> free(&array[-1]);
>>>>>> }

In this case the callback is called. Inside the Malloc checker, we detect that the pointer is being invalidated because of a call to free. Malloc checker still wants to track the pointer since it knows the semantics of free, so we return early from the callback:
  // If we know that the call does not free memory, keep tracking the top
  // level arguments.       
  if (Call && doesNotFreeMemory(Call, State))
    return State;

On Jan 2, 2013, at 8:08 PM, Branden Archer <b.m.archer4 at gmail.com> wrote:

> I am trying to catch up to your discussion. 
> 
> I debugged the examples. The callback is called in both cases.
> 
> In the second case, we correctly determine that the pointer is being freed.
> 
> In the first case, we are not as precise as we could be. We decide that the pointer escapes, when 'data' is passed to 'free'. This is part of general pessimistic logic, which decides that everything accessible though a pointer passed to a function can escape. In this case, we could have added more logic to assume that 'free' does not invalidate any pointers. In order to do this, we would need to extend the pointerEscapes callback with a bool that let's us know if the invalidation is direct or not. Currently, we always set Call to NULL, when the invalidation is indirect.
> 
> 
> Oops. You are correct, the callback is being called for both cases. My mistake. 
> 
> I am observing that the callback is unable to determine what type of non-function call based escape has occurred. For example, the second case is one example:
> 
>   int * data = malloc(sizeof(int)*2);
>   data += 1;
>   free(data);
> 
> For that example, I am assuming the callback is being invoked when the pointer is being changed, but before it is passed to a function.

It is called when evaluating the function. See the explanation above.

> Or maybe it is occurring when free is being called with an invalid pointer. The callback does not mention a function, setting Call to NULL, so I am not sure.
> Another example that results in a NULL Call is passing data to a global structure. Passing the pointer to a global structure is quite fine, but it is indistinguishable from the problem case I am interested in detecting in example 2.
> 
> 
> Free is special but neither the MallocChecker nor the callback are dealing with it.
> 
> - The callback could pass non-NULL Call and a special flag saying that the invalidation is not direct.
> - The malloc could check if Call is to 'free' and never act upon a pointer escape if it is. This could probably be generalized for other functions we know about. Ex, it's not malloc checker or 'free' specific - we always know that 'free' could only touch the argument. It might be better to find a more general solution for this. 
> 
> It is not clear to me in which cases the callback will be invoked when the escape is not due to a function call. That is, what is the difference between a 'direct' and 'indirect' invalidation.  

Direct invalidation is when the pointer is one of the arguments. Indirect is everything else: assigned to a global, accessible though one of the arguments and more. We could change the callback to be more specific, however, there were no need for it so far.

> 
> I think that the solution you propose may work. That is, passing a non-NULL Call and a flag (to distinguish example 2 from other situations). For my curiosity, what are some examples where the Call would still be NULL, and would the value of the flag matter in such a case?
> 
> The invalidation does happen (same as before). However, ideally, we would not invalidate and report a leak in example 1.
> 
> What criteria is being used to determine if a pointer is to be invalidated?
> 
> - Branden

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20130103/faafca3d/attachment.html>