[cfe-dev] Static Analyzer Rocks Hard

[Matthew Jimenez]

On Jun 23, 2008, at 1:05 AM, Ted Kremenek wrote:

<snip />
>
> On Jun 19, 2008, at 10:36 PM, Matthew Jimenez wrote:
>>
>> This behavior does make sense, but should it really extend past the
>> first usage of the of the variable or after the first pass of a loop?
>> There may be cases where extending it might be a good idea, but I'd
>> imagine those cases too be very rare in code.
>
> I myself am often surprised how logic bugs can cross loop  
> iterations.  Programmers sometimes make assumptions that hold on the  
> first loop iteration and not the second.
>

Ok, fair enough.

<snip />
>
>> Adding the asserts would probably be the best case in the projects I
>> work on, but there are going to be many cases of this, and the
>> original case was a little confusing at first glance.
>
> Just a thought: In a way, adding a flag to the static analyzer to  
> assume that all allocations succeed is almost the same as adding an  
> assertion after every allocation.  The only difference is that there  
> is no run-time check that an assertion provides.  The question is  
> whether or not observing the assertion actually firing would be  
> useful, since it is often obvious that a program is running out of  
> memory.  This of course depends on the application and the  
> preferences of the developer.
>
> While I don't like the idea of having a proliferation of options to  
> the checker which could overly complicate its interface, it does  
> seem the me that "assuming allocations don't fail" would be a very  
> common escape hatch when suppressing false positives for memory  
> leaks and other bugs.  It's an assumption that holds 99% of the time  
> --- except when it doesn't --- in which case most programs are  
> screwed because they aren't written to handle such failure (this  
> isn't really a bug, but a design decision).  Having a flag for this  
> to the static analyzer allows one to assert this across an entire  
> project.  In many ways its a personal decision about how software  
> should be written, and whether or not a bug-finding tool should  
> always scold you about it.
>
> Just a thought.
>

It's been my experience that developers assume success without much  
thought as to the result of failure until they actually witness it.  
Allocation failure is especially nasty since the chances of  
reproducing the error is so slim, and Objective-C might be even  
nastier as nil is a valid receiver. So the need for a tool to check  
this exists, but for better or worse, I still wouldn't imagine  
developers to be proactive about handling such situations.

Strangely enough, I have seen many developers be much more paranoid  
when allocating memory directly rather than through objects.

>> It's definitely made me think about the way I code, but it would make
>> me think the best option would be to always assert against all
>> allocations - which would be madness in a large pre-existing code
>> base :-)
>
> I think that's awesome if you (and others) want to document your  
> code so thoroughly with assertions.  The problem I see is that you  
> may need a lot of these assertions since allocations are frequent.   
> I'm concerned that some programmers may feel that it is necessary to  
> add an assertion after every allocation just to reduce the noise  
> from the checker.  That's an extra line of code for every  
> allocation; that's cumbersome for both pre-existing and new code.
>
> I'm curious about the following: if I add an option to have the  
> static analyzer assume that all allocations succeed, what portion of  
> your false warnings disappear?  If it is a significant number, I  
> think this option is worth including (or even it including it as the  
> default behavior).  Would you be interested in running this  
> experiment?

I wish I could give good numbers on this at the moment, but I only had  
the chance to check the parts that compile on my mac so far (objective- 
c and windows only - it's rather silly).
However I from what I did manage to compile, I only saw this once out  
of about fifty other reported problems. I figure this accounts for  
about a tenth of the total code, so I might see this ten more times,  
maybe twenty.

I plan on testing the code base throughly at some point in the next  
month or so, and I am definitely interested in helping if I can.

>> Seriously though, my case may still be rare enough outside of the  
>> code
>> I work on that it could be successfully ignored. I'll gladly ignore
>> the bug if it comes to that. I'm already ignoring some cases of dead
>> stores and memory leaks that are in code that was not relevant
>> (obvious because the original author named the function
>> "never_called_but_needed"), and I'm not aiming for complete
>> coverage... at least not yet.
>
> I really appreciate your perspective.  As you continue to use the  
> static analyzer (especially as it evolves) I'm very interested to  
> hear any feedback on how the workflow of the tool can be improved to  
> fit into your model of finding and fixing bugs.

Sure thing.

Thanks,
-Matthew