<br><br><div class="gmail_quote">On 12 May 2011 16:08, Ted Kremenek <span dir="ltr"><<a href="mailto:kremenek@apple.com">kremenek@apple.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div style="word-wrap:break-word"><div class="im"><div><div>On May 12, 2011, at 11:19 AM, John Smith wrote:</div><br><blockquote type="cite"><span style="border-collapse:separate;font-family:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium">But my main point wasnt really finding bugs in the projects<br>
themselves, but finding & fixing bugs in the analyzer (by decreasing<br>the potential for false positives).</span></blockquote></div><div><br></div></div><div>Thanks John. That's what I am hopeful for as well.</div>
<div><br></div><div>To make this exercise the most constructive, we need actual bug reports against the analyzer. Diagnosing a sea of reports, and complaining that there are too many false positives just really isn't constructive or helpful on its own.</div>
<div><br></div><div>Typically the bug reports have the following characteristics:</div><div><br></div><div>a) have a concise but precise diagnosis of what the analyzer isn't reasoning about correctly</div><div><br></div>
<div>b) provides a test case of a preprocessed file that can be used later to reproduce the issue. (also include the platform/arch you are on when filing the report)</div></div></blockquote><div><br></div><div>One thing that could be very usefully added to the output of scan-build is exactly this prepocessed file, which is otherwise painul to prepare...</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div style="word-wrap:break-word"><div><br></div><div>The scan-build results are useful, but they ultimately lack the ability to be replayed in a debugger session, which is useful when debugging the analyzer. Typically, I have found three kinds of analyzer false positives:</div>
<div><br></div><div>1) The analyzer doesn't know about some higher-level program invariant that the developer knows about and is implicitly relying upon. The discussion there should be how to help the analyzer become more educated about such invariants. Sometimes the answer is interprocedural analysis, sometimes its annotations, etc. There is some actual design tradeoffs to be made here in "fixing" these kind of issues. In some cases, restructuring the original code to make it easier to be reasoned about is the best answer (but that depends on who is voicing the opinion and on what codebase).</div>
</div></blockquote><div><br></div><div>I think restructuring the code is a perfectly valid thing to ask for - it would help if there were some body of knowledge about what kind of restructuring might help. Certainly in OpenSSL I often add unneeded initialisers simply to silence compiler warnings, for example.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div style="word-wrap:break-word"><div><br></div><div>2) The analyzer has an outright bug in handling a specific edge cases. Typically these require a modest amount of change to the analyzer, but having a test case is really key to diagnosing these issues. These are honestly the easiest issues to fix.</div>
<div><br></div><div>3) The analyzer has an algorithmic problem with reasoning about some code. For example, the analyzer doesn't currently reason about bit fields. It also lacks the ability to reason about linear constraints (e.g., a + b > c). Some of these are known issues, others are not. Having concrete examples really helps.</div>
<div><br></div><div>Beyond filing static analyzer bug reports, it would also be great if anyone wanted to help with any of the following projects:</div><div><br></div><div>1) Update or overhaul <a href="http://clang-analyzer.llvm.org" target="_blank">http://clang-analyzer.llvm.org</a> have more information about extracting maximum value from the analyzer.</div>
<div><br></div><div>2) Making scan-build more awesome by making it more turn key, or having a much better way of presenting analysis results. There's a ton of stuff we could do here. I'm not a web developer, so scan-build's HTML reports are what they are because I don't have the expertise to make them better.</div>
</div></blockquote><div><br></div><div>FWIW I think they're pretty reasonable, but ways to make bug reporting cheaper would definitely be nice.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div style="word-wrap:break-word"><div><br></div><div>3) Integrating the analyzer into other IDEs, such as Eclipse.</div><div><br></div><div>4) Working on helping to make the analyzer's precision better, or working on new checkers.</div>
</div></blockquote><div><br></div><div>For anyone contemplating this: clang's static analysis code is a pleasure to work with: I recommend the experience (and wish I had more spare time myself).</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div style="word-wrap:break-word"><div><br></div><div>I really should document all of this on the clang-analyzer website.</div><div><br></div><div>Anyhow, thanks again everyone for running the analyzer on these projects. I do appreciate your level of enthusiasm; my main concern is channeling that enthusiasm in a way that has maximum value.</div>
<div><br></div><div>Cheers,</div><div>Ted</div></div><br>_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@cs.uiuc.edu">cfe-dev@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev</a><br>
<br></blockquote></div><br>