<div dir="ltr">Dear Dr. Alexandre,<div><br></div><div>The leak warning is raised by MallocChecker and not the RetainCountChecker (which performs reference counting).</div><div>Adding 'rc_ownership_trusted_implementation' annotation to 'bar' and returning true (successfully evaluating 'bar') from evalCall() callback in RetainCountChecker prevents the analyzer from analyzing bar's body. This causes problems for the MallocChecker as it is unable to find a call to 'free' for the call to 'malloc' in foo().</div><div><br></div><div>Using evalCall(), we'll be able to suppress leak false positives raised in ISL due to obj_free(), obj_cow() and obj_copy(). Now, we want our solution (to suppress such false positives) to be as generalized as possible. By generalized, I mean using the same "trusted" annotation across different codebases.</div><div>However, this approach (using evalCall()) can't be generalized for other codebases in C as it's possible for some codebase to have code similar to the one mentioned in my previous email. Such a test-case will result in the MallocChecker raising false positives.</div></div><div hspace="streak-pt-mark" style="max-height:1px"><img alt="" style="width:0px;max-height:0px;overflow:hidden" src="https://mailfoogae.appspot.com/t?sender=aY3MxM2IxMDMxQGlpdGguYWMuaW4%3D&type=zerocontent&guid=cf72707e-8870-4a33-b744-8c71f511e486"><font color="#ffffff" size="1">ᐧ</font></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jul 10, 2017 at 6:43 PM, Alexandre Isoard <span dir="ltr"><<a href="mailto:alexandre.isoard@gmail.com" target="_blank">alexandre.isoard@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">Hi Malhar,</p>
<p dir="ltr">I think the idea of trusted implementation is to trust blindly that the implementation respects the release/acquire annotation without inlining nor looking inside the function. That means you need those functions to be properly annotated.</p>
<p dir="ltr">Your bar function do not have any annotations to "trust". Hence the problem.</p><div class="HOEnZb"><div class="h5">
<br><div class="gmail_quote"><div dir="ltr">On Mon, Jul 10, 2017, 11:57 Malhar Thakkar via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>I've been thinking a lot lately and I feel evalCall-ing is not the right way to proceed (even though it works as desired) as I feel I won't be able to generalize it to work for different kinds of codebases written in C. For example, it won't work for the following case (as you pointed out before).</div><div></div><b><br></b><div><div class="m_7179209788484697977m_1468660041653885838hljs m_7179209788484697977m_1468660041653885838cpp" style="display:block;overflow-x:auto;padding:0.5em;background:rgb(248,248,248);font-family:monospace"><div style="color:rgb(51,51,51)"><b>// evalCall-ing based on annotation</b></div><div style="color:rgb(51,51,51)"><br></div><div><div style="color:rgb(51,51,51)"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)">#<span class="m_7179209788484697977m_1468660041653885838hljs-meta-keyword" style="font-weight:bold">include</span> <span class="m_7179209788484697977m_1468660041653885838hljs-meta-string" style="color:rgb(77,153,191)"><stdlib.h></span></span></div><div style="color:rgb(51,51,51)"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)">typedef struct</span></div><div style="color:rgb(51,51,51)"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)">{</span></div><div style="color:rgb(51,51,51)"><span style="white-space:pre-wrap"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)"> </span></span><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)">int ref_count;</span></div><div style="color:rgb(51,51,51)"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)">} rc_struct;</span></div><div style="color:rgb(51,51,51)"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)">void free(void *);</span></div><div style="color:rgb(51,51,51)"><br></div><div style="color:rgb(51,51,51)"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)">__attribute__((annotate(<span class="m_7179209788484697977m_1468660041653885838hljs-string" style="color:rgb(136,0,0)">"rc_<wbr>ownership_trusted_<wbr>implementation"</span>))) rc_struct *bar(rc_struct *r) {</span></div><div style="color:rgb(51,51,51)"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)"> if (!r)</span></div><div style="color:rgb(51,51,51)"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)"> return NULL;</span></div><div style="color:rgb(51,51,51)"><br></div><div style="color:rgb(51,51,51)"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)"> if (--r->ref_count > 0)</span></div><div style="color:rgb(51,51,51)"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)"> return NULL;</span></div><div style="color:rgb(51,51,51)"><br></div><div style="color:rgb(51,51,51)"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)"> free(r);</span></div><div style="color:rgb(51,51,51)"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)"> return NULL;</span></div><div style="color:rgb(51,51,51)"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)">}</span></div><div style="color:rgb(51,51,51)"><br></div><div style="color:rgb(51,51,51)"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)">void foo() {</span></div><div style="color:rgb(51,51,51)"><span style="white-space:pre-wrap"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)"> </span></span><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)">rc_struct *r = (rc_struct *)malloc(sizeof(rc_struct));</span></div><div style="color:rgb(51,51,51)"><span style="white-space:pre-wrap"><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)"> </span></span><span class="m_7179209788484697977m_1468660041653885838hljs-meta" style="color:rgb(31,113,153)">bar(r);</span></div><div><span class="m_7179209788484697977m_1468660041653885838hljs-meta"><font color="#1f7199">} </font><b><font color="#ff0000">// Leak warning raised for 'r'</font></b></span></div></div></div></div><br><div><div></div></div><div><br></div>Is it possible to check if the RetainCountChecker entered a function containing a certain annotation by walking through the diagnostic path just before emitting a bug report?<div><br></div><div class="gmail_extra"><br><div class="gmail_quote"></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Fri, Jul 7, 2017 at 6:33 PM, Malhar Thakkar <span dir="ltr"><<a href="mailto:cs13b1031@iith.ac.in" target="_blank">cs13b1031@iith.ac.in</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="m_7179209788484697977m_1468660041653885838gmail-">On Fri, Jul 7, 2017 at 3:37 PM, Artem Dergachev <span dir="ltr"><<a href="mailto:noqnoqneo@gmail.com" target="_blank">noqnoqneo@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">For now RetainCountChecker avoids inlining CoreFoundation's CFRetain() by looking at its annotation and performing evalCall(). </blockquote></span><div>Oh, I see. </div><span class="m_7179209788484697977m_1468660041653885838gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">evalCall is the checker callback that allows the checker to completely model the function's effects; if the function is evalCall-ed by the checker, the analyzer core doesn't try to inline it. I think this is exactly what you need to do: the function you're modeling is *the* release, and the analyzer doesn't need to know anything about how it is implemented.<br></blockquote></span><div>Yes, I think, this is exactly what I need but I just read that only one checker can evaluate a call at a given instance and that, as you mentioned with your example containing MallocChecker, can cause some problems. I just tried evalCall-ing functions which contain a certain annotation and it seems to be working on the test-cases which I wrote. But, would that solution be acceptable to the community(i.e., commit-worthy)? If not, I'll have to think of something else.</div><div><div class="m_7179209788484697977m_1468660041653885838gmail-h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
However, by evalCall-ing based on annotation, you essentially say that all functions that wear this annotation are to be modeled similarly. If some of them have additional side effects, you may fail to model them. For example, if the analyzer has seen the malloc() that corresponds to the free() in isl_basic_map_free(), it'd make MallocChecker unhappy - unlike the case when you neither evalCall nor inline but evaluate conservatively, causing the malloc-ed pointer to "escape" into this function. </blockquote></div></div></div></div></div></blockquote></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>What exactly do you mean by evaluating conservatively?</div></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="m_7179209788484697977m_1468660041653885838gmail-h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">If this becomes a problem, you may need to look into our body farms - the mechanism that synthesizes ASTs of various functions for analysis purposes. I imagine farming a function that only does releases, and then farming more functions that call this function and do additional side effects.</blockquote></div></div></div></div></div></blockquote></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>How would farming a function work exactly? Also, the ISL codebase contains a lot of different functions (one for each ISL data type) which perform releases. So, farming functions for each of them doesn't seem like a good option.</div></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="m_7179209788484697977m_1468660041653885838gmail-h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span><br>
<br>
On 7/7/17 10:23 AM, Malhar Thakkar via cfe-dev wrote:<br>
</span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span>
Dear all,<br>
<br>
I wish to prevent the RetainCountChecker from analyzing function bodies if these functions have certain annotate attributes. Consider the following example to get a better idea of why I wish to do that.<br>
<br>
Below is a small snippet from the Integer Set Library (ISL).<br>
<br>
typedef struct {<br>
int ref;<br>
} isl_basic_map;<br>
<br>
__attribute__((cf_returns_<wbr>retained))<br>
isl_basic_map *isl_basic_map_copy(isl_basic_<wbr>map *bmap);<br>
<br>
__attribute__((cf_returns_<wbr>retained))<br>
isl_basic_map *isl_basic_map_cow<br>
(__attribute__((cf_consumed)) isl_basic_map *bmap);<br>
<br>
void free(void *);<br>
<br>
__attribute__((annotate("rc_<wbr>ownership_trusted_<wbr>implementation"))) isl_basic_map *isl_basic_map_free<br>
(__attribute__((cf_consumed)) isl_basic_map *bmap) {<br>
if (!bmap)<br>
return NULL;<br>
<br>
if (--bmap->ref > 0)<br>
return NULL;<br>
<br>
free(bmap);<br>
return NULL;<br>
}<br>
<br>
__attribute__((cf_returns_<wbr>retained))<br>
isl_basic_map *foo<br>
(__attribute__((cf_consumed)) isl_basic_map *bmap) {<br></span>
// *After this call, 'temp' has a +1 reference count.*<br>
isl_basic_map *temp = isl_basic_map_copy(bmap);<br>
// *After this call, 'bmap' has a +1 reference count.*<br>
bmap = isl_basic_map_cow(bmap);<br>
// *After this call, assuming the predicate of the second if branch to be true, 'bmap' has a +1 reference count.*<br>
isl_basic_map_free(bmap);<br>
return temp; *// Object leaked: 'bmap'*<span><br>
}<br>
<br>
While running the RetainCountChecker on the above example, it raises a leak warning for 'bmap' in function 'foo'. This warning is a true positive from the checker's perspective in the sense that the reference count of 'bmap' obtained from 'isl_basic_map_cow' is not decremented(from the checker's perspective) in 'isl_basic_map_free' even though it takes the argument 'bmap' as '__attribute__((cf_consumed))'<wbr>.<br>
<br>
Actually, '--bmap->ref' does decrement the reference count (from ISL's perspective). Hence, to prevent such false positives (from ISL's perspective) to be raised, I wish to prevent the RetainCountChecker to analyze the bodies of the functions having 'rc_ownership_trusted_<wbr>implementation' annotate attribute. I want the checker to just look at the declaration of such functions (and not go inside their bodies) to get the necessary information about reference counting.<br>
<br>
Could someone suggest me a way to achieve my objective?<br>
<br>
<br>
Thank you.<br>
<br>
<br>
Regards,<br>
Malhar Thakkar<br></span>
ᐧ<br>
<br>
<br>
______________________________<wbr>_________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-dev</a><br>
</blockquote>
<br>
</blockquote></div></div></div><br></div></div><div hspace="streak-pt-mark" style="max-height:1px"><img alt="" style="width:0px;max-height:0px;overflow:hidden" src="https://mailfoogae.appspot.com/t?sender=aY3MxM2IxMDMxQGlpdGguYWMuaW4%3D&type=zerocontent&guid=ac7df2d2-c869-4772-864c-996b0eca79ad"><font color="#ffffff" size="1">ᐧ</font></div>
</blockquote></div></div></div><div hspace="streak-pt-mark" style="max-height:1px"><img alt="" style="width:0px;max-height:0px;overflow:hidden" src="https://mailfoogae.appspot.com/t?sender=aY3MxM2IxMDMxQGlpdGguYWMuaW4%3D&type=zerocontent&guid=e703ec0a-59d7-44f4-bf14-ca7c3240163b"><font color="#ffffff" size="1">ᐧ</font></div>
______________________________<wbr>_________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-dev</a><br>
</blockquote></div>
</div></div></blockquote></div><br></div>