<div dir="ltr"><div>A bit out of scope question. Has someone ever faced problem with declaring/implementing ento::register${CHECKER_NAME} function (if I recall correctly Aleksei already has). I don't understand what's wrong with it, but I have to wrap it into 'namespace clang { namespace ento { ... } }' explicitly which probably works only because linker finds it somehow. In addition I have no chance to register any other checker as the dependency via calling register${OTHER_CHECKER_NAME} even if I add appropriate includes. Thanks in advance!<br><br></div>Sample clang output:<br><br><div><div style="margin-left:40px">error: 'void clang::ento::registerSmartPtrInitChecker(clang::ento::CheckerManager&)' should have been declared inside 'clang::ento'<br></div><br>Regards, Alexey K<div class="gmail_extra"><br><div class="gmail_quote">17 Мар 2018 г. 17:25 пользователь "Alexey Knyshev" <<a href="mailto:alexey.knyshev@gmail.com" target="_blank">alexey.knyshev@gmail.com</a>> написал:<br type="attribution"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">This should already be supported by MallocChecker - i.e. it warns when
you allocate memory with, say, new() and release it with free() or
delete[]().<br>
<br>
If methods of a smart pointer are "inlined" during analysis, these bugs will already be caught automatically.</blockquote><div><br></div><div>I took a look at your recent work & status report posted there in mailing lists. And I have a question about how it would work in particular cases, e.g.:<br></div><div>1. Returning smart_ptr by value from function that has it's implementation in analyzed translation unit but it isn't called from any other function in the same TU. <br><br><div style="margin-left:40px">std::shared_ptr<S> createInstance() {</div><div style="margin-left:40px"> return std::shared_ptr<S>(new S, free);<br></div><div style="margin-left:40px">}</div><div style="margin-left:40px"><br></div>If I understand correctly, compiler is free to apply copy-elison optimization on shared_ptr which is returned by value. So destructor call won't be modeled properly because it's not inlined during analysis. And bug won't be caught.<br><br></div><div>2. As you mentioned before, report would be fired only if constructor and destructor have been inlined (which is not guaranteed to be done). As the result such check won't work in general case.<br><br></div><div>Actually my point is about modeling smart pointers behavior specifically and don't inline any calls to constructor / destructor & methods those modeled by checker. <span id="gmail-m_-5908515395320095639m_7747275425523572015m_1328483257789609973gmail-result_box" class="gmail-m_-5908515395320095639m_7747275425523572015m_1328483257789609973gmail-short_text" lang="en"><span>And the question immediately arises</span></span>: is there way to control inline policy?<br></div><div>And sorry for any possible misunderstanding from my side as I'm not quite familiar with CSA internals. <br></div><div><br></div><div>Regards, Alexey K<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">2018-03-17 5:21 GMT+03:00 Artem Dergachev <span dir="ltr"><<a href="mailto:noqnoqneo@gmail.com" target="_blank">noqnoqneo@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span><br>
<br>
On 16/03/2018 12:30 PM, Alexey Knyshev wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Sorry, accidentally removed part of message before sending.<br>
<br>
Aleksei,<br>
<br>
Could you share your ideas of checker design? It is possible that<br>
problems you met can be solved in different (maybe even better)<br>
way if we know the whole picture.<br>
<br>
<br>
I'm currently have no clear idea how to implement it in the "right manner". But first of all I would aim to track the origin (malloc, new, new [], etc) of SVal and check if Deleter is the expected corresponding way to deallocate such SVal.<br>
</blockquote>
<br></span>
This should already be supported by MallocChecker - i.e. it warns when you allocate memory with, say, new() and release it with free() or delete[]().<br>
<br>
If methods of a smart pointer are "inlined" during analysis, these bugs will already be caught automatically.<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Regards, Alexey K<br>
<br>
2018-03-16 22:22 GMT+03:00 Alexey Knyshev <<a href="mailto:alexey.knyshev@gmail.com" target="_blank">alexey.knyshev@gmail.com</a> <mailto:<a href="mailto:alexey.knyshev@gmail.com" target="_blank">alexey.knyshev@gmail.c<wbr>om</a>>>:<span><br>
<br>
Hello guys,<br>
<br>
And thanks for your attention<br>
<br>
Aleksei,<br>
<br>
Could you share your ideas of checker design? It is possible<br>
that problems you met can be solved in different (maybe even<br>
better) way if we know the whole picture.<br>
<br>
<br>
As I said, I'm interested in improving current state of dynamic<br>
memory modeling. Especially stuff related to smart pointers (e.g.<br>
shared, unique) which also mentioned in list of potential checkers<br>
as smartptr.SmartPtrInit<br></span>
<<a href="https://clang-analyzer.llvm.org/potential_checkers.html" rel="noreferrer" target="_blank">https://clang-analyzer.llvm.o<wbr>rg/potential_checkers.html</a>>*<br>
<br>
*<br>
<br>
**You can see an example of how state trait is exported in<span><br>
GenericTaintChecker or MPIChecker. Generally, you just create<br>
a named ProgramStateTrait in the header.<br>
<br>
<br>
Briefly looked at MPITypes.h. Does it mean we should move<br>
RegionState to separate file and register it via Traits in the<br>
same manner to make it avaliable from other checkers (not other TU<br>
as mentioned in ento::mpi::RequestMap)?<br>
<br>
</span></blockquote>
<br>
GenericTaintChecker is made in a fairly intrusive manner by putting stuff into the program state class directly, which isn't very sane. I'd definitely prefer something similar to dynamic type propagation (see DynamicTypeMap.cpp, DynamicTypeMap.h). You can use the usual REGISTER_MAP_WITH_PROGRAMSTATE (etc.) macros in the .cpp file as long as you provide accessor methods declared in the .h file that other checkers could include.<span><br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Artem,<br>
<br>
you need to keep all of this in mind when writing a checker.<br>
<br>
<br>
Sure, but on the other hand I think it's possible to implement and<br>
improve modeling of various API calls' effects step-by-step. Let's<br>
say, it case of SmartPtrInit checker mentioned before the bare<br>
minimum would be modeling of construction (and destruction to<br>
avoid leak report from existing related checkers).<br>
<br>
which is why the few experimental C++ checkers that we<br>
currently have required heavy hacks to become possible. But<br>
for now you'd rather keep an eye on these problems.<br>
<br>
<br>
Does it mean it's better to wait a bit for Core improvements from<br>
main contributors? Does it make sense to make an efforts to<br>
implement SmartPtrItnit checker in current state of things?<br>
<br>
</blockquote>
<br></span>
You should feel free to start working on the checker in an incremental manner (everything is better in an incremental manner!), but in some cases i may prefer improving the checker API or fixing core modeling bugs instead of writing large portions of checker code to work around inconvenient APIs or bugs, and this is something i might not be immediately capable of doing myself (due to being busy with other things), which may potentially delay your progress.<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span>
Thanks in advance and regards,<br>
Alexey K<br>
<br>
2018-03-16 21:47 GMT+03:00 Artem Dergachev <<a href="mailto:noqnoqneo@gmail.com" target="_blank">noqnoqneo@gmail.com</a><br></span>
<mailto:<a href="mailto:noqnoqneo@gmail.com" target="_blank">noqnoqneo@gmail.com</a>>>:<div><div class="gmail-m_-5908515395320095639m_7747275425523572015m_1328483257789609973h5"><br>
<br>
Dynamic memory management is pretty much done at this point -<br>
we have very good support for malloc()-like stuff and<br>
reasonably good support for operator new() (support for<br>
operator new[]() is currently missing because it requires<br>
calling an indefinite number of constructors which is not yet<br>
implemented). There is also a known issue with custom operator<br>
new(...) returning null pointers - in this case we should not<br>
be calling the constructor but for now we evaluate the<br>
constructor conservatively.<br>
<br>
Your real problem will be managing smart pointers themselves<br>
because they are C++ objects that have a myriad of methods,<br>
they can be copied, moved, assigned, move-assigned, they<br>
destroyed, they lifetime-extended, you need to keep all of<br>
this in mind when writing a checker. This is slowly getting<br>
easier because i'm currently working on that, but until<br>
recently it wasn't working correctly in the core, let alone<br>
checkers, which is why the few experimental C++ checkers that<br>
we currently have required heavy hacks to become possible. But<br>
for now you'd rather keep an eye on these problems.<br>
<br>
On 16/03/2018 3:57 AM, Aleksei Sidorin via cfe-dev wrote:<br>
<br>
Hi Alexey!<br>
<br>
Could you share your ideas of checker design? It is<br>
possible that problems you met can be solved in different<br>
(maybe even better) way if we know the whole picture.<br>
Regarding your questions, you can find some answers below.<br>
<br>
1. a) The first way for checker communication is to share<br>
their program state trait. You can see an example of how<br>
state trait is exported in GenericTaintChecker or<br>
MPIChecker. Generally, you just create a named<br>
ProgramStateTrait in the header. You can take a look at<br>
TaintManager.h and MPITypes.h and how they are used.<br>
b) To set a dependency from another checker, you can just<br>
register it while registering your checker. An example can<br>
be found in MallocChecker where register$Checker also<br>
calls registerCStringCheckerBasic to register a checker it<br>
depends on.<br>
As you pointed, inter-checker communication can become a<br>
source of some problems. Most of them are discussed in<br>
this conversation:<br>
<a href="http://clang-developers.42468.n3.nabble.com/analyzer-RFC-Design-idea-separate-modelling-from-checking-td4059122.html" rel="noreferrer" target="_blank">http://clang-developers.42468.<wbr>n3.nabble.com/analyzer-RFC-Des<wbr>ign-idea-separate-modelling-fr<wbr>om-checking-td4059122.html</a><br>
<<a href="http://clang-developers.42468.n3.nabble.com/analyzer-RFC-Design-idea-separate-modelling-from-checking-td4059122.html" rel="noreferrer" target="_blank">http://clang-developers.42468<wbr>.n3.nabble.com/analyzer-RFC-De<wbr>sign-idea-separate-modelling-f<wbr>rom-checking-td4059122.html</a>><br>
<br>
2. I think there is nothing bad in sharing RegionState<br>
across checkers in the way shown in 1a.<br>
<br>
3. Artem Dergachev has done some excellent work on<br>
improvement of operator 'new' processing in CSA engine.<br>
Regarding checkers, I can see some on<br>
<a href="https://clang-analyzer.llvm.org/potential_checkers.html" rel="noreferrer" target="_blank">https://clang-analyzer.llvm.or<wbr>g/potential_checkers.html</a><br>
<<a href="https://clang-analyzer.llvm.org/potential_checkers.html" rel="noreferrer" target="_blank">https://clang-analyzer.llvm.o<wbr>rg/potential_checkers.html</a>>:<br>
for example, undefbehavior.AutoptrsOwnSameO<wbr>bj. You can<br>
search this list to find more.<br>
<br>
<br>
15.03.2018 23:54, Alexey Knyshev via cfe-dev пишет:<br>
<br>
Hi there!<br>
<br>
While thinking about how it would be possible to<br>
implement various smart ptr related checkers I tried<br>
to review current state of MallocChecker and came up<br>
with that it would be great to have RegionState info<br>
available to other checkers. Could you please share<br>
your points of view and comments on the following<br>
statements / questions:<br>
<br>
1. Is there any right way for chaining checkers? How<br>
they are expected to communicate between each other<br>
(excluding generation of nodes / ProgramStates). I've<br>
heard that there are couple of problems caused by<br>
inlining functions, constructors / descructors.<br>
2. What do you think about moving RegionState to the<br>
Core of CSA or providing optional extended info in<br>
MemRegion about the source of such region (new<br>
opearator / array new, malloc, alloca, etc). So it<br>
would be available to all checkers.<br>
3. Is there any roadmap for CSA and especially for<br>
dynamic memory management modeling & related checkers?<br>
<br>
Regards, Alexey K<br>
<br></div></div>
-- <a href="http://linkedin.com/profile" rel="noreferrer" target="_blank">linkedin.com/profile</a> <<a href="http://linkedin.com/profile" rel="noreferrer" target="_blank">http://linkedin.com/profile</a>><br>
<<a href="https://www.linkedin.com/profile/view?id=AAMAABn6oKQBDhBteiQnWsYm-S9yxT7wQkfWhSw" rel="noreferrer" target="_blank">https://www.linkedin.com/prof<wbr>ile/view?id=AAMAABn6oKQBDhBtei<wbr>QnWsYm-S9yxT7wQkfWhSw</a><span><br>
<<a href="https://www.linkedin.com/profile/view?id=AAMAABn6oKQBDhBteiQnWsYm-S9yxT7wQkfWhSw" rel="noreferrer" target="_blank">https://www.linkedin.com/prof<wbr>ile/view?id=AAMAABn6oKQBDhBtei<wbr>QnWsYm-S9yxT7wQkfWhSw</a>>><br>
<br>
<a href="http://github.com/alexeyknyshev" rel="noreferrer" target="_blank">github.com/alexeyknyshev</a><br>
<<a href="http://github.com/alexeyknyshev" rel="noreferrer" target="_blank">http://github.com/alexeyknysh<wbr>ev</a>><br></span>
<<a href="http://github.com/alexeyknyshev" rel="noreferrer" target="_blank">http://github.com/alexeyknysh<wbr>ev</a><br>
<<a href="http://github.com/alexeyknyshev" rel="noreferrer" target="_blank">http://github.com/alexeyknysh<wbr>ev</a>>><br>
<a href="http://bitbucket.org/alexeyknyshev" rel="noreferrer" target="_blank">bitbucket.org/alexeyknyshev</a><br>
<<a href="http://bitbucket.org/alexeyknyshev" rel="noreferrer" target="_blank">http://bitbucket.org/alexeykn<wbr>yshev</a>><br>
<<a href="https://bitbucket.org/alexeyknyshev/" rel="noreferrer" target="_blank">https://bitbucket.org/alexeyk<wbr>nyshev/</a><span><br>
<<a href="https://bitbucket.org/alexeyknyshev/" rel="noreferrer" target="_blank">https://bitbucket.org/alexeyk<wbr>nyshev/</a>>><br>
<br>
<br>
______________________________<wbr>_________________<br>
cfe-dev mailing list<br></span>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a> <mailto:<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><wbr>><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-dev</a><span><br>
<<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin<wbr>/mailman/listinfo/cfe-dev</a>><br>
<br>
<br>
<br>
-- Best regards,<br>
Aleksei Sidorin,<br>
SRR, Samsung Electronics<br>
<br>
<br>
______________________________<wbr>_________________<br>
cfe-dev mailing list<br></span>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a> <mailto:<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><wbr>><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-dev</a><br>
<<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin<wbr>/mailman/listinfo/cfe-dev</a>><span><br>
<br>
<br>
<br>
<br>
<br>
-- <a href="http://linkedin.com/profile" rel="noreferrer" target="_blank">linkedin.com/profile</a><br>
<<a href="https://www.linkedin.com/profile/view?id=AAMAABn6oKQBDhBteiQnWsYm-S9yxT7wQkfWhSw" rel="noreferrer" target="_blank">https://www.linkedin.com/prof<wbr>ile/view?id=AAMAABn6oKQBDhBtei<wbr>QnWsYm-S9yxT7wQkfWhSw</a>><br>
<br>
<a href="http://github.com/alexeyknyshev" rel="noreferrer" target="_blank">github.com/alexeyknyshev</a> <<a href="http://github.com/alexeyknyshev" rel="noreferrer" target="_blank">http://github.com/alexeyknysh<wbr>ev</a>><br>
<a href="http://bitbucket.org/alexeyknyshev" rel="noreferrer" target="_blank">bitbucket.org/alexeyknyshev</a> <<a href="https://bitbucket.org/alexeyknyshev/" rel="noreferrer" target="_blank">https://bitbucket.org/alexeyk<wbr>nyshev/</a>><br>
<br>
<br>
<br>
<br>
-- <br>
<a href="http://linkedin.com/profile" rel="noreferrer" target="_blank">linkedin.com/profile</a> <<a href="https://www.linkedin.com/profile/view?id=AAMAABn6oKQBDhBteiQnWsYm-S9yxT7wQkfWhSw" rel="noreferrer" target="_blank">https://www.linkedin.com/prof<wbr>ile/view?id=AAMAABn6oKQBDhBtei<wbr>QnWsYm-S9yxT7wQkfWhSw</a>><br>
<br>
<a href="http://github.com/alexeyknyshev" rel="noreferrer" target="_blank">github.com/alexeyknyshev</a> <<a href="http://github.com/alexeyknyshev" rel="noreferrer" target="_blank">http://github.com/alexeyknysh<wbr>ev</a>><br>
<a href="http://bitbucket.org/alexeyknyshev" rel="noreferrer" target="_blank">bitbucket.org/alexeyknyshev</a> <<a href="https://bitbucket.org/alexeyknyshev/" rel="noreferrer" target="_blank">https://bitbucket.org/alexeyk<wbr>nyshev/</a>><br>
</span></blockquote>
<br>
</blockquote></div><br><br clear="all"><br>-- <br><div class="gmail-m_-5908515395320095639m_7747275425523572015m_1328483257789609973gmail_signature"><div dir="ltr"><a href="https://www.linkedin.com/profile/view?id=AAMAABn6oKQBDhBteiQnWsYm-S9yxT7wQkfWhSw" target="_blank">linkedin.com/profile</a><br><br><a href="http://github.com/alexeyknyshev" target="_blank">github.com/alexeyknyshev</a><span></span><span></span><br><a href="https://bitbucket.org/alexeyknyshev/" target="_blank">bitbucket.org/alexeyknyshev</a><br></div></div>
</div>
</blockquote></div></div>
</div></div>