[cfe-commits] [PATCH] Additional output for AnalyzerStatsChecker

Thu Feb 23 16:29:38 PST 2012

Sorry Anna yes I meant ProgramPoint.

I managed to reproduce the situation I was talking about in gdb, and I used a file in the LLVM tree so hopefully you can reproduce it.

I apologise for the wall of text! A brief run-through of what I did:
- added an assert to find when I cannot get a PostStmt from the ExplodedNode
- Tracked down an example to llvm/lib/Support/DeltaAlgorithm.cpp in function Search
- The third aborted block is the one that triggers my assertion.

In the gdb run-through below, you can see that when the aborted block is added (node=0x1075499b8, stmt=0x10680d8e8) that everything is as expected. I added a watchpoint for the ProgramPoint::Kind of the node. After a number of reads triggering it, we can see a write that changes the Location to a BlockEdge. At this point I did a backtrace. When the VisitEndAnalysis happens, we see that the same node=0x1075499b8 now has a BlockEdge as its Location and the stmt=0x10680d8e8 has been lost.

Let me know if you need any other information.

Tom

------------

Starting program: /Users/tcare/Projects/llvm-debug/bin/clang++ -cc1 -triple x86_64-apple-macosx10.7.0 -analyze -disable-free -main-file-name DeltaAlgorithm.cpp -analyzer-store=region -analyzer-opt-analyze-nested-blocks -analyzer-eagerly-assume -analyzer-checker=core -analyzer-checker=unix -analyzer-checker=osx -analyzer-checker=deadcode -analyzer-checker=security.insecureAPI.UncheckedReturn -analyzer-checker=security.insecureAPI.getpw -analyzer-checker=security.insecureAPI.gets -analyzer-checker=security.insecureAPI.mktemp -analyzer-checker=security.insecureAPI.mkstemp -analyzer-checker=security.insecureAPI.vfork -analyzer-output plist -w -pic-level 1 -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu core2 -resource-dir /Users/tcare/Projects/llvm-debug/bin/../lib/clang/3.1 -D _DEBUG -D _GNU_SOURCE -D __STDC_CONSTANT_MACROS -D __STDC_FORMAT_MACROS -D __STDC_LIMIT_MACROS -I /Users/tcare/Projects/llvm-analysis/include -I /Users/tcare/Projects/llvm-analysis/lib/Support -I /Users/tcare/Projects/llvm/include -I /Users/tcare/Projects/llvm/lib/Support -fmodule-cache-path /var/folders/cz/fgrz7bdd7mn19gj5hl3w25rw0000gn/T/clang-module-cache -Wno-unused-parameter -fdeprecated-macro -fdebug-compilation-dir /Users/tcare/Projects/llvm-analysis/lib/Support -ferror-limit 19 -fmessage-length 0 -stack-protector 1 -mstackrealign -fblocks -fobjc-runtime-has-arc -fobjc-runtime-has-weak -fobjc-dispatch-method=mixed -fcxx-exceptions -fexceptions -fdiagnostics-show-option -analyzer-display-progress -analyzer-inline-call -analyzer-checker debug.Stats -analyzer-output=html -x c++ /Users/tcare/Projects/llvm/lib/Support/DeltaAlgorithm.cpp -analyze-function Search
ANALYZE: /Users/tcare/Projects/llvm/lib/Support/DeltaAlgorithm.cpp Search

Breakpoint 2, clang::ento::CoreEngine::addAbortedBlock (this=0x7fff5fbfce08, block=0x10733e400, node=0x1073cf268, stmt=0x10680d8e8) at CoreEngine.h:149
149	    blocksAborted.push_back(std::make_pair(block, std::make_pair(node, stmt)));
(gdb) c
Continuing.

Breakpoint 2, clang::ento::CoreEngine::addAbortedBlock (this=0x7fff5fbfce08, block=0x10733f778, node=0x1075115a0, stmt=0x1072d0418) at CoreEngine.h:149
149	    blocksAborted.push_back(std::make_pair(block, std::make_pair(node, stmt)));
(gdb) c
Continuing.

Breakpoint 2, clang::ento::CoreEngine::addAbortedBlock (this=0x7fff5fbfce08, block=0x10733e400, node=0x1075499b8, stmt=0x10680d8e8) at CoreEngine.h:149
149	    blocksAborted.push_back(std::make_pair(block, std::make_pair(node, stmt)));
(gdb) p stmt->dump()
(CXXBindTemporaryExpr 0x10680d8e8 'class DeltaAlgorithm::changeset_ty':'class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >' (CXXTemporary 0x10680d8e0)
  (CXXMemberCallExpr 0x10680d870 'class DeltaAlgorithm::changeset_ty':'class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >'
    (MemberExpr 0x10680d7e8 '<bound member function type>' ->Delta 0x1072c80d0
      (CXXThisExpr 0x10680d7d0 'class llvm::DeltaAlgorithm *' this))
    (ImplicitCastExpr 0x10680d8a8 'const changeset_ty':'const class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >' lvalue <NoOp>
      (DeclRefExpr 0x10680d820 'changeset_ty':'class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >' lvalue Var 0x1072d0700 'Complement' 'changeset_ty':'class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >'))
    (ImplicitCastExpr 0x10680d8c0 'const changesetlist_ty':'const class std::vector<class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >, class std::allocator<class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> > > >' lvalue <NoOp>
      (DeclRefExpr 0x10680d848 'changesetlist_ty':'class std::vector<class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >, class std::allocator<class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> > > >' lvalue Var 0x1072d3020 'ComplementSets' 'changesetlist_ty':'class std::vector<class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >, class std::allocator<class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> > > >'))))
$2 = void
(gdb) p *node
$3 = {
  <llvm::FoldingSetImpl::Node> = {
    NextInFoldingSetBucket = 0x107423da0
  }, 
  members of clang::ento::ExplodedNode: 
  Location = {
    Data = {
      first = 0x10680d8e8, 
      second = 0x0
    }, 
    K = clang::ProgramPoint::PostStmtKind, 
    L = 0x106580d30, 
    Tag = 0x0
  }, 
  State = {
    Obj = 0x1075498f8
  }, 
  Preds = {
    P = 4417952112
  }, 
  Succs = {
    P = 0
  }
}
(gdb) p &node->Location->K
$4 = ('clang::ProgramPoint::Kind' *) 0x1075499d0
(gdb) awatch *$4
Hardware access (read/write) watchpoint 3: *$4
(gdb) d 2
(gdb) c
Continuing.
Hardware access (read/write) watchpoint 3: *$4

Value = clang::ProgramPoint::PostStmtKind
0x00007fff92af4cde in memmove$VARIANT$sse42 ()
(gdb) 
Continuing.
...several hits later...
Hardware access (read/write) watchpoint 3: *$4

Old value = clang::ProgramPoint::PostStmtKind
New value = clang::ProgramPoint::BlockEdgeKind
0x00007fff92af4ce4 in memmove$VARIANT$sse42 ()
(gdb) d 3
(gdb) bt
#0  0x00007fff92af4ce4 in memmove$VARIANT$sse42 ()
#1  0x0000000101090594 in clang::ento::ExplodedNode::ExplodedNode (this=0x1075499b8, loc=@0x7fff5fbfc4f8, state=@0x7fff5fbfc2f0, IsSink=false) at ExplodedGraph.h:122
#2  0x000000010108d1cc in clang::ento::ExplodedNode::ExplodedNode (this=0x1075499b8, loc=@0x7fff5fbfc4f8, state=<value temporarily unavailable, due to optimizations>, IsSink=false) at ExplodedGraph.h:125
#3  0x000000010108c160 in clang::ento::ExplodedGraph::getNode (this=0x10657eec0, L=@0x7fff5fbfc4f8, State=@0x7fff5fbfc438, IsSink=false, IsNew=0x7fff5fbfc44e) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Core/ExplodedGraph.cpp:257
#4  0x0000000101081992 in clang::ento::NodeBuilder::generateNodeImpl (this=0x7fff5fbfc6c8, Loc=@0x7fff5fbfc4f8, State=@0x7fff5fbfc4c0, FromN=0x107549a48, MarkAsSink=false) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Core/CoreEngine.cpp:535
#5  0x0000000101081c96 in clang::ento::BranchNodeBuilder::generateNode (this=0x7fff5fbfc6c8, State=@0x7fff5fbfc5f0, branch=false, NodePred=0x107549a48) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Core/CoreEngine.cpp:568
#6  0x0000000101097044 in clang::ento::ExprEngine::processBranch (this=0x7fff5fbfcdf0, Condition=0x10730a140, Term=0x10730b180, BldCtx=@0x7fff5fbfc8f0, Pred=0x107549a48, Dst=@0x7fff5fbfc888, DstT=0x10738d610, DstF=0x10738c528) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Core/ExprEngine.cpp:1125
#7  0x00000001010812dd in clang::ento::CoreEngine::HandleBranch (this=0x7fff5fbfce08, Cond=0x10730a140, Term=0x10730b180, B=0x10738c770, Pred=0x107549a48) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Core/CoreEngine.cpp:406
#8  0x0000000101081021 in clang::ento::CoreEngine::HandleBlockExit (this=0x7fff5fbfce08, B=0x10738c770, Pred=0x107549a48) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Core/CoreEngine.cpp:388
#9  0x00000001010809e6 in clang::ento::CoreEngine::HandlePostStmt (this=0x7fff5fbfce08, B=0x10738c770, StmtIdx=1, Pred=0x107549a48) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Core/CoreEngine.cpp:417
#10 0x000000010108027a in clang::ento::CoreEngine::ExecuteWorkList (this=0x7fff5fbfce08, L=0x106580d30, Steps=146026, InitState=@0x7fff5fbfcda0) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Core/CoreEngine.cpp:228
#11 0x0000000100f3e0fd in clang::ento::ExprEngine::ExecuteWorkList (this=0x7fff5fbfcdf0, L=0x106580d30, Steps=150000) at ExprEngine.h:97
#12 0x0000000100f3d2bc in ActionExprEngine (C=@0x1065063a0, mgr=@0x10650d280, D=0x1072ceac0, ObjCGCEnabled=false) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Frontend/AnalysisConsumer.cpp:325
#13 0x0000000100f3d16f in RunPathSensitiveChecks (C=@0x1065063a0, mgr=@0x10650d280, D=0x1072ceac0) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Frontend/AnalysisConsumer.cpp:344
#14 0x0000000100f3cfd7 in (anonymous namespace)::AnalysisConsumer::HandleCode (this=0x1065063a0, D=0x1072ceac0) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Frontend/AnalysisConsumer.cpp:300
#15 0x0000000100f3c839 in (anonymous namespace)::AnalysisConsumer::HandleDeclContextDecl (this=0x1065063a0, C=@0x106829a00, D=0x1072ceac0) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Frontend/AnalysisConsumer.cpp:206
#16 0x0000000100f3c574 in (anonymous namespace)::AnalysisConsumer::HandleDeclContext (this=0x1065063a0, C=@0x106829a00, dc=0x10682c4f0) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Frontend/AnalysisConsumer.cpp:177
#17 0x0000000100f3c452 in (anonymous namespace)::AnalysisConsumer::HandleTranslationUnit (this=0x1065063a0, C=@0x106829a00) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Frontend/AnalysisConsumer.cpp:244
#18 0x0000000100634bb4 in clang::ParseAST (S=@0x106840200, PrintStats=false) at /Users/tcare/Projects/llvm/tools/clang/lib/Parse/ParseAST.cpp:106
#19 0x0000000100589488 in clang::ASTFrontendAction::ExecuteAction (this=0x106503ea0) at /Users/tcare/Projects/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:414
#20 0x00000001005890d7 in clang::FrontendAction::Execute (this=0x106503ea0) at /Users/tcare/Projects/llvm/tools/clang/lib/Frontend/FrontendAction.cpp:334
#21 0x00000001005557b5 in clang::CompilerInstance::ExecuteAction (this=0x1065029b0, Act=@0x106503ea0) at /Users/tcare/Projects/llvm/tools/clang/lib/Frontend/CompilerInstance.cpp:653
#22 0x00000001000181cf in clang::ExecuteCompilerInvocation (Clang=0x1065029b0) at /Users/tcare/Projects/llvm/tools/clang/lib/FrontendTool/ExecuteCompilerInvocation.cpp:176
#23 0x0000000100009749 in cc1_main (ArgBegin=0x7fff5fbfea40, ArgEnd=0x7fff5fbfecb8, Argv0=0x106500d08 "/Users/tcare/Projects/llvm-debug/bin/clang++", MainAddr=0x100001a40) at /Users/tcare/Projects/llvm/tools/clang/tools/driver/cc1_main.cpp:165
#24 0x0000000100001c8d in main (argc_=81, argv_=0x7fff5fbff2b0) at /Users/tcare/Projects/llvm/tools/clang/tools/driver/driver.cpp:352
(gdb) c
Continuing.

Breakpoint 1, (anonymous namespace)::AnalyzerStatsChecker::emitAbortedStats (this=0x10650e050, Eng=@0x7fff5fbfcdf0, B=@0x7fff5fbfcfd8, SM=@0x106504c30) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Checkers/AnalyzerStatsChecker.cpp:170
170	    const CoreEngine::AbortedStmtPair aborted = I->second;
(gdb) c
Continuing.

Breakpoint 1, (anonymous namespace)::AnalyzerStatsChecker::emitAbortedStats (this=0x10650e050, Eng=@0x7fff5fbfcdf0, B=@0x7fff5fbfcfd8, SM=@0x106504c30) at /Users/tcare/Projects/llvm/tools/clang/lib/StaticAnalyzer/Checkers/AnalyzerStatsChecker.cpp:170
170	    const CoreEngine::AbortedStmtPair aborted = I->second;
(gdb) n 
171	    const LocationContext *LC = aborted.first->getLocationContext();
(gdb) n
173	    const PostStmt *PS = aborted.first->getLocationAs<PostStmt>();
(gdb) n
174	    assert(PS);
(gdb) p PS
$5 = (const 'clang::PostStmt' *) 0x0
(gdb) p aborted.first
$6 = (const 'clang::ento::ExplodedNode' *) 0x1075499b8
(gdb) p *aborted.first
$7 = {
  <llvm::FoldingSetImpl::Node> = {
    NextInFoldingSetBucket = 0x1067326a9
  }, 
  members of clang::ento::ExplodedNode: 
  Location = {
    Data = {
      first = 0x10738c770, 
      second = 0x10738c528
    }, 
    K = clang::ProgramPoint::BlockEdgeKind, 
    L = 0x106584050, 
    Tag = 0x0
  }, 
  State = {
    Obj = 0x10755da98
  }, 
  Preds = {
    P = 4417952328
  }, 
  Succs = {
    P = 4417950920
  }
}
(gdb) p aborted->second
$8 = (const 'clang::Stmt' *) 0x10680d8e8
(gdb) p aborted->second->dump()
(CXXBindTemporaryExpr 0x10680d8e8 'class DeltaAlgorithm::changeset_ty':'class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >' (CXXTemporary 0x10680d8e0)
  (CXXMemberCallExpr 0x10680d870 'class DeltaAlgorithm::changeset_ty':'class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >'
    (MemberExpr 0x10680d7e8 '<bound member function type>' ->Delta 0x1072c80d0
      (CXXThisExpr 0x10680d7d0 'class llvm::DeltaAlgorithm *' this))
    (ImplicitCastExpr 0x10680d8a8 'const changeset_ty':'const class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >' lvalue <NoOp>
      (DeclRefExpr 0x10680d820 'changeset_ty':'class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >' lvalue Var 0x1072d0700 'Complement' 'changeset_ty':'class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >'))
    (ImplicitCastExpr 0x10680d8c0 'const changesetlist_ty':'const class std::vector<class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >, class std::allocator<class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> > > >' lvalue <NoOp>
      (DeclRefExpr 0x10680d848 'changesetlist_ty':'class std::vector<class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >, class std::allocator<class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> > > >' lvalue Var 0x1072d3020 'ComplementSets' 'changesetlist_ty':'class std::vector<class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> >, class std::allocator<class std::set<unsigned int, struct std::less<unsigned int>, class std::allocator<unsigned int> > > >'))))
$9 = void

On 23/02/2012, at 11:26 PM, Anna Zaks wrote:

> 
> On Feb 23, 2012, at 1:50 PM, Tom Care wrote:
> 
>> The reason I added a Stmt* was because the LocationContext of the node (which starts out as a PostStmt) can be overwritten to become a BlockEdge. The Stmt* pointer is lost in that case, and it becomes difficult to guess where the original Stmt was. Since we only add aborted blocks/nodes when a Stmt was unsupported, I thought that it was important to keep the original pointer to find out why we stopped.
>> 
> 
> Tom, you must be talking about ProgramPoint/Location, not LocationContext, correct? Can you investigate where it gets overwritten? As far as I understand, it should never change, in fact, it's declared as const. We are only adding predecessors and successors to the ExplodedNodes; they should stay unchanged otherwise.
> 
> class ExplodedNode : public llvm::FoldingSetNode {
> ....
>   /// Location - The program location (within a function body) associated
>   ///  with this node.
>   const ProgramPoint Location;
> 
>> Example of the new output:
>> 
>> /Users/tcare/Projects/llvm/tools/clang/lib/Sema/SemaDeclAttr.cpp:218:5: warning: The analyzer stopped at this point due to an unsupported statement of type CXXBindTemporaryExpr
>>    S.Diag(Attr.getLoc(), diag::err_attribute_too_few_arguments) << Num;
>>    ^
>> 
>> /Users/tcare/Projects/llvm/tools/clang/lib/Sema/SemaDeclAttr.cpp:308:28: warning: The analyzer stopped because this block was visited too many times
>>  for(unsigned Idx = Sidx; Idx < Attr.getNumArgs(); ++Idx) {
>>                           ^
>> 
>> /Users/tcare/Projects/llvm/tools/clang/lib/Sema/SemaDeclAttr.cpp:934:11: warning: The analyzer stoppedat this point due to visitation limits
>>      if (T->isAnyPointerType() || T->isBlockPointerType())
>>          ^
>> 
>> (need to add a space in the last one)
>> 
>> I'm testing over all of LLVM/Clang right now as I am collecting data.
>> 
>> Tom
>> 
>> On 23/02/2012, at 6:04 PM, Anna Zaks wrote:
>> 
>>> On Feb 23, 2012, at 7:27 AM, Ted Kremenek wrote:
>>> 
>>>> Looks good to me.  Do you have an example of the new output?
>>>> 
>>>> On Feb 23, 2012, at 7:08 AM, Tom Care <tom.care at uqconnect.edu.au> wrote:
>>>> 
>>>>> I made some changes to AnalyzerStatsChecker to make it a bit more useful. Changelog:
>>>>> 
>>>>> Improved the information output by AnalyzerStatsChecker. In addition to the analysis stats for Decls, AnalyzerStatsChecker now also outputs:
>>>>> - Exhausted blocks and sinks, including their approximate location
>>>>> - Aborted blocks and their statement locations
>>>>> - When and where the analyzer reaches the max block limit
>>>>> Some supporting infrastructure changes:
>>>>> - CoreEngine now stores Stmt* with the ExplodedNode* in the aborted block data, since the node can change between generation and VisitEndAnalysis
>>> 
>>> My understanding is that, here, we add a bit of overhead on every analyzes run to make stats more precise. 
>>> Could you elaborate/investigate a bit more on why it is absolutely necessary in this case? Is the ExplodedNode you are storing here deleted by the time you reach VisitEndAnalysis? (How is the node changed?)
>>> 
>>>>> - Exposed 'block limit exceeded' tag to allow AnalyzerStatsChecker to detect a sink caused by exceeding the max visits for a block
>>>>> 
>>>>> Regarding test cases, any test case made for this checker would break quite frequently as the analyzer is changed. I can't think of a good way to solve this.
>>>>> 
>>> 
>>> Would be great if you could (manually) test this with -analyzer-inline-call.
>>> 
>>>>> Been a while since my last commit, so a review is much appreciated :)
>>>>> 
>>>>> Tom
>>>>> <AnalyzerStatsChecker.patch>_______________________________________________
>>>>> cfe-commits mailing list
>>>>> cfe-commits at cs.uiuc.edu
>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>>>> 
>>>> _______________________________________________
>>>> cfe-commits mailing list
>>>> cfe-commits at cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>>> 
>>> 
>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20120224/ef81589f/attachment.html>