[cfe-dev] [GSoC 2018] Using the Z3 SMT Solver to Validate Bugs Reported by the Clang Static Analyzer
Mikhail Ramalho via cfe-dev
cfe-dev at lists.llvm.org
Thu May 24 07:11:15 PDT 2018
Hi all,
This is my first report to the community, comparing the results with and
without the Z3 refutation when analyzing a number of projects.
~
First of all, I'd like to thank Réka Kovács as the first version of the
refutation using Z3 was created by her (https://reviews.llvm.org/D45517)!
Thank you very much!
After applying patch D45517, you can use the refutation check by
using -analyzer-config crosscheck-with-z3=true. Obviously, you need a
version of clang built with Z3.
~
I'm currently analyzing 7 C projects (unfortunately, there's a bug
preventing us from analyzing FFmpeg):
1. Tmux
2. Redis
3. OpenSSL
4. Twin
5. Git
6. Postgresql
7. Sqlite3
In short, the refutation check slows down the verification by ~20%. It
removed 6 FPs from Redis, 6 FPs from git and 1 FP from Sqlite3 (FP means
false positive). Surprisingly enough, some analysis were faster with the
crosscheck, but I'm not sure why (maybe ccache?).
Attached is a spreadsheet (report1.ods) with some number (total time,
average time per check, # of reported bugs) and a txt with all the raw data
from the analysis (raw.txt). I'll add these data to google drive for the
next report.
In order to generate the raw data, you need to use a version of clang with
assertions enabled, call scan-build.py with '-analyzer-config
serialize-stats=true' and you need to apply patch
https://reviews.llvm.org/D43134.
Thank you very much,
2018-05-01 15:27 GMT+01:00 Mikhail Ramalho <mikhail.ramalho at gmail.com>:
> Hello all,
>
> My proposal for GSoC 2018 [0] about Bug Validation in the Clang Static
> Analyzer using the Z3 SMT Solver was accepted.
>
> I'll work with George Karpenkov to improve the bug reports that the static
> analyzer produces by reducing the number of false bugs.
>
> Thank you,
>
> [0] https://docs.google.com/document/d/1-zNSv0l4WyoxYpJUAw8L
> FnQq_TY4AGjIpPu1VPkmO-g
>
> --
>
> Mikhail Ramalho.
>
--
Mikhail Ramalho.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20180524/16ecc667/attachment.html>
-------------- next part --------------
AnalysisConsumer.MaxCFGSize
max 244.000 -> 244.000
mean 98.000 -> 98.000
median 64.000 -> 64.000
min 27.000 -> 27.000
AnalysisConsumer.NumBlocksInAnalyzedFunctions
max 1938.000 -> 1938.000
mean 523.000 -> 523.000
median 294.000 -> 294.000
min 55.000 -> 55.000
AnalysisConsumer.NumFunctionTopLevel
max 131.000 -> 131.000
mean 54.000 -> 54.000
median 48.000 -> 48.000
min 28.000 -> 28.000
AnalysisConsumer.NumFunctionsAnalyzed
max 57.000 -> 57.000
mean 13.000 -> 13.000
median 10.000 -> 10.000
min 1.000 -> 1.000
AnalysisConsumer.NumVisitedBlocksInAnalyzedFunctions
max 1701.000 -> 1701.000
mean 411.000 -> 411.000
median 183.000 -> 183.000
min 52.000 -> 52.000
AnalysisConsumer.PercentReachableBlocks
max 94.000 -> 94.000
mean 77.000 -> 77.000
median 82.000 -> 82.000
min 47.000 -> 47.000
BugReporter.MaxBugClassSize
max 571.000 -> 571.000
mean 85.000 -> 85.000
median 44.000 -> 44.000
min 1.000 -> 1.000
BugReporter.MaxValidBugClassSize
max 571.000 -> 571.000
mean 76.000 -> 76.000
median 6.000 -> 6.000
min 1.000 -> 1.000
CoreEngine.NumPathsExplored
max 1106.000 -> 1106.000
mean 277.000 -> 277.000
median 166.000 -> 166.000
min 8.000 -> 8.000
CoreEngine.NumReachedMaxSteps
max 9.000 -> 9.000
mean 3.000 -> 3.000
median 3.000 -> 3.000
min 1.000 -> 1.000
CoreEngine.NumSteps
max 2759088.000 -> 2759088.000
mean 558118.000 -> 558118.000
median 232619.000 -> 232619.000
min 29544.000 -> 29544.000
ExprEngine.NumInlinedCalls
max 8245.000 -> 8245.000
mean 1474.000 -> 1474.000
median 800.000 -> 800.000
min 1.000 -> 1.000
ExprEngine.NumMaxBlockCountReached
max 10177.000 -> 10177.000
mean 1227.000 -> 1227.000
median 499.000 -> 499.000
min 7.000 -> 7.000
ExprEngine.NumReachedInlineCountMax
max 897.000 -> 897.000
mean 428.000 -> 428.000
median 465.000 -> 465.000
min 20.000 -> 20.000
ExprEngine.NumRemoveDeadBindings
max 475759.000 -> 475759.000
mean 95749.000 -> 95749.000
median 46844.000 -> 46844.000
min 4454.000 -> 4454.000
ExprEngine.NumTimesRetriedWithoutInlining
max 192.000 -> 192.000
mean 30.000 -> 30.000
median 17.000 -> 17.000
min 1.000 -> 1.000
time.analyzer.time.sys
max 0.237 -> 0.313
mean 0.076 -> 0.074
median 0.050 -> 0.047
min 0.007 -> 0.008
time.analyzer.time.user
max 27.253 -> 56.251
mean 7.405 -> 11.281
median 4.044 -> 3.875
min 0.410 -> 0.431
time.analyzer.time.wall
max 27.701 -> 56.788
mean 7.656 -> 11.505
median 4.195 -> 3.949
min 0.442 -> 0.440
TOTAL REPORTS: 32
TOTAL ADDED: 0
TOTAL REMOVED: 0
Diagnostic comparison complete (time: 0.34).
Completed tests for project tmux (time: 153.36).
--- Building project redis
Log file: /home/mramalho/llvm/tools/clang/utils/analyzer/redis/ScanBuildResults/Logs/run_static_analyzer.log
Output directory: /home/mramalho/llvm/tools/clang/utils/analyzer/redis/ScanBuildResults
No local patches.
Executing: /home/mramalho/llvm/tools/clang/utils/analyzer/redis/cleanup_run_static_analyzer.sh
Build complete (time: 193.43). See the log for more details: /home/mramalho/llvm/tools/clang/utils/analyzer/redis/ScanBuildResults/Logs/run_static_analyzer.log
Number of bug reports (non-empty plist files) produced: 59
AnalysisConsumer.MaxCFGSize
max 3152.000 -> 3152.000
mean 169.000 -> 169.000
median 60.000 -> 60.000
min 3.000 -> 3.000
AnalysisConsumer.NumBlocksInAnalyzedFunctions
max 5347.000 -> 5347.000
mean 706.000 -> 706.000
median 342.000 -> 342.000
min 0.000 -> 0.000
AnalysisConsumer.NumFunctionTopLevel
max 410.000 -> 410.000
mean 148.000 -> 148.000
median 91.000 -> 91.000
min 2.000 -> 2.000
AnalysisConsumer.NumFunctionsAnalyzed
max 141.000 -> 141.000
mean 15.000 -> 15.000
median 8.000 -> 8.000
min 1.000 -> 1.000
AnalysisConsumer.NumVisitedBlocksInAnalyzedFunctions
max 2899.000 -> 2899.000
mean 490.000 -> 490.000
median 282.000 -> 282.000
min 0.000 -> 0.000
AnalysisConsumer.PercentReachableBlocks
max 100.000 -> 100.000
mean 78.000 -> 78.000
median 83.000 -> 83.000
min 31.000 -> 31.000
BugReporter.MaxBugClassSize
max 499.000 -> 499.000
mean 56.000 -> 56.000
median 16.000 -> 16.000
min 1.000 -> 1.000
BugReporter.MaxValidBugClassSize
max 338.000 -> 338.000
mean 35.000 -> 35.000
median 8.000 -> 7.000
min 1.000 -> 1.000
CoreEngine.NumPathsExplored
max 1674.000 -> 1674.000
mean 238.000 -> 238.000
median 63.000 -> 63.000
min 1.000 -> 1.000
CoreEngine.NumReachedMaxSteps
max 15.000 -> 15.000
mean 3.000 -> 3.000
median 2.000 -> 2.000
min 1.000 -> 1.000
CoreEngine.NumSteps
max 4019007.000 -> 4019007.000
mean 549749.000 -> 549749.000
median 236569.000 -> 236569.000
min 23.000 -> 23.000
ExprEngine.NumInlinedCalls
max 44448.000 -> 44448.000
mean 5195.000 -> 5195.000
median 1647.000 -> 1647.000
min 1.000 -> 1.000
ExprEngine.NumMaxBlockCountReached
max 4872.000 -> 4872.000
mean 469.000 -> 469.000
median 197.000 -> 197.000
min 2.000 -> 2.000
ExprEngine.NumReachedInlineCountMax
max 14769.000 -> 14772.000
mean 2376.000 -> 2377.000
median 839.000 -> 839.000
min 20.000 -> 20.000
ExprEngine.NumRemoveDeadBindings
max 703505.000 -> 703505.000
mean 103328.000 -> 103329.000
median 46312.000 -> 46312.000
min 9.000 -> 9.000
ExprEngine.NumTimesRetriedWithoutInlining
max 109.000 -> 109.000
mean 23.000 -> 23.000
median 13.000 -> 13.000
min 1.000 -> 1.000
time.analyzer.time.sys
max 0.337 -> 0.299
mean 0.057 -> 0.057
median 0.039 -> 0.036
min 0.000 -> 0.000
time.analyzer.time.user
max 42.719 -> 46.861
mean 7.083 -> 7.621
median 3.643 -> 2.782
min 0.001 -> 0.002
time.analyzer.time.wall
max 43.856 -> 47.165
mean 7.271 -> 7.728
median 3.729 -> 2.834
min 0.002 -> 0.003
REMOVED: 'src/chunk.c#chunk_recycle:271:7, Logic error: Called function pointer is null (null dereference)'
REMOVED: 'src/chunk.c#chunk_recycle:298:20, Logic error: Called function pointer is null (null dereference)'
REMOVED: 'src/chunk.c#chunk_recycle:252:6, Logic error: Called function pointer is null (null dereference)'
REMOVED: 'src/arena.c#arena_avail_tree_remove:202:1, Logic error: Dereference of undefined pointer value'
REMOVED: 'include/jemalloc/internal/jemalloc_internal.h#je_size2index_compute:552:38, Logic error: The result of the left shift is undefined because the left operand is negative'
REMOVED: 'include/jemalloc/internal/jemalloc_internal.h#je_size2index_compute:552:38, Logic error: The result of the left shift is undefined because the left operand is negative'
TOTAL REPORTS: 140
TOTAL ADDED: 0
TOTAL REMOVED: 6
Warning: 6 differences in diagnostics.
Diagnostic comparison complete (time: 1.21).
Completed tests for project redis (time: 195.11).
--- Building project openssl
Log file: /home/mramalho/llvm/tools/clang/utils/analyzer/openssl/ScanBuildResults/Logs/run_static_analyzer.log
Output directory: /home/mramalho/llvm/tools/clang/utils/analyzer/openssl/ScanBuildResults
No local patches.
Executing: /home/mramalho/llvm/tools/clang/utils/analyzer/openssl/cleanup_run_static_analyzer.sh
Build complete (time: 213.53). See the log for more details: /home/mramalho/llvm/tools/clang/utils/analyzer/openssl/ScanBuildResults/Logs/run_static_analyzer.log
Number of bug reports (non-empty plist files) produced: 105
AnalysisConsumer.MaxCFGSize
max 3043.000 -> 3043.000
mean 157.000 -> 157.000
median 58.000 -> 58.000
min 3.000 -> 3.000
AnalysisConsumer.NumBlocksInAnalyzedFunctions
max 4130.000 -> 4130.000
mean 447.000 -> 447.000
median 170.000 -> 170.000
min 0.000 -> 0.000
AnalysisConsumer.NumFunctionTopLevel
max 1915.000 -> 1915.000
mean 635.000 -> 635.000
median 395.000 -> 395.000
min 2.000 -> 2.000
AnalysisConsumer.NumFunctionsAnalyzed
max 245.000 -> 245.000
mean 10.000 -> 10.000
median 4.000 -> 4.000
min 1.000 -> 1.000
AnalysisConsumer.NumVisitedBlocksInAnalyzedFunctions
max 2272.000 -> 2272.000
mean 305.000 -> 305.000
median 153.000 -> 153.000
min 0.000 -> 0.000
AnalysisConsumer.PercentReachableBlocks
max 100.000 -> 100.000
mean 84.000 -> 84.000
median 92.000 -> 92.000
min 34.000 -> 34.000
BugReporter.MaxBugClassSize
max 201.000 -> 201.000
mean 38.000 -> 38.000
median 12.000 -> 12.000
min 1.000 -> 1.000
BugReporter.MaxValidBugClassSize
max 201.000 -> 201.000
mean 38.000 -> 38.000
median 12.000 -> 12.000
min 1.000 -> 1.000
CoreEngine.NumPathsExplored
max 1159.000 -> 1159.000
mean 82.000 -> 82.000
median 28.000 -> 28.000
min 1.000 -> 1.000
CoreEngine.NumReachedMaxSteps
max 4.000 -> 4.000
mean 1.000 -> 1.000
median 1.000 -> 1.000
min 1.000 -> 1.000
CoreEngine.NumSteps
max 916708.000 -> 916708.000
mean 207945.000 -> 207945.000
median 183915.000 -> 183915.000
min 236.000 -> 236.000
ExprEngine.NumInlinedCalls
max 8344.000 -> 8344.000
mean 1539.000 -> 1539.000
median 928.000 -> 928.000
min 1.000 -> 1.000
ExprEngine.NumMaxBlockCountReached
max 2682.000 -> 2682.000
mean 341.000 -> 341.000
median 76.000 -> 76.000
min 1.000 -> 1.000
ExprEngine.NumReachedInlineCountMax
max 5335.000 -> 5335.000
mean 914.000 -> 914.000
median 399.000 -> 399.000
min 29.000 -> 29.000
ExprEngine.NumRemoveDeadBindings
max 193805.000 -> 193805.000
mean 38643.000 -> 38643.000
median 34604.000 -> 34604.000
min 29.000 -> 29.000
ExprEngine.NumTimesRetriedWithoutInlining
max 77.000 -> 77.000
mean 12.000 -> 12.000
median 6.000 -> 6.000
min 1.000 -> 1.000
time.analyzer.time.sys
max 0.177 -> 0.154
mean 0.042 -> 0.035
median 0.037 -> 0.028
min 0.000 -> 0.000
time.analyzer.time.user
max 45.252 -> 39.195
mean 3.310 -> 3.099
median 2.182 -> 1.916
min 0.003 -> 0.005
time.analyzer.time.wall
max 46.526 -> 39.480
mean 3.412 -> 3.152
median 2.232 -> 1.955
min 0.005 -> 0.005
TOTAL REPORTS: 204
TOTAL ADDED: 0
TOTAL REMOVED: 0
Diagnostic comparison complete (time: 1.25).
Completed tests for project openssl (time: 215.19).
--- Building project twin
Log file: /home/mramalho/llvm/tools/clang/utils/analyzer/twin/ScanBuildResults/Logs/run_static_analyzer.log
Output directory: /home/mramalho/llvm/tools/clang/utils/analyzer/twin/ScanBuildResults
No local patches.
Executing: /home/mramalho/llvm/tools/clang/utils/analyzer/twin/cleanup_run_static_analyzer.sh
Build complete (time: 133.83). See the log for more details: /home/mramalho/llvm/tools/clang/utils/analyzer/twin/ScanBuildResults/Logs/run_static_analyzer.log
Number of bug reports (non-empty plist files) produced: 37
AnalysisConsumer.MaxCFGSize
max 344.000 -> 344.000
mean 127.000 -> 127.000
median 116.000 -> 116.000
min 14.000 -> 14.000
AnalysisConsumer.NumBlocksInAnalyzedFunctions
max 1875.000 -> 1875.000
mean 655.000 -> 655.000
median 335.000 -> 335.000
min 66.000 -> 66.000
AnalysisConsumer.NumFunctionTopLevel
max 238.000 -> 238.000
mean 50.000 -> 50.000
median 37.000 -> 37.000
min 4.000 -> 4.000
AnalysisConsumer.NumFunctionsAnalyzed
max 118.000 -> 118.000
mean 17.000 -> 17.000
median 8.000 -> 8.000
min 1.000 -> 1.000
AnalysisConsumer.NumVisitedBlocksInAnalyzedFunctions
max 1670.000 -> 1670.000
mean 593.000 -> 593.000
median 292.000 -> 292.000
min 57.000 -> 57.000
AnalysisConsumer.PercentReachableBlocks
max 97.000 -> 97.000
mean 90.000 -> 90.000
median 92.000 -> 92.000
min 80.000 -> 80.000
BugReporter.MaxBugClassSize
max 386.000 -> 386.000
mean 76.000 -> 76.000
median 40.000 -> 40.000
min 1.000 -> 1.000
BugReporter.MaxValidBugClassSize
max 386.000 -> 386.000
mean 72.000 -> 72.000
median 32.000 -> 32.000
min 1.000 -> 1.000
CoreEngine.NumPathsExplored
max 2796.000 -> 2796.000
mean 677.000 -> 676.000
median 411.000 -> 411.000
min 9.000 -> 9.000
CoreEngine.NumReachedMaxSteps
max 9.000 -> 9.000
mean 2.000 -> 2.000
median 2.000 -> 2.000
min 1.000 -> 1.000
CoreEngine.NumSteps
max 2254531.000 -> 2254531.000
mean 667515.000 -> 667216.000
median 466417.000 -> 466417.000
min 2802.000 -> 2802.000
ExprEngine.NumInlinedCalls
max 16309.000 -> 16309.000
mean 2308.000 -> 2304.000
median 994.000 -> 994.000
min 4.000 -> 4.000
ExprEngine.NumMaxBlockCountReached
max 6300.000 -> 6300.000
mean 783.000 -> 783.000
median 186.000 -> 186.000
min 1.000 -> 1.000
ExprEngine.NumReachedInlineCountMax
max 8049.000 -> 8049.000
mean 1350.000 -> 1350.000
median 465.000 -> 465.000
min 10.000 -> 10.000
ExprEngine.NumRemoveDeadBindings
max 364166.000 -> 364166.000
mean 107047.000 -> 106978.000
median 66720.000 -> 66720.000
min 723.000 -> 723.000
ExprEngine.NumTimesRetriedWithoutInlining
max 80.000 -> 80.000
mean 29.000 -> 29.000
median 15.000 -> 15.000
min 6.000 -> 6.000
time.analyzer.time.sys
max 0.204 -> 0.239
mean 0.067 -> 0.060
median 0.051 -> 0.045
min 0.000 -> 0.000
time.analyzer.time.user
max 19.082 -> 18.453
mean 6.593 -> 6.790
median 4.123 -> 5.559
min 0.034 -> 0.031
time.analyzer.time.wall
max 19.227 -> 18.666
mean 6.696 -> 6.882
median 4.203 -> 5.628
min 0.036 -> 0.032
TOTAL REPORTS: 138
TOTAL ADDED: 0
TOTAL REMOVED: 0
Diagnostic comparison complete (time: 1.10).
Completed tests for project twin (time: 135.25).
--- Building project git
Log file: /home/mramalho/llvm/tools/clang/utils/analyzer/git/ScanBuildResults/Logs/run_static_analyzer.log
Output directory: /home/mramalho/llvm/tools/clang/utils/analyzer/git/ScanBuildResults
No local patches.
Executing: /home/mramalho/llvm/tools/clang/utils/analyzer/git/cleanup_run_static_analyzer.sh
Build complete (time: 289.59). See the log for more details: /home/mramalho/llvm/tools/clang/utils/analyzer/git/ScanBuildResults/Logs/run_static_analyzer.log
Number of bug reports (non-empty plist files) produced: 60
AnalysisConsumer.MaxCFGSize
max 4040.000 -> 4040.000
mean 332.000 -> 345.000
median 134.000 -> 147.000
min 15.000 -> 15.000
AnalysisConsumer.NumBlocksInAnalyzedFunctions
max 7784.000 -> 7784.000
mean 1169.000 -> 1178.000
median 760.000 -> 760.000
min 11.000 -> 11.000
AnalysisConsumer.NumFunctionTopLevel
max 345.000 -> 345.000
mean 199.000 -> 197.000
median 191.000 -> 191.000
min 144.000 -> 144.000
AnalysisConsumer.NumFunctionsAnalyzed
max 57.000 -> 47.000
mean 13.000 -> 12.000
median 9.000 -> 9.000
min 1.000 -> 1.000
AnalysisConsumer.NumVisitedBlocksInAnalyzedFunctions
max 3720.000 -> 3720.000
mean 692.000 -> 692.000
median 540.000 -> 540.000
min 8.000 -> 8.000
AnalysisConsumer.PercentReachableBlocks
max 96.000 -> 96.000
mean 66.000 -> 65.000
median 66.000 -> 66.000
min 39.000 -> 39.000
BugReporter.MaxBugClassSize
max 304.000 -> 304.000
mean 48.000 -> 47.000
median 17.000 -> 17.000
min 1.000 -> 1.000
BugReporter.MaxValidBugClassSize
max 304.000 -> 304.000
mean 46.000 -> 46.000
median 12.000 -> 12.000
min 1.000 -> 1.000
CoreEngine.NumPathsExplored
max 2446.000 -> 1399.000
mean 267.000 -> 220.000
median 140.000 -> 140.000
min 1.000 -> 1.000
CoreEngine.NumReachedMaxSteps
max 13.000 -> 13.000
mean 2.000 -> 2.000
median 2.000 -> 2.000
min 1.000 -> 1.000
CoreEngine.NumSteps
max 3234601.000 -> 3234601.000
mean 625419.000 -> 615461.000
median 454900.000 -> 454900.000
min 186.000 -> 186.000
ExprEngine.NumInlinedCalls
max 31453.000 -> 31453.000
mean 6010.000 -> 5938.000
median 4903.000 -> 4661.000
min 2.000 -> 2.000
ExprEngine.NumMaxBlockCountReached
max 3370.000 -> 3370.000
mean 445.000 -> 403.000
median 226.000 -> 187.000
min 3.000 -> 3.000
ExprEngine.NumReachedInlineCountMax
max 7133.000 -> 7133.000
mean 852.000 -> 848.000
median 449.000 -> 432.000
min 1.000 -> 1.000
ExprEngine.NumRemoveDeadBindings
max 620230.000 -> 620230.000
mean 119594.000 -> 116858.000
median 90917.000 -> 90917.000
min 27.000 -> 27.000
ExprEngine.NumTimesRetriedWithoutInlining
max 187.000 -> 187.000
mean 38.000 -> 37.000
median 25.000 -> 25.000
min 1.000 -> 1.000
time.analyzer.time.sys
max 0.281 -> 0.201
mean 0.075 -> 0.062
median 0.060 -> 0.057
min 0.003 -> 0.003
time.analyzer.time.user
max 42.402 -> 43.366
mean 8.520 -> 7.924
median 6.741 -> 6.662
min 0.035 -> 0.037
time.analyzer.time.wall
max 43.006 -> 43.678
mean 8.670 -> 8.023
median 6.838 -> 6.726
min 0.039 -> 0.040
REMOVED: "diffcore-pickaxe.c#diff_grep:49:31, Logic error: Access to field 'ptr' results in a dereference of a null pointer (loaded from variable 'two')"
REMOVED: 'commit.c#remove_redundant:910:7, Logic error: Dereference of null pointer'
REMOVED: 'commit.c#paint_down_to_common:785:25, Logic error: Dereference of null pointer'
REMOVED: 'commit.c#paint_down_to_common:777:20, Logic error: Dereference of null pointer'
REMOVED: 'combine-diff.c#coalesce_lines:284:2, Memory error: Use of memory after it is freed'
REMOVED: "./cache.h#hashcpy:992:2, API: Null pointer passed as an argument to a 'nonnull' parameter"
TOTAL REPORTS: 90
TOTAL ADDED: 0
TOTAL REMOVED: 6
Warning: 6 differences in diagnostics.
Diagnostic comparison complete (time: 1.18).
Completed tests for project git (time: 291.23).
--- Building project postgresql
Log file: /home/mramalho/llvm/tools/clang/utils/analyzer/postgresql/ScanBuildResults/Logs/run_static_analyzer.log
Output directory: /home/mramalho/llvm/tools/clang/utils/analyzer/postgresql/ScanBuildResults
No local patches.
Executing: /home/mramalho/llvm/tools/clang/utils/analyzer/postgresql/cleanup_run_static_analyzer.sh
Build complete (time: 902.86). See the log for more details: /home/mramalho/llvm/tools/clang/utils/analyzer/postgresql/ScanBuildResults/Logs/run_static_analyzer.log
Number of bug reports (non-empty plist files) produced: 221
AnalysisConsumer.MaxCFGSize
max 2883.000 -> 2883.000
mean 215.000 -> 215.000
median 126.000 -> 126.000
min 5.000 -> 5.000
AnalysisConsumer.NumBlocksInAnalyzedFunctions
max 8748.000 -> 8748.000
mean 1019.000 -> 1019.000
median 593.000 -> 593.000
min 5.000 -> 5.000
AnalysisConsumer.NumFunctionTopLevel
max 331.000 -> 331.000
mean 112.000 -> 112.000
median 112.000 -> 112.000
min 29.000 -> 29.000
AnalysisConsumer.NumFunctionsAnalyzed
max 206.000 -> 206.000
mean 17.000 -> 17.000
median 9.000 -> 9.000
min 1.000 -> 1.000
AnalysisConsumer.NumVisitedBlocksInAnalyzedFunctions
max 4571.000 -> 4571.000
mean 707.000 -> 707.000
median 437.000 -> 437.000
min 4.000 -> 4.000
AnalysisConsumer.PercentReachableBlocks
max 100.000 -> 100.000
mean 75.000 -> 75.000
median 80.000 -> 80.000
min 32.000 -> 32.000
BugReporter.MaxBugClassSize
max 622.000 -> 622.000
mean 57.000 -> 57.000
median 29.000 -> 29.000
min 1.000 -> 1.000
BugReporter.MaxValidBugClassSize
max 622.000 -> 622.000
mean 39.000 -> 39.000
median 21.000 -> 21.000
min 1.000 -> 1.000
CoreEngine.NumPathsExplored
max 4235.000 -> 4235.000
mean 301.000 -> 301.000
median 101.000 -> 101.000
min 7.000 -> 7.000
CoreEngine.NumReachedMaxSteps
max 24.000 -> 24.000
mean 3.000 -> 3.000
median 2.000 -> 2.000
min 1.000 -> 1.000
CoreEngine.NumSteps
max 6265096.000 -> 6265096.000
mean 685303.000 -> 685296.000
median 450190.000 -> 450190.000
min 557.000 -> 557.000
ExprEngine.NumInlinedCalls
max 18815.000 -> 18815.000
mean 2612.000 -> 2612.000
median 1344.000 -> 1344.000
min 1.000 -> 1.000
ExprEngine.NumMaxBlockCountReached
max 10649.000 -> 10649.000
mean 921.000 -> 921.000
median 218.000 -> 218.000
min 1.000 -> 1.000
ExprEngine.NumReachedInlineCountMax
max 9455.000 -> 9439.000
mean 1216.000 -> 1216.000
median 675.000 -> 675.000
min 7.000 -> 7.000
ExprEngine.NumRemoveDeadBindings
max 992208.000 -> 992208.000
mean 122631.000 -> 122630.000
median 76747.000 -> 76747.000
min 101.000 -> 101.000
ExprEngine.NumTimesRetriedWithoutInlining
max 229.000 -> 229.000
mean 33.000 -> 33.000
median 19.000 -> 19.000
min 1.000 -> 1.000
time.analyzer.time.sys
max 0.525 -> 0.539
mean 0.079 -> 0.077
median 0.057 -> 0.055
min 0.000 -> 0.000
time.analyzer.time.user
max 82.656 -> 81.805
mean 8.482 -> 9.694
median 5.277 -> 5.348
min 0.012 -> 0.012
time.analyzer.time.wall
max 85.166 -> 83.548
mean 8.631 -> 9.863
median 5.445 -> 5.435
min 0.013 -> 0.014
TOTAL REPORTS: 676
TOTAL ADDED: 0
TOTAL REMOVED: 0
Diagnostic comparison complete (time: 3.99).
Completed tests for project postgresql (time: 908.56).
--- Building project sqlite3
Log file: /home/mramalho/llvm/tools/clang/utils/analyzer/sqlite3/ScanBuildResults/Logs/run_static_analyzer.log
Output directory: /home/mramalho/llvm/tools/clang/utils/analyzer/sqlite3/ScanBuildResults
No local patches.
Executing: /home/mramalho/llvm/tools/clang/utils/analyzer/sqlite3/cleanup_run_static_analyzer.sh
Build complete (time: 1255.98). See the log for more details: /home/mramalho/llvm/tools/clang/utils/analyzer/sqlite3/ScanBuildResults/Logs/run_static_analyzer.log
Number of bug reports (non-empty plist files) produced: 4
AnalysisConsumer.MaxCFGSize
max 1302.000 -> 1302.000
mean 1214.000 -> 1214.000
median 1302.000 -> 1302.000
min 952.000 -> 952.000
AnalysisConsumer.NumBlocksInAnalyzedFunctions
max 36347.000 -> 36347.000
mean 28405.000 -> 28405.000
median 35850.000 -> 35850.000
min 5575.000 -> 5575.000
AnalysisConsumer.NumFunctionTopLevel
max 2502.000 -> 2502.000
mean 1934.000 -> 1934.000
median 2456.000 -> 2456.000
min 324.000 -> 324.000
AnalysisConsumer.NumFunctionsAnalyzed
max 605.000 -> 605.000
mean 475.000 -> 475.000
median 578.000 -> 578.000
min 139.000 -> 139.000
AnalysisConsumer.NumVisitedBlocksInAnalyzedFunctions
max 30042.000 -> 30042.000
mean 23470.000 -> 23470.000
median 29627.000 -> 29627.000
min 4584.000 -> 4584.000
AnalysisConsumer.PercentReachableBlocks
max 82.000 -> 82.000
mean 82.000 -> 82.000
median 82.000 -> 82.000
min 82.000 -> 82.000
BugReporter.MaxBugClassSize
max 213.000 -> 213.000
mean 163.000 -> 163.000
median 213.000 -> 213.000
min 16.000 -> 16.000
BugReporter.MaxValidBugClassSize
max 213.000 -> 213.000
mean 163.000 -> 163.000
median 213.000 -> 213.000
min 16.000 -> 16.000
CoreEngine.NumPathsExplored
max 11219.000 -> 11219.000
mean 8345.000 -> 8345.000
median 10510.000 -> 10510.000
min 1144.000 -> 1144.000
CoreEngine.NumReachedMaxSteps
max 192.000 -> 192.000
mean 143.000 -> 143.000
median 184.000 -> 184.000
min 12.000 -> 12.000
CoreEngine.NumSteps
max 48541096.000 -> 48541096.000
mean 36077727.000 -> 36077580.000
median 46119689.000 -> 46119101.000
min 3531025.000 -> 3531025.000
ExprEngine.NumInlinedCalls
max 794296.000 -> 794292.000
mean 577433.000 -> 577434.000
median 748979.000 -> 748986.000
min 17481.000 -> 17481.000
ExprEngine.NumMaxBlockCountReached
max 16502.000 -> 16490.000
mean 12844.000 -> 12841.000
median 16490.000 -> 16490.000
min 2634.000 -> 2634.000
ExprEngine.NumReachedInlineCountMax
max 65590.000 -> 65579.000
mean 49067.000 -> 49065.000
median 65578.000 -> 65578.000
min 1200.000 -> 1200.000
ExprEngine.NumRemoveDeadBindings
max 10734124.000 -> 10734117.000
mean 7925195.000 -> 7925142.000
median 10183177.000 -> 10183057.000
min 600423.000 -> 600423.000
ExprEngine.NumTimesRetriedWithoutInlining
max 1425.000 -> 1425.000
mean 1125.000 -> 1125.000
median 1371.000 -> 1371.000
min 333.000 -> 333.000
time.analyzer.time.sys
max 0.430 -> 0.664
mean 0.262 -> 0.430
median 0.258 -> 0.522
min 0.114 -> 0.046
time.analyzer.time.user
max 487.971 -> 557.899
mean 368.446 -> 407.763
median 478.111 -> 529.470
min 32.184 -> 32.014
time.analyzer.time.wall
max 490.523 -> 560.594
mean 370.486 -> 409.688
median 480.502 -> 531.777
min 32.465 -> 32.175
REMOVED: "sqlite3.c#sqlite3VdbeSwap:74310:9, Logic error: Dereference of null pointer (loaded from variable 'pA')"
TOTAL REPORTS: 199
TOTAL ADDED: 0
TOTAL REMOVED: 1
Warning: 1 differences in diagnostics.
Diagnostic comparison complete (time: 3.17).
Completed tests for project sqlite3 (time: 1260.85).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: report1.ods
Type: application/vnd.oasis.opendocument.spreadsheet
Size: 19243 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20180524/16ecc667/attachment.ods>
More information about the cfe-dev
mailing list