<div dir="ltr">Artem,<div><br></div><div>I think you broke the build. Could you take a look please?</div><div><br></div><div><a href="http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/11051">http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/11051</a><br></div><div><a href="http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/11051/steps/build%20clang/logs/stdio">http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/11051/steps/build%20clang/logs/stdio</a><br></div><div><br></div><div><pre style="font-family:"courier new",courier,monotype,monospace;font-size:medium;line-height:normal"><span class="inbox-inbox-stdout">[80/140] Building CXX object tools/clang/lib/StaticAnalyzer/Checkers/CMakeFiles/clangStaticAnalyzerCheckers.dir/CloneChecker.cpp.o
FAILED: tools/clang/lib/StaticAnalyzer/Checkers/CMakeFiles/clangStaticAnalyzerCheckers.dir/CloneChecker.cpp.o
/usr/bin/c++ -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_OBJC_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -DGTEST_HAS_RTTI=0 -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/clang/lib/StaticAnalyzer/Checkers -I/mnt/b/sanitizer-buildbo</span><span class="inbox-inbox-stdout">t5/sanitizer-x86_64-linux-fuzzer/build/llvm/tools/clang/lib/StaticAnalyzer/Checkers -I/mnt/b/sanitizer-buildbot5/sanitizer-x86_64-linux-fuzzer/build/llvm/tools/clang/include -Itools/clang/include -Iinclude -I/mnt/b/sanitizer-buildbot5/sanitizer-x86_64-linux-fuzzer/build/llvm/include -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -fno-strict-aliasing -O3 -DNDEBUG -fno-exceptions -fno-rtti -MMD -MT tools/clang/lib/StaticAnalyzer/Checkers/CMakeFiles/clangStaticAnalyzerCheckers.dir/CloneChecker.cpp.o -MF tools/clang/lib/StaticAnalyzer/Checkers/CMakeFiles/clangStaticAnalyzerCheckers.dir/CloneChecker.cpp.o.d -o tools/clang/lib/StaticAnalyzer/Checkers/CMakeFiles/clangStaticAnalyzerCheckers.dir/CloneChecker.cpp.o -c /mnt/b/sanitizer-buildbot5/sanitizer-x86_64-linux-fuzzer/build/llvm/tools/clang/lib/StaticAnalyzer/Checkers/CloneChecker.cpp
</span><span class="inbox-inbox-stdout">/mnt/b/sanitizer-buildbot5/sanitizer-x86_64-linux-fuzzer/build/llvm/tools/clang/lib/StaticAnalyzer/Checkers/CloneChecker.cpp:29:25: error: declaration of ‘clang::CloneDetector {anonymous}::CloneChecker::CloneDetector’ [-fpermissive]
mutable CloneDetector CloneDetector;
^
In file included from /mnt/b/sanitizer-buildbot5/sanitizer-x86_64-linux-fuzzer/build/llvm/tools/clang/lib/StaticAnalyzer/Checkers/CloneChecker.cpp:17:0:
/mnt/b/sanitizer-buildbot5/sanitizer-x86_64-linux-fuzzer/build/llvm/tools/clang/include/clang/Analysis/CloneDetection.h:158:7: error: changes meaning of ‘CloneDetector’ from ‘class clang::CloneDetector’ [-fpermissive]
class CloneDetector {
^
</span></pre><br class="inbox-inbox-Apple-interchange-newline"></div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Jul 26, 2016 at 11:28 AM Artem Dergachev via cfe-commits <<a href="mailto:cfe-commits@lists.llvm.org">cfe-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: dergachev<br>
Date: Tue Jul 26 13:13:12 2016<br>
New Revision: 276782<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=276782&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=276782&view=rev</a><br>
Log:<br>
[analyzer] Add basic capabilities to detect source code clones.<br>
<br>
This patch adds the CloneDetector class which allows searching source code<br>
for clones.<br>
<br>
For every statement or group of statements within a compound statement,<br>
CloneDetector computes a hash value, and finds clones by detecting<br>
identical hash values.<br>
<br>
This initial patch only provides a simple hashing mechanism<br>
that hashes the kind of each sub-statement.<br>
<br>
This patch also adds CloneChecker - a simple static analyzer checker<br>
that uses CloneDetector to report copy-pasted code.<br>
<br>
Patch by Raphael Isemann!<br>
<br>
Differential Revision: <a href="https://reviews.llvm.org/D20795" rel="noreferrer" target="_blank">https://reviews.llvm.org/D20795</a><br>
<br>
Added:<br>
cfe/trunk/include/clang/Analysis/CloneDetection.h<br>
cfe/trunk/lib/Analysis/CloneDetection.cpp<br>
cfe/trunk/lib/StaticAnalyzer/Checkers/CloneChecker.cpp<br>
cfe/trunk/test/Analysis/copypaste/<br>
cfe/trunk/test/Analysis/copypaste/blocks.cpp<br>
cfe/trunk/test/Analysis/copypaste/false-positives.cpp<br>
cfe/trunk/test/Analysis/copypaste/functions.cpp<br>
cfe/trunk/test/Analysis/copypaste/objc-methods.m<br>
cfe/trunk/test/Analysis/copypaste/sub-sequences.cpp<br>
Modified:<br>
cfe/trunk/include/clang/StaticAnalyzer/Checkers/Checkers.td<br>
cfe/trunk/lib/Analysis/CMakeLists.txt<br>
cfe/trunk/lib/StaticAnalyzer/Checkers/CMakeLists.txt<br>
<br>
Added: cfe/trunk/include/clang/Analysis/CloneDetection.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Analysis/CloneDetection.h?rev=276782&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Analysis/CloneDetection.h?rev=276782&view=auto</a><br>
==============================================================================<br>
--- cfe/trunk/include/clang/Analysis/CloneDetection.h (added)<br>
+++ cfe/trunk/include/clang/Analysis/CloneDetection.h Tue Jul 26 13:13:12 2016<br>
@@ -0,0 +1,235 @@<br>
+//===--- CloneDetection.h - Finds code clones in an AST ---------*- C++ -*-===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+///<br>
+/// /file<br>
+/// This file defines classes for searching and anlyzing source code clones.<br>
+///<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#ifndef LLVM_CLANG_AST_CLONEDETECTION_H<br>
+#define LLVM_CLANG_AST_CLONEDETECTION_H<br>
+<br>
+#include "clang/Basic/SourceLocation.h"<br>
+#include "llvm/ADT/StringMap.h"<br>
+<br>
+#include <vector><br>
+<br>
+namespace clang {<br>
+<br>
+class Stmt;<br>
+class Decl;<br>
+class ASTContext;<br>
+class CompoundStmt;<br>
+<br>
+/// \brief Identifies a list of statements.<br>
+///<br>
+/// Can either identify a single arbitrary Stmt object, a continuous sequence of<br>
+/// child statements inside a CompoundStmt or no statements at all.<br>
+class StmtSequence {<br>
+ /// If this object identifies a sequence of statements inside a CompoundStmt,<br>
+ /// S points to this CompoundStmt. If this object only identifies a single<br>
+ /// Stmt, then S is a pointer to this Stmt.<br>
+ const Stmt *S;<br>
+<br>
+ /// The related ASTContext for S.<br>
+ ASTContext *Context;<br>
+<br>
+ /// If EndIndex is non-zero, then S is a CompoundStmt and this StmtSequence<br>
+ /// instance is representing the CompoundStmt children inside the array<br>
+ /// [StartIndex, EndIndex).<br>
+ unsigned StartIndex;<br>
+ unsigned EndIndex;<br>
+<br>
+public:<br>
+ /// \brief Constructs a StmtSequence holding multiple statements.<br>
+ ///<br>
+ /// The resulting StmtSequence identifies a continuous sequence of statements<br>
+ /// in the body of the given CompoundStmt. Which statements of the body should<br>
+ /// be identified needs to be specified by providing a start and end index<br>
+ /// that describe a non-empty sub-array in the body of the given CompoundStmt.<br>
+ ///<br>
+ /// \param Stmt A CompoundStmt that contains all statements in its body.<br>
+ /// \param Context The ASTContext for the given CompoundStmt.<br>
+ /// \param StartIndex The inclusive start index in the children array of<br>
+ /// \p Stmt<br>
+ /// \param EndIndex The exclusive end index in the children array of \p Stmt.<br>
+ StmtSequence(const CompoundStmt *Stmt, ASTContext &Context,<br>
+ unsigned StartIndex, unsigned EndIndex);<br>
+<br>
+ /// \brief Constructs a StmtSequence holding a single statement.<br>
+ ///<br>
+ /// \param Stmt An arbitrary Stmt.<br>
+ /// \param Context The ASTContext for the given Stmt.<br>
+ StmtSequence(const Stmt *Stmt, ASTContext &Context);<br>
+<br>
+ /// \brief Constructs an empty StmtSequence.<br>
+ StmtSequence();<br>
+<br>
+ typedef const Stmt *const *iterator;<br>
+<br>
+ /// Returns an iterator pointing to the first statement in this sequence.<br>
+ iterator begin() const;<br>
+<br>
+ /// Returns an iterator pointing behind the last statement in this sequence.<br>
+ iterator end() const;<br>
+<br>
+ /// Returns the first statement in this sequence.<br>
+ ///<br>
+ /// This method should only be called on a non-empty StmtSequence object.<br>
+ const Stmt *front() const {<br>
+ assert(!empty());<br>
+ return begin()[0];<br>
+ }<br>
+<br>
+ /// Returns the last statement in this sequence.<br>
+ ///<br>
+ /// This method should only be called on a non-empty StmtSequence object.<br>
+ const Stmt *back() const {<br>
+ assert(!empty());<br>
+ return begin()[size() - 1];<br>
+ }<br>
+<br>
+ /// Returns the number of statements this object holds.<br>
+ unsigned size() const {<br>
+ if (holdsSequence())<br>
+ return EndIndex - StartIndex;<br>
+ if (S == nullptr)<br>
+ return 0;<br>
+ return 1;<br>
+ }<br>
+<br>
+ /// Returns true if and only if this StmtSequence contains no statements.<br>
+ bool empty() const { return size() == 0; }<br>
+<br>
+ /// Returns the related ASTContext for the stored Stmts.<br>
+ ASTContext &getASTContext() const {<br>
+ assert(Context);<br>
+ return *Context;<br>
+ }<br>
+<br>
+ /// Returns true if this objects holds a list of statements.<br>
+ bool holdsSequence() const { return EndIndex != 0; }<br>
+<br>
+ /// Returns the start sourcelocation of the first statement in this sequence.<br>
+ ///<br>
+ /// This method should only be called on a non-empty StmtSequence object.<br>
+ SourceLocation getStartLoc() const;<br>
+<br>
+ /// Returns the end sourcelocation of the last statement in this sequence.<br>
+ ///<br>
+ /// This method should only be called on a non-empty StmtSequence object.<br>
+ SourceLocation getEndLoc() const;<br>
+<br>
+ bool operator==(const StmtSequence &Other) const {<br>
+ return std::tie(S, StartIndex, EndIndex) ==<br>
+ std::tie(Other.S, Other.StartIndex, Other.EndIndex);<br>
+ }<br>
+<br>
+ bool operator!=(const StmtSequence &Other) const {<br>
+ return std::tie(S, StartIndex, EndIndex) !=<br>
+ std::tie(Other.S, Other.StartIndex, Other.EndIndex);<br>
+ }<br>
+<br>
+ /// Returns true if and only if this sequence covers a source range that<br>
+ /// contains the source range of the given sequence \p Other.<br>
+ ///<br>
+ /// This method should only be called on a non-empty StmtSequence object<br>
+ /// and passed a non-empty StmtSequence object.<br>
+ bool contains(const StmtSequence &Other) const;<br>
+};<br>
+<br>
+/// \brief Searches for clones in source code.<br>
+///<br>
+/// First, this class needs a translation unit which is passed via<br>
+/// \p analyzeTranslationUnit . It will then generate and store search data<br>
+/// for all statements inside the given translation unit.<br>
+/// Afterwards the generated data can be used to find code clones by calling<br>
+/// \p findClones .<br>
+///<br>
+/// This class only searches for clones in exectuable source code<br>
+/// (e.g. function bodies). Other clones (e.g. cloned comments or declarations)<br>
+/// are not supported.<br>
+class CloneDetector {<br>
+public:<br>
+ /// Holds the data about a StmtSequence that is needed during the search for<br>
+ /// code clones.<br>
+ struct CloneSignature {<br>
+ /// \brief Holds all relevant data of a StmtSequence.<br>
+ ///<br>
+ /// If this variable is equal for two different StmtSequences, then they can<br>
+ /// be considered clones of each other.<br>
+ std::vector<unsigned> Data;<br>
+<br>
+ /// \brief The complexity of the StmtSequence.<br>
+ ///<br>
+ /// This scalar value serves as a simple way of filtering clones that are<br>
+ /// too small to be reported. A greater value indicates that the related<br>
+ /// StmtSequence is probably more interesting to the user.<br>
+ unsigned Complexity;<br>
+<br>
+ /// \brief Creates an empty CloneSignature without any data.<br>
+ CloneSignature() : Complexity(1) {}<br>
+<br>
+ CloneSignature(const std::vector<unsigned> &Data, unsigned Complexity)<br>
+ : Data(Data), Complexity(Complexity) {}<br>
+<br>
+ /// \brief Adds the data from the given CloneSignature to this one.<br>
+ void add(const CloneSignature &Other) {<br>
+ Data.insert(Data.end(), Other.Data.begin(), Other.Data.end());<br>
+ Complexity += Other.Complexity;<br>
+ }<br>
+ };<br>
+<br>
+ /// Holds group of StmtSequences that are clones of each other and the<br>
+ /// complexity value (see CloneSignature::Complexity) that all stored<br>
+ /// StmtSequences have in common.<br>
+ struct CloneGroup {<br>
+ std::vector<StmtSequence> Sequences;<br>
+ unsigned Complexity;<br>
+<br>
+ CloneGroup(const StmtSequence &Seq, unsigned Complexity)<br>
+ : Complexity(Complexity) {<br>
+ Sequences.push_back(Seq);<br>
+ }<br>
+<br>
+ /// \brief Returns false if and only if this group should be skipped when<br>
+ /// searching for clones.<br>
+ bool isValid() const {<br>
+ // A clone group with only one member makes no sense, so we skip them.<br>
+ return Sequences.size() > 1;<br>
+ }<br>
+ };<br>
+<br>
+ /// \brief Generates and stores search data for all statements in the body of<br>
+ /// the given Decl.<br>
+ void analyzeCodeBody(const Decl *D);<br>
+<br>
+ /// \brief Stores the CloneSignature to allow future querying.<br>
+ void add(const StmtSequence &S, const CloneSignature &Signature);<br>
+<br>
+ /// \brief Searches the provided statements for clones.<br>
+ ///<br>
+ /// \param Result Output parameter that is filled with a list of found<br>
+ /// clone groups. Each group contains multiple StmtSequences<br>
+ /// that were identified to be clones of each other.<br>
+ /// \param MinGroupComplexity Only return clones which have at least this<br>
+ /// complexity value.<br>
+ void findClones(std::vector<CloneGroup> &Result, unsigned MinGroupComplexity);<br>
+<br>
+private:<br>
+ /// Stores all found clone groups including invalid groups with only a single<br>
+ /// statement.<br>
+ std::vector<CloneGroup> CloneGroups;<br>
+ /// Maps search data to its related index in the \p CloneGroups vector.<br>
+ llvm::StringMap<std::size_t> CloneGroupIndexes;<br>
+};<br>
+<br>
+} // end namespace clang<br>
+<br>
+#endif // LLVM_CLANG_AST_CLONEDETECTION_H<br>
<br>
Modified: cfe/trunk/include/clang/StaticAnalyzer/Checkers/Checkers.td<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/StaticAnalyzer/Checkers/Checkers.td?rev=276782&r1=276781&r2=276782&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/StaticAnalyzer/Checkers/Checkers.td?rev=276782&r1=276781&r2=276782&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/include/clang/StaticAnalyzer/Checkers/Checkers.td (original)<br>
+++ cfe/trunk/include/clang/StaticAnalyzer/Checkers/Checkers.td Tue Jul 26 13:13:12 2016<br>
@@ -77,6 +77,8 @@ def MPI : Package<"mpi">, InPackage<OptI<br>
def LLVM : Package<"llvm">;<br>
def Debug : Package<"debug">;<br>
<br>
+def CloneDetectionAlpha : Package<"clone">, InPackage<Alpha>, Hidden;<br>
+<br>
//===----------------------------------------------------------------------===//<br>
// Core Checkers.<br>
//===----------------------------------------------------------------------===//<br>
@@ -661,3 +663,17 @@ def BugHashDumper : Checker<"DumpBugHash<br>
DescFile<"DebugCheckers.cpp">;<br>
<br>
} // end "debug"<br>
+<br>
+<br>
+//===----------------------------------------------------------------------===//<br>
+// Clone Detection<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+let ParentPackage = CloneDetectionAlpha in {<br>
+<br>
+def CloneChecker : Checker<"CloneChecker">,<br>
+ HelpText<"Reports similar pieces of code.">,<br>
+ DescFile<"CloneChecker.cpp">;<br>
+<br>
+} // end "clone"<br>
+<br>
<br>
Modified: cfe/trunk/lib/Analysis/CMakeLists.txt<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Analysis/CMakeLists.txt?rev=276782&r1=276781&r2=276782&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Analysis/CMakeLists.txt?rev=276782&r1=276781&r2=276782&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/Analysis/CMakeLists.txt (original)<br>
+++ cfe/trunk/lib/Analysis/CMakeLists.txt Tue Jul 26 13:13:12 2016<br>
@@ -9,6 +9,7 @@ add_clang_library(clangAnalysis<br>
CFGReachabilityAnalysis.cpp<br>
CFGStmtMap.cpp<br>
CallGraph.cpp<br>
+ CloneDetection.cpp<br>
CocoaConventions.cpp<br>
Consumed.cpp<br>
CodeInjector.cpp<br>
<br>
Added: cfe/trunk/lib/Analysis/CloneDetection.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Analysis/CloneDetection.cpp?rev=276782&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Analysis/CloneDetection.cpp?rev=276782&view=auto</a><br>
==============================================================================<br>
--- cfe/trunk/lib/Analysis/CloneDetection.cpp (added)<br>
+++ cfe/trunk/lib/Analysis/CloneDetection.cpp Tue Jul 26 13:13:12 2016<br>
@@ -0,0 +1,277 @@<br>
+//===--- CloneDetection.cpp - Finds code clones in an AST -------*- C++ -*-===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+///<br>
+/// This file implements classes for searching and anlyzing source code clones.<br>
+///<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#include "clang/Analysis/CloneDetection.h"<br>
+<br>
+#include "clang/AST/ASTContext.h"<br>
+#include "clang/AST/RecursiveASTVisitor.h"<br>
+#include "clang/AST/Stmt.h"<br>
+#include "llvm/ADT/StringRef.h"<br>
+<br>
+using namespace clang;<br>
+<br>
+StmtSequence::StmtSequence(const CompoundStmt *Stmt, ASTContext &Context,<br>
+ unsigned StartIndex, unsigned EndIndex)<br>
+ : S(Stmt), Context(&Context), StartIndex(StartIndex), EndIndex(EndIndex) {<br>
+ assert(Stmt && "Stmt must not be a nullptr");<br>
+ assert(StartIndex < EndIndex && "Given array should not be empty");<br>
+ assert(EndIndex <= Stmt->size() && "Given array too big for this Stmt");<br>
+}<br>
+<br>
+StmtSequence::StmtSequence(const Stmt *Stmt, ASTContext &Context)<br>
+ : S(Stmt), Context(&Context), StartIndex(0), EndIndex(0) {}<br>
+<br>
+StmtSequence::StmtSequence()<br>
+ : S(nullptr), Context(nullptr), StartIndex(0), EndIndex(0) {}<br>
+<br>
+bool StmtSequence::contains(const StmtSequence &Other) const {<br>
+ // If both sequences reside in different translation units, they can never<br>
+ // contain each other.<br>
+ if (Context != Other.Context)<br>
+ return false;<br>
+<br>
+ const SourceManager &SM = Context->getSourceManager();<br>
+<br>
+ // Otherwise check if the start and end locations of the current sequence<br>
+ // surround the other sequence.<br>
+ bool StartIsInBounds =<br>
+ SM.isBeforeInTranslationUnit(getStartLoc(), Other.getStartLoc()) ||<br>
+ getStartLoc() == Other.getStartLoc();<br>
+ if (!StartIsInBounds)<br>
+ return false;<br>
+<br>
+ bool EndIsInBounds =<br>
+ SM.isBeforeInTranslationUnit(Other.getEndLoc(), getEndLoc()) ||<br>
+ Other.getEndLoc() == getEndLoc();<br>
+ return EndIsInBounds;<br>
+}<br>
+<br>
+StmtSequence::iterator StmtSequence::begin() const {<br>
+ if (!holdsSequence()) {<br>
+ return &S;<br>
+ }<br>
+ auto CS = cast<CompoundStmt>(S);<br>
+ return CS->body_begin() + StartIndex;<br>
+}<br>
+<br>
+StmtSequence::iterator StmtSequence::end() const {<br>
+ if (!holdsSequence()) {<br>
+ return &S + 1;<br>
+ }<br>
+ auto CS = cast<CompoundStmt>(S);<br>
+ return CS->body_begin() + EndIndex;<br>
+}<br>
+<br>
+SourceLocation StmtSequence::getStartLoc() const {<br>
+ return front()->getLocStart();<br>
+}<br>
+<br>
+SourceLocation StmtSequence::getEndLoc() const { return back()->getLocEnd(); }<br>
+<br>
+namespace {<br>
+/// Generates CloneSignatures for a set of statements and stores the results in<br>
+/// a CloneDetector object.<br>
+class CloneSignatureGenerator {<br>
+<br>
+ CloneDetector &CD;<br>
+ ASTContext &Context;<br>
+<br>
+ /// \brief Generates CloneSignatures for all statements in the given statement<br>
+ /// tree and stores them in the CloneDetector.<br>
+ ///<br>
+ /// \param S The root of the given statement tree.<br>
+ /// \return The CloneSignature of the root statement.<br>
+ CloneDetector::CloneSignature generateSignatures(const Stmt *S) {<br>
+ // Create an empty signature that will be filled in this method.<br>
+ CloneDetector::CloneSignature Signature;<br>
+<br>
+ // The only relevant data for now is the class of the statement.<br>
+ // TODO: Collect statement class specific data.<br>
+ Signature.Data.push_back(S->getStmtClass());<br>
+<br>
+ // Storage for the signatures of the direct child statements. This is only<br>
+ // needed if the current statement is a CompoundStmt.<br>
+ std::vector<CloneDetector::CloneSignature> ChildSignatures;<br>
+ const CompoundStmt *CS = dyn_cast<const CompoundStmt>(S);<br>
+<br>
+ // The signature of a statement includes the signatures of its children.<br>
+ // Therefore we create the signatures for every child and add them to the<br>
+ // current signature.<br>
+ for (const Stmt *Child : S->children()) {<br>
+ // Some statements like 'if' can have nullptr children that we will skip.<br>
+ if (!Child)<br>
+ continue;<br>
+<br>
+ // Recursive call to create the signature of the child statement. This<br>
+ // will also create and store all clone groups in this child statement.<br>
+ auto ChildSignature = generateSignatures(Child);<br>
+<br>
+ // Add the collected data to the signature of the current statement.<br>
+ Signature.add(ChildSignature);<br>
+<br>
+ // If the current statement is a CompoundStatement, we need to store the<br>
+ // signature for the generation of the sub-sequences.<br>
+ if (CS)<br>
+ ChildSignatures.push_back(ChildSignature);<br>
+ }<br>
+<br>
+ // If the current statement is a CompoundStmt, we also need to create the<br>
+ // clone groups from the sub-sequences inside the children.<br>
+ if (CS)<br>
+ handleSubSequences(CS, ChildSignatures);<br>
+<br>
+ // Save the signature for the current statement in the CloneDetector object.<br>
+ CD.add(StmtSequence(S, Context), Signature);<br>
+<br>
+ return Signature;<br>
+ }<br>
+<br>
+ /// \brief Adds all possible sub-sequences in the child array of the given<br>
+ /// CompoundStmt to the CloneDetector.<br>
+ /// \param CS The given CompoundStmt.<br>
+ /// \param ChildSignatures A list of calculated signatures for each child in<br>
+ /// the given CompoundStmt.<br>
+ void handleSubSequences(<br>
+ const CompoundStmt *CS,<br>
+ const std::vector<CloneDetector::CloneSignature> &ChildSignatures) {<br>
+<br>
+ // FIXME: This function has quadratic runtime right now. Check if skipping<br>
+ // this function for too long CompoundStmts is an option.<br>
+<br>
+ // The length of the sub-sequence. We don't need to handle sequences with<br>
+ // the length 1 as they are already handled in CollectData().<br>
+ for (unsigned Length = 2; Length <= CS->size(); ++Length) {<br>
+ // The start index in the body of the CompoundStmt. We increase the<br>
+ // position until the end of the sub-sequence reaches the end of the<br>
+ // CompoundStmt body.<br>
+ for (unsigned Pos = 0; Pos <= CS->size() - Length; ++Pos) {<br>
+ // Create an empty signature and add the signatures of all selected<br>
+ // child statements to it.<br>
+ CloneDetector::CloneSignature SubSignature;<br>
+<br>
+ for (unsigned i = Pos; i < Pos + Length; ++i) {<br>
+ SubSignature.add(ChildSignatures[i]);<br>
+ }<br>
+<br>
+ // Save the signature together with the information about what children<br>
+ // sequence we selected.<br>
+ CD.add(StmtSequence(CS, Context, Pos, Pos + Length), SubSignature);<br>
+ }<br>
+ }<br>
+ }<br>
+<br>
+public:<br>
+ explicit CloneSignatureGenerator(CloneDetector &CD, ASTContext &Context)<br>
+ : CD(CD), Context(Context) {}<br>
+<br>
+ /// \brief Generates signatures for all statements in the given function body.<br>
+ void consumeCodeBody(const Stmt *S) { generateSignatures(S); }<br>
+};<br>
+} // end anonymous namespace<br>
+<br>
+void CloneDetector::analyzeCodeBody(const Decl *D) {<br>
+ assert(D);<br>
+ assert(D->hasBody());<br>
+ CloneSignatureGenerator Generator(*this, D->getASTContext());<br>
+ Generator.consumeCodeBody(D->getBody());<br>
+}<br>
+<br>
+void CloneDetector::add(const StmtSequence &S,<br>
+ const CloneSignature &Signature) {<br>
+ // StringMap only works with StringRefs, so we create one for our data vector.<br>
+ auto &Data = Signature.Data;<br>
+ StringRef DataRef = StringRef(reinterpret_cast<const char *>(Data.data()),<br>
+ Data.size() * sizeof(unsigned));<br>
+<br>
+ // Search with the help of the signature if we already have encountered a<br>
+ // clone of the given StmtSequence.<br>
+ auto I = CloneGroupIndexes.find(DataRef);<br>
+ if (I == CloneGroupIndexes.end()) {<br>
+ // We haven't found an existing clone group, so we create a new clone group<br>
+ // for this StmtSequence and store the index of it in our search map.<br>
+ CloneGroupIndexes[DataRef] = CloneGroups.size();<br>
+ CloneGroups.emplace_back(S, Signature.Complexity);<br>
+ return;<br>
+ }<br>
+<br>
+ // We have found an existing clone group and can expand it with the given<br>
+ // StmtSequence.<br>
+ CloneGroups[I->getValue()].Sequences.push_back(S);<br>
+}<br>
+<br>
+namespace {<br>
+/// \brief Returns true if and only if \p Stmt contains at least one other<br>
+/// sequence in the \p Group.<br>
+bool containsAnyInGroup(StmtSequence &Stmt,<br>
+ CloneDetector::CloneGroup &Group) {<br>
+ for (StmtSequence &GroupStmt : Group.Sequences) {<br>
+ if (Stmt.contains(GroupStmt))<br>
+ return true;<br>
+ }<br>
+ return false;<br>
+}<br>
+<br>
+/// \brief Returns true if and only if all sequences in \p OtherGroup are<br>
+/// contained by a sequence in \p Group.<br>
+bool containsGroup(CloneDetector::CloneGroup &Group,<br>
+ CloneDetector::CloneGroup &OtherGroup) {<br>
+ // We have less sequences in the current group than we have in the other,<br>
+ // so we will never fulfill the requirement for returning true. This is only<br>
+ // possible because we know that a sequence in Group can contain at most<br>
+ // one sequence in OtherGroup.<br>
+ if (Group.Sequences.size() < OtherGroup.Sequences.size())<br>
+ return false;<br>
+<br>
+ for (StmtSequence &Stmt : Group.Sequences) {<br>
+ if (!containsAnyInGroup(Stmt, OtherGroup))<br>
+ return false;<br>
+ }<br>
+ return true;<br>
+}<br>
+} // end anonymous namespace<br>
+<br>
+void CloneDetector::findClones(std::vector<CloneGroup> &Result,<br>
+ unsigned MinGroupComplexity) {<br>
+ // Add every valid clone group that fulfills the complexity requirement.<br>
+ for (const CloneGroup &Group : CloneGroups) {<br>
+ if (Group.isValid() && Group.Complexity >= MinGroupComplexity) {<br>
+ Result.push_back(Group);<br>
+ }<br>
+ }<br>
+<br>
+ std::vector<unsigned> IndexesToRemove;<br>
+<br>
+ // Compare every group in the result with the rest. If one groups contains<br>
+ // another group, we only need to return the bigger group.<br>
+ // Note: This doesn't scale well, so if possible avoid calling any heavy<br>
+ // function from this loop to minimize the performance impact.<br>
+ for (unsigned i = 0; i < Result.size(); ++i) {<br>
+ for (unsigned j = 0; j < Result.size(); ++j) {<br>
+ // Don't compare a group with itself.<br>
+ if (i == j)<br>
+ continue;<br>
+<br>
+ if (containsGroup(Result[j], Result[i])) {<br>
+ IndexesToRemove.push_back(i);<br>
+ break;<br>
+ }<br>
+ }<br>
+ }<br>
+<br>
+ // Erasing a list of indexes from the vector should be done with decreasing<br>
+ // indexes. As IndexesToRemove is constructed with increasing values, we just<br>
+ // reverse iterate over it to get the desired order.<br>
+ for (auto I = IndexesToRemove.rbegin(); I != IndexesToRemove.rend(); ++I) {<br>
+ Result.erase(Result.begin() + *I);<br>
+ }<br>
+}<br>
<br>
Modified: cfe/trunk/lib/StaticAnalyzer/Checkers/CMakeLists.txt<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/StaticAnalyzer/Checkers/CMakeLists.txt?rev=276782&r1=276781&r2=276782&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/StaticAnalyzer/Checkers/CMakeLists.txt?rev=276782&r1=276781&r2=276782&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/StaticAnalyzer/Checkers/CMakeLists.txt (original)<br>
+++ cfe/trunk/lib/StaticAnalyzer/Checkers/CMakeLists.txt Tue Jul 26 13:13:12 2016<br>
@@ -22,6 +22,7 @@ add_clang_library(clangStaticAnalyzerChe<br>
CheckerDocumentation.cpp<br>
ChrootChecker.cpp<br>
ClangCheckers.cpp<br>
+ CloneChecker.cpp<br>
CXXSelfAssignmentChecker.cpp<br>
DeadStoresChecker.cpp<br>
DebugCheckers.cpp<br>
<br>
Added: cfe/trunk/lib/StaticAnalyzer/Checkers/CloneChecker.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/StaticAnalyzer/Checkers/CloneChecker.cpp?rev=276782&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/StaticAnalyzer/Checkers/CloneChecker.cpp?rev=276782&view=auto</a><br>
==============================================================================<br>
--- cfe/trunk/lib/StaticAnalyzer/Checkers/CloneChecker.cpp (added)<br>
+++ cfe/trunk/lib/StaticAnalyzer/Checkers/CloneChecker.cpp Tue Jul 26 13:13:12 2016<br>
@@ -0,0 +1,96 @@<br>
+//===--- CloneChecker.cpp - Clone detection checker -------------*- C++ -*-===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+///<br>
+/// \file<br>
+/// CloneChecker is a checker that reports clones in the current translation<br>
+/// unit.<br>
+///<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#include "ClangSACheckers.h"<br>
+#include "clang/Analysis/CloneDetection.h"<br>
+#include "clang/Basic/Diagnostic.h"<br>
+#include "clang/StaticAnalyzer/Core/Checker.h"<br>
+#include "clang/StaticAnalyzer/Core/CheckerManager.h"<br>
+#include "clang/StaticAnalyzer/Core/PathSensitive/CheckerContext.h"<br>
+<br>
+using namespace clang;<br>
+using namespace ento;<br>
+<br>
+namespace {<br>
+class CloneChecker<br>
+ : public Checker<check::ASTCodeBody, check::EndOfTranslationUnit> {<br>
+ mutable CloneDetector CloneDetector;<br>
+<br>
+public:<br>
+ void checkASTCodeBody(const Decl *D, AnalysisManager &Mgr,<br>
+ BugReporter &BR) const;<br>
+<br>
+ void checkEndOfTranslationUnit(const TranslationUnitDecl *TU,<br>
+ AnalysisManager &Mgr, BugReporter &BR) const;<br>
+};<br>
+} // end anonymous namespace<br>
+<br>
+void CloneChecker::checkASTCodeBody(const Decl *D, AnalysisManager &Mgr,<br>
+ BugReporter &BR) const {<br>
+ // Every statement that should be included in the search for clones needs to<br>
+ // be passed to the CloneDetector.<br>
+ CloneDetector.analyzeCodeBody(D);<br>
+}<br>
+<br>
+void CloneChecker::checkEndOfTranslationUnit(const TranslationUnitDecl *TU,<br>
+ AnalysisManager &Mgr,<br>
+ BugReporter &BR) const {<br>
+ // At this point, every statement in the translation unit has been analyzed by<br>
+ // the CloneDetector. The only thing left to do is to report the found clones.<br>
+<br>
+ int MinComplexity = Mgr.getAnalyzerOptions().getOptionAsInteger(<br>
+ "MinimumCloneComplexity", 10, this);<br>
+<br>
+ assert(MinComplexity >= 0);<br>
+<br>
+ SourceManager &SM = BR.getSourceManager();<br>
+<br>
+ std::vector<CloneDetector::CloneGroup> CloneGroups;<br>
+ CloneDetector.findClones(CloneGroups, MinComplexity);<br>
+<br>
+ DiagnosticsEngine &DiagEngine = Mgr.getDiagnostic();<br>
+<br>
+ unsigned WarnID = DiagEngine.getCustomDiagID(DiagnosticsEngine::Warning,<br>
+ "Detected code clone.");<br>
+<br>
+ unsigned NoteID = DiagEngine.getCustomDiagID(DiagnosticsEngine::Note,<br>
+ "Related code clone is here.");<br>
+<br>
+ for (CloneDetector::CloneGroup &Group : CloneGroups) {<br>
+ // For readability reasons we sort the clones by line numbers.<br>
+ std::sort(Group.Sequences.begin(), Group.Sequences.end(),<br>
+ [&SM](const StmtSequence &LHS, const StmtSequence &RHS) {<br>
+ return SM.isBeforeInTranslationUnit(LHS.getStartLoc(),<br>
+ RHS.getStartLoc()) &&<br>
+ SM.isBeforeInTranslationUnit(LHS.getEndLoc(),<br>
+ RHS.getEndLoc());<br>
+ });<br>
+<br>
+ // We group the clones by printing the first as a warning and all others<br>
+ // as a note.<br>
+ DiagEngine.Report(Group.Sequences.front().getStartLoc(), WarnID);<br>
+ for (unsigned i = 1; i < Group.Sequences.size(); ++i) {<br>
+ DiagEngine.Report(Group.Sequences[i].getStartLoc(), NoteID);<br>
+ }<br>
+ }<br>
+}<br>
+<br>
+//===----------------------------------------------------------------------===//<br>
+// Register CloneChecker<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+void ento::registerCloneChecker(CheckerManager &Mgr) {<br>
+ Mgr.registerChecker<CloneChecker>();<br>
+}<br>
<br>
Added: cfe/trunk/test/Analysis/copypaste/blocks.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Analysis/copypaste/blocks.cpp?rev=276782&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Analysis/copypaste/blocks.cpp?rev=276782&view=auto</a><br>
==============================================================================<br>
--- cfe/trunk/test/Analysis/copypaste/blocks.cpp (added)<br>
+++ cfe/trunk/test/Analysis/copypaste/blocks.cpp Tue Jul 26 13:13:12 2016<br>
@@ -0,0 +1,19 @@<br>
+// RUN: %clang_cc1 -analyze -fblocks -std=c++11 -analyzer-checker=alpha.clone.CloneChecker -verify %s<br>
+<br>
+// This tests if we search for clones in blocks.<br>
+<br>
+void log();<br>
+<br>
+auto BlockA = ^(int a, int b){ // expected-warning{{Detected code clone.}}<br>
+ log();<br>
+ if (a > b)<br>
+ return a;<br>
+ return b;<br>
+};<br>
+<br>
+auto BlockB = ^(int a, int b){ // expected-note{{Related code clone is here.}}<br>
+ log();<br>
+ if (a > b)<br>
+ return a;<br>
+ return b;<br>
+};<br>
<br>
Added: cfe/trunk/test/Analysis/copypaste/false-positives.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Analysis/copypaste/false-positives.cpp?rev=276782&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Analysis/copypaste/false-positives.cpp?rev=276782&view=auto</a><br>
==============================================================================<br>
--- cfe/trunk/test/Analysis/copypaste/false-positives.cpp (added)<br>
+++ cfe/trunk/test/Analysis/copypaste/false-positives.cpp Tue Jul 26 13:13:12 2016<br>
@@ -0,0 +1,29 @@<br>
+// RUN: %clang_cc1 -analyze -std=c++11 -analyzer-checker=alpha.clone.CloneChecker -verify %s<br>
+<br>
+// This test contains false-positive reports from the CloneChecker that need to<br>
+// be fixed.<br>
+<br>
+void log();<br>
+<br>
+int max(int a, int b) { // expected-warning{{Detected code clone.}}<br>
+ log();<br>
+ if (a > b)<br>
+ return a;<br>
+ return b;<br>
+}<br>
+<br>
+// FIXME: Detect different binary operator kinds.<br>
+int min1(int a, int b) { // expected-note{{Related code clone is here.}}<br>
+ log();<br>
+ if (a < b)<br>
+ return a;<br>
+ return b;<br>
+}<br>
+<br>
+// FIXME: Detect different variable patterns.<br>
+int min2(int a, int b) { // expected-note{{Related code clone is here.}}<br>
+ log();<br>
+ if (b > a)<br>
+ return a;<br>
+ return b;<br>
+}<br>
<br>
Added: cfe/trunk/test/Analysis/copypaste/functions.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Analysis/copypaste/functions.cpp?rev=276782&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Analysis/copypaste/functions.cpp?rev=276782&view=auto</a><br>
==============================================================================<br>
--- cfe/trunk/test/Analysis/copypaste/functions.cpp (added)<br>
+++ cfe/trunk/test/Analysis/copypaste/functions.cpp Tue Jul 26 13:13:12 2016<br>
@@ -0,0 +1,25 @@<br>
+// RUN: %clang_cc1 -analyze -std=c++11 -analyzer-checker=alpha.clone.CloneChecker -verify %s<br>
+<br>
+// This tests if we search for clones in functions.<br>
+<br>
+void log();<br>
+<br>
+int max(int a, int b) { // expected-warning{{Detected code clone.}}<br>
+ log();<br>
+ if (a > b)<br>
+ return a;<br>
+ return b;<br>
+}<br>
+<br>
+int maxClone(int x, int y) { // expected-note{{Related code clone is here.}}<br>
+ log();<br>
+ if (x > y)<br>
+ return x;<br>
+ return y;<br>
+}<br>
+<br>
+// Functions below are not clones and should not be reported.<br>
+<br>
+int foo(int a, int b) { // no-warning<br>
+ return a + b;<br>
+}<br>
<br>
Added: cfe/trunk/test/Analysis/copypaste/objc-methods.m<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Analysis/copypaste/objc-methods.m?rev=276782&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Analysis/copypaste/objc-methods.m?rev=276782&view=auto</a><br>
==============================================================================<br>
--- cfe/trunk/test/Analysis/copypaste/objc-methods.m (added)<br>
+++ cfe/trunk/test/Analysis/copypaste/objc-methods.m Tue Jul 26 13:13:12 2016<br>
@@ -0,0 +1,27 @@<br>
+// RUN: %clang_cc1 -analyze -Wno-objc-root-class -analyzer-checker=alpha.clone.CloneChecker -verify %s<br>
+<br>
+// This tests if we search for clones in Objective-C methods.<br>
+<br>
+@interface A<br>
+- (int) setOk : (int) a : (int) b;<br>
+@end<br>
+<br>
+@implementation A<br>
+- (int) setOk : (int) a : (int) b { // expected-warning{{Detected code clone.}}<br>
+ if (a > b)<br>
+ return a;<br>
+ return b;<br>
+}<br>
+@end<br>
+<br>
+@interface B<br>
+- (int) setOk : (int) a : (int) b;<br>
+@end<br>
+<br>
+@implementation B<br>
+- (int) setOk : (int) a : (int) b { // expected-note{{Related code clone is here.}}<br>
+ if (a > b)<br>
+ return a;<br>
+ return b;<br>
+}<br>
+@end<br>
<br>
Added: cfe/trunk/test/Analysis/copypaste/sub-sequences.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Analysis/copypaste/sub-sequences.cpp?rev=276782&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Analysis/copypaste/sub-sequences.cpp?rev=276782&view=auto</a><br>
==============================================================================<br>
--- cfe/trunk/test/Analysis/copypaste/sub-sequences.cpp (added)<br>
+++ cfe/trunk/test/Analysis/copypaste/sub-sequences.cpp Tue Jul 26 13:13:12 2016<br>
@@ -0,0 +1,27 @@<br>
+// RUN: %clang_cc1 -analyze -std=c++11 -analyzer-checker=alpha.clone.CloneChecker -verify %s<br>
+<br>
+// This tests if sub-sequences can match with normal sequences.<br>
+<br>
+void log2(int a);<br>
+void log();<br>
+<br>
+int max(int a, int b) {<br>
+ log2(a);<br>
+ log(); // expected-warning{{Detected code clone.}}<br>
+ if (a > b)<br>
+ return a;<br>
+ return b;<br>
+}<br>
+<br>
+int maxClone(int a, int b) {<br>
+ log(); // expected-note{{Related code clone is here.}}<br>
+ if (a > b)<br>
+ return a;<br>
+ return b;<br>
+}<br>
+<br>
+// Functions below are not clones and should not be reported.<br>
+<br>
+int foo(int a, int b) { // no-warning<br>
+ return a + b;<br>
+}<br>
<br>
<br>
_______________________________________________<br>
cfe-commits mailing list<br>
<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits</a><br>
</blockquote></div><div dir="ltr">-- <br></div><div data-smartmail="gmail_signature">Mike<br>Sent from phone</div>