[clang-tools-extra] r186938 - cp11-migrate: Integration with LibFormat
Guillaume Papin
guillaume.papin at epitech.eu
Wed Jul 24 16:15:00 PDT 2013
Hum, I'm the responsible of this failure but I'm not sure about the
proper fix. I can shed some light and let Edwin choose a proper fix
since I won't be available tomorrow.
The fault comes from this line in test/cpp11-migrate/Core/Reformatting.cpp:
// RUN: not cpp11-migrate -format-style=/tmp/ -use-auto %t.cpp -- -std=c++11
"/tmp/" being a directory and not a file I wanted to test that errors
are checked when reading an invalid configuration files. But putting
"/tmp/" was a stupid idea. The test should probably create a
non-readable file first and use this one instead. Or at least using the
directory '.' which should be available on all platform (unlike /tmp/).
But I'm surprised about the error though:
YAML:1:1: error: Got empty plain scalar
The code doing the reading in cpp11-migrate/tool/Cpp11Migrate.cpp is:
llvm::StringRef ConfigFilePath = FormatStyleOpt; // FormatStyleOpt is "/tmp/" in this failure
llvm::OwningPtr<llvm::MemoryBuffer> Text;
llvm::error_code ec;
ec = llvm::MemoryBuffer::getFile(ConfigFilePath, Text); // Does this function do proper error checking on FreeBSD? In particular regarding to directories? A bug?
if (!ec)
ec = format::parseConfiguration(Text->getBuffer(), &Style);
if (ec) {
llvm::errs() << ProgName << ": invalid format style " << FormatStyleOpt
<< ": " << ec.message() << "\n";
Error = true;
return 0;
}
Galina Kistanova <gkistanova at gmail.com> writes:
> Hello Edwin,
>
>>Added:
>> clang-tools-extra/trunk/cpp11-migrate/Core/Reformatting.cpp
>> clang-tools-extra/trunk/cpp11-migrate/Core/Reformatting.h
>> clang-tools-extra/trunk/test/cpp11-migrate/Core/Reformatting.cpp
>> clang-tools-extra/trunk/unittests/cpp11-migrate/ReformattingTest.cpp
>
> cpp11-migrate/Core/Reformatting.cpp fails on one of the builders:
> http://lab.llvm.org:8011/builders/clang-X86_64-freebsd
> Please have a look at this?
>
> Thanks
>
> Galina
>
>
>
> On Tue, Jul 23, 2013 at 5:50 AM, Edwin Vane <edwin.vane at intel.com> wrote:
>
> Author: revane
> Date: Tue Jul 23 07:50:03 2013
> New Revision: 186938
>
> URL: http://llvm.org/viewvc/llvm-project?rev=186938&view=rev
> Log:
> cp11-migrate: Integration with LibFormat
>
> Adding a feature to optionally reformat code changed by the migrator. Like
> LibFormat, can choose between built-in styles (LLVM, Mozilla, Google,
> Chromium)
> or use a YAML-format config file.
>
> Now with no dependency on iostream by the Reformatting.cpp LIT test.
>
> Author: Guillaume Papin <guillaume.papin at epitech.eu>
>
>
> Added:
> clang-tools-extra/trunk/cpp11-migrate/Core/Reformatting.cpp
> clang-tools-extra/trunk/cpp11-migrate/Core/Reformatting.h
> clang-tools-extra/trunk/test/cpp11-migrate/Core/Reformatting.cpp
> clang-tools-extra/trunk/unittests/cpp11-migrate/ReformattingTest.cpp
> Modified:
> clang-tools-extra/trunk/cpp11-migrate/Core/CMakeLists.txt
> clang-tools-extra/trunk/cpp11-migrate/Core/FileOverrides.cpp
> clang-tools-extra/trunk/cpp11-migrate/Core/FileOverrides.h
> clang-tools-extra/trunk/cpp11-migrate/tool/Cpp11Migrate.cpp
> clang-tools-extra/trunk/cpp11-migrate/tool/Makefile
> clang-tools-extra/trunk/docs/MigratorUsage.rst
> clang-tools-extra/trunk/unittests/cpp11-migrate/CMakeLists.txt
> clang-tools-extra/trunk/unittests/cpp11-migrate/FileOverridesTest.cpp
> clang-tools-extra/trunk/unittests/cpp11-migrate/Makefile
> clang-tools-extra/trunk/unittests/cpp11-migrate/TransformTest.cpp
>
> Modified: clang-tools-extra/trunk/cpp11-migrate/Core/CMakeLists.txt
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/cpp11-migrate/Core/CMakeLists.
> txt?rev=186938&r1=186937&r2=186938&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/cpp11-migrate/Core/CMakeLists.txt (original)
> +++ clang-tools-extra/trunk/cpp11-migrate/Core/CMakeLists.txt Tue Jul 23
> 07:50:03 2013
> @@ -7,9 +7,12 @@ add_clang_library(migrateCore
> Transform.cpp
> IncludeExcludeInfo.cpp
> PerfSupport.cpp
> + Reformatting.cpp
> )
> target_link_libraries(migrateCore
> + clangFormat
> clangTooling
> clangBasic
> clangASTMatchers
> + clangRewriteFrontend
> )
>
> Modified: clang-tools-extra/trunk/cpp11-migrate/Core/FileOverrides.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/cpp11-migrate/Core/FileOverrides.
> cpp?rev=186938&r1=186937&r2=186938&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/cpp11-migrate/Core/FileOverrides.cpp (original)
> +++ clang-tools-extra/trunk/cpp11-migrate/Core/FileOverrides.cpp Tue Jul 23
> 07:50:03 2013
> @@ -17,8 +17,6 @@
> #include "clang/Basic/Diagnostic.h"
> #include "clang/Basic/DiagnosticOptions.h"
> #include "clang/Basic/SourceManager.h"
> -#include "clang/Format/Format.h"
> -#include "clang/Lex/Lexer.h"
> #include "clang/Rewrite/Core/Rewriter.h"
> #include "clang/Tooling/Tooling.h"
> #include "llvm/Support/FileSystem.h"
> @@ -29,10 +27,11 @@
> using namespace clang;
> using namespace clang::tooling;
>
> -SourceOverrides::SourceOverrides(llvm::StringRef MainFileName)
> - : MainFileName(MainFileName) {}
> +SourceOverrides::SourceOverrides(llvm::StringRef MainFileName,
> + bool TrackChanges)
> + : MainFileName(MainFileName), TrackChanges(TrackChanges) {}
>
> -void SourceOverrides::applyReplacements(tooling::Replacements &Replaces) {
> +void SourceOverrides::applyReplacements(Replacements &Replaces) {
> llvm::IntrusiveRefCntPtr<clang::DiagnosticOptions> DiagOpts(
> new DiagnosticOptions());
> DiagnosticsEngine Diagnostics(
> @@ -57,6 +56,8 @@ void SourceOverrides::applyReplacements(
> llvm::errs() << "error: failed to apply some replacements.";
>
> applyRewrites(Rewrites);
> + if (TrackChanges)
> + adjustChangedRanges(Replaces);
> }
>
> void SourceOverrides::applyRewrites(Rewriter &Rewrites) {
> @@ -96,6 +97,30 @@ void SourceOverrides::applyRewrites(Rewr
> }
> }
>
> +void SourceOverrides::adjustChangedRanges(const Replacements &Replaces) {
> + // Start by grouping replacements by file name
> + Replacements MainFileReplaces;
> + llvm::StringMap<Replacements> HeadersReplaces;
> +
> + for (Replacements::iterator I = Replaces.begin(), E = Replaces.end(); I
> != E;
> + ++I) {
> + llvm::StringRef ReplacementFileName = I->getFilePath();
> +
> + if (ReplacementFileName == MainFileName)
> + MainFileReplaces.insert(*I);
> + else
> + HeadersReplaces[ReplacementFileName].insert(*I);
> + }
> +
> + // Then adjust the changed ranges for each individual file
> + MainFileChanges.adjustChangedRanges(Replaces);
> + for (llvm::StringMap<Replacements>::iterator I = HeadersReplaces.begin(),
> + E = HeadersReplaces.end();
> + I != E; ++I) {
> + Headers[I->getKey()].Changes.adjustChangedRanges(I->getValue());
> + }
> +}
> +
> void SourceOverrides::applyOverrides(SourceManager &SM) const {
> FileManager &FM = SM.getFileManager();
>
> @@ -154,6 +179,109 @@ SourceOverrides &FileOverrides::getOrCre
> SourceOverrides *&Override = Overrides[Filename];
>
> if (Override == NULL)
> - Override = new SourceOverrides(Filename);
> + Override = new SourceOverrides(Filename, TrackChanges);
> return *Override;
> }
> +
> +namespace {
> +
> +/// \brief Comparator to be able to order tooling::Range based on their
> offset.
> +bool rangeLess(clang::tooling::Range A, clang::tooling::Range B) {
> + if (A.getOffset() == B.getOffset())
> + return A.getLength() < B.getLength();
> + return A.getOffset() < B.getOffset();
> +}
> +
> +/// \brief Functor that returns the given range without its overlaps with
> the
> +/// replacement given in the constructor.
> +struct RangeReplacedAdjuster {
> + RangeReplacedAdjuster(const tooling::Replacement &Replace)
> + : Replace(Replace.getOffset(), Replace.getLength()),
> + ReplaceNewSize(Replace.getReplacementText().size()) {}
> +
> + tooling::Range operator()(clang::tooling::Range Range) const {
> + if (!Range.overlapsWith(Replace))
> + return Range;
> + // range inside replacement -> make the range length null
> + if (Replace.contains(Range))
> + return tooling::Range(Range.getOffset(), 0);
> + // replacement inside range -> resize the range
> + if (Range.contains(Replace)) {
> + int Difference = ReplaceNewSize - Replace.getLength();
> + return tooling::Range(Range.getOffset(), Range.getLength() +
> Difference);
> + }
> + // beginning of the range replaced -> truncate range beginning
> + if (Range.getOffset() > Replace.getOffset()) {
> + unsigned ReplaceEnd = Replace.getOffset() + Replace.getLength();
> + unsigned RangeEnd = Range.getOffset() + Range.getLength();
> + return tooling::Range(ReplaceEnd, RangeEnd - ReplaceEnd);
> + }
> + // end of the range replaced -> truncate range end
> + if (Range.getOffset() < Replace.getOffset())
> + return tooling::Range(Range.getOffset(),
> + Replace.getOffset() - Range.getOffset());
> + llvm_unreachable("conditions not handled properly");
> + }
> +
> + const tooling::Range Replace;
> + const unsigned ReplaceNewSize;
> +};
> +
> +} // end anonymous namespace
> +
> +void ChangedRanges::adjustChangedRanges(const tooling::Replacements
> &Replaces) {
> + // first adjust existing ranges in case they overlap with the
> replacements
> + for (Replacements::iterator I = Replaces.begin(), E = Replaces.end(); I
> != E;
> + ++I) {
> + const tooling::Replacement &Replace = *I;
> +
> + std::transform(Ranges.begin(), Ranges.end(), Ranges.begin(),
> + RangeReplacedAdjuster(Replace));
> + }
> +
> + // then shift existing ranges to reflect the new positions
> + for (RangeVec::iterator I = Ranges.begin(), E = Ranges.end(); I != E;
> ++I) {
> + unsigned ShiftedOffset =
> + tooling::shiftedCodePosition(Replaces, I->getOffset());
> + *I = tooling::Range(ShiftedOffset, I->getLength());
> + }
> +
> + // then generate the new ranges from the replacements
> + for (Replacements::iterator I = Replaces.begin(), E = Replaces.end(); I
> != E;
> + ++I) {
> + const tooling::Replacement &R = *I;
> + unsigned Offset = tooling::shiftedCodePosition(Replaces, R.getOffset
> ());
> + unsigned Length = R.getReplacementText().size();
> +
> + Ranges.push_back(tooling::Range(Offset, Length));
> + }
> +
> + // cleanups unecessary ranges to finish
> + coalesceRanges();
> +}
> +
> +void ChangedRanges::coalesceRanges() {
> + // sort the ranges by offset and then for each group of
> adjacent/overlapping
> + // ranges the first one in the group is extended to cover the whole
> group.
> + std::sort(Ranges.begin(), Ranges.end(), &rangeLess);
> + RangeVec::iterator FirstInGroup = Ranges.begin();
> + assert(!Ranges.empty() && "unexpected empty vector");
> + for (RangeVec::iterator I = Ranges.begin() + 1, E = Ranges.end(); I != E;
> + ++I) {
> + unsigned GroupEnd = FirstInGroup->getOffset() + FirstInGroup->getLength
> ();
> +
> + // no contact
> + if (I->getOffset() > GroupEnd)
> + FirstInGroup = I;
> + else {
> + unsigned GrpBegin = FirstInGroup->getOffset();
> + unsigned GrpEnd = std::max(GroupEnd, I->getOffset() + I->getLength
> ());
> + *FirstInGroup = tooling::Range(GrpBegin, GrpEnd - GrpBegin);
> + }
> + }
> +
> + // remove the ranges that are covered by the first member of the group
> + Ranges.erase(std::unique(Ranges.begin(), Ranges.end(),
> + std::mem_fun_ref(&Range::contains)),
> + Ranges.end());
> +}
>
> Modified: clang-tools-extra/trunk/cpp11-migrate/Core/FileOverrides.h
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/cpp11-migrate/Core/FileOverrides.
> h?rev=186938&r1=186937&r2=186938&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/cpp11-migrate/Core/FileOverrides.h (original)
> +++ clang-tools-extra/trunk/cpp11-migrate/Core/FileOverrides.h Tue Jul 23
> 07:50:03 2013
> @@ -29,6 +29,34 @@ class SourceManager;
> class Rewriter;
> } // namespace clang
>
> +/// \brief Class encapsulating a list of \c tooling::Range with some
> +/// convenience methods.
> +///
> +/// The ranges stored are used to keep track of the overriden parts of a
> file.
> +class ChangedRanges {
> + typedef std::vector<clang::tooling::Range> RangeVec;
> +
> +public:
> + typedef RangeVec::const_iterator const_iterator;
> +
> + /// \brief Create new ranges from the replacements and adjust existing
> one
> + /// to remove replaced parts.
> + ///
> + /// Note that all replacements should come from the same file.
> + void adjustChangedRanges(const clang::tooling::Replacements &Replaces);
> +
> + /// \brief Iterators.
> + /// @{
> + const_iterator begin() const { return Ranges.begin(); }
> + const_iterator end() const { return Ranges.end(); }
> + /// @}
> +
> +private:
> + void coalesceRanges();
> +
> + RangeVec Ranges;
> +};
> +
> /// \brief Container for storing override information for a single headers.
> struct HeaderOverride {
> HeaderOverride() {}
> @@ -36,6 +64,7 @@ struct HeaderOverride {
>
> std::string FileName;
> std::string FileOverride;
> + ChangedRanges Changes;
> };
>
> /// \brief Container mapping header file names to override information.
> @@ -46,12 +75,18 @@ typedef llvm::StringMap<HeaderOverride>
> /// which changes have been made.
> class SourceOverrides {
> public:
> - SourceOverrides(llvm::StringRef MainFileName);
> + SourceOverrides(llvm::StringRef MainFileName, bool TrackChanges);
>
> /// \brief Accessors.
> /// @{
> llvm::StringRef getMainFileName() const { return MainFileName; }
> llvm::StringRef getMainFileContent() const { return MainFileOverride; }
> + const ChangedRanges &getChangedRanges() const { return MainFileChanges; }
> +
> + /// \brief Is file change tracking enabled?
> + ///
> + /// Tracking file changes can be useful to reformat the code for example.
> + bool isTrackingFileChanges() const { return TrackChanges; }
> /// @}
>
> /// \brief Indicates if the source file has been overridden.
> @@ -87,8 +122,14 @@ private:
> /// file content overrides.
> void applyRewrites(clang::Rewriter &Rewrite);
>
> + /// \brief Adjust the changed ranges to reflect the parts of the files
> that
> + /// have been replaced.
> + void adjustChangedRanges(const clang::tooling::Replacements &Replaces);
> +
> const std::string MainFileName;
> std::string MainFileOverride;
> + const bool TrackChanges;
> + ChangedRanges MainFileChanges;
> HeaderOverrides Headers;
> };
>
> @@ -98,7 +139,11 @@ public:
> typedef llvm::StringMap<SourceOverrides *> SourceOverridesMap;
> typedef SourceOverridesMap::const_iterator const_iterator;
>
> - FileOverrides() {}
> + /// \brief Construct the SourceOverrides manager.
> + ///
> + /// \param TrackChanges Wether or not the \c SourceOverrides should keep
> track
> + /// of changes. See \c SourceOverrides::isTrackingFileChanges().
> + FileOverrides(bool TrackChanges) : TrackChanges(TrackChanges) {}
> ~FileOverrides();
>
> const_iterator find(llvm::StringRef Filename) const {
> @@ -120,6 +165,7 @@ private:
> FileOverrides &operator=(const FileOverrides &) LLVM_DELETED_FUNCTION;
>
> SourceOverridesMap Overrides;
> + const bool TrackChanges;
> };
>
> /// \brief Generate a unique filename to store the replacements.
>
> Added: clang-tools-extra/trunk/cpp11-migrate/Core/Reformatting.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/cpp11-migrate/Core/Reformatting.
> cpp?rev=186938&view=auto
> ==============================================================================
> --- clang-tools-extra/trunk/cpp11-migrate/Core/Reformatting.cpp (added)
> +++ clang-tools-extra/trunk/cpp11-migrate/Core/Reformatting.cpp Tue Jul 23
> 07:50:03 2013
> @@ -0,0 +1,76 @@
> +//===-- Core/Reformatting.cpp - LibFormat integration
> ---------------------===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +///
> +/// \file
> +/// \brief This file provides the LibFormat integration used to reformat
> +/// migrated code.
> +///
> +//===----------------------------------------------------------------------===//
> +
> +#include "Core/Reformatting.h"
> +#include "Core/FileOverrides.h"
> +#include "clang/Basic/Diagnostic.h"
> +#include "clang/Basic/DiagnosticOptions.h"
> +#include "clang/Basic/SourceManager.h"
> +#include "clang/Lex/Lexer.h"
> +
> +using namespace clang;
> +
> +void Reformatter::reformatChanges(SourceOverrides &Overrides) {
> + llvm::IntrusiveRefCntPtr<clang::DiagnosticOptions> DiagOpts(
> + new DiagnosticOptions());
> + DiagnosticsEngine Diagnostics(
> + llvm::IntrusiveRefCntPtr<DiagnosticIDs>(new DiagnosticIDs()),
> + DiagOpts.getPtr());
> + FileManager Files((FileSystemOptions()));
> + SourceManager SM(Diagnostics, Files);
> +
> + reformatChanges(Overrides, SM);
> +}
> +
> +void Reformatter::reformatChanges(SourceOverrides &Overrides,
> + clang::SourceManager &SM) {
> + tooling::Replacements Replaces;
> + Overrides.applyOverrides(SM);
> + if (Overrides.isSourceOverriden())
> + Replaces = reformatSingleFile(Overrides.getMainFileName(),
> + Overrides.getChangedRanges(), SM);
> +
> + for (HeaderOverrides::const_iterator I = Overrides.headers_begin(),
> + E = Overrides.headers_end();
> + I != E; ++I) {
> + const HeaderOverride &Header = I->getValue();
> + const tooling::Replacements &HeaderReplaces =
> + reformatSingleFile(Header.FileName, Header.Changes, SM);
> + Replaces.insert(HeaderReplaces.begin(), HeaderReplaces.end());
> + }
> + Overrides.applyReplacements(Replaces, SM);
> +}
> +
> +tooling::Replacements Reformatter::reformatSingleFile(
> + llvm::StringRef FileName, const ChangedRanges &Changes, SourceManager
> &SM) {
> + const clang::FileEntry *Entry = SM.getFileManager().getFile(FileName);
> + assert(Entry && "expected an existing file");
> +
> + FileID ID = SM.translateFile(Entry);
> + if (ID.isInvalid())
> + ID = SM.createFileID(Entry, SourceLocation(), clang::SrcMgr::C_User);
> +
> + std::vector<CharSourceRange> ReformatRanges;
> + SourceLocation StartOfFile = SM.getLocForStartOfFile(ID);
> + for (ChangedRanges::const_iterator I = Changes.begin(), E = Changes.end
> ();
> + I != E; ++I) {
> + SourceLocation Start = StartOfFile.getLocWithOffset(I->getOffset());
> + SourceLocation End = Start.getLocWithOffset(I->getLength());
> + ReformatRanges.push_back(CharSourceRange::getCharRange(Start, End));
> + }
> +
> + Lexer Lex(ID, SM.getBuffer(ID), SM, getFormattingLangOpts
> (Style.Standard));
> + return format::reformat(Style, Lex, SM, ReformatRanges);
> +}
>
> Added: clang-tools-extra/trunk/cpp11-migrate/Core/Reformatting.h
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/cpp11-migrate/Core/Reformatting.
> h?rev=186938&view=auto
> ==============================================================================
> --- clang-tools-extra/trunk/cpp11-migrate/Core/Reformatting.h (added)
> +++ clang-tools-extra/trunk/cpp11-migrate/Core/Reformatting.h Tue Jul 23
> 07:50:03 2013
> @@ -0,0 +1,59 @@
> +//===-- Core/Reformatting.h - LibFormat integration -------------*- C++
> -*-===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +///
> +/// \file
> +/// \brief This file provides the LibFormat integration used to reformat
> +/// migrated code.
> +///
> +//===----------------------------------------------------------------------===//
> +
> +#ifndef CPP11_MIGRATE_REFORMATTING_H
> +#define CPP11_MIGRATE_REFORMATTING_H
> +
> +#include "clang/Format/Format.h"
> +
> +class SourceOverrides;
> +class ChangedRanges;
> +
> +class Reformatter {
> +public:
> + Reformatter(const clang::format::FormatStyle &Style) : Style(Style) {}
> +
> + /// \brief Reformat the changes made to the file overrides.
> + ///
> + /// \param Overrides Overriden source files to reformat. Note that since
> only
> + /// the changes are reformatted, file change tracking has to be enabled.
> + /// \param SM A SourceManager where the overridens files can be found.
> + ///
> + /// \sa \c SourceOverrides::isTrackingFileChanges()
> + void reformatChanges(SourceOverrides &Overrides, clang::SourceManager
> &SM);
> +
> + /// \brief Overload of \c reformatChanges() providing it's own
> + /// \c SourceManager.
> + void reformatChanges(SourceOverrides &Overrides);
> +
> + /// \brief Produce a list of replacements to apply on \p FileName, only
> the
> + /// ranges in \p Changes are replaced.
> + ///
> + /// Since this routine use \c clang::format::reformat() the rules that
> applies
> + /// on the ranges are identical:
> + ///
> + /// <blockquote> Each range is extended on either end to its next bigger
> logic
> + /// unit, i.e. everything that might influence its formatting or might be
> + /// influenced by its formatting.
> + /// -- \c clang::format::reformat()</blockquote>
> + clang::tooling::Replacements reformatSingleFile(llvm::StringRef FileName,
> + const ChangedRanges
> &Changes,
> + clang::SourceManager
> &SM);
> +
> +private:
> + clang::format::FormatStyle Style;
> +};
> +
> +#endif // CPP11_MIGRATE_REFORMATTING_H
>
> Modified: clang-tools-extra/trunk/cpp11-migrate/tool/Cpp11Migrate.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/cpp11-migrate/tool/Cpp11Migrate.
> cpp?rev=186938&r1=186937&r2=186938&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/cpp11-migrate/tool/Cpp11Migrate.cpp (original)
> +++ clang-tools-extra/trunk/cpp11-migrate/tool/Cpp11Migrate.cpp Tue Jul 23
> 07:50:03 2013
> @@ -20,6 +20,7 @@
> #include "Core/SyntaxCheck.h"
> #include "Core/Transform.h"
> #include "Core/Transforms.h"
> +#include "Core/Reformatting.h"
> #include "LoopConvert/LoopConvert.h"
> #include "UseNullptr/UseNullptr.h"
> #include "UseAuto/UseAuto.h"
> @@ -28,6 +29,7 @@
> #include "clang/Frontend/FrontendActions.h"
> #include "clang/Tooling/CommonOptionsParser.h"
> #include "clang/Tooling/Tooling.h"
> +#include "llvm/Support/MemoryBuffer.h"
> #include "llvm/Support/Signals.h"
>
> namespace cl = llvm::cl;
> @@ -43,9 +45,9 @@ static cl::extrahelp MoreHelp(
> " cpp11-migrate -use-auto path/to/file.cpp -- -Ipath/to/include/\n"
> "\n"
> "Convert for loops to the new ranged-based for loops on all files in a
> "
> - "subtree:\n\n"
> + "subtree\nand reformat the code automatically using the LLVM
> style:\n\n"
> " find path/in/subtree -name '*.cpp' -exec \\\n"
> - " cpp11-migrate -p build/path -loop-convert {} ';'\n"
> + " cpp11-migrate -p build/path -format-style=LLVM -loop-convert {}
> ';'\n"
> "\n"
> "Make use of both nullptr and the override specifier, using git
> ls-files:\n"
> "\n"
> @@ -70,6 +72,14 @@ static cl::opt<bool> FinalSyntaxCheck(
> cl::desc("Check for correct syntax after applying transformations"),
> cl::init(false));
>
> +static cl::opt<std::string> FormatStyleOpt(
> + "format-style",
> + cl::desc("Coding style to use on the replacements, either a builtin
> style\n"
> + "or a YAML config file (see: clang-format -dump-config).\n"
> + "Currently supports 4 builtins style:\n"
> + " LLVM, Google, Chromium, Mozilla.\n"),
> + cl::value_desc("string"));
> +
> static cl::opt<bool>
> SummaryMode("summary", cl::desc("Print transform summary"),
> cl::init(false));
> @@ -108,6 +118,40 @@ static cl::opt<bool, /*ExternalStorage=*
> cl::location(GlobalOptions.EnableHeaderModifications),
> cl::init(false));
>
> +/// \brief Creates the Reformatter if the format style option is provided,
> +/// return a null pointer otherwise.
> +///
> +/// \param ProgName The name of the program, \c argv[0], used to print
> errors.
> +/// \param Error If the \c -format-style is provided but with wrong
> parameters
> +/// this is parameter is set to \c true, left untouched otherwise. An error
> +/// message is printed with an explanation.
> +static Reformatter *handleFormatStyle(const char *ProgName, bool &Error) {
> + if (FormatStyleOpt.getNumOccurrences() > 0) {
> + format::FormatStyle Style;
> + if (!format::getPredefinedStyle(FormatStyleOpt, &Style)) {
> + llvm::StringRef ConfigFilePath = FormatStyleOpt;
> + llvm::OwningPtr<llvm::MemoryBuffer> Text;
> + llvm::error_code ec;
> +
> + ec = llvm::MemoryBuffer::getFile(ConfigFilePath, Text);
> + if (!ec)
> + ec = parseConfiguration(Text->getBuffer(), &Style);
> +
> + if (ec) {
> + llvm::errs() << ProgName << ": invalid format style " <<
> FormatStyleOpt
> + << ": " << ec.message() << "\n";
> + Error = true;
> + return 0;
> + }
> + }
> +
> + // force mode to C++11
> + Style.Standard = clang::format::FormatStyle::LS_Cpp11;
> + return new Reformatter(Style);
> + }
> + return 0;
> +}
> +
> int main(int argc, const char **argv) {
> llvm::sys::PrintStackTraceOnErrorSignal();
> Transforms TransformManager;
> @@ -137,6 +181,13 @@ int main(int argc, const char **argv) {
> // against the default value when the command line option is not
> specified.
> GlobalOptions.EnableTiming = (TimingDirectoryName != NoTiming);
>
> + // Check the reformatting style option
> + bool BadStyle = false;
> + llvm::OwningPtr<Reformatter> ChangesReformatter(
> + handleFormatStyle(argv[0], BadStyle));
> + if (BadStyle)
> + return 1;
> +
> // Populate the ModifiableHeaders structure if header modifications are
> // enabled.
> if (GlobalOptions.EnableHeaderModifications) {
> @@ -153,7 +204,10 @@ int main(int argc, const char **argv) {
> return 1;
> }
>
> - FileOverrides FileStates;
> + // if reformatting is enabled we wants to track file changes so that it's
> + // possible to reformat them.
> + bool TrackReplacements = static_cast<bool>(ChangesReformatter);
> + FileOverrides FileStates(TrackReplacements);
> SourcePerfData PerfData;
>
> // Apply transforms.
> @@ -183,6 +237,15 @@ int main(int argc, const char **argv) {
> }
> }
>
> + // Reformat changes if a reformatter is provided.
> + if (ChangesReformatter)
> + for (FileOverrides::const_iterator I = FileStates.begin(),
> + E = FileStates.end();
> + I != E; ++I) {
> + SourceOverrides &Overrides = *I->second;
> + ChangesReformatter->reformatChanges(Overrides);
> + }
> +
> if (FinalSyntaxCheck)
> if (!doSyntaxCheck(OptionsParser.getCompilations(),
> OptionsParser.getSourcePathList(), FileStates))
>
> Modified: clang-tools-extra/trunk/cpp11-migrate/tool/Makefile
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/cpp11-migrate/tool/Makefile?
> rev=186938&r1=186937&r2=186938&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/cpp11-migrate/tool/Makefile (original)
> +++ clang-tools-extra/trunk/cpp11-migrate/tool/Makefile Tue Jul 23 07:50:03
> 2013
> @@ -34,10 +34,10 @@ SOURCES += $(addprefix ../ReplaceAutoPtr
> BUILT_SOURCES += $(ObjDir)/../ReplaceAutoPtr/.objdir
>
> LINK_COMPONENTS := $(TARGETS_TO_BUILD) asmparser bitreader support mc
> mcparser option
> -USEDLIBS = migrateCore.a clangTooling.a clangFrontend.a
> clangSerialization.a clangDriver.a \
> - clangRewriteFrontend.a clangRewriteCore.a clangParse.a \
> - clangSema.a clangAnalysis.a \
> - clangAST.a clangASTMatchers.a clangEdit.a clangLex.a
> clangBasic.a
> +USEDLIBS = migrateCore.a clangFormat.a clangTooling.a clangFrontend.a \
> + clangSerialization.a clangDriver.a clangRewriteFrontend.a \
> + clangRewriteCore.a clangParse.a clangSema.a clangAnalysis.a \
> + clangAST.a clangASTMatchers.a clangEdit.a clangLex.a clangBasic.a
>
> include $(CLANG_LEVEL)/Makefile
>
>
> Modified: clang-tools-extra/trunk/docs/MigratorUsage.rst
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/docs/MigratorUsage.
> rst?rev=186938&r1=186937&r2=186938&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/docs/MigratorUsage.rst (original)
> +++ clang-tools-extra/trunk/docs/MigratorUsage.rst Tue Jul 23 07:50:03 2013
> @@ -66,6 +66,43 @@ General Command Line Options
> earlier transforms are already caught when subsequent transforms parse
> the
> file.
>
> +.. option:: -format-style=<string>
> +
> + After all transformations have been applied, reformat the changes using
> the
> + style ``string`` given as argument to the option. The style can be a
> builtin
> + style, one of LLVM, Google, Chromium, Mozilla; or a YAML configuration
> file.
> +
> + If you want a place to start for using your own custom configuration
> file,
> + ClangFormat_ can generate a file with ``clang-format -dump-config``.
> +
> + Example:
> +
> + .. code-block:: c++
> + :emphasize-lines: 10-12,18
> +
> + // file.cpp
> + for (std::vector<int>::const_iterator I = my_container.begin(),
> + E = my_container.end();
> + I != E; ++I) {
> + std::cout << *I << std::endl;
> + }
> +
> + // No reformatting:
> + // cpp11-migrate -use-auto file.cpp --
> + for (auto I = my_container.begin(),
> + E = my_container.end();
> + I != E; ++I) {
> + std::cout << *I << std::endl;
> + }
> +
> + // With reformatting enabled:
> + // cpp11-migrate -format-style=LLVM -use-auto file.cpp --
> + for (auto I = my_container.begin(), E = my_container.end(); I != E;
> ++I) {
> + std::cout << *I << std::endl;
> + }
> +
> +.. _ClangFormat: http://clang.llvm.org/docs/ClangFormat.html
> +
> .. option:: -summary
>
> Displays a summary of the number of changes each transform made or could
> have
>
> Added: clang-tools-extra/trunk/test/cpp11-migrate/Core/Reformatting.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/cpp11-migrate/Core/Reformatting.
> cpp?rev=186938&view=auto
> ==============================================================================
> --- clang-tools-extra/trunk/test/cpp11-migrate/Core/Reformatting.cpp (added)
> +++ clang-tools-extra/trunk/test/cpp11-migrate/Core/Reformatting.cpp Tue Jul
> 23 07:50:03 2013
> @@ -0,0 +1,15 @@
> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
> +// RUN: not cpp11-migrate -format-style=FOO -use-auto %t.cpp -- -std=c++11
> +// RUN: not cpp11-migrate -format-style=/tmp/ -use-auto %t.cpp --
> -std=c++11
> +// RUN: cpp11-migrate -format-style=LLVM -use-auto %t.cpp -- -std=c++11
> +// RUN: FileCheck --strict-whitespace -input-file=%t.cpp %s
> +
> +class MyType012345678901234567890123456789 {};
> +
> +int f() {
> + MyType012345678901234567890123456789 *a =
> + new MyType012345678901234567890123456789();
> + // CHECK: {{^\ \ auto\ a\ \=\ new\ MyType012345678901234567890123456789\
> (\);}}
> +
> + delete a;
> +}
>
> Modified: clang-tools-extra/trunk/unittests/cpp11-migrate/CMakeLists.txt
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/cpp11-migrate/CMakeLists.
> txt?rev=186938&r1=186937&r2=186938&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/unittests/cpp11-migrate/CMakeLists.txt
> (original)
> +++ clang-tools-extra/trunk/unittests/cpp11-migrate/CMakeLists.txt Tue Jul
> 23 07:50:03 2013
> @@ -8,6 +8,7 @@ include_directories(${CPP11_MIGRATE_SOUR
>
> add_extra_unittest(Cpp11MigrateTests
> FileOverridesTest.cpp
> + ReformattingTest.cpp
> IncludeExcludeTest.cpp
> PerfSupportTest.cpp
> TransformTest.cpp
> @@ -16,7 +17,9 @@ add_extra_unittest(Cpp11MigrateTests
>
> target_link_libraries(Cpp11MigrateTests
> migrateCore
> + clangFormat
> clangTooling
> clangBasic
> clangASTMatchers
> + clangRewriteFrontend
> )
>
> Modified:
> clang-tools-extra/trunk/unittests/cpp11-migrate/FileOverridesTest.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/cpp11-migrate/FileOverridesTest.
> cpp?rev=186938&r1=186937&r2=186938&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/unittests/cpp11-migrate/FileOverridesTest.cpp
> (original)
> +++ clang-tools-extra/trunk/unittests/cpp11-migrate/FileOverridesTest.cpp
> Tue Jul 23 07:50:03 2013
> @@ -21,10 +21,11 @@ TEST(SourceOverridesTest, Interface) {
> FileName,
> "std::vector<such_a_long_name_for_a_type>::const_iterator long_type
> =\n"
> " vec.begin();\n");
> - SourceOverrides Overrides(FileName);
> + SourceOverrides Overrides(FileName, /*TrackFileChanges=*/false);
>
> EXPECT_EQ(FileName, Overrides.getMainFileName());
> EXPECT_FALSE(Overrides.isSourceOverriden());
> + EXPECT_FALSE(Overrides.isTrackingFileChanges());
>
> Replacements Replaces;
> unsigned ReplacementLength =
> @@ -37,3 +38,103 @@ TEST(SourceOverridesTest, Interface) {
> " vec.begin();\n";
> EXPECT_EQ(ExpectedContent, Overrides.getMainFileContent());
> }
> +
> +namespace {
> +Replacement makeReplacement(unsigned Offset, unsigned Length,
> + unsigned ReplacementLength) {
> + return Replacement("", Offset, Length, std::string(ReplacementLength, '~')
> );
> +}
> +
> +// generate a set of replacements containing one element
> +Replacements makeReplacements(unsigned Offset, unsigned Length,
> + unsigned ReplacementLength) {
> + Replacements Replaces;
> + Replaces.insert(makeReplacement(Offset, Length, ReplacementLength));
> + return Replaces;
> +}
> +
> +bool equalRanges(Range A, Range B) {
> + return A.getOffset() == B.getOffset() && A.getLength() == B.getLength();
> +}
> +} // end anonymous namespace
> +
> +TEST(ChangedRangesTest, adjustChangedRangesShrink) {
> + ChangedRanges Changes;
> + Changes.adjustChangedRanges(makeReplacements(0, 0, 4));
> + EXPECT_NE(Changes.begin(), Changes.end());
> + EXPECT_TRUE(equalRanges(Range(0, 4), *Changes.begin()));
> + // create a replacement that cuts the end of the last insertion
> + Changes.adjustChangedRanges(makeReplacements(2, 4, 0));
> + Range ExpectedChanges[] = { Range(0, 2) };
> + EXPECT_TRUE(
> + std::equal(Changes.begin(), Changes.end(), ExpectedChanges,
> equalRanges));
> +}
> +
> +TEST(ChangedRangesTest, adjustChangedRangesExtend) {
> + ChangedRanges Changes;
> + Changes.adjustChangedRanges(makeReplacements(1, 0, 4));
> + // cut the old one by a bigger one
> + Changes.adjustChangedRanges(makeReplacements(3, 4, 6));
> + Range ExpectedChanges[] = { Range(1, 8) };
> + EXPECT_TRUE(
> + std::equal(Changes.begin(), Changes.end(), ExpectedChanges,
> equalRanges));
> +}
> +
> +TEST(ChangedRangesTest, adjustChangedRangesNoOverlap) {
> + ChangedRanges Changes;
> + Changes.adjustChangedRanges(makeReplacements(0, 0, 4));
> + Changes.adjustChangedRanges(makeReplacements(6, 0, 4));
> + Range ExpectedChanges[] = { Range(0, 4), Range(6, 4) };
> + EXPECT_TRUE(
> + std::equal(Changes.begin(), Changes.end(), ExpectedChanges,
> equalRanges));
> +}
> +
> +TEST(ChangedRangesTest, adjustChangedRangesNullRange) {
> + ChangedRanges Changes;
> + Changes.adjustChangedRanges(makeReplacements(0, 4, 0));
> + Range ExpectedChanges[] = { Range(0, 0) };
> + EXPECT_TRUE(
> + std::equal(Changes.begin(), Changes.end(), ExpectedChanges,
> equalRanges));
> +}
> +
> +TEST(ChangedRangesTest, adjustChangedRangesExtendExisting) {
> + ChangedRanges Changes;
> + Changes.adjustChangedRanges(makeReplacements(0, 0, 3));
> + Changes.adjustChangedRanges(makeReplacements(2, 5, 8));
> + Range ExpectedChanges[] = { Range(0, 10) };
> + EXPECT_TRUE(
> + std::equal(Changes.begin(), Changes.end(), ExpectedChanges,
> equalRanges));
> +}
> +
> +TEST(ChangedRangesTest, adjustChangedRangesSplit) {
> + ChangedRanges Changes;
> + Changes.adjustChangedRanges(makeReplacements(0, 0, 3));
> + Changes.adjustChangedRanges(makeReplacements(1, 1, 0));
> + Range ExpectedChanges[] = { Range(0, 2) };
> + EXPECT_TRUE(
> + std::equal(Changes.begin(), Changes.end(), ExpectedChanges,
> equalRanges));
> +}
> +
> +TEST(ChangedRangesTest, adjustChangedRangesRangeContained) {
> + ChangedRanges Changes;
> + Changes.adjustChangedRanges(makeReplacements(3, 0, 2));
> + Changes.adjustChangedRanges(makeReplacements(1, 4, 5));
> + Range ExpectedChanges[] = { Range(1, 5) };
> + EXPECT_TRUE(
> + std::equal(Changes.begin(), Changes.end(), ExpectedChanges,
> equalRanges));
> +}
> +
> +TEST(ChangedRangesTest, adjustChangedRangesRangeResized) {
> + ChangedRanges Changes;
> + Changes.adjustChangedRanges(makeReplacements(2, 0, 5));
> + // first make the range bigger
> + Changes.adjustChangedRanges(makeReplacements(4, 1, 3));
> + Range ExpectedChanges[] = { Range(2, 7) };
> + EXPECT_TRUE(
> + std::equal(Changes.begin(), Changes.end(), ExpectedChanges,
> equalRanges));
> + // then smaller
> + Changes.adjustChangedRanges(makeReplacements(3, 3, 1));
> + ExpectedChanges[0] = Range(2, 5);
> + EXPECT_TRUE(
> + std::equal(Changes.begin(), Changes.end(), ExpectedChanges,
> equalRanges));
> +}
>
> Modified: clang-tools-extra/trunk/unittests/cpp11-migrate/Makefile
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/cpp11-migrate/Makefile?
> rev=186938&r1=186937&r2=186938&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/unittests/cpp11-migrate/Makefile (original)
> +++ clang-tools-extra/trunk/unittests/cpp11-migrate/Makefile Tue Jul 23
> 07:50:03 2013
> @@ -12,10 +12,11 @@ include $(CLANG_LEVEL)/../../Makefile.co
>
> TESTNAME = Cpp11MigrateTests
> LINK_COMPONENTS := asmparser bitreader support MC MCParser option
> -USEDLIBS = migrateCore.a clangTooling.a clangFrontend.a
> clangSerialization.a clangDriver.a \
> - clangRewriteFrontend.a clangRewriteCore.a clangParse.a \
> - clangSema.a clangAnalysis.a \
> - clangAST.a clangASTMatchers.a clangEdit.a clangLex.a
> clangBasic.a
> +USEDLIBS = migrateCore.a clangFormat.a clangTooling.a clangFrontend.a \
> + clangSerialization.a clangDriver.a clangRewriteFrontend.a \
> + clangRewriteCore.a clangParse.a clangSema.a clangAnalysis.a
> \
> + clangAST.a clangASTMatchers.a clangEdit.a clangLex.a \
> + clangBasic.a
>
> include $(CLANG_LEVEL)/Makefile
> MAKEFILE_UNITTEST_NO_INCLUDE_COMMON := 1
>
> Added: clang-tools-extra/trunk/unittests/cpp11-migrate/ReformattingTest.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/cpp11-migrate/ReformattingTest.
> cpp?rev=186938&view=auto
> ==============================================================================
> --- clang-tools-extra/trunk/unittests/cpp11-migrate/ReformattingTest.cpp
> (added)
> +++ clang-tools-extra/trunk/unittests/cpp11-migrate/ReformattingTest.cpp Tue
> Jul 23 07:50:03 2013
> @@ -0,0 +1,50 @@
> +//===- cpp11-migrate/ReformattingTest.cpp - Reformatting unit tests
> -------===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#include "Core/Reformatting.h"
> +#include "Core/FileOverrides.h"
> +#include "gtest/gtest.h"
> +#include "VirtualFileHelper.h"
> +
> +using namespace clang;
> +using namespace clang::tooling;
> +
> +namespace {
> +// convenience function to create a ChangedRanges containing one Range
> +ChangedRanges makeChangedRanges(unsigned Offset, unsigned Length) {
> + ChangedRanges Changes;
> + Replacements Replaces;
> +
> + Replaces.insert(Replacement("", Offset, 0, std::string(Length, '~')));
> + Changes.adjustChangedRanges(Replaces);
> + return Changes;
> +}
> +} // end anonymous namespace
> +
> +TEST(Reformatter, SingleReformat) {
> + VirtualFileHelper VFHelper;
> + llvm::StringRef FileName = "<test>";
> + VFHelper.mapFile(FileName, "int a;\n"
> + "int b;\n");
> +
> + Reformatter ChangesReformatter(format::getLLVMStyle());
> + ChangedRanges Changes = makeChangedRanges(0, 6);
> + tooling::Replacements Replaces = ChangesReformatter.reformatSingleFile(
> + FileName, Changes, VFHelper.getNewSourceManager());
> +
> + SourceOverrides Overrides(FileName, /*TrackChanges=*/false);
> + Overrides.applyReplacements(Replaces, VFHelper.getNewSourceManager());
> +
> + std::string Expected, Result;
> +
> + Expected = "int a;\n"
> + "int b;\n";
> + Result = Overrides.getMainFileContent();
> + EXPECT_EQ(Expected, Result);
> +}
>
> Modified: clang-tools-extra/trunk/unittests/cpp11-migrate/TransformTest.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/cpp11-migrate/TransformTest.
> cpp?rev=186938&r1=186937&r2=186938&view=diff
> ==============================================================================
> --- clang-tools-extra/trunk/unittests/cpp11-migrate/TransformTest.cpp
> (original)
> +++ clang-tools-extra/trunk/unittests/cpp11-migrate/TransformTest.cpp Tue
> Jul 23 07:50:03 2013
> @@ -161,7 +161,7 @@ TEST(Transform, Timings) {
>
> // Transform's handle* functions require FileOverrides to be set, even if
> // there aren't any.
> - FileOverrides Overrides;
> + FileOverrides Overrides(/*TrackFileChanges=*/false);
> T.setOverrides(Overrides);
>
> Tool.run(clang::tooling::newFrontendActionFactory(&Factory, &Callbacks));
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>
>
--
Guillaume Papin
More information about the cfe-commits
mailing list