[clang-tools-extra] r180939 - Add support to read include/exclude paths from file
Manuel Klimek
klimek at google.com
Thu May 2 23:19:32 PDT 2013
I'd be interested whether we can find a simpler solution than the DATADIR
magic :)
In this specific case, where the test is not actually run with lit // RUN
lines, but just a simple unit test, why not have the test data created from
the unit test so it stays self contained?
On Thu, May 2, 2013 at 9:02 PM, Edwin Vane <edwin.vane at intel.com> wrote:
> Author: revane
> Date: Thu May 2 14:02:02 2013
> New Revision: 180939
>
> URL: http://llvm.org/viewvc/llvm-project?rev=180939&view=rev
> Log:
> Add support to read include/exclude paths from file
>
> Files containing the list of paths to be included and excluded can now be
> specified through -include-from=<filename> and -exclude-from=<filename>
> command
> line options in cpp11-migrate.
>
> Added support for data files for cpp11-migrate unittests. The
> Cpp11MigrateTests
> executable just requires a DATADIR environment variable to be set which
> specifies the directory where data files are stored. This is handled
> automatically when using LIT.
>
> Author: Jack Yang <jack.yang at intel.com>, Edwin Vane <edwin.vane at intel.com>
>
> Added:
> clang-tools-extra/trunk/unittests/cpp11-migrate/Data/
> clang-tools-extra/trunk/unittests/cpp11-migrate/Data/ExcludeData.in
> clang-tools-extra/trunk/unittests/cpp11-migrate/Data/ExcludeDataCRLF.in
> clang-tools-extra/trunk/unittests/cpp11-migrate/Data/IncludeData.in
> clang-tools-extra/trunk/unittests/cpp11-migrate/Data/IncludeDataCRLF.in
> clang-tools-extra/trunk/unittests/cpp11-migrate/lit.local.cfg
> Modified:
> clang-tools-extra/trunk/cpp11-migrate/Core/IncludeExcludeInfo.cpp
> clang-tools-extra/trunk/cpp11-migrate/Core/IncludeExcludeInfo.h
> clang-tools-extra/trunk/cpp11-migrate/tool/Cpp11Migrate.cpp
> clang-tools-extra/trunk/test/Unit/lit.cfg
> clang-tools-extra/trunk/test/Unit/lit.site.cfg.in
> clang-tools-extra/trunk/unittests/cpp11-migrate/CMakeLists.txt
> clang-tools-extra/trunk/unittests/cpp11-migrate/IncludeExcludeTest.cpp
> clang-tools-extra/trunk/unittests/cpp11-migrate/Makefile
>
> Modified: clang-tools-extra/trunk/cpp11-migrate/Core/IncludeExcludeInfo.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/cpp11-migrate/Core/IncludeExcludeInfo.cpp?rev=180939&r1=180938&r2=180939&view=diff
>
> ==============================================================================
> --- clang-tools-extra/trunk/cpp11-migrate/Core/IncludeExcludeInfo.cpp
> (original)
> +++ clang-tools-extra/trunk/cpp11-migrate/Core/IncludeExcludeInfo.cpp Thu
> May 2 14:02:02 2013
> @@ -16,7 +16,9 @@
> #include "IncludeExcludeInfo.h"
> #include "llvm/Support/CommandLine.h"
> #include "llvm/Support/FileSystem.h"
> +#include "llvm/Support/MemoryBuffer.h"
> #include "llvm/Support/Path.h"
> +#include "llvm/Support/raw_ostream.h"
>
> using namespace llvm;
>
> @@ -46,25 +48,59 @@ bool fileHasPathPrefix(StringRef File, S
> return true;
> }
>
> -/// \brief Helper function to parse a string of comma seperated paths into
> +/// \brief Helper function to tokenize a string of paths and populate
> /// the vector.
> -void parseCLInput(StringRef Line, std::vector<std::string> &List) {
> +error_code parseCLInput(StringRef Line, std::vector<std::string> &List,
> + StringRef Separator) {
> SmallVector<StringRef, 32> Tokens;
> - Line.split(Tokens, ",", /*MaxSplit=*/-1, /*KeepEmpty=*/false);
> + Line.split(Tokens, Separator, /*MaxSplit=*/ -1, /*KeepEmpty=*/ false);
> for (SmallVectorImpl<StringRef>::iterator I = Tokens.begin(),
> E = Tokens.end();
> I != E; ++I) {
> // Convert each path to its absolute path.
> - SmallString<64> AbsolutePath = *I;
> - sys::fs::make_absolute(AbsolutePath);
> + SmallString<64> AbsolutePath = I->rtrim();
> + if (error_code Err = sys::fs::make_absolute(AbsolutePath))
> + return Err;
> List.push_back(std::string(AbsolutePath.str()));
> }
> + return error_code::success();
> }
> } // end anonymous namespace
>
> -IncludeExcludeInfo::IncludeExcludeInfo(StringRef Include, StringRef
> Exclude) {
> - parseCLInput(Include, IncludeList);
> - parseCLInput(Exclude, ExcludeList);
> +error_code IncludeExcludeInfo::readListFromString(StringRef IncludeString,
> + StringRef
> ExcludeString) {
> + if (error_code Err = parseCLInput(IncludeString, IncludeList,
> + /*Separator=*/ ","))
> + return Err;
> + if (error_code Err = parseCLInput(ExcludeString, ExcludeList,
> + /*Separator=*/ ","))
> + return Err;
> + return error_code::success();
> +}
> +
> +error_code IncludeExcludeInfo::readListFromFile(StringRef IncludeListFile,
> + StringRef
> ExcludeListFile) {
> + if (!IncludeListFile.empty()) {
> + OwningPtr<MemoryBuffer> FileBuf;
> + if (error_code Err = MemoryBuffer::getFile(IncludeListFile, FileBuf))
> {
> + errs() << "Unable to read from include file.\n";
> + return Err;
> + }
> + if (error_code Err = parseCLInput(FileBuf->getBuffer(), IncludeList,
> + /*Separator=*/ "\n"))
> + return Err;
> + }
> + if (!ExcludeListFile.empty()) {
> + OwningPtr<MemoryBuffer> FileBuf;
> + if (error_code Err = MemoryBuffer::getFile(ExcludeListFile, FileBuf))
> {
> + errs() << "Unable to read from exclude file.\n";
> + return Err;
> + }
> + if (error_code Err = parseCLInput(FileBuf->getBuffer(), ExcludeList,
> + /*Separator=*/ "\n"))
> + return Err;
> + }
> + return error_code::success();
> }
>
> bool IncludeExcludeInfo::isFileIncluded(StringRef FilePath) {
>
> Modified: clang-tools-extra/trunk/cpp11-migrate/Core/IncludeExcludeInfo.h
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/cpp11-migrate/Core/IncludeExcludeInfo.h?rev=180939&r1=180938&r2=180939&view=diff
>
> ==============================================================================
> --- clang-tools-extra/trunk/cpp11-migrate/Core/IncludeExcludeInfo.h
> (original)
> +++ clang-tools-extra/trunk/cpp11-migrate/Core/IncludeExcludeInfo.h Thu
> May 2 14:02:02 2013
> @@ -16,18 +16,30 @@
> #define LLVM_TOOLS_CLANG_TOOLS_EXTRA_CPP11_MIGRATE_INCLUDEEXCLUDEINFO_H
>
> #include "llvm/ADT/StringRef.h"
> +#include "llvm/Support/system_error.h"
> #include <vector>
>
> /// \brief Class encapsulating the handling of include and exclude paths
> /// provided by the user through command line options.
> class IncludeExcludeInfo {
> public:
> - /// \brief Determine if the given file is safe to transform.
> + /// \brief Read and parse a comma-seperated lists of paths from
> + /// \a IncludeString and \a ExcludeString.
> ///
> - /// \a Include and \a Exclude must be formatted as a comma-seperated
> list.
> - IncludeExcludeInfo(llvm::StringRef Include, llvm::StringRef Exclude);
> + /// Returns error_code::success() on successful parse of the strings or
> + /// an error_code indicating the encountered error.
> + llvm::error_code readListFromString(llvm::StringRef IncludeString,
> + llvm::StringRef ExcludeString);
>
> - /// \brief Determine if the given filepath is in the list of include
> paths but
> + /// \brief Read and parse the lists of paths from \a IncludeListFile
> + /// and \a ExcludeListFile. Each file should contain one path per line.
> + ///
> + /// Returns error_code::success() on successful read and parse of both
> files
> + /// or an error_code indicating the encountered error.
> + llvm::error_code readListFromFile(llvm::StringRef IncludeListFile,
> + llvm::StringRef ExcludeListFile);
> +
> + /// \brief Determine if the given path is in the list of include paths
> but
> /// not in the list of exclude paths.
> bool isFileIncluded(llvm::StringRef FilePath);
>
>
> Modified: clang-tools-extra/trunk/cpp11-migrate/tool/Cpp11Migrate.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/cpp11-migrate/tool/Cpp11Migrate.cpp?rev=180939&r1=180938&r2=180939&view=diff
>
> ==============================================================================
> --- clang-tools-extra/trunk/cpp11-migrate/tool/Cpp11Migrate.cpp (original)
> +++ clang-tools-extra/trunk/cpp11-migrate/tool/Cpp11Migrate.cpp Thu May 2
> 14:02:02 2013
> @@ -54,12 +54,20 @@ SummaryMode("summary", cl::desc("Print t
> // options are implemented in the tool.
> static cl::opt<std::string>
> IncludePaths("include", cl::Hidden,
> - cl::desc("Comma seperated list of filepaths to consider to
> be "
> + cl::desc("Comma seperated list of paths to consider to be "
> "transformed"));
> static cl::opt<std::string>
> ExcludePaths("exclude", cl::Hidden,
> - cl::desc("Comma seperated list of filepaths that can not "
> + cl::desc("Comma seperated list of paths that can not "
> "be transformed"));
> +static cl::opt<std::string>
> +IncludeFromFile("include-from", cl::Hidden, cl::value_desc("filename"),
> + cl::desc("File containing a list of paths to consider to "
> + "be transformed"));
> +static cl::opt<std::string>
> +ExcludeFromFile("exclude-from", cl::Hidden, cl::value_desc("filename"),
> + cl::desc("File containing a list of paths that can not be
> "
> + "transforms"));
>
> class EndSyntaxArgumentsAdjuster : public ArgumentsAdjuster {
> CommandLineArguments Adjust(const CommandLineArguments &Args) {
>
> Modified: clang-tools-extra/trunk/test/Unit/lit.cfg
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/Unit/lit.cfg?rev=180939&r1=180938&r2=180939&view=diff
>
> ==============================================================================
> --- clang-tools-extra/trunk/test/Unit/lit.cfg (original)
> +++ clang-tools-extra/trunk/test/Unit/lit.cfg Thu May 2 14:02:02 2013
> @@ -7,7 +7,7 @@ config.suffixes = [] # Seems not to matt
> # test binaries are built.
> extra_tools_obj_dir = getattr(config, 'extra_tools_obj_dir', None)
> if extra_tools_obj_dir is not None:
> - config.test_source_root = os.path.join(extra_tools_obj_dir, 'unittests')
> + config.test_source_root = extra_tools_obj_dir
> config.test_exec_root = config.test_source_root
>
> # All GoogleTests are named to have 'Tests' as their suffix. The '.'
> option is
>
> Modified: clang-tools-extra/trunk/test/Unit/lit.site.cfg.in
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/Unit/lit.site.cfg.in?rev=180939&r1=180938&r2=180939&view=diff
>
> ==============================================================================
> --- clang-tools-extra/trunk/test/Unit/lit.site.cfg.in (original)
> +++ clang-tools-extra/trunk/test/Unit/lit.site.cfg.in Thu May 2 14:02:02
> 2013
> @@ -1,6 +1,13 @@
> ## Autogenerated by LLVM/Clang configuration.
> # Do not edit!
> -config.extra_tools_obj_dir = "@CLANG_TOOLS_BINARY_DIR@"
> +config.extra_tools_obj_dir = "@CLANG_TOOLS_BINARY_DIR@/unittests"
> +config.extra_tools_src_dir = "@CLANG_TOOLS_SOURCE_DIR@/unittests"
> config.target_triple = "@TARGET_TRIPLE@"
>
> +# Make sure any custom vars defined above that are required in
> lit.local.cfg
> +# files are made available.
> +def on_clone(parent, clone, path):
> + clone.extra_tools_src_dir = parent.extra_tools_src_dir
> +
> +config.on_clone = on_clone
> lit.load_config(config, "@CLANG_TOOLS_SOURCE_DIR@/test/Unit/lit.cfg")
>
> Modified: clang-tools-extra/trunk/unittests/cpp11-migrate/CMakeLists.txt
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/cpp11-migrate/CMakeLists.txt?rev=180939&r1=180938&r2=180939&view=diff
>
> ==============================================================================
> --- clang-tools-extra/trunk/unittests/cpp11-migrate/CMakeLists.txt
> (original)
> +++ clang-tools-extra/trunk/unittests/cpp11-migrate/CMakeLists.txt Thu May
> 2 14:02:02 2013
> @@ -17,3 +17,5 @@ target_link_libraries(Cpp11MigrateTests
> clangASTMatchers
> )
>
> +configure_file(${CMAKE_CURRENT_SOURCE_DIR}/lit.local.cfg
> + ${CMAKE_CURRENT_BINARY_DIR} COPYONLY)
>
> Added: clang-tools-extra/trunk/unittests/cpp11-migrate/Data/ExcludeData.in
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/cpp11-migrate/Data/ExcludeData.in?rev=180939&view=auto
>
> ==============================================================================
> --- clang-tools-extra/trunk/unittests/cpp11-migrate/Data/ExcludeData.in
> (added)
> +++ clang-tools-extra/trunk/unittests/cpp11-migrate/Data/ExcludeData.in
> Thu May 2 14:02:02 2013
> @@ -0,0 +1,4 @@
> +a/af.cpp
> +a/a2
> +b/b2/b2f.cpp
> +c/c2
>
> Added:
> clang-tools-extra/trunk/unittests/cpp11-migrate/Data/ExcludeDataCRLF.in
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/cpp11-migrate/Data/ExcludeDataCRLF.in?rev=180939&view=auto
>
> ==============================================================================
> ---
> clang-tools-extra/trunk/unittests/cpp11-migrate/Data/ExcludeDataCRLF.in
> (added)
> +++
> clang-tools-extra/trunk/unittests/cpp11-migrate/Data/ExcludeDataCRLF.in Thu
> May 2 14:02:02 2013
> @@ -0,0 +1,4 @@
> +a/af.cpp
> +a/a2
> +b/b2/b2f.cpp
> +c/c2
>
> Added: clang-tools-extra/trunk/unittests/cpp11-migrate/Data/IncludeData.in
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/cpp11-migrate/Data/IncludeData.in?rev=180939&view=auto
>
> ==============================================================================
> --- clang-tools-extra/trunk/unittests/cpp11-migrate/Data/IncludeData.in
> (added)
> +++ clang-tools-extra/trunk/unittests/cpp11-migrate/Data/IncludeData.in
> Thu May 2 14:02:02 2013
> @@ -0,0 +1,3 @@
> +a
> +b/b2
> +c/c2
>
> Added:
> clang-tools-extra/trunk/unittests/cpp11-migrate/Data/IncludeDataCRLF.in
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/cpp11-migrate/Data/IncludeDataCRLF.in?rev=180939&view=auto
>
> ==============================================================================
> ---
> clang-tools-extra/trunk/unittests/cpp11-migrate/Data/IncludeDataCRLF.in
> (added)
> +++
> clang-tools-extra/trunk/unittests/cpp11-migrate/Data/IncludeDataCRLF.in Thu
> May 2 14:02:02 2013
> @@ -0,0 +1,3 @@
> +a
> +b/b2
> +c/c2
>
> Modified:
> clang-tools-extra/trunk/unittests/cpp11-migrate/IncludeExcludeTest.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/cpp11-migrate/IncludeExcludeTest.cpp?rev=180939&r1=180938&r2=180939&view=diff
>
> ==============================================================================
> --- clang-tools-extra/trunk/unittests/cpp11-migrate/IncludeExcludeTest.cpp
> (original)
> +++ clang-tools-extra/trunk/unittests/cpp11-migrate/IncludeExcludeTest.cpp
> Thu May 2 14:02:02 2013
> @@ -1,24 +1,25 @@
> #include "Core/IncludeExcludeInfo.h"
> #include "gtest/gtest.h"
> +#include "llvm/Support/Path.h"
>
> -IncludeExcludeInfo IEManager(/*include=*/ "a,b/b2,c/c2/c3",
> - /*exclude=*/
> "a/af.cpp,a/a2,b/b2/b2f.cpp,c/c2/c3");
> +TEST(IncludeExcludeTest, ParseString) {
> + IncludeExcludeInfo IEManager;
> + llvm::error_code Err = IEManager.readListFromString(
> + /*include=*/ "a,b/b2,c/c2",
> + /*exclude=*/ "a/af.cpp,a/a2,b/b2/b2f.cpp,c/c2");
> +
> + ASSERT_EQ(Err, llvm::error_code::success());
>
> -TEST(IncludeExcludeTest, NoMatchOnIncludeList) {
> // If the file does not appear on the include list then it is not safe
> to
> // transform. Files are not safe to transform by default.
> EXPECT_FALSE(IEManager.isFileIncluded("f.cpp"));
> EXPECT_FALSE(IEManager.isFileIncluded("b/dir/f.cpp"));
> -}
>
> -TEST(IncludeExcludeTest, MatchOnIncludeList) {
> // If the file appears on only the include list then it is safe to
> transform.
> EXPECT_TRUE(IEManager.isFileIncluded("a/f.cpp"));
> EXPECT_TRUE(IEManager.isFileIncluded("a/dir/f.cpp"));
> EXPECT_TRUE(IEManager.isFileIncluded("b/b2/f.cpp"));
> -}
>
> -TEST(IncludeExcludeTest, MatchOnBothLists) {
> // If the file appears on both the include or exclude list then it is
> not
> // safe to transform.
> EXPECT_FALSE(IEManager.isFileIncluded("a/af.cpp"));
> @@ -27,3 +28,51 @@ TEST(IncludeExcludeTest, MatchOnBothList
> EXPECT_FALSE(IEManager.isFileIncluded("b/b2/b2f.cpp"));
> EXPECT_FALSE(IEManager.isFileIncluded("c/c2/c3/f.cpp"));
> }
> +
> +// The IncludeExcludeTest suite requires data files. The location of these
> +// files must be provided in the 'DATADIR' environment variable.
> +class IncludeExcludeFileTest : public ::testing::Test {
> +public:
> + virtual void SetUp() {
> + DataDir = getenv("DATADIR");
> + if (DataDir == 0) {
> + FAIL()
> + << "IncludeExcludeFileTest requires the DATADIR environment
> variable "
> + "to be set.";
> + }
> + }
> +
> + const char *DataDir;
> +};
> +
> +TEST_F(IncludeExcludeFileTest, UNIXFile) {
> + llvm::SmallString<128> IncludeData(DataDir);
> + llvm::SmallString<128> ExcludeData(IncludeData);
> + llvm::sys::path::append(IncludeData, "IncludeData.in");
> + llvm::sys::path::append(ExcludeData, "ExcludeData.in");
> +
> + IncludeExcludeInfo IEManager;
> + llvm::error_code Err = IEManager.readListFromFile(IncludeData,
> ExcludeData);
> +
> + ASSERT_EQ(Err, llvm::error_code::success());
> +
> + EXPECT_FALSE(IEManager.isFileIncluded("f.cpp"));
> + EXPECT_TRUE(IEManager.isFileIncluded("a/f.cpp"));
> + EXPECT_FALSE(IEManager.isFileIncluded("a/af.cpp"));
> +}
> +
> +TEST_F(IncludeExcludeFileTest, DOSFile) {
> + llvm::SmallString<128> IncludeData(DataDir);
> + llvm::SmallString<128> ExcludeData(IncludeData);
> + llvm::sys::path::append(IncludeData, "IncludeDataCRLF.in");
> + llvm::sys::path::append(ExcludeData, "ExcludeDataCRLF.in");
> +
> + IncludeExcludeInfo IEManager;
> + llvm::error_code Err = IEManager.readListFromFile(IncludeData,
> ExcludeData);
> +
> + ASSERT_EQ(Err, llvm::error_code::success());
> +
> + EXPECT_FALSE(IEManager.isFileIncluded("f.cpp"));
> + EXPECT_TRUE(IEManager.isFileIncluded("a/f.cpp"));
> + EXPECT_FALSE(IEManager.isFileIncluded("a/af.cpp"));
> +}
>
> Modified: clang-tools-extra/trunk/unittests/cpp11-migrate/Makefile
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/cpp11-migrate/Makefile?rev=180939&r1=180938&r2=180939&view=diff
>
> ==============================================================================
> --- clang-tools-extra/trunk/unittests/cpp11-migrate/Makefile (original)
> +++ clang-tools-extra/trunk/unittests/cpp11-migrate/Makefile Thu May 2
> 14:02:02 2013
> @@ -22,3 +22,8 @@ include $(CLANG_LEVEL)/Makefile
> MAKEFILE_UNITTEST_NO_INCLUDE_COMMON := 1
> CPP.Flags += -I$(PROJ_SRC_DIR)/../../cpp11-migrate
> include $(LLVM_SRC_ROOT)/unittests/Makefile.unittest
> +
> +$(PROJ_OBJ_DIR)/lit.local.cfg: $(PROJ_SRC_DIR)/lit.local.cfg
> + @cp $< $@
> +
> +all:: $(PROJ_OBJ_DIR)/lit.local.cfg
>
> Added: clang-tools-extra/trunk/unittests/cpp11-migrate/lit.local.cfg
> URL:
> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/unittests/cpp11-migrate/lit.local.cfg?rev=180939&view=auto
>
> ==============================================================================
> --- clang-tools-extra/trunk/unittests/cpp11-migrate/lit.local.cfg (added)
> +++ clang-tools-extra/trunk/unittests/cpp11-migrate/lit.local.cfg Thu May
> 2 14:02:02 2013
> @@ -0,0 +1,4 @@
> +# Some tests require access to data files which are stored in the 'Data'
> +# subdirectory. This environment variable indicates where to find those
> files.
> +config.environment['DATADIR'] =
> os.path.normpath(os.path.join(config.extra_tools_src_dir,
> +
> 'cpp11-migrate', 'Data'))
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20130503/0d3206d4/attachment.html>
More information about the cfe-commits
mailing list