r217792 - Teach Clang how to use response files when calling other tools
Reid Kleckner
reid at kleckner.net
Mon Sep 15 10:45:40 PDT 2014
Author: rnk
Date: Mon Sep 15 12:45:39 2014
New Revision: 217792
URL: http://llvm.org/viewvc/llvm-project?rev=217792&view=rev
Log:
Teach Clang how to use response files when calling other tools
Patch by Rafael Auler!
This patch addresses PR15171 and teaches Clang how to call other tools
with response files, when the command line exceeds system limits. This
is a problem for Windows systems, whose maximum command-line length is
32kb.
I introduce the concept of "response file support" for each Tool object.
A given Tool may have full support for response files (e.g. MSVC's
link.exe) or only support file names inside response files, but no flags
(e.g. Apple's ld64, as commented in PR15171), or no support at all (the
default case). Therefore, if you implement a toolchain in the clang
driver and you want clang to be able to use response files in your
tools, you must override a method (getReponseFileSupport()) to tell so.
I designed it to support different kinds of tools and
internationalisation needs:
- VS response files ( UTF-16 )
- GNU tools ( uses system's current code page, windows' legacy intl.
support, with escaped backslashes. On unix, fallback to UTF-8 )
- Clang itself ( UTF-16 on windows, UTF-8 on unix )
- ld64 response files ( only a limited file list, UTF-8 on unix )
With this design, I was able to test input file names with spaces and
international characters for Windows. When the linker input is large
enough, it creates a response file with the correct encoding. On a Mac,
to test ld64, I temporarily changed Clang's behavior to always use
response files regardless of the command size limit (avoiding using huge
command line inputs). I tested clang with the LLVM test suite (compiling
benchmarks) and it did fine.
Test Plan: A LIT test that tests proper response files support. This is
tricky, since, for Unix systems, we need a 2MB response file, otherwise
Clang will simply use regular arguments instead of a response file. To
do this, my LIT test generate the file on the fly by cloning many -DTEST
parameters until we have a 2MB file. I found out that processing 2MB of
arguments is pretty slow, it takes 1 minute using my notebook in a debug
build, or 10s in a Release build. Therefore, I also added "REQUIRES:
long_tests", so it will only run when the user wants to run long tests.
In the full discussion in
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20130408/171463.html,
Rafael Espindola discusses a proper way to test
llvm::sys::argumentsFitWithinSystemLimits(), and, there, Chandler
suggests to use 10 times the current system limit (20MB resp file), so
we guarantee that the system will always use response file, even if a
new linux comes up that can handle a few more bytes of arguments.
However, by testing with a 20MB resp file, the test takes long 8 minutes
just to perform a silly check to see if the driver will use a response
file. I found it to be unreasonable. Thus, I discarded this approach and
uses a 2MB response file, which should be enough.
Reviewers: asl, rafael, silvas
Reviewed By: silvas
Subscribers: silvas, rnk, thakis, cfe-commits
Differential Revision: http://reviews.llvm.org/D4897
Added:
cfe/trunk/test/Driver/Inputs/gen-response.c
cfe/trunk/test/Driver/response-file.c
Modified:
cfe/trunk/include/clang/Driver/Driver.h
cfe/trunk/include/clang/Driver/Job.h
cfe/trunk/include/clang/Driver/Tool.h
cfe/trunk/lib/Driver/Compilation.cpp
cfe/trunk/lib/Driver/Driver.cpp
cfe/trunk/lib/Driver/Job.cpp
cfe/trunk/lib/Driver/Tool.cpp
cfe/trunk/lib/Driver/Tools.cpp
cfe/trunk/lib/Driver/Tools.h
Modified: cfe/trunk/include/clang/Driver/Driver.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/Driver.h?rev=217792&r1=217791&r2=217792&view=diff
==============================================================================
--- cfe/trunk/include/clang/Driver/Driver.h (original)
+++ cfe/trunk/include/clang/Driver/Driver.h Mon Sep 15 12:45:39 2014
@@ -42,6 +42,7 @@ namespace driver {
class Command;
class Compilation;
class InputInfo;
+ class Job;
class JobAction;
class SanitizerArgs;
class ToolChain;
@@ -190,6 +191,9 @@ private:
phases::ID getFinalPhase(const llvm::opt::DerivedArgList &DAL,
llvm::opt::Arg **FinalPhaseArg = nullptr) const;
+ // Before executing jobs, sets up response files for commands that need them.
+ void setUpResponseFiles(Compilation &C, Job &J);
+
public:
Driver(StringRef _ClangExecutable,
StringRef _DefaultTargetTriple,
@@ -291,10 +295,10 @@ public:
/// arguments and return an appropriate exit code.
///
/// This routine handles additional processing that must be done in addition
- /// to just running the subprocesses, for example reporting errors, removing
- /// temporary files, etc.
- int ExecuteCompilation(const Compilation &C,
- SmallVectorImpl< std::pair<int, const Command *> > &FailingCommands) const;
+ /// to just running the subprocesses, for example reporting errors, setting
+ /// up response files, removing temporary files, etc.
+ int ExecuteCompilation(Compilation &C,
+ SmallVectorImpl< std::pair<int, const Command *> > &FailingCommands);
/// generateCompilationDiagnostics - Generate diagnostics information
/// including preprocessed source file(s).
Modified: cfe/trunk/include/clang/Driver/Job.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/Job.h?rev=217792&r1=217791&r2=217792&view=diff
==============================================================================
--- cfe/trunk/include/clang/Driver/Job.h (original)
+++ cfe/trunk/include/clang/Driver/Job.h Mon Sep 15 12:45:39 2014
@@ -72,6 +72,30 @@ class Command : public Job {
/// argument, which will be the executable).
llvm::opt::ArgStringList Arguments;
+ /// Response file name, if this command is set to use one, or nullptr
+ /// otherwise
+ const char *ResponseFile;
+
+ /// The input file list in case we need to emit a file list instead of a
+ /// proper response file
+ llvm::opt::ArgStringList InputFileList;
+
+ /// String storage if we need to create a new argument to specify a response
+ /// file
+ std::string ResponseFileFlag;
+
+ /// When a response file is needed, we try to put most arguments in an
+ /// exclusive file, while others remains as regular command line arguments.
+ /// This functions fills a vector with the regular command line arguments,
+ /// argv, excluding the ones passed in a response file.
+ void buildArgvForResponseFile(llvm::SmallVectorImpl<const char *> &Out) const;
+
+ /// Encodes an array of C strings into a single string separated by whitespace.
+ /// This function will also put in quotes arguments that have whitespaces and
+ /// will escape the regular backslashes (used in Windows paths) and quotes.
+ /// The results are the contents of a response file, written into a raw_ostream.
+ void writeResponseFile(raw_ostream &OS) const;
+
public:
Command(const Action &_Source, const Tool &_Creator, const char *_Executable,
const llvm::opt::ArgStringList &_Arguments);
@@ -88,6 +112,15 @@ public:
/// getCreator - Return the Tool which caused the creation of this job.
const Tool &getCreator() const { return Creator; }
+ /// Set to pass arguments via a response file when launching the command
+ void setResponseFile(const char *FileName);
+
+ /// Set an input file list, necessary if we need to use a response file but
+ /// the tool being called only supports input files lists.
+ void setInputFileList(llvm::opt::ArgStringList List) {
+ InputFileList = std::move(List);
+ }
+
const char *getExecutable() const { return Executable; }
const llvm::opt::ArgStringList &getArguments() const { return Arguments; }
Modified: cfe/trunk/include/clang/Driver/Tool.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/Tool.h?rev=217792&r1=217791&r2=217792&view=diff
==============================================================================
--- cfe/trunk/include/clang/Driver/Tool.h (original)
+++ cfe/trunk/include/clang/Driver/Tool.h Mon Sep 15 12:45:39 2014
@@ -11,6 +11,7 @@
#define LLVM_CLANG_DRIVER_TOOL_H
#include "clang/Basic/LLVM.h"
+#include "llvm/Support/Program.h"
namespace llvm {
namespace opt {
@@ -31,6 +32,24 @@ namespace driver {
/// Tool - Information on a specific compilation tool.
class Tool {
+public:
+ // Documents the level of support for response files in this tool.
+ // Response files are necessary if the command line gets too large,
+ // requiring the arguments to be transfered to a file.
+ enum ResponseFileSupport {
+ // Provides full support for response files, which means we can transfer
+ // all tool input arguments to a file. E.g.: clang, gcc, binutils and MSVC
+ // tools.
+ RF_Full,
+ // Input file names can live in a file, but flags can't. E.g.: ld64 (Mac
+ // OS X linker).
+ RF_FileList,
+ // Does not support response files: all arguments must be passed via
+ // command line.
+ RF_None
+ };
+
+private:
/// The tool name (for debugging).
const char *Name;
@@ -40,9 +59,20 @@ class Tool {
/// The tool chain this tool is a part of.
const ToolChain &TheToolChain;
+ /// The level of support for response files seen in this tool
+ const ResponseFileSupport ResponseSupport;
+
+ /// The encoding to use when writing response files for this tool on Windows
+ const llvm::sys::WindowsEncodingMethod ResponseEncoding;
+
+ /// The flag used to pass a response file via command line to this tool
+ const char *const ResponseFlag;
+
public:
- Tool(const char *Name, const char *ShortName,
- const ToolChain &TC);
+ Tool(const char *Name, const char *ShortName, const ToolChain &TC,
+ ResponseFileSupport ResponseSupport = RF_None,
+ llvm::sys::WindowsEncodingMethod ResponseEncoding = llvm::sys::WEM_UTF8,
+ const char *ResponseFlag = "@");
public:
virtual ~Tool();
@@ -57,6 +87,29 @@ public:
virtual bool hasIntegratedCPP() const = 0;
virtual bool isLinkJob() const { return false; }
virtual bool isDsymutilJob() const { return false; }
+ /// \brief Returns the level of support for response files of this tool,
+ /// whether it accepts arguments to be passed via a file on disk.
+ ResponseFileSupport getResponseFilesSupport() const {
+ return ResponseSupport;
+ }
+ /// \brief Returns which encoding the response file should use. This is only
+ /// relevant on Windows platforms where there are different encodings being
+ /// accepted for different tools. On UNIX, UTF8 is universal.
+ ///
+ /// Windows use cases: - GCC and Binutils on mingw only accept ANSI response
+ /// files encoded with the system current code page.
+ /// - MSVC's CL.exe and LINK.exe accept UTF16 on Windows.
+ /// - Clang accepts both UTF8 and UTF16.
+ ///
+ /// FIXME: When GNU tools learn how to parse UTF16 on Windows, we should
+ /// always use UTF16 for Windows, which is the Windows official encoding for
+ /// international characters.
+ llvm::sys::WindowsEncodingMethod getResponseFileEncoding() const {
+ return ResponseEncoding;
+ }
+ /// \brief Returns which prefix to use when passing the name of a response
+ /// file as a parameter to this tool.
+ const char *getResponseFileFlag() const { return ResponseFlag; }
/// \brief Does this tool have "good" standardized diagnostics, or should the
/// driver add an additional "command failed" diagnostic on failures.
Modified: cfe/trunk/lib/Driver/Compilation.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/Compilation.cpp?rev=217792&r1=217791&r2=217792&view=diff
==============================================================================
--- cfe/trunk/lib/Driver/Compilation.cpp (original)
+++ cfe/trunk/lib/Driver/Compilation.cpp Mon Sep 15 12:45:39 2014
@@ -88,7 +88,7 @@ bool Compilation::CleanupFile(const char
// Failure is only failure if the file exists and is "regular". We checked
// for it being regular before, and llvm::sys::fs::remove ignores ENOENT,
// so we don't need to check again.
-
+
if (IssueErrors)
getDriver().Diag(clang::diag::err_drv_unable_to_remove_file)
<< EC.message();
Modified: cfe/trunk/lib/Driver/Driver.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/Driver.cpp?rev=217792&r1=217791&r2=217792&view=diff
==============================================================================
--- cfe/trunk/lib/Driver/Driver.cpp (original)
+++ cfe/trunk/lib/Driver/Driver.cpp Mon Sep 15 12:45:39 2014
@@ -576,8 +576,31 @@ void Driver::generateCompilationDiagnost
}
}
-int Driver::ExecuteCompilation(const Compilation &C,
- SmallVectorImpl< std::pair<int, const Command *> > &FailingCommands) const {
+void Driver::setUpResponseFiles(Compilation &C, Job &J) {
+ if (JobList *Jobs = dyn_cast<JobList>(&J)) {
+ for (JobList::iterator I = Jobs->begin(), E = Jobs->end(); I != E; ++I)
+ setUpResponseFiles(C, **I);
+ return;
+ }
+
+ Command *CurCommand = dyn_cast<Command>(&J);
+ if (!CurCommand)
+ return;
+
+ // Since argumentsFitWithinSystemLimits() may underestimate system's capacity
+ // if the tool does not support response files, there is a chance/ that things
+ // will just work without a response file, so we silently just skip it.
+ if (CurCommand->getCreator().getResponseFilesSupport() == Tool::RF_None ||
+ llvm::sys::argumentsFitWithinSystemLimits(CurCommand->getArguments()))
+ return;
+
+ std::string TmpName = GetTemporaryPath("response", "txt");
+ CurCommand->setResponseFile(C.addTempFile(C.getArgs().MakeArgString(
+ TmpName.c_str())));
+}
+
+int Driver::ExecuteCompilation(Compilation &C,
+ SmallVectorImpl< std::pair<int, const Command *> > &FailingCommands) {
// Just print if -### was present.
if (C.getArgs().hasArg(options::OPT__HASH_HASH_HASH)) {
C.getJobs().Print(llvm::errs(), "\n", true);
@@ -588,6 +611,9 @@ int Driver::ExecuteCompilation(const Com
if (Diags.hasErrorOccurred())
return 1;
+ // Set up response file names for each command, if necessary
+ setUpResponseFiles(C, C.getJobs());
+
C.ExecuteJob(C.getJobs(), FailingCommands);
// Remove temp files.
Modified: cfe/trunk/lib/Driver/Job.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/Job.cpp?rev=217792&r1=217791&r2=217792&view=diff
==============================================================================
--- cfe/trunk/lib/Driver/Job.cpp (original)
+++ cfe/trunk/lib/Driver/Job.cpp Mon Sep 15 12:45:39 2014
@@ -12,8 +12,10 @@
#include "clang/Driver/Job.h"
#include "clang/Driver/Tool.h"
#include "clang/Driver/ToolChain.h"
+#include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/StringSet.h"
#include "llvm/ADT/StringSwitch.h"
#include "llvm/Support/Program.h"
#include "llvm/Support/raw_ostream.h"
@@ -21,6 +23,7 @@
using namespace clang::driver;
using llvm::raw_ostream;
using llvm::StringRef;
+using llvm::ArrayRef;
Job::~Job() {}
@@ -28,7 +31,8 @@ Command::Command(const Action &_Source,
const char *_Executable,
const ArgStringList &_Arguments)
: Job(CommandClass), Source(_Source), Creator(_Creator),
- Executable(_Executable), Arguments(_Arguments) {}
+ Executable(_Executable), Arguments(_Arguments),
+ ResponseFile(nullptr) {}
static int skipArgs(const char *Flag) {
// These flags are all of the form -Flag <Arg> and are treated as two
@@ -93,14 +97,74 @@ static void PrintArg(raw_ostream &OS, co
OS << '"';
}
+void Command::writeResponseFile(raw_ostream &OS) const {
+ // In a file list, we only write the set of inputs to the response file
+ if (Creator.getResponseFilesSupport() == Tool::RF_FileList) {
+ for (const char *Arg : InputFileList) {
+ OS << Arg << '\n';
+ }
+ return;
+ }
+
+ // In regular response files, we send all arguments to the response file
+ for (const char *Arg : Arguments) {
+ OS << '"';
+
+ for (; *Arg != '\0'; Arg++) {
+ if (*Arg == '\"' || *Arg == '\\') {
+ OS << '\\';
+ }
+ OS << *Arg;
+ }
+
+ OS << "\" ";
+ }
+}
+
+void Command::buildArgvForResponseFile(
+ llvm::SmallVectorImpl<const char *> &Out) const {
+ // When not a file list, all arguments are sent to the response file.
+ // This leaves us to set the argv to a single parameter, requesting the tool
+ // to read the response file.
+ if (Creator.getResponseFilesSupport() != Tool::RF_FileList) {
+ Out.push_back(Executable);
+ Out.push_back(ResponseFileFlag.c_str());
+ return;
+ }
+
+ llvm::StringSet<> Inputs;
+ for (const char *InputName : InputFileList)
+ Inputs.insert(InputName);
+ Out.push_back(Executable);
+ // In a file list, build args vector ignoring parameters that will go in the
+ // response file (elements of the InputFileList vector)
+ bool FirstInput = true;
+ for (const char *Arg : Arguments) {
+ if (Inputs.count(Arg) == 0) {
+ Out.push_back(Arg);
+ } else if (FirstInput) {
+ FirstInput = false;
+ Out.push_back(Creator.getResponseFileFlag());
+ Out.push_back(ResponseFile);
+ }
+ }
+}
+
void Command::Print(raw_ostream &OS, const char *Terminator, bool Quote,
bool CrashReport) const {
// Always quote the exe.
OS << ' ';
PrintArg(OS, Executable, /*Quote=*/true);
- for (size_t i = 0, e = Arguments.size(); i < e; ++i) {
- const char *const Arg = Arguments[i];
+ llvm::ArrayRef<const char *> Args = Arguments;
+ llvm::SmallVector<const char *, 128> ArgsRespFile;
+ if (ResponseFile != nullptr) {
+ buildArgvForResponseFile(ArgsRespFile);
+ Args = ArrayRef<const char *>(ArgsRespFile).slice(1); // no executable name
+ }
+
+ for (size_t i = 0, e = Args.size(); i < e; ++i) {
+ const char *const Arg = Args[i];
if (CrashReport) {
if (int Skip = skipArgs(Arg)) {
@@ -114,19 +178,65 @@ void Command::Print(raw_ostream &OS, con
if (CrashReport && quoteNextArg(Arg) && i + 1 < e) {
OS << ' ';
- PrintArg(OS, Arguments[++i], true);
+ PrintArg(OS, Args[++i], true);
}
}
+
+ if (ResponseFile != nullptr) {
+ OS << "\n Arguments passed via response file:\n";
+ writeResponseFile(OS);
+ // Avoiding duplicated newline terminator, since FileLists are
+ // newline-separated.
+ if (Creator.getResponseFilesSupport() != Tool::RF_FileList)
+ OS << "\n";
+ OS << " (end of response file)";
+ }
+
OS << Terminator;
}
+void Command::setResponseFile(const char *FileName) {
+ ResponseFile = FileName;
+ ResponseFileFlag = Creator.getResponseFileFlag();
+ ResponseFileFlag += FileName;
+}
+
int Command::Execute(const StringRef **Redirects, std::string *ErrMsg,
bool *ExecutionFailed) const {
SmallVector<const char*, 128> Argv;
- Argv.push_back(Executable);
- for (size_t i = 0, e = Arguments.size(); i != e; ++i)
- Argv.push_back(Arguments[i]);
+
+ if (ResponseFile == nullptr) {
+ Argv.push_back(Executable);
+ for (size_t i = 0, e = Arguments.size(); i != e; ++i)
+ Argv.push_back(Arguments[i]);
+ Argv.push_back(nullptr);
+
+ return llvm::sys::ExecuteAndWait(Executable, Argv.data(), /*env*/ nullptr,
+ Redirects, /*secondsToWait*/ 0,
+ /*memoryLimit*/ 0, ErrMsg,
+ ExecutionFailed);
+ }
+
+ // We need to put arguments in a response file (command is too large)
+ // Open stream to store the response file contents
+ std::string RespContents;
+ llvm::raw_string_ostream SS(RespContents);
+
+ // Write file contents and build the Argv vector
+ writeResponseFile(SS);
+ buildArgvForResponseFile(Argv);
Argv.push_back(nullptr);
+ SS.flush();
+
+ // Save the response file in the appropriate encoding
+ if (std::error_code EC = writeFileWithEncoding(
+ ResponseFile, RespContents, Creator.getResponseFileEncoding())) {
+ if (ErrMsg)
+ *ErrMsg = EC.message();
+ if (ExecutionFailed)
+ *ExecutionFailed = true;
+ return -1;
+ }
return llvm::sys::ExecuteAndWait(Executable, Argv.data(), /*env*/ nullptr,
Redirects, /*secondsToWait*/ 0,
Modified: cfe/trunk/lib/Driver/Tool.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/Tool.cpp?rev=217792&r1=217791&r2=217792&view=diff
==============================================================================
--- cfe/trunk/lib/Driver/Tool.cpp (original)
+++ cfe/trunk/lib/Driver/Tool.cpp Mon Sep 15 12:45:39 2014
@@ -11,11 +11,13 @@
using namespace clang::driver;
-Tool::Tool(const char *_Name, const char *_ShortName,
- const ToolChain &TC) : Name(_Name), ShortName(_ShortName),
- TheToolChain(TC)
-{
-}
+Tool::Tool(const char *_Name, const char *_ShortName, const ToolChain &TC,
+ ResponseFileSupport _ResponseSupport,
+ llvm::sys::WindowsEncodingMethod _ResponseEncoding,
+ const char *_ResponseFlag)
+ : Name(_Name), ShortName(_ShortName), TheToolChain(TC),
+ ResponseSupport(_ResponseSupport), ResponseEncoding(_ResponseEncoding),
+ ResponseFlag(_ResponseFlag) {}
Tool::~Tool() {
}
Modified: cfe/trunk/lib/Driver/Tools.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/Tools.cpp?rev=217792&r1=217791&r2=217792&view=diff
==============================================================================
--- cfe/trunk/lib/Driver/Tools.cpp (original)
+++ cfe/trunk/lib/Driver/Tools.cpp Mon Sep 15 12:45:39 2014
@@ -506,7 +506,7 @@ static void getARMHWDivFeatures(const Dr
} else
D.Diag(diag::err_drv_clang_unsupported) << A->getAsString(Args);
}
-
+
// Handle -mfpu=.
//
// FIXME: Centralize feature selection, defaulting shouldn't be also in the
@@ -4928,6 +4928,8 @@ void ClangAs::ConstructJob(Compilation &
SplitDebugName(Args, Inputs));
}
+void GnuTool::anchor() {}
+
void gcc::Common::ConstructJob(Compilation &C, const JobAction &JA,
const InputInfo &Output,
const InputInfoList &Inputs,
@@ -5782,6 +5784,12 @@ void darwin::Link::ConstructJob(Compilat
const char *LinkingOutput) const {
assert(Output.getType() == types::TY_Image && "Invalid linker output type.");
+ // If the number of arguments surpasses the system limits, we will encode the
+ // input files in a separate file, shortening the command line. To this end,
+ // build a list of input file names that can be passed via a file with the
+ // -filelist linker option.
+ llvm::opt::ArgStringList InputFileList;
+
// The logic here is derived from gcc's behavior; most of which
// comes from specs (starting with link_command). Consult gcc for
// more information.
@@ -5850,7 +5858,23 @@ void darwin::Link::ConstructJob(Compilat
}
AddLinkerInputs(getToolChain(), Inputs, Args, CmdArgs);
-
+ // Build the input file for -filelist (list of linker input files) in case we
+ // need it later
+ for (const auto &II : Inputs) {
+ if (!II.isFilename()) {
+ // This is a linker input argument.
+ // We cannot mix input arguments and file names in a -filelist input, thus
+ // we prematurely stop our list (remaining files shall be passed as
+ // arguments).
+ if (InputFileList.size() > 0)
+ break;
+
+ continue;
+ }
+
+ InputFileList.push_back(II.getFilename());
+ }
+
if (isObjCRuntimeLinked(Args) &&
!Args.hasArg(options::OPT_nostdlib) &&
!Args.hasArg(options::OPT_nodefaultlibs)) {
@@ -5893,7 +5917,10 @@ void darwin::Link::ConstructJob(Compilat
const char *Exec =
Args.MakeArgString(getToolChain().GetLinkerPath());
- C.addCommand(llvm::make_unique<Command>(JA, *this, Exec, CmdArgs));
+ std::unique_ptr<Command> Cmd =
+ llvm::make_unique<Command>(JA, *this, Exec, CmdArgs);
+ Cmd->setInputFileList(std::move(InputFileList));
+ C.addCommand(std::move(Cmd));
}
void darwin::Lipo::ConstructJob(Compilation &C, const JobAction &JA,
Modified: cfe/trunk/lib/Driver/Tools.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/Tools.h?rev=217792&r1=217791&r2=217792&view=diff
==============================================================================
--- cfe/trunk/lib/Driver/Tools.h (original)
+++ cfe/trunk/lib/Driver/Tools.h Mon Sep 15 12:45:39 2014
@@ -90,7 +90,7 @@ using llvm::opt::ArgStringList;
mutable std::unique_ptr<visualstudio::Compile> CLFallback;
public:
- Clang(const ToolChain &TC) : Tool("clang", "clang frontend", TC) {}
+ Clang(const ToolChain &TC) : Tool("clang", "clang frontend", TC, RF_Full) {}
bool hasGoodDiagnostics() const override { return true; }
bool hasIntegratedAssembler() const override { return true; }
@@ -106,7 +106,8 @@ using llvm::opt::ArgStringList;
class LLVM_LIBRARY_VISIBILITY ClangAs : public Tool {
public:
ClangAs(const ToolChain &TC) : Tool("clang::as",
- "clang integrated assembler", TC) {}
+ "clang integrated assembler", TC,
+ RF_Full) {}
bool hasGoodDiagnostics() const override { return true; }
bool hasIntegratedAssembler() const override { return false; }
@@ -118,12 +119,22 @@ using llvm::opt::ArgStringList;
const char *LinkingOutput) const override;
};
+ /// \brief Base class for all GNU tools that provide the same behavior when
+ /// it comes to response files support
+ class GnuTool : public Tool {
+ virtual void anchor();
+
+ public:
+ GnuTool(const char *Name, const char *ShortName, const ToolChain &TC)
+ : Tool(Name, ShortName, TC, RF_Full, llvm::sys::WEM_CurrentCodePage) {}
+ };
+
/// gcc - Generic GCC tool implementations.
namespace gcc {
- class LLVM_LIBRARY_VISIBILITY Common : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Common : public GnuTool {
public:
Common(const char *Name, const char *ShortName,
- const ToolChain &TC) : Tool(Name, ShortName, TC) {}
+ const ToolChain &TC) : GnuTool(Name, ShortName, TC) {}
void ConstructJob(Compilation &C, const JobAction &JA,
const InputInfo &Output,
@@ -178,9 +189,9 @@ namespace gcc {
namespace hexagon {
// For Hexagon, we do not need to instantiate tools for PreProcess, PreCompile and Compile.
// We simply use "clang -cc1" for those actions.
- class LLVM_LIBRARY_VISIBILITY Assemble : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Assemble : public GnuTool {
public:
- Assemble(const ToolChain &TC) : Tool("hexagon::Assemble",
+ Assemble(const ToolChain &TC) : GnuTool("hexagon::Assemble",
"hexagon-as", TC) {}
bool hasIntegratedCPP() const override { return false; }
@@ -193,9 +204,9 @@ namespace hexagon {
const char *LinkingOutput) const override;
};
- class LLVM_LIBRARY_VISIBILITY Link : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Link : public GnuTool {
public:
- Link(const ToolChain &TC) : Tool("hexagon::Link",
+ Link(const ToolChain &TC) : GnuTool("hexagon::Link",
"hexagon-ld", TC) {}
bool hasIntegratedCPP() const override { return false; }
@@ -248,8 +259,13 @@ namespace darwin {
}
public:
- MachOTool(const char *Name, const char *ShortName,
- const ToolChain &TC) : Tool(Name, ShortName, TC) {}
+ MachOTool(
+ const char *Name, const char *ShortName, const ToolChain &TC,
+ ResponseFileSupport ResponseSupport = RF_None,
+ llvm::sys::WindowsEncodingMethod ResponseEncoding = llvm::sys::WEM_UTF8,
+ const char *ResponseFlag = "@")
+ : Tool(Name, ShortName, TC, ResponseSupport, ResponseEncoding,
+ ResponseFlag) {}
};
class LLVM_LIBRARY_VISIBILITY Assemble : public MachOTool {
@@ -272,7 +288,9 @@ namespace darwin {
const InputInfoList &Inputs) const;
public:
- Link(const ToolChain &TC) : MachOTool("darwin::Link", "linker", TC) {}
+ Link(const ToolChain &TC) : MachOTool("darwin::Link", "linker", TC,
+ RF_FileList, llvm::sys::WEM_UTF8,
+ "-filelist") {}
bool hasIntegratedCPP() const override { return false; }
bool isLinkJob() const override { return true; }
@@ -327,9 +345,9 @@ namespace darwin {
/// openbsd -- Directly call GNU Binutils assembler and linker
namespace openbsd {
- class LLVM_LIBRARY_VISIBILITY Assemble : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Assemble : public GnuTool {
public:
- Assemble(const ToolChain &TC) : Tool("openbsd::Assemble", "assembler",
+ Assemble(const ToolChain &TC) : GnuTool("openbsd::Assemble", "assembler",
TC) {}
bool hasIntegratedCPP() const override { return false; }
@@ -340,9 +358,9 @@ namespace openbsd {
const llvm::opt::ArgList &TCArgs,
const char *LinkingOutput) const override;
};
- class LLVM_LIBRARY_VISIBILITY Link : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Link : public GnuTool {
public:
- Link(const ToolChain &TC) : Tool("openbsd::Link", "linker", TC) {}
+ Link(const ToolChain &TC) : GnuTool("openbsd::Link", "linker", TC) {}
bool hasIntegratedCPP() const override { return false; }
bool isLinkJob() const override { return true; }
@@ -356,9 +374,9 @@ namespace openbsd {
/// bitrig -- Directly call GNU Binutils assembler and linker
namespace bitrig {
- class LLVM_LIBRARY_VISIBILITY Assemble : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Assemble : public GnuTool {
public:
- Assemble(const ToolChain &TC) : Tool("bitrig::Assemble", "assembler",
+ Assemble(const ToolChain &TC) : GnuTool("bitrig::Assemble", "assembler",
TC) {}
bool hasIntegratedCPP() const override { return false; }
@@ -368,9 +386,9 @@ namespace bitrig {
const llvm::opt::ArgList &TCArgs,
const char *LinkingOutput) const override;
};
- class LLVM_LIBRARY_VISIBILITY Link : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Link : public GnuTool {
public:
- Link(const ToolChain &TC) : Tool("bitrig::Link", "linker", TC) {}
+ Link(const ToolChain &TC) : GnuTool("bitrig::Link", "linker", TC) {}
bool hasIntegratedCPP() const override { return false; }
bool isLinkJob() const override { return true; }
@@ -384,9 +402,9 @@ namespace bitrig {
/// freebsd -- Directly call GNU Binutils assembler and linker
namespace freebsd {
- class LLVM_LIBRARY_VISIBILITY Assemble : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Assemble : public GnuTool {
public:
- Assemble(const ToolChain &TC) : Tool("freebsd::Assemble", "assembler",
+ Assemble(const ToolChain &TC) : GnuTool("freebsd::Assemble", "assembler",
TC) {}
bool hasIntegratedCPP() const override { return false; }
@@ -396,9 +414,9 @@ namespace freebsd {
const llvm::opt::ArgList &TCArgs,
const char *LinkingOutput) const override;
};
- class LLVM_LIBRARY_VISIBILITY Link : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Link : public GnuTool {
public:
- Link(const ToolChain &TC) : Tool("freebsd::Link", "linker", TC) {}
+ Link(const ToolChain &TC) : GnuTool("freebsd::Link", "linker", TC) {}
bool hasIntegratedCPP() const override { return false; }
bool isLinkJob() const override { return true; }
@@ -412,11 +430,11 @@ namespace freebsd {
/// netbsd -- Directly call GNU Binutils assembler and linker
namespace netbsd {
- class LLVM_LIBRARY_VISIBILITY Assemble : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Assemble : public GnuTool {
public:
Assemble(const ToolChain &TC)
- : Tool("netbsd::Assemble", "assembler", TC) {}
+ : GnuTool("netbsd::Assemble", "assembler", TC) {}
bool hasIntegratedCPP() const override { return false; }
@@ -425,11 +443,11 @@ namespace netbsd {
const llvm::opt::ArgList &TCArgs,
const char *LinkingOutput) const override;
};
- class LLVM_LIBRARY_VISIBILITY Link : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Link : public GnuTool {
public:
Link(const ToolChain &TC)
- : Tool("netbsd::Link", "linker", TC) {}
+ : GnuTool("netbsd::Link", "linker", TC) {}
bool hasIntegratedCPP() const override { return false; }
bool isLinkJob() const override { return true; }
@@ -443,9 +461,9 @@ namespace netbsd {
/// Directly call GNU Binutils' assembler and linker.
namespace gnutools {
- class LLVM_LIBRARY_VISIBILITY Assemble : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Assemble : public GnuTool {
public:
- Assemble(const ToolChain &TC) : Tool("GNU::Assemble", "assembler", TC) {}
+ Assemble(const ToolChain &TC) : GnuTool("GNU::Assemble", "assembler", TC) {}
bool hasIntegratedCPP() const override { return false; }
@@ -455,9 +473,9 @@ namespace gnutools {
const llvm::opt::ArgList &TCArgs,
const char *LinkingOutput) const override;
};
- class LLVM_LIBRARY_VISIBILITY Link : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Link : public GnuTool {
public:
- Link(const ToolChain &TC) : Tool("GNU::Link", "linker", TC) {}
+ Link(const ToolChain &TC) : GnuTool("GNU::Link", "linker", TC) {}
bool hasIntegratedCPP() const override { return false; }
bool isLinkJob() const override { return true; }
@@ -471,9 +489,9 @@ namespace gnutools {
}
/// minix -- Directly call GNU Binutils assembler and linker
namespace minix {
- class LLVM_LIBRARY_VISIBILITY Assemble : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Assemble : public GnuTool {
public:
- Assemble(const ToolChain &TC) : Tool("minix::Assemble", "assembler",
+ Assemble(const ToolChain &TC) : GnuTool("minix::Assemble", "assembler",
TC) {}
bool hasIntegratedCPP() const override { return false; }
@@ -484,9 +502,9 @@ namespace minix {
const llvm::opt::ArgList &TCArgs,
const char *LinkingOutput) const override;
};
- class LLVM_LIBRARY_VISIBILITY Link : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Link : public GnuTool {
public:
- Link(const ToolChain &TC) : Tool("minix::Link", "linker", TC) {}
+ Link(const ToolChain &TC) : GnuTool("minix::Link", "linker", TC) {}
bool hasIntegratedCPP() const override { return false; }
bool isLinkJob() const override { return true; }
@@ -529,9 +547,9 @@ namespace solaris {
/// dragonfly -- Directly call GNU Binutils assembler and linker
namespace dragonfly {
- class LLVM_LIBRARY_VISIBILITY Assemble : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Assemble : public GnuTool {
public:
- Assemble(const ToolChain &TC) : Tool("dragonfly::Assemble", "assembler",
+ Assemble(const ToolChain &TC) : GnuTool("dragonfly::Assemble", "assembler",
TC) {}
bool hasIntegratedCPP() const override { return false; }
@@ -541,9 +559,9 @@ namespace dragonfly {
const llvm::opt::ArgList &TCArgs,
const char *LinkingOutput) const override;
};
- class LLVM_LIBRARY_VISIBILITY Link : public Tool {
+ class LLVM_LIBRARY_VISIBILITY Link : public GnuTool {
public:
- Link(const ToolChain &TC) : Tool("dragonfly::Link", "linker", TC) {}
+ Link(const ToolChain &TC) : GnuTool("dragonfly::Link", "linker", TC) {}
bool hasIntegratedCPP() const override { return false; }
bool isLinkJob() const override { return true; }
@@ -560,7 +578,8 @@ namespace dragonfly {
namespace visualstudio {
class LLVM_LIBRARY_VISIBILITY Link : public Tool {
public:
- Link(const ToolChain &TC) : Tool("visualstudio::Link", "linker", TC) {}
+ Link(const ToolChain &TC) : Tool("visualstudio::Link", "linker", TC,
+ RF_Full, llvm::sys::WEM_UTF16) {}
bool hasIntegratedCPP() const override { return false; }
bool isLinkJob() const override { return true; }
@@ -573,7 +592,8 @@ namespace visualstudio {
class LLVM_LIBRARY_VISIBILITY Compile : public Tool {
public:
- Compile(const ToolChain &TC) : Tool("visualstudio::Compile", "compiler", TC) {}
+ Compile(const ToolChain &TC) : Tool("visualstudio::Compile", "compiler", TC,
+ RF_Full, llvm::sys::WEM_UTF16) {}
bool hasIntegratedAssembler() const override { return true; }
bool hasIntegratedCPP() const override { return true; }
Added: cfe/trunk/test/Driver/Inputs/gen-response.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/Inputs/gen-response.c?rev=217792&view=auto
==============================================================================
--- cfe/trunk/test/Driver/Inputs/gen-response.c (added)
+++ cfe/trunk/test/Driver/Inputs/gen-response.c Mon Sep 15 12:45:39 2014
@@ -0,0 +1,8 @@
+#define M -DTEST
+#define M1 M M M M M M M M M M
+#define M2 M1 M1 M1 M1 M1 M1 M1 M1 M1 M1
+#define M3 M2 M2 M2 M2 M2 M2 M2 M2 M2 M2
+#define M4 M3 M3 M3 M3 M3 M3 M3 M3 M3 M3
+#define M5 M4 M4 M4 M4 M4 M4 M4 M4 M4 M4
+#define TEXT M5 M5 M5
+TEXT
Added: cfe/trunk/test/Driver/response-file.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/response-file.c?rev=217792&view=auto
==============================================================================
--- cfe/trunk/test/Driver/response-file.c (added)
+++ cfe/trunk/test/Driver/response-file.c Mon Sep 15 12:45:39 2014
@@ -0,0 +1,23 @@
+// REQUIRES: long_tests
+
+// Check that clang is able to process short response files
+// Since this is a short response file, clang must not use a response file
+// to pass its parameters to other tools. This is only necessary for a large
+// number of parameters.
+// RUN: echo "-DTEST" >> %t.0.txt
+// RUN: %clang -E @%t.0.txt %s -v 2>&1 | FileCheck %s -check-prefix=SHORT
+// SHORT-NOT: Arguments passed via response file
+// SHORT: extern int it_works;
+
+// Check that clang is able to process long response files, routing a long
+// sequence of arguments to other tools by using response files as well.
+// We generate a 2MB response file to be big enough to surpass any system
+// limit.
+// RUN: %clang -E %S/Inputs/gen-response.c | grep DTEST > %t.1.txt
+// RUN: %clang -E @%t.1.txt %s -v 2>&1 | FileCheck %s -check-prefix=LONG
+// LONG: Arguments passed via response file
+// LONG: extern int it_works;
+
+#ifdef TEST
+extern int it_works;
+#endif
More information about the cfe-commits
mailing list