[clang] 155c49e - [Driver] Print process statistics report on CC_PRINT_PROC_STAT env variable.
Vladimir Vereschaka via cfe-commits
cfe-commits at lists.llvm.org
Fri Feb 26 16:17:11 PST 2021
Author: Vladimir Vereschaka
Date: 2021-02-26T16:16:00-08:00
New Revision: 155c49e0878de667b8021b9ba685390151dee11e
URL: https://github.com/llvm/llvm-project/commit/155c49e0878de667b8021b9ba685390151dee11e
DIFF: https://github.com/llvm/llvm-project/commit/155c49e0878de667b8021b9ba685390151dee11e.diff
LOG: [Driver] Print process statistics report on CC_PRINT_PROC_STAT env variable.
Added supporting CC_PRINT_PROC_STAT and CC_PRINT_PROC_STAT_FILE
environment variables to trigger clang driver reporting the process
statistics into specified file (alternate for -fproc-stat-report
option).
Differential Revision: https://reviews.llvm.org/D97094
Added:
clang/test/Driver/cc-print-proc-stat.c
Modified:
clang/docs/UsersManual.rst
clang/include/clang/Driver/Driver.h
clang/lib/Driver/Driver.cpp
clang/test/Driver/lit.local.cfg
clang/tools/driver/driver.cpp
Removed:
################################################################################
diff --git a/clang/docs/UsersManual.rst b/clang/docs/UsersManual.rst
index 6f593cab8f1a..b762bf9a542b 100644
--- a/clang/docs/UsersManual.rst
+++ b/clang/docs/UsersManual.rst
@@ -764,12 +764,12 @@ compilations steps.
is sent to. If it specifies a regular file, the data are saved to this file in
CSV format:
-.. code-block:: console
+ .. code-block:: console
- $ clang -fproc-stat-report=abc foo.c
- $ cat abc
- clang-11,"/tmp/foo-123456.o",92000,84000,87536
- ld,"a.out",900,8000,53568
+ $ clang -fproc-stat-report=abc foo.c
+ $ cat abc
+ clang-11,"/tmp/foo-123456.o",92000,84000,87536
+ ld,"a.out",900,8000,53568
The data on each row represent:
@@ -780,19 +780,38 @@ compilations steps.
* peak memory usage in Kb.
It is possible to specify this option without any value. In this case statistics
- is printed on standard output in human readable format:
+ are printed on standard output in human readable format:
-.. code-block:: console
+ .. code-block:: console
- $ clang -fproc-stat-report foo.c
- clang-11: output=/tmp/foo-855a8e.o, total=68.000 ms, user=60.000 ms, mem=86920 Kb
- ld: output=a.out, total=8.000 ms, user=4.000 ms, mem=52320 Kb
+ $ clang -fproc-stat-report foo.c
+ clang-11: output=/tmp/foo-855a8e.o, total=68.000 ms, user=60.000 ms, mem=86920 Kb
+ ld: output=a.out, total=8.000 ms, user=4.000 ms, mem=52320 Kb
The report file specified in the option is locked for write, so this option
can be used to collect statistics in parallel builds. The report file is not
cleared, new data is appended to it, thus making posible to accumulate build
statistics.
+ You can also use environment variables to control the process statistics reporting.
+ Setting ``CC_PRINT_PROC_STAT`` to ``1`` enables the feature, the report goes to
+ stdout in human readable format.
+ Setting ``CC_PRINT_PROC_STAT_FILE`` to a fully qualified file path makes it report
+ process statistics to the given file in the CSV format. Specifying a relative
+ path will likely lead to multiple files with the same name created in
diff erent
+ directories, since the path is relative to a changing working directory.
+
+ These environment variables are handy when you need to request the statistics
+ report without changing your build scripts or alter the existing set of compiler
+ options. Note that ``-fproc-stat-report`` take precedence over ``CC_PRINT_PROC_STAT``
+ and ``CC_PRINT_PROC_STAT_FILE``.
+
+ .. code-block:: console
+
+ $ export CC_PRINT_PROC_STAT=1
+ $ export CC_PRINT_PROC_STAT_FILE=~/project-build-proc-stat.csv
+ $ make
+
Other Options
-------------
Clang options that don't fit neatly into other categories.
diff --git a/clang/include/clang/Driver/Driver.h b/clang/include/clang/Driver/Driver.h
index 74a9cf3dab81..54c20620910b 100644
--- a/clang/include/clang/Driver/Driver.h
+++ b/clang/include/clang/Driver/Driver.h
@@ -156,6 +156,9 @@ class Driver {
/// Information about the host which can be overridden by the user.
std::string HostBits, HostMachine, HostSystem, HostRelease;
+ /// The file to log CC_PRINT_PROC_STAT_FILE output to, if enabled.
+ const char *CCPrintStatReportFilename;
+
/// The file to log CC_PRINT_OPTIONS output to, if enabled.
const char *CCPrintOptionsFilename;
@@ -204,6 +207,10 @@ class Driver {
/// Whether the driver is generating diagnostics for debugging purposes.
unsigned CCGenDiagnostics : 1;
+ /// Set CC_PRINT_PROC_STAT mode, which causes the driver to dump
+ /// performance report to CC_PRINT_PROC_STAT_FILE or to stdout.
+ unsigned CCPrintProcessStats : 1;
+
/// Pointer to the ExecuteCC1Tool function, if available.
/// When the clangDriver lib is used through clang.exe, this provides a
/// shortcut for executing the -cc1 command-line directly, in the same
diff --git a/clang/lib/Driver/Driver.cpp b/clang/lib/Driver/Driver.cpp
index 84bee2ae9d07..acea40f05182 100644
--- a/clang/lib/Driver/Driver.cpp
+++ b/clang/lib/Driver/Driver.cpp
@@ -137,8 +137,9 @@ Driver::Driver(StringRef ClangExecutable, StringRef TargetTriple,
ClangExecutable(ClangExecutable), SysRoot(DEFAULT_SYSROOT),
DriverTitle(Title), CCPrintOptionsFilename(nullptr),
CCPrintHeadersFilename(nullptr), CCLogDiagnosticsFilename(nullptr),
- CCCPrintBindings(false), CCPrintOptions(false), CCPrintHeaders(false),
- CCLogDiagnostics(false), CCGenDiagnostics(false),
+ CCPrintStatReportFilename(nullptr), CCCPrintBindings(false),
+ CCPrintOptions(false), CCPrintHeaders(false), CCLogDiagnostics(false),
+ CCGenDiagnostics(false), CCPrintProcessStats(false),
TargetTriple(TargetTriple), CCCGenericGCCName(""), Saver(Alloc),
CheckInputsExist(true), GenReproducer(false),
SuppressMissingInputWarning(false) {
@@ -1096,6 +1097,15 @@ Compilation *Driver::BuildCompilation(ArrayRef<const char *> ArgList) {
GenReproducer = Args.hasFlag(options::OPT_gen_reproducer,
options::OPT_fno_crash_diagnostics,
!!::getenv("FORCE_CLANG_DIAGNOSTICS_CRASH"));
+
+ // Process -fproc-stat-report options.
+ if (const Arg *A = Args.getLastArg(options::OPT_fproc_stat_report_EQ)) {
+ CCPrintProcessStats = true;
+ CCPrintStatReportFilename = A->getValue();
+ }
+ if (Args.hasArg(options::OPT_fproc_stat_report))
+ CCPrintProcessStats = true;
+
// FIXME: TargetTriple is used by the target-prefixed calls to as/ld
// and getToolChain is const.
if (IsCLMode()) {
@@ -4009,62 +4019,58 @@ void Driver::BuildJobs(Compilation &C) const {
/*TargetDeviceOffloadKind*/ Action::OFK_None);
}
- StringRef StatReportFile;
- bool PrintProcessStat = false;
- if (const Arg *A = C.getArgs().getLastArg(options::OPT_fproc_stat_report_EQ))
- StatReportFile = A->getValue();
- if (C.getArgs().hasArg(options::OPT_fproc_stat_report))
- PrintProcessStat = true;
-
// If we have more than one job, then disable integrated-cc1 for now. Do this
// also when we need to report process execution statistics.
- if (C.getJobs().size() > 1 || !StatReportFile.empty() || PrintProcessStat)
+ if (C.getJobs().size() > 1 || CCPrintProcessStats)
for (auto &J : C.getJobs())
J.InProcess = false;
- if (!StatReportFile.empty() || PrintProcessStat) {
+ if (CCPrintProcessStats) {
C.setPostCallback([=](const Command &Cmd, int Res) {
Optional<llvm::sys::ProcessStatistics> ProcStat =
Cmd.getProcessStatistics();
if (!ProcStat)
return;
- if (PrintProcessStat) {
+
+ const char *LinkingOutput = nullptr;
+ if (FinalOutput)
+ LinkingOutput = FinalOutput->getValue();
+ else if (!Cmd.getOutputFilenames().empty())
+ LinkingOutput = Cmd.getOutputFilenames().front().c_str();
+ else
+ LinkingOutput = getDefaultImageName();
+
+ if (!CCPrintStatReportFilename) {
using namespace llvm;
// Human readable output.
outs() << sys::path::filename(Cmd.getExecutable()) << ": "
- << "output=";
- if (Cmd.getOutputFilenames().empty())
- outs() << "\"\"";
- else
- outs() << Cmd.getOutputFilenames().front();
+ << "output=" << LinkingOutput;
outs() << ", total="
<< format("%.3f", ProcStat->TotalTime.count() / 1000.) << " ms"
<< ", user="
<< format("%.3f", ProcStat->UserTime.count() / 1000.) << " ms"
<< ", mem=" << ProcStat->PeakMemory << " Kb\n";
- }
- if (!StatReportFile.empty()) {
+ } else {
// CSV format.
std::string Buffer;
llvm::raw_string_ostream Out(Buffer);
llvm::sys::printArg(Out, llvm::sys::path::filename(Cmd.getExecutable()),
/*Quote*/ true);
Out << ',';
- if (Cmd.getOutputFilenames().empty())
- Out << "\"\"";
- else
- llvm::sys::printArg(Out, Cmd.getOutputFilenames().front(), true);
+ llvm::sys::printArg(Out, LinkingOutput, true);
Out << ',' << ProcStat->TotalTime.count() << ','
<< ProcStat->UserTime.count() << ',' << ProcStat->PeakMemory
<< '\n';
Out.flush();
std::error_code EC;
- llvm::raw_fd_ostream OS(StatReportFile, EC, llvm::sys::fs::OF_Append);
+ llvm::raw_fd_ostream OS(CCPrintStatReportFilename, EC,
+ llvm::sys::fs::OF_Append);
if (EC)
return;
auto L = OS.lock();
if (!L) {
- llvm::errs() << "ERROR: Cannot lock file " << StatReportFile << ": "
+ llvm::errs() << "ERROR: Cannot lock file "
+ << CCPrintStatReportFilename << ": "
<< toString(L.takeError()) << "\n";
return;
}
diff --git a/clang/test/Driver/cc-print-proc-stat.c b/clang/test/Driver/cc-print-proc-stat.c
new file mode 100644
index 000000000000..f3a38559b78a
--- /dev/null
+++ b/clang/test/Driver/cc-print-proc-stat.c
@@ -0,0 +1,9 @@
+// RUN: env CC_PRINT_PROC_STAT=1 \
+// RUN: CC_PRINT_PROC_STAT_FILE=%t.csv \
+// RUN: %clang -no-canonical-prefixes -S -o %t.s %s
+// RUN: FileCheck --check-prefix=CHECK-CSV %s < %t.csv
+// CHECK-CSV: clang{{.*}},"{{.*}}.s",{{[0-9]+}},{{[0-9]+}},{{[0-9]+}}
+
+// RUN: env CC_PRINT_PROC_STAT=1 \
+// RUN: %clang -c -fintegrated-as %s | FileCheck %s
+// CHECK: clang{{.*}}: output={{.*}}.o, total={{[0-9.]+}} ms, user={{[0-9.]+}} ms, mem={{[0-9]+}} Kb
diff --git a/clang/test/Driver/lit.local.cfg b/clang/test/Driver/lit.local.cfg
index ad3f0e92cacd..7ab6c29ec948 100644
--- a/clang/test/Driver/lit.local.cfg
+++ b/clang/test/Driver/lit.local.cfg
@@ -11,7 +11,7 @@ driver_overwrite_env_vars = ['MACOSX_DEPLOYMENT_TARGET',
'IPHONEOS_DEPLOYMENT_TARGET',
'SDKROOT', 'CCC_OVERRIDE_OPTIONS',
'CC_PRINT_OPTIONS', 'CC_PRINT_HEADERS',
- 'CC_LOG_DIAGNOSTICS']
+ 'CC_LOG_DIAGNOSTICS', 'CC_PRINT_PROC_STAT']
for name in driver_overwrite_env_vars:
if name in config.environment:
diff --git a/clang/tools/driver/driver.cpp b/clang/tools/driver/driver.cpp
index a13c1c4a3834..c3a3aab066ac 100644
--- a/clang/tools/driver/driver.cpp
+++ b/clang/tools/driver/driver.cpp
@@ -258,6 +258,11 @@ static void SetBackdoorDriverOutputsFromEnvVars(Driver &TheDriver) {
TheDriver.CCLogDiagnostics = !!::getenv("CC_LOG_DIAGNOSTICS");
if (TheDriver.CCLogDiagnostics)
TheDriver.CCLogDiagnosticsFilename = ::getenv("CC_LOG_DIAGNOSTICS_FILE");
+
+ // Handle CC_PRINT_PROC_STAT and CC_PRINT_PROC_STAT_FILE.
+ TheDriver.CCPrintProcessStats = !!::getenv("CC_PRINT_PROC_STAT");
+ if (TheDriver.CCPrintProcessStats)
+ TheDriver.CCPrintStatReportFilename = ::getenv("CC_PRINT_PROC_STAT_FILE");
}
static void FixupDiagPrefixExeName(TextDiagnosticPrinter *DiagClient,
More information about the cfe-commits
mailing list