r344199 - Add a flag to remap manglings when reading profile data information.
Richard Smith via cfe-commits
cfe-commits at lists.llvm.org
Wed Oct 10 16:13:35 PDT 2018
Author: rsmith
Date: Wed Oct 10 16:13:35 2018
New Revision: 344199
URL: http://llvm.org/viewvc/llvm-project?rev=344199&view=rev
Log:
Add a flag to remap manglings when reading profile data information.
This can be used to preserve profiling information across codebase
changes that have widespread impact on mangled names, but across which
most profiling data should still be usable. For example, when switching
from libstdc++ to libc++, or from the old libstdc++ ABI to the new ABI,
or even from a 32-bit to a 64-bit build.
The user can provide a remapping file specifying parts of mangled names
that should be treated as equivalent (eg, std::__1 should be treated as
equivalent to std::__cxx11), and profile data will be treated as
applying to a particular function if its name is equivalent to the name
of a function in the profile data under the provided equivalences. See
the documentation change for a description of how this is configured.
Remapping is supported for both sample-based profiling and instruction
profiling. We do not support remapping indirect branch target
information, but all other profile data should be remapped
appropriately.
Support is only added for the new pass manager. If someone wants to also
add support for this for the old pass manager, doing so should be
straightforward.
Added:
cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map
cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext
cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples
cfe/trunk/test/CodeGenCXX/profile-remap.cpp
Modified:
cfe/trunk/docs/ReleaseNotes.rst
cfe/trunk/docs/UsersManual.rst
cfe/trunk/include/clang/Driver/Options.td
cfe/trunk/include/clang/Frontend/CodeGenOptions.h
cfe/trunk/lib/CodeGen/BackendUtil.cpp
cfe/trunk/lib/CodeGen/CodeGenModule.cpp
cfe/trunk/lib/Driver/ToolChains/Clang.cpp
cfe/trunk/lib/Frontend/CompilerInvocation.cpp
cfe/trunk/test/Driver/clang_f_opts.c
Modified: cfe/trunk/docs/ReleaseNotes.rst
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/ReleaseNotes.rst?rev=344199&r1=344198&r2=344199&view=diff
==============================================================================
--- cfe/trunk/docs/ReleaseNotes.rst (original)
+++ cfe/trunk/docs/ReleaseNotes.rst Wed Oct 10 16:13:35 2018
@@ -46,6 +46,11 @@ sections with improvements to Clang's su
Major New Features
------------------
+- Clang supports use of a profile remapping file, which permits
+ profile data captured for one version of a program to be applied
+ when building another version where symbols have changed (for
+ example, due to renaming a class or namespace).
+ See the :doc:`UsersManual` for details.
Improvements to Clang's diagnostics
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Modified: cfe/trunk/docs/UsersManual.rst
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/UsersManual.rst?rev=344199&r1=344198&r2=344199&view=diff
==============================================================================
--- cfe/trunk/docs/UsersManual.rst (original)
+++ cfe/trunk/docs/UsersManual.rst Wed Oct 10 16:13:35 2018
@@ -1799,6 +1799,69 @@ In these cases, you can use the flag ``-
Note that these flags should appear after the corresponding profile
flags to have an effect.
+Profile remapping
+^^^^^^^^^^^^^^^^^
+
+When the program is compiled after a change that affects many symbol names,
+pre-existing profile data may no longer match the program. For example:
+
+ * switching from libstdc++ to libc++ will result in the mangled names of all
+ functions taking standard library types to change
+ * renaming a widely-used type in C++ will result in the mangled names of all
+ functions that have parameters involving that type to change
+ * moving from a 32-bit compilation to a 64-bit compilation may change the
+ underlying type of ``size_t`` and similar types, resulting in changes to
+ manglings
+
+Clang allows use of a profile remapping file to specify that such differences
+in mangled names should be ignored when matching the profile data against the
+program.
+
+.. option:: -fprofile-remapping-file=<file>
+
+ Specifies a file containing profile remapping information, that will be
+ used to match mangled names in the profile data to mangled names in the
+ program.
+
+The profile remapping file is a text file containing lines of the form
+
+.. code-block::
+
+ fragmentkind fragment1 fragment2
+
+where ``fragmentkind`` is one of ``name``, ``type``, or ``encoding``,
+indicating whether the following mangled name fragments are
+<`name <http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.name>`>s,
+<`type <http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.type>`>s, or
+<`encoding <http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.encoding>`>s,
+respectively.
+
+Blank lines and lines starting with ``#`` are ignored.
+
+For example, to specify that ``absl::string_view`` and ``std::string_view``
+should be treated as equivalent when matching profile data, the following
+remapping file could be used:
+
+.. code-block::
+
+ # absl::string_view is considered equivalent to std::string_view
+ type N4absl11string_viewE St17basic_string_viewIcSt11char_traitsIcEE
+
+ # std:: might be std::__1:: in libc++ or std::__cxx11:: in libstdc++
+ name 3std St3__1
+ name 3std St7__cxx11
+
+Matching profile data using a profile remapping file is supported on a
+best-effort basis. For example, information regarding indirect call targets is
+currently not remapped. For best results, you are encouraged to generate new
+profile data matching the updated program.
+
+.. note::
+
+ Profile data remapping is currently only supported for C++ mangled names
+ following the Itanium C++ ABI mangling scheme. This covers all C++ targets
+ supported by Clang other than Windows.
+
Controlling Debug Information
-----------------------------
Modified: cfe/trunk/include/clang/Driver/Options.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/Options.td?rev=344199&r1=344198&r2=344199&view=diff
==============================================================================
--- cfe/trunk/include/clang/Driver/Options.td (original)
+++ cfe/trunk/include/clang/Driver/Options.td Wed Oct 10 16:13:35 2018
@@ -735,6 +735,11 @@ def fprofile_instr_use : Flag<["-"], "fp
def fprofile_instr_use_EQ : Joined<["-"], "fprofile-instr-use=">,
Group<f_Group>, Flags<[CoreOption]>,
HelpText<"Use instrumentation data for profile-guided optimization">;
+def fprofile_remapping_file_EQ : Joined<["-"], "fprofile-remapping-file=">,
+ Group<f_Group>, Flags<[CC1Option, CoreOption]>, MetaVarName<"<file>">,
+ HelpText<"Use the remappings described in <file> to match the profile data against names in the program">;
+def fprofile_remapping_file : Separate<["-"], "fprofile-remapping-file">,
+ Group<f_Group>, Flags<[CoreOption]>, Alias<fprofile_remapping_file_EQ>;
def fcoverage_mapping : Flag<["-"], "fcoverage-mapping">,
Group<f_Group>, Flags<[CC1Option, CoreOption]>,
HelpText<"Generate coverage mapping to enable code coverage analysis">;
Modified: cfe/trunk/include/clang/Frontend/CodeGenOptions.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/CodeGenOptions.h?rev=344199&r1=344198&r2=344199&view=diff
==============================================================================
--- cfe/trunk/include/clang/Frontend/CodeGenOptions.h (original)
+++ cfe/trunk/include/clang/Frontend/CodeGenOptions.h Wed Oct 10 16:13:35 2018
@@ -200,6 +200,10 @@ public:
/// Name of the profile file to use as input for -fprofile-instr-use
std::string ProfileInstrumentUsePath;
+ /// Name of the profile remapping file to apply to the profile data supplied
+ /// by -fprofile-sample-use or -fprofile-instr-use.
+ std::string ProfileRemappingFile;
+
/// Name of the function summary index file to use for ThinLTO function
/// importing.
std::string ThinLTOIndexFile;
Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=344199&r1=344198&r2=344199&view=diff
==============================================================================
--- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original)
+++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Wed Oct 10 16:13:35 2018
@@ -930,18 +930,21 @@ void EmitAssemblyHelper::EmitAssemblyWit
PGOOpt = PGOOptions(CodeGenOpts.InstrProfileOutput.empty()
? DefaultProfileGenName
: CodeGenOpts.InstrProfileOutput,
- "", "", true, CodeGenOpts.DebugInfoForProfiling);
+ "", "", "", true,
+ CodeGenOpts.DebugInfoForProfiling);
else if (CodeGenOpts.hasProfileIRUse())
// -fprofile-use.
- PGOOpt = PGOOptions("", CodeGenOpts.ProfileInstrumentUsePath, "", false,
+ PGOOpt = PGOOptions("", CodeGenOpts.ProfileInstrumentUsePath, "",
+ CodeGenOpts.ProfileRemappingFile, false,
CodeGenOpts.DebugInfoForProfiling);
else if (!CodeGenOpts.SampleProfileFile.empty())
// -fprofile-sample-use
- PGOOpt = PGOOptions("", "", CodeGenOpts.SampleProfileFile, false,
+ PGOOpt = PGOOptions("", "", CodeGenOpts.SampleProfileFile,
+ CodeGenOpts.ProfileRemappingFile, false,
CodeGenOpts.DebugInfoForProfiling);
else if (CodeGenOpts.DebugInfoForProfiling)
// -fdebug-info-for-profiling
- PGOOpt = PGOOptions("", "", "", false, true);
+ PGOOpt = PGOOptions("", "", "", "", false, true);
PassBuilder PB(TM.get(), PGOOpt);
@@ -1130,6 +1133,7 @@ static void runThinLTOBackend(ModuleSumm
const LangOptions &LOpts,
std::unique_ptr<raw_pwrite_stream> OS,
std::string SampleProfile,
+ std::string ProfileRemapping,
BackendAction Action) {
StringMap<DenseMap<GlobalValue::GUID, GlobalValueSummary *>>
ModuleToDefinedGVSummaries;
@@ -1202,6 +1206,7 @@ static void runThinLTOBackend(ModuleSumm
Conf.CGOptLevel = getCGOptLevel(CGOpts);
initTargetOptions(Conf.Options, CGOpts, TOpts, LOpts, HeaderOpts);
Conf.SampleProfile = std::move(SampleProfile);
+ Conf.ProfileRemapping = std::move(ProfileRemapping);
Conf.UseNewPM = CGOpts.ExperimentalNewPassManager;
Conf.DebugPassManager = CGOpts.DebugPassManager;
Conf.RemarksWithHotness = CGOpts.DiagnosticsWithHotness;
@@ -1268,7 +1273,7 @@ void clang::EmitBackendOutput(Diagnostic
if (!CombinedIndex->skipModuleByDistributedBackend()) {
runThinLTOBackend(CombinedIndex.get(), M, HeaderOpts, CGOpts, TOpts,
LOpts, std::move(OS), CGOpts.SampleProfileFile,
- Action);
+ CGOpts.ProfileRemappingFile, Action);
return;
}
// Distributed indexing detected that nothing from the module is needed
Modified: cfe/trunk/lib/CodeGen/CodeGenModule.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenModule.cpp?rev=344199&r1=344198&r2=344199&view=diff
==============================================================================
--- cfe/trunk/lib/CodeGen/CodeGenModule.cpp (original)
+++ cfe/trunk/lib/CodeGen/CodeGenModule.cpp Wed Oct 10 16:13:35 2018
@@ -154,7 +154,7 @@ CodeGenModule::CodeGenModule(ASTContext
if (CodeGenOpts.hasProfileClangUse()) {
auto ReaderOrErr = llvm::IndexedInstrProfReader::create(
- CodeGenOpts.ProfileInstrumentUsePath);
+ CodeGenOpts.ProfileInstrumentUsePath, CodeGenOpts.ProfileRemappingFile);
if (auto E = ReaderOrErr.takeError()) {
unsigned DiagID = Diags.getCustomDiagID(DiagnosticsEngine::Error,
"Could not read profile %0: %1");
Modified: cfe/trunk/lib/Driver/ToolChains/Clang.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/ToolChains/Clang.cpp?rev=344199&r1=344198&r2=344199&view=diff
==============================================================================
--- cfe/trunk/lib/Driver/ToolChains/Clang.cpp (original)
+++ cfe/trunk/lib/Driver/ToolChains/Clang.cpp Wed Oct 10 16:13:35 2018
@@ -4355,6 +4355,7 @@ void Clang::ConstructJob(Compilation &C,
else
A->render(Args, CmdArgs);
}
+ Args.AddLastArg(CmdArgs, options::OPT_fprofile_remapping_file_EQ);
RenderBuiltinOptions(TC, RawTriple, Args, CmdArgs);
Modified: cfe/trunk/lib/Frontend/CompilerInvocation.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/CompilerInvocation.cpp?rev=344199&r1=344198&r2=344199&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/CompilerInvocation.cpp (original)
+++ cfe/trunk/lib/Frontend/CompilerInvocation.cpp Wed Oct 10 16:13:35 2018
@@ -657,6 +657,13 @@ static bool ParseCodeGenArgs(CodeGenOpti
Args.getLastArgValue(OPT_fprofile_instrument_use_path_EQ);
if (!Opts.ProfileInstrumentUsePath.empty())
setPGOUseInstrumentor(Opts, Opts.ProfileInstrumentUsePath);
+ Opts.ProfileRemappingFile =
+ Args.getLastArgValue(OPT_fprofile_remapping_file_EQ);
+ if (!Opts.ProfileRemappingFile.empty() && !Opts.ExperimentalNewPassManager) {
+ Diags.Report(diag::err_drv_argument_only_allowed_with)
+ << Args.getLastArg(OPT_fprofile_remapping_file_EQ)->getAsString(Args)
+ << "-fexperimental-new-pass-manager";
+ }
Opts.CoverageMapping =
Args.hasFlag(OPT_fcoverage_mapping, OPT_fno_coverage_mapping, false);
Added: cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map?rev=344199&view=auto
==============================================================================
--- cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map (added)
+++ cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map Wed Oct 10 16:13:35 2018
@@ -0,0 +1,2 @@
+name 3Foo 3Bar
+type N3Foo1XE N3Baz1YE
Added: cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext?rev=344199&view=auto
==============================================================================
--- cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext (added)
+++ cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext Wed Oct 10 16:13:35 2018
@@ -0,0 +1,7 @@
+:ir
+_ZN3Foo8functionENS_1XE
+29667547796
+2
+10
+90
+
Added: cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples?rev=344199&view=auto
==============================================================================
--- cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples (added)
+++ cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples Wed Oct 10 16:13:35 2018
@@ -0,0 +1,3 @@
+_ZN3Bar8functionEN3Baz1YE:100:0
+ 2: 10
+ 4: 90
Added: cfe/trunk/test/CodeGenCXX/profile-remap.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/profile-remap.cpp?rev=344199&view=auto
==============================================================================
--- cfe/trunk/test/CodeGenCXX/profile-remap.cpp (added)
+++ cfe/trunk/test/CodeGenCXX/profile-remap.cpp Wed Oct 10 16:13:35 2018
@@ -0,0 +1,27 @@
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fprofile-sample-use=%S/Inputs/profile-remap.samples -fprofile-remapping-file=%S/Inputs/profile-remap.map -fexperimental-new-pass-manager -O2 %s -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,CHECK-SAMPLES
+// RUN: llvm-profdata merge -output %T.profdata %S/Inputs/profile-remap.proftext
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fprofile-instrument-use-path=%T.profdata -fprofile-remapping-file=%S/Inputs/profile-remap.map -fexperimental-new-pass-manager -O2 %s -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,CHECK-INSTR
+
+namespace Foo {
+ struct X {};
+ bool cond();
+ void bar();
+ void baz();
+ void function(X x) {
+ if (cond())
+ bar();
+ else
+ baz();
+ }
+}
+
+// CHECK: define {{.*}} @_ZN3Foo8functionENS_1XE() {{.*}} !prof [[FUNC_ENTRY:![0-9]*]]
+// CHECK: br i1 {{.*}} !prof [[BR_WEIGHTS:![0-9]*]]
+//
+// FIXME: Laplace's rule of succession is applied to sample profiles...
+// CHECK-SAMPLES-DAG: [[FUNC_ENTRY]] = !{!"function_entry_count", i64 1}
+// CHECK-SAMPLES-DAG: [[BR_WEIGHTS]] = !{!"branch_weights", i32 11, i32 91}
+//
+// ... but not to instruction profiles.
+// CHECK-INSTR-DAG: [[FUNC_ENTRY]] = !{!"function_entry_count", i64 100}
+// CHECK-INSTR-DAG: [[BR_WEIGHTS]] = !{!"branch_weights", i32 10, i32 90}
Modified: cfe/trunk/test/Driver/clang_f_opts.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/clang_f_opts.c?rev=344199&r1=344198&r2=344199&view=diff
==============================================================================
--- cfe/trunk/test/Driver/clang_f_opts.c (original)
+++ cfe/trunk/test/Driver/clang_f_opts.c Wed Oct 10 16:13:35 2018
@@ -116,6 +116,7 @@
// RUN: %clang -### -S -fcoverage-mapping %s 2>&1 | FileCheck -check-prefix=CHECK-COVERAGE-AND-GEN %s
// RUN: %clang -### -S -fcoverage-mapping -fno-coverage-mapping %s 2>&1 | FileCheck -check-prefix=CHECK-DISABLE-COVERAGE %s
// RUN: %clang -### -S -fprofile-instr-generate -fcoverage-mapping -fno-coverage-mapping %s 2>&1 | FileCheck -check-prefix=CHECK-DISABLE-COVERAGE %s
+// RUN: %clang -### -S -fprofile-remapping-file foo/bar.txt %s 2>&1 | FileCheck -check-prefix=CHECK-PROFILE-REMAP %s
// CHECK-PROFILE-GENERATE: "-fprofile-instrument=clang"
// CHECK-PROFILE-GENERATE-LLVM: "-fprofile-instrument=llvm"
// CHECK-PROFILE-GENERATE-DIR: "-fprofile-instrument-path=/some/dir{{/|\\\\}}{{.*}}"
@@ -126,6 +127,7 @@
// CHECK-DISABLE-USE-NOT: "-fprofile-instr-use"
// CHECK-COVERAGE-AND-GEN: '-fcoverage-mapping' only allowed with '-fprofile-instr-generate'
// CHECK-DISABLE-COVERAGE-NOT: "-fcoverage-mapping"
+// CHECK-PROFILE-REMAP: "-fprofile-remapping-file=foo/bar.txt"
// RUN: %clang -### -S -fprofile-use %s 2>&1 | FileCheck -check-prefix=CHECK-PROFILE-USE %s
// RUN: %clang -### -S -fprofile-instr-use %s 2>&1 | FileCheck -check-prefix=CHECK-PROFILE-USE %s
More information about the cfe-commits
mailing list