r344199 - Add a flag to remap manglings when reading profile data information.
Yvan Roux via cfe-commits
cfe-commits at lists.llvm.org
Wed Oct 10 23:13:34 PDT 2018
Hi Richard,
This patch broke ARM bots which don't have x86_64 target available.
Logs are available here:
http://lab.llvm.org:8011/builders/clang-cmake-armv7-quick/builds/4326/steps/ninja%20check%201/logs/FAIL%3A%20Clang%3A%3Aprofile-remap.cpp
Cheers,
Yvan
On Thu, 11 Oct 2018 at 01:15, Richard Smith via cfe-commits
<cfe-commits at lists.llvm.org> wrote:
>
> Author: rsmith
> Date: Wed Oct 10 16:13:35 2018
> New Revision: 344199
>
> URL: http://llvm.org/viewvc/llvm-project?rev=344199&view=rev
> Log:
> Add a flag to remap manglings when reading profile data information.
>
> This can be used to preserve profiling information across codebase
> changes that have widespread impact on mangled names, but across which
> most profiling data should still be usable. For example, when switching
> from libstdc++ to libc++, or from the old libstdc++ ABI to the new ABI,
> or even from a 32-bit to a 64-bit build.
>
> The user can provide a remapping file specifying parts of mangled names
> that should be treated as equivalent (eg, std::__1 should be treated as
> equivalent to std::__cxx11), and profile data will be treated as
> applying to a particular function if its name is equivalent to the name
> of a function in the profile data under the provided equivalences. See
> the documentation change for a description of how this is configured.
>
> Remapping is supported for both sample-based profiling and instruction
> profiling. We do not support remapping indirect branch target
> information, but all other profile data should be remapped
> appropriately.
>
> Support is only added for the new pass manager. If someone wants to also
> add support for this for the old pass manager, doing so should be
> straightforward.
>
> Added:
> cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map
> cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext
> cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples
> cfe/trunk/test/CodeGenCXX/profile-remap.cpp
> Modified:
> cfe/trunk/docs/ReleaseNotes.rst
> cfe/trunk/docs/UsersManual.rst
> cfe/trunk/include/clang/Driver/Options.td
> cfe/trunk/include/clang/Frontend/CodeGenOptions.h
> cfe/trunk/lib/CodeGen/BackendUtil.cpp
> cfe/trunk/lib/CodeGen/CodeGenModule.cpp
> cfe/trunk/lib/Driver/ToolChains/Clang.cpp
> cfe/trunk/lib/Frontend/CompilerInvocation.cpp
> cfe/trunk/test/Driver/clang_f_opts.c
>
> Modified: cfe/trunk/docs/ReleaseNotes.rst
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/ReleaseNotes.rst?rev=344199&r1=344198&r2=344199&view=diff
> ==============================================================================
> --- cfe/trunk/docs/ReleaseNotes.rst (original)
> +++ cfe/trunk/docs/ReleaseNotes.rst Wed Oct 10 16:13:35 2018
> @@ -46,6 +46,11 @@ sections with improvements to Clang's su
> Major New Features
> ------------------
>
> +- Clang supports use of a profile remapping file, which permits
> + profile data captured for one version of a program to be applied
> + when building another version where symbols have changed (for
> + example, due to renaming a class or namespace).
> + See the :doc:`UsersManual` for details.
>
> Improvements to Clang's diagnostics
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> Modified: cfe/trunk/docs/UsersManual.rst
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/UsersManual.rst?rev=344199&r1=344198&r2=344199&view=diff
> ==============================================================================
> --- cfe/trunk/docs/UsersManual.rst (original)
> +++ cfe/trunk/docs/UsersManual.rst Wed Oct 10 16:13:35 2018
> @@ -1799,6 +1799,69 @@ In these cases, you can use the flag ``-
> Note that these flags should appear after the corresponding profile
> flags to have an effect.
>
> +Profile remapping
> +^^^^^^^^^^^^^^^^^
> +
> +When the program is compiled after a change that affects many symbol names,
> +pre-existing profile data may no longer match the program. For example:
> +
> + * switching from libstdc++ to libc++ will result in the mangled names of all
> + functions taking standard library types to change
> + * renaming a widely-used type in C++ will result in the mangled names of all
> + functions that have parameters involving that type to change
> + * moving from a 32-bit compilation to a 64-bit compilation may change the
> + underlying type of ``size_t`` and similar types, resulting in changes to
> + manglings
> +
> +Clang allows use of a profile remapping file to specify that such differences
> +in mangled names should be ignored when matching the profile data against the
> +program.
> +
> +.. option:: -fprofile-remapping-file=<file>
> +
> + Specifies a file containing profile remapping information, that will be
> + used to match mangled names in the profile data to mangled names in the
> + program.
> +
> +The profile remapping file is a text file containing lines of the form
> +
> +.. code-block::
> +
> + fragmentkind fragment1 fragment2
> +
> +where ``fragmentkind`` is one of ``name``, ``type``, or ``encoding``,
> +indicating whether the following mangled name fragments are
> +<`name <http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.name>`>s,
> +<`type <http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.type>`>s, or
> +<`encoding <http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.encoding>`>s,
> +respectively.
> +
> +Blank lines and lines starting with ``#`` are ignored.
> +
> +For example, to specify that ``absl::string_view`` and ``std::string_view``
> +should be treated as equivalent when matching profile data, the following
> +remapping file could be used:
> +
> +.. code-block::
> +
> + # absl::string_view is considered equivalent to std::string_view
> + type N4absl11string_viewE St17basic_string_viewIcSt11char_traitsIcEE
> +
> + # std:: might be std::__1:: in libc++ or std::__cxx11:: in libstdc++
> + name 3std St3__1
> + name 3std St7__cxx11
> +
> +Matching profile data using a profile remapping file is supported on a
> +best-effort basis. For example, information regarding indirect call targets is
> +currently not remapped. For best results, you are encouraged to generate new
> +profile data matching the updated program.
> +
> +.. note::
> +
> + Profile data remapping is currently only supported for C++ mangled names
> + following the Itanium C++ ABI mangling scheme. This covers all C++ targets
> + supported by Clang other than Windows.
> +
> Controlling Debug Information
> -----------------------------
>
>
> Modified: cfe/trunk/include/clang/Driver/Options.td
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/Options.td?rev=344199&r1=344198&r2=344199&view=diff
> ==============================================================================
> --- cfe/trunk/include/clang/Driver/Options.td (original)
> +++ cfe/trunk/include/clang/Driver/Options.td Wed Oct 10 16:13:35 2018
> @@ -735,6 +735,11 @@ def fprofile_instr_use : Flag<["-"], "fp
> def fprofile_instr_use_EQ : Joined<["-"], "fprofile-instr-use=">,
> Group<f_Group>, Flags<[CoreOption]>,
> HelpText<"Use instrumentation data for profile-guided optimization">;
> +def fprofile_remapping_file_EQ : Joined<["-"], "fprofile-remapping-file=">,
> + Group<f_Group>, Flags<[CC1Option, CoreOption]>, MetaVarName<"<file>">,
> + HelpText<"Use the remappings described in <file> to match the profile data against names in the program">;
> +def fprofile_remapping_file : Separate<["-"], "fprofile-remapping-file">,
> + Group<f_Group>, Flags<[CoreOption]>, Alias<fprofile_remapping_file_EQ>;
> def fcoverage_mapping : Flag<["-"], "fcoverage-mapping">,
> Group<f_Group>, Flags<[CC1Option, CoreOption]>,
> HelpText<"Generate coverage mapping to enable code coverage analysis">;
>
> Modified: cfe/trunk/include/clang/Frontend/CodeGenOptions.h
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/CodeGenOptions.h?rev=344199&r1=344198&r2=344199&view=diff
> ==============================================================================
> --- cfe/trunk/include/clang/Frontend/CodeGenOptions.h (original)
> +++ cfe/trunk/include/clang/Frontend/CodeGenOptions.h Wed Oct 10 16:13:35 2018
> @@ -200,6 +200,10 @@ public:
> /// Name of the profile file to use as input for -fprofile-instr-use
> std::string ProfileInstrumentUsePath;
>
> + /// Name of the profile remapping file to apply to the profile data supplied
> + /// by -fprofile-sample-use or -fprofile-instr-use.
> + std::string ProfileRemappingFile;
> +
> /// Name of the function summary index file to use for ThinLTO function
> /// importing.
> std::string ThinLTOIndexFile;
>
> Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=344199&r1=344198&r2=344199&view=diff
> ==============================================================================
> --- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original)
> +++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Wed Oct 10 16:13:35 2018
> @@ -930,18 +930,21 @@ void EmitAssemblyHelper::EmitAssemblyWit
> PGOOpt = PGOOptions(CodeGenOpts.InstrProfileOutput.empty()
> ? DefaultProfileGenName
> : CodeGenOpts.InstrProfileOutput,
> - "", "", true, CodeGenOpts.DebugInfoForProfiling);
> + "", "", "", true,
> + CodeGenOpts.DebugInfoForProfiling);
> else if (CodeGenOpts.hasProfileIRUse())
> // -fprofile-use.
> - PGOOpt = PGOOptions("", CodeGenOpts.ProfileInstrumentUsePath, "", false,
> + PGOOpt = PGOOptions("", CodeGenOpts.ProfileInstrumentUsePath, "",
> + CodeGenOpts.ProfileRemappingFile, false,
> CodeGenOpts.DebugInfoForProfiling);
> else if (!CodeGenOpts.SampleProfileFile.empty())
> // -fprofile-sample-use
> - PGOOpt = PGOOptions("", "", CodeGenOpts.SampleProfileFile, false,
> + PGOOpt = PGOOptions("", "", CodeGenOpts.SampleProfileFile,
> + CodeGenOpts.ProfileRemappingFile, false,
> CodeGenOpts.DebugInfoForProfiling);
> else if (CodeGenOpts.DebugInfoForProfiling)
> // -fdebug-info-for-profiling
> - PGOOpt = PGOOptions("", "", "", false, true);
> + PGOOpt = PGOOptions("", "", "", "", false, true);
>
> PassBuilder PB(TM.get(), PGOOpt);
>
> @@ -1130,6 +1133,7 @@ static void runThinLTOBackend(ModuleSumm
> const LangOptions &LOpts,
> std::unique_ptr<raw_pwrite_stream> OS,
> std::string SampleProfile,
> + std::string ProfileRemapping,
> BackendAction Action) {
> StringMap<DenseMap<GlobalValue::GUID, GlobalValueSummary *>>
> ModuleToDefinedGVSummaries;
> @@ -1202,6 +1206,7 @@ static void runThinLTOBackend(ModuleSumm
> Conf.CGOptLevel = getCGOptLevel(CGOpts);
> initTargetOptions(Conf.Options, CGOpts, TOpts, LOpts, HeaderOpts);
> Conf.SampleProfile = std::move(SampleProfile);
> + Conf.ProfileRemapping = std::move(ProfileRemapping);
> Conf.UseNewPM = CGOpts.ExperimentalNewPassManager;
> Conf.DebugPassManager = CGOpts.DebugPassManager;
> Conf.RemarksWithHotness = CGOpts.DiagnosticsWithHotness;
> @@ -1268,7 +1273,7 @@ void clang::EmitBackendOutput(Diagnostic
> if (!CombinedIndex->skipModuleByDistributedBackend()) {
> runThinLTOBackend(CombinedIndex.get(), M, HeaderOpts, CGOpts, TOpts,
> LOpts, std::move(OS), CGOpts.SampleProfileFile,
> - Action);
> + CGOpts.ProfileRemappingFile, Action);
> return;
> }
> // Distributed indexing detected that nothing from the module is needed
>
> Modified: cfe/trunk/lib/CodeGen/CodeGenModule.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenModule.cpp?rev=344199&r1=344198&r2=344199&view=diff
> ==============================================================================
> --- cfe/trunk/lib/CodeGen/CodeGenModule.cpp (original)
> +++ cfe/trunk/lib/CodeGen/CodeGenModule.cpp Wed Oct 10 16:13:35 2018
> @@ -154,7 +154,7 @@ CodeGenModule::CodeGenModule(ASTContext
>
> if (CodeGenOpts.hasProfileClangUse()) {
> auto ReaderOrErr = llvm::IndexedInstrProfReader::create(
> - CodeGenOpts.ProfileInstrumentUsePath);
> + CodeGenOpts.ProfileInstrumentUsePath, CodeGenOpts.ProfileRemappingFile);
> if (auto E = ReaderOrErr.takeError()) {
> unsigned DiagID = Diags.getCustomDiagID(DiagnosticsEngine::Error,
> "Could not read profile %0: %1");
>
> Modified: cfe/trunk/lib/Driver/ToolChains/Clang.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/ToolChains/Clang.cpp?rev=344199&r1=344198&r2=344199&view=diff
> ==============================================================================
> --- cfe/trunk/lib/Driver/ToolChains/Clang.cpp (original)
> +++ cfe/trunk/lib/Driver/ToolChains/Clang.cpp Wed Oct 10 16:13:35 2018
> @@ -4355,6 +4355,7 @@ void Clang::ConstructJob(Compilation &C,
> else
> A->render(Args, CmdArgs);
> }
> + Args.AddLastArg(CmdArgs, options::OPT_fprofile_remapping_file_EQ);
>
> RenderBuiltinOptions(TC, RawTriple, Args, CmdArgs);
>
>
> Modified: cfe/trunk/lib/Frontend/CompilerInvocation.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/CompilerInvocation.cpp?rev=344199&r1=344198&r2=344199&view=diff
> ==============================================================================
> --- cfe/trunk/lib/Frontend/CompilerInvocation.cpp (original)
> +++ cfe/trunk/lib/Frontend/CompilerInvocation.cpp Wed Oct 10 16:13:35 2018
> @@ -657,6 +657,13 @@ static bool ParseCodeGenArgs(CodeGenOpti
> Args.getLastArgValue(OPT_fprofile_instrument_use_path_EQ);
> if (!Opts.ProfileInstrumentUsePath.empty())
> setPGOUseInstrumentor(Opts, Opts.ProfileInstrumentUsePath);
> + Opts.ProfileRemappingFile =
> + Args.getLastArgValue(OPT_fprofile_remapping_file_EQ);
> + if (!Opts.ProfileRemappingFile.empty() && !Opts.ExperimentalNewPassManager) {
> + Diags.Report(diag::err_drv_argument_only_allowed_with)
> + << Args.getLastArg(OPT_fprofile_remapping_file_EQ)->getAsString(Args)
> + << "-fexperimental-new-pass-manager";
> + }
>
> Opts.CoverageMapping =
> Args.hasFlag(OPT_fcoverage_mapping, OPT_fno_coverage_mapping, false);
>
> Added: cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map?rev=344199&view=auto
> ==============================================================================
> --- cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map (added)
> +++ cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map Wed Oct 10 16:13:35 2018
> @@ -0,0 +1,2 @@
> +name 3Foo 3Bar
> +type N3Foo1XE N3Baz1YE
>
> Added: cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext?rev=344199&view=auto
> ==============================================================================
> --- cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext (added)
> +++ cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext Wed Oct 10 16:13:35 2018
> @@ -0,0 +1,7 @@
> +:ir
> +_ZN3Foo8functionENS_1XE
> +29667547796
> +2
> +10
> +90
> +
>
> Added: cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples?rev=344199&view=auto
> ==============================================================================
> --- cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples (added)
> +++ cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples Wed Oct 10 16:13:35 2018
> @@ -0,0 +1,3 @@
> +_ZN3Bar8functionEN3Baz1YE:100:0
> + 2: 10
> + 4: 90
>
> Added: cfe/trunk/test/CodeGenCXX/profile-remap.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/profile-remap.cpp?rev=344199&view=auto
> ==============================================================================
> --- cfe/trunk/test/CodeGenCXX/profile-remap.cpp (added)
> +++ cfe/trunk/test/CodeGenCXX/profile-remap.cpp Wed Oct 10 16:13:35 2018
> @@ -0,0 +1,27 @@
> +// RUN: %clang_cc1 -triple x86_64-linux-gnu -fprofile-sample-use=%S/Inputs/profile-remap.samples -fprofile-remapping-file=%S/Inputs/profile-remap.map -fexperimental-new-pass-manager -O2 %s -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,CHECK-SAMPLES
> +// RUN: llvm-profdata merge -output %T.profdata %S/Inputs/profile-remap.proftext
> +// RUN: %clang_cc1 -triple x86_64-linux-gnu -fprofile-instrument-use-path=%T.profdata -fprofile-remapping-file=%S/Inputs/profile-remap.map -fexperimental-new-pass-manager -O2 %s -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,CHECK-INSTR
> +
> +namespace Foo {
> + struct X {};
> + bool cond();
> + void bar();
> + void baz();
> + void function(X x) {
> + if (cond())
> + bar();
> + else
> + baz();
> + }
> +}
> +
> +// CHECK: define {{.*}} @_ZN3Foo8functionENS_1XE() {{.*}} !prof [[FUNC_ENTRY:![0-9]*]]
> +// CHECK: br i1 {{.*}} !prof [[BR_WEIGHTS:![0-9]*]]
> +//
> +// FIXME: Laplace's rule of succession is applied to sample profiles...
> +// CHECK-SAMPLES-DAG: [[FUNC_ENTRY]] = !{!"function_entry_count", i64 1}
> +// CHECK-SAMPLES-DAG: [[BR_WEIGHTS]] = !{!"branch_weights", i32 11, i32 91}
> +//
> +// ... but not to instruction profiles.
> +// CHECK-INSTR-DAG: [[FUNC_ENTRY]] = !{!"function_entry_count", i64 100}
> +// CHECK-INSTR-DAG: [[BR_WEIGHTS]] = !{!"branch_weights", i32 10, i32 90}
>
> Modified: cfe/trunk/test/Driver/clang_f_opts.c
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/clang_f_opts.c?rev=344199&r1=344198&r2=344199&view=diff
> ==============================================================================
> --- cfe/trunk/test/Driver/clang_f_opts.c (original)
> +++ cfe/trunk/test/Driver/clang_f_opts.c Wed Oct 10 16:13:35 2018
> @@ -116,6 +116,7 @@
> // RUN: %clang -### -S -fcoverage-mapping %s 2>&1 | FileCheck -check-prefix=CHECK-COVERAGE-AND-GEN %s
> // RUN: %clang -### -S -fcoverage-mapping -fno-coverage-mapping %s 2>&1 | FileCheck -check-prefix=CHECK-DISABLE-COVERAGE %s
> // RUN: %clang -### -S -fprofile-instr-generate -fcoverage-mapping -fno-coverage-mapping %s 2>&1 | FileCheck -check-prefix=CHECK-DISABLE-COVERAGE %s
> +// RUN: %clang -### -S -fprofile-remapping-file foo/bar.txt %s 2>&1 | FileCheck -check-prefix=CHECK-PROFILE-REMAP %s
> // CHECK-PROFILE-GENERATE: "-fprofile-instrument=clang"
> // CHECK-PROFILE-GENERATE-LLVM: "-fprofile-instrument=llvm"
> // CHECK-PROFILE-GENERATE-DIR: "-fprofile-instrument-path=/some/dir{{/|\\\\}}{{.*}}"
> @@ -126,6 +127,7 @@
> // CHECK-DISABLE-USE-NOT: "-fprofile-instr-use"
> // CHECK-COVERAGE-AND-GEN: '-fcoverage-mapping' only allowed with '-fprofile-instr-generate'
> // CHECK-DISABLE-COVERAGE-NOT: "-fcoverage-mapping"
> +// CHECK-PROFILE-REMAP: "-fprofile-remapping-file=foo/bar.txt"
>
> // RUN: %clang -### -S -fprofile-use %s 2>&1 | FileCheck -check-prefix=CHECK-PROFILE-USE %s
> // RUN: %clang -### -S -fprofile-instr-use %s 2>&1 | FileCheck -check-prefix=CHECK-PROFILE-USE %s
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
More information about the cfe-commits
mailing list