r337552 - Implement cpu_dispatch/cpu_specific Multiversioning
Richard Smith via cfe-commits
cfe-commits at lists.llvm.org
Fri Sep 7 14:44:11 PDT 2018
(Noticed while reviewing https://reviews.llvm.org/D51812)
Should these attributes be disallowed on lambdas? Currently we allow them,
but that seems questionable since the call operator of a lambda can't be
overloaded. Eg:
auto x = []() __attribute__((cpu_dispatch(atom))) {};
void f() { x(); }
... generates an ifuc calling a resolver that can never be defined (and
likewise for cpu_specific).
On Fri, 20 Jul 2018 at 07:18, Erich Keane via cfe-commits <
cfe-commits at lists.llvm.org> wrote:
> Author: erichkeane
> Date: Fri Jul 20 07:13:28 2018
> New Revision: 337552
>
> URL: http://llvm.org/viewvc/llvm-project?rev=337552&view=rev
> Log:
> Implement cpu_dispatch/cpu_specific Multiversioning
>
> As documented here: https://software.intel.com/en-us/node/682969 and
> https://software.intel.com/en-us/node/523346. cpu_dispatch multiversioning
> is an ICC feature that provides for function multiversioning.
>
> This feature is implemented with two attributes: First, cpu_specific,
> which specifies the individual function versions. Second, cpu_dispatch,
> which specifies the location of the resolver function and the list of
> resolvable functions.
>
> This is valuable since it provides a mechanism where the resolver's TU
> can be specified in one location, and the individual implementions
> each in their own translation units.
>
> The goal of this patch is to be source-compatible with ICC, so this
> implementation diverges from the ICC implementation in a few ways:
> 1- Linux x86/64 only: This implementation uses ifuncs in order to
> properly dispatch functions. This is is a valuable performance benefit
> over the ICC implementation. A future patch will be provided to enable
> this feature on Windows, but it will obviously more closely fit ICC's
> implementation.
> 2- CPU Identification functions: ICC uses a set of custom functions to
> identify
> the feature list of the host processor. This patch uses the cpu_supports
> functionality in order to better align with 'target' multiversioning.
> 1- cpu_dispatch function def/decl: ICC's cpu_dispatch requires that the
> function
> marked cpu_dispatch be an empty definition. This patch supports that as
> well,
> however declarations are also permitted, since the linker will solve the
> issue of multiple emissions.
>
> Differential Revision: https://reviews.llvm.org/D47474
>
> Added:
> cfe/trunk/test/CodeGen/attr-cpuspecific.c
> cfe/trunk/test/Sema/attr-cpuspecific.c
> cfe/trunk/test/SemaCXX/attr-cpuspecific.cpp
> Modified:
> cfe/trunk/include/clang/AST/Decl.h
> cfe/trunk/include/clang/Basic/Attr.td
> cfe/trunk/include/clang/Basic/AttrDocs.td
> cfe/trunk/include/clang/Basic/DiagnosticGroups.td
> cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td
> cfe/trunk/include/clang/Basic/TargetInfo.h
> cfe/trunk/include/clang/Basic/X86Target.def
> cfe/trunk/lib/AST/Decl.cpp
> cfe/trunk/lib/Basic/Targets/X86.cpp
> cfe/trunk/lib/Basic/Targets/X86.h
> cfe/trunk/lib/CodeGen/CGBuiltin.cpp
> cfe/trunk/lib/CodeGen/CodeGenFunction.cpp
> cfe/trunk/lib/CodeGen/CodeGenFunction.h
> cfe/trunk/lib/CodeGen/CodeGenModule.cpp
> cfe/trunk/lib/CodeGen/CodeGenModule.h
> cfe/trunk/lib/Parse/ParseDecl.cpp
> cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp
> cfe/trunk/lib/Sema/Sema.cpp
> cfe/trunk/lib/Sema/SemaDecl.cpp
> cfe/trunk/lib/Sema/SemaDeclAttr.cpp
> cfe/trunk/lib/Sema/SemaExpr.cpp
> cfe/trunk/lib/Sema/SemaOverload.cpp
> cfe/trunk/test/Misc/pragma-attribute-supported-attributes-list.test
> cfe/trunk/utils/TableGen/ClangAttrEmitter.cpp
>
> Modified: cfe/trunk/include/clang/AST/Decl.h
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/AST/Decl.h?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/include/clang/AST/Decl.h (original)
> +++ cfe/trunk/include/clang/AST/Decl.h Fri Jul 20 07:13:28 2018
> @@ -2209,6 +2209,13 @@ public:
> getCanonicalDecl()->IsMultiVersion = V;
> }
>
> + /// True if this function is a multiversioned dispatch function as a
> part of
> + /// the cpu_specific/cpu_dispatch functionality.
> + bool isCPUDispatchMultiVersion() const;
> + /// True if this function is a multiversioned processor specific
> function as a
> + /// part of the cpu_specific/cpu_dispatch functionality.
> + bool isCPUSpecificMultiVersion() const;
> +
> void setPreviousDeclaration(FunctionDecl * PrevDecl);
>
> FunctionDecl *getCanonicalDecl() override;
>
> Modified: cfe/trunk/include/clang/Basic/Attr.td
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/Attr.td?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/include/clang/Basic/Attr.td (original)
> +++ cfe/trunk/include/clang/Basic/Attr.td Fri Jul 20 07:13:28 2018
> @@ -168,6 +168,7 @@ class UnsignedArgument<string name, bit
> class VariadicUnsignedArgument<string name> : Argument<name, 1>;
> class VariadicExprArgument<string name> : Argument<name, 1>;
> class VariadicStringArgument<string name> : Argument<name, 1>;
> +class VariadicIdentifierArgument<string name> : Argument<name, 1>;
>
> // Like VariadicUnsignedArgument except values are ParamIdx.
> class VariadicParamIdxArgument<string name> : Argument<name, 1>;
> @@ -845,6 +846,27 @@ def Constructor : InheritableAttr {
> let Documentation = [Undocumented];
> }
>
> +def CPUSpecific : InheritableAttr {
> + let Spellings = [Clang<"cpu_specific">];
> + let Args = [VariadicIdentifierArgument<"Cpus">];
> + let Subjects = SubjectList<[Function]>;
> + let Documentation = [CPUSpecificCPUDispatchDocs];
> + let AdditionalMembers = [{
> + unsigned ActiveArgIndex = 0;
> +
> + IdentifierInfo *getCurCPUName() const {
> + return *(cpus_begin() + ActiveArgIndex);
> + }
> + }];
> +}
> +
> +def CPUDispatch : InheritableAttr {
> + let Spellings = [Clang<"cpu_dispatch">];
> + let Args = [VariadicIdentifierArgument<"Cpus">];
> + let Subjects = SubjectList<[Function]>;
> + let Documentation = [CPUSpecificCPUDispatchDocs];
> +}
> +
> // CUDA attributes are spelled __attribute__((attr)) or
> __declspec(__attr__),
> // and they do not receive a [[]] spelling.
> def CUDAConstant : InheritableAttr {
>
> Modified: cfe/trunk/include/clang/Basic/AttrDocs.td
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/AttrDocs.td?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/include/clang/Basic/AttrDocs.td (original)
> +++ cfe/trunk/include/clang/Basic/AttrDocs.td Fri Jul 20 07:13:28 2018
> @@ -191,6 +191,65 @@ in generation of more efficient code.
> }];
> }
>
> +def CPUSpecificCPUDispatchDocs : Documentation {
> + let Category = DocCatFunction;
> + let Content = [{
> +The ``cpu_specific`` and ``cpu_dispatch`` attributes are used to define
> and
> +resolve multiversioned functions. This form of multiversioning provides a
> +mechanism for declaring versions across translation units and manually
> +specifying the resolved function list. A specified CPU defines a set of
> minimum
> +features that are required for the function to be called. The result of
> this is
> +that future processors execute the most restrictive version of the
> function the
> +new processor can execute.
> +
> +Function versions are defined with ``cpu_specific``, which takes one or
> more CPU
> +names as a parameter. For example:
> +
> +.. code-block:: c
> +
> + // Declares and defines the ivybridge version of single_cpu.
> + __attribute__((cpu_specific(ivybridge)))
> + void single_cpu(void){}
> +
> + // Declares and defines the atom version of single_cpu.
> + __attribute__((cpu_specific(atom)))
> + void single_cpu(void){}
> +
> + // Declares and defines both the ivybridge and atom version of
> multi_cpu.
> + __attribute__((cpu_specific(ivybridge, atom)))
> + void multi_cpu(void){}
> +
> +A dispatching (or resolving) function can be declared anywhere in a
> project's
> +source code with ``cpu_dispatch``. This attribute takes one or more CPU
> names
> +as a parameter (like ``cpu_specific``). Functions marked with
> ``cpu_dispatch``
> +are not expected to be defined, only declared. If such a marked function
> has a
> +definition, any side effects of the function are ignored; trivial function
> +bodies are permissible for ICC compatibility.
> +
> +.. code-block:: c
> +
> + // Creates a resolver for single_cpu above.
> + __attribute__((cpu_dispatch(ivybridge, atom)))
> + void single_cpu(void){}
> +
> + // Creates a resolver for multi_cpu, but adds a 3rd version defined in
> another
> + // translation unit.
> + __attribute__((cpu_dispatch(ivybridge, atom, sandybridge)))
> + void multi_cpu(void){}
> +
> +Note that it is possible to have a resolving function that dispatches
> based on
> +more or fewer options than are present in the program. Specifying fewer
> will
> +result in the omitted options not being considered during resolution.
> Specifying
> +a version for resolution that isn't defined in the program will result in
> a
> +linking failure.
> +
> +It is also possible to specify a CPU name of ``generic`` which will be
> resolved
> +if the executing processor doesn't satisfy the features required in the
> CPU
> +name. The behavior of a program executing on a processor that doesn't
> satisfy
> +any option of a multiversioned function is undefined.
> + }];
> +}
> +
> def C11NoReturnDocs : Documentation {
> let Category = DocCatFunction;
> let Content = [{
>
> Modified: cfe/trunk/include/clang/Basic/DiagnosticGroups.td
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticGroups.td?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/include/clang/Basic/DiagnosticGroups.td (original)
> +++ cfe/trunk/include/clang/Basic/DiagnosticGroups.td Fri Jul 20 07:13:28
> 2018
> @@ -1022,3 +1022,7 @@ def SpirCompat : DiagGroup<"spir-compat"
>
> // Warning for the experimental-isel options.
> def ExperimentalISel : DiagGroup<"experimental-isel">;
> +
> +// A warning group specifically for warnings related to function
> +// multiversioning.
> +def FunctionMultiVersioning : DiagGroup<"function-multiversion">;
>
> Modified: cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td (original)
> +++ cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td Fri Jul 20
> 07:13:28 2018
> @@ -608,6 +608,8 @@ def err_builtin_redeclare : Error<"canno
> def err_arm_invalid_specialreg : Error<"invalid special register for
> builtin">;
> def err_invalid_cpu_supports : Error<"invalid cpu feature string for
> builtin">;
> def err_invalid_cpu_is : Error<"invalid cpu name for builtin">;
> +def err_invalid_cpu_specific_dispatch_value : Error<
> +"invalid option '%0' for %select{cpu_specific|cpu_dispatch}1">;
> def err_builtin_needs_feature : Error<"%0 needs target feature %1">;
> def err_function_needs_feature
> : Error<"always_inline function %1 requires target feature '%2', but
> would "
> @@ -3788,8 +3790,8 @@ def err_ovl_no_viable_subscript :
> def err_ovl_no_oper :
> Error<"type %0 does not provide a %select{subscript|call}1 operator">;
> def err_ovl_unresolvable : Error<
> - "reference to overloaded function could not be resolved; "
> - "did you mean to call it%select{| with no arguments}0?">;
> + "reference to %select{overloaded|multiversioned}1 function could not be
> "
> + "resolved; did you mean to call it%select{| with no arguments}0?">;
> def err_bound_member_function : Error<
> "reference to non-static member function must be called"
> "%select{|; did you mean to call it with no arguments?}0">;
> @@ -9355,9 +9357,9 @@ def warn_shadow_field :
> InGroup<ShadowField>, DefaultIgnore;
> def note_shadow_field : Note<"declared here">;
>
> -def err_target_required_in_redecl : Error<
> - "function declaration is missing 'target' attribute in a multiversioned
> "
> - "function">;
> +def err_multiversion_required_in_redecl : Error<
> + "function declaration is missing %select{'target'|'cpu_specific' or "
> + "'cpu_dispatch'}0 attribute in a multiversioned function">;
> def note_multiversioning_caused_here : Note<
> "function multiversioning caused by this declaration">;
> def err_multiversion_after_used : Error<
> @@ -9371,20 +9373,33 @@ def err_multiversion_duplicate : Error<
> def err_multiversion_noproto : Error<
> "multiversioned function must have a prototype">;
> def err_multiversion_no_other_attrs : Error<
> - "attribute 'target' multiversioning cannot be combined with other "
> - "attributes">;
> + "attribute '%select{target|cpu_specific|cpu_dispatch}0' multiversioning
> cannot be combined"
> + " with other attributes">;
> def err_multiversion_diff : Error<
> "multiversioned function declaration has a different %select{calling
> convention"
> "|return type|constexpr specification|inline specification|storage
> class|"
> "linkage}0">;
> def err_multiversion_doesnt_support : Error<
> - "multiversioned functions do not yet support %select{function
> templates|"
> - "virtual functions|deduced return types|constructors|destructors|"
> - "deleted functions|defaulted functions}0">;
> + "attribute '%select{target|cpu_specific|cpu_dispatch}0' multiversioned
> functions do not "
> + "yet support %select{function templates|virtual functions|"
> + "deduced return types|constructors|destructors|deleted functions|"
> + "defaulted functions|constexpr functions}1">;
> def err_multiversion_not_allowed_on_main : Error<
> "'main' cannot be a multiversioned function">;
> def err_multiversion_not_supported : Error<
> "function multiversioning is not supported on the current target">;
> +def err_multiversion_types_mixed : Error<
> + "multiversioning attributes cannot be combined">;
> +def err_cpu_dispatch_mismatch : Error<
> + "'cpu_dispatch' function redeclared with different CPUs">;
> +def err_cpu_specific_multiple_defs : Error<
> + "multiple 'cpu_specific' functions cannot specify the same CPU: %0">;
> +def warn_multiversion_duplicate_entries : Warning<
> + "CPU list contains duplicate entries; attribute ignored">,
> + InGroup<FunctionMultiVersioning>;
> +def warn_dispatch_body_ignored : Warning<
> + "body of cpu_dispatch function will be ignored">,
> + InGroup<FunctionMultiVersioning>;
>
> // three-way comparison operator diagnostics
> def err_implied_comparison_category_type_not_found : Error<
>
> Modified: cfe/trunk/include/clang/Basic/TargetInfo.h
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/TargetInfo.h?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/include/clang/Basic/TargetInfo.h (original)
> +++ cfe/trunk/include/clang/Basic/TargetInfo.h Fri Jul 20 07:13:28 2018
> @@ -1092,6 +1092,27 @@ public:
> // argument.
> virtual bool validateCpuIs(StringRef Name) const { return false; }
>
> + // Validate a cpu_dispatch/cpu_specific CPU option, which is a
> different list
> + // from cpu_is, since it checks via features rather than CPUs directly.
> + virtual bool validateCPUSpecificCPUDispatch(StringRef Name) const {
> + return false;
> + }
> +
> + // Get the character to be added for mangling purposes for cpu_specific.
> + virtual char CPUSpecificManglingCharacter(StringRef Name) const {
> + llvm_unreachable(
> + "cpu_specific Multiversioning not implemented on this target");
> + }
> +
> + // Get a list of the features that make up the CPU option for
> + // cpu_specific/cpu_dispatch so that it can be passed to llvm as
> optimization
> + // options.
> + virtual void getCPUSpecificCPUDispatchFeatures(
> + StringRef Name, llvm::SmallVectorImpl<StringRef> &Features) const {
> + llvm_unreachable(
> + "cpu_specific Multiversioning not implemented on this target");
> + }
> +
> // Returns maximal number of args passed in registers.
> unsigned getRegParmMax() const {
> assert(RegParmMax < 7 && "RegParmMax value is larger than AST can
> handle");
>
> Modified: cfe/trunk/include/clang/Basic/X86Target.def
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/X86Target.def?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/include/clang/Basic/X86Target.def (original)
> +++ cfe/trunk/include/clang/Basic/X86Target.def Fri Jul 20 07:13:28 2018
> @@ -29,6 +29,14 @@
> #define FEATURE(ENUM)
> #endif
>
> +#ifndef CPU_SPECIFIC
> +#define CPU_SPECIFIC(NAME, MANGLING, FEATURES)
> +#endif
> +
> +#ifndef CPU_SPECIFIC_ALIAS
> +#define CPU_SPECIFIC_ALIAS(NEW_NAME, NAME)
> +#endif
> +
> #define PROC_64_BIT true
> #define PROC_32_BIT false
>
> @@ -276,6 +284,45 @@ FEATURE(FEATURE_AVX5124VNNIW)
> FEATURE(FEATURE_AVX5124FMAPS)
> FEATURE(FEATURE_AVX512VPOPCNTDQ)
>
> +
> +// FIXME: When commented out features are supported in LLVM, enable them
> here.
> +CPU_SPECIFIC("generic", 'A', "")
> +CPU_SPECIFIC("pentium", 'B', "")
> +CPU_SPECIFIC("pentium_pro", 'C', "+cmov")
> +CPU_SPECIFIC("pentium_mmx", 'D', "+mmx")
> +CPU_SPECIFIC("pentium_ii", 'E', "+cmov,+mmx")
> +CPU_SPECIFIC("pentium_iii", 'H', "+cmov,+mmx,+sse")
> +CPU_SPECIFIC("pentium_iii_no_xmm_regs", 'H',"+cmov,+sse")
> +CPU_SPECIFIC("pentium_4", 'J', "+cmov,+mmx,+sse,+sse2")
> +CPU_SPECIFIC("pentium_m", 'K', "+cmov,+mmx,+sse,+sse2")
> +CPU_SPECIFIC("pentium_4_sse3", 'L', "+cmov,+mmx,+sse,+sse2,+sse3")
> +CPU_SPECIFIC("core_2_duo_ssse3", 'M',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3")
> +CPU_SPECIFIC("core_2_duo_sse4_1", 'N',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1")
> +CPU_SPECIFIC("atom", 'O', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+movbe")
> +CPU_SPECIFIC("atom_sse4_2", 'c',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+popcnt")
> +CPU_SPECIFIC("core_i7_sse4_2", 'P',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+popcnt")
> +CPU_SPECIFIC("core_aes_pclmulqdq", 'Q',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+popcnt")
> +CPU_SPECIFIC("atom_sse4_2_movbe", 'd',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt")
> +CPU_SPECIFIC("goldmont", 'i',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt")
> +CPU_SPECIFIC("sandybridge", 'R',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+popcnt,+avx")
> +CPU_SPECIFIC_ALIAS("core_2nd_gen_avx", "sandybridge")
> +CPU_SPECIFIC("ivybridge", 'S',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+popcnt,+f16c,+avx")
> +CPU_SPECIFIC_ALIAS("core_3rd_gen_avx", "ivybridge")
> +CPU_SPECIFIC("haswell", 'V',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2")
> +CPU_SPECIFIC_ALIAS("core_4th_gen_avx", "haswell")
> +CPU_SPECIFIC("core_4th_gen_avx_tsx", 'W',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2")
> +CPU_SPECIFIC("broadwell", 'X',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2,+adx")
> +CPU_SPECIFIC_ALIAS("core_5th_gen_avx", "broadwell")
> +CPU_SPECIFIC("core_5th_gen_avx_tsx", 'Y',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2,+adx")
> +CPU_SPECIFIC("knl", 'Z',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2,+avx512f,+adx,+avx512er,+avx512pf,+avx512cd")
> +CPU_SPECIFIC_ALIAS("mic_avx512", "knl")
> +CPU_SPECIFIC("skylake", 'b',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2,+adx,+mpx")
> +CPU_SPECIFIC( "skylake_avx512", 'a',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2,+avx512dq,+avx512f,+adx,+avx512cd,+avx512bw,+avx512vl,+clwb")
> +CPU_SPECIFIC("cannonlake", 'e',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2,+avx512dq,+avx512f,+adx,+avx512ifma,+avx512cd,+avx512bw,+avx512vl,+avx512vbmi")
> +CPU_SPECIFIC("knm", 'j',
> "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2,+avx512f,+adx,+avx512er,+avx512pf,+avx512cd,+avx5124fmaps,+avx5124vnniw,+avx512vpopcntdq")
> +
> +#undef CPU_SPECIFIC_ALIAS
> +#undef CPU_SPECIFIC
> #undef PROC_64_BIT
> #undef PROC_32_BIT
> #undef FEATURE
>
> Modified: cfe/trunk/lib/AST/Decl.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/AST/Decl.cpp?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/AST/Decl.cpp (original)
> +++ cfe/trunk/lib/AST/Decl.cpp Fri Jul 20 07:13:28 2018
> @@ -2873,6 +2873,14 @@ bool FunctionDecl::isNoReturn() const {
> return false;
> }
>
> +bool FunctionDecl::isCPUDispatchMultiVersion() const {
> + return isMultiVersion() && hasAttr<CPUDispatchAttr>();
> +}
> +
> +bool FunctionDecl::isCPUSpecificMultiVersion() const {
> + return isMultiVersion() && hasAttr<CPUSpecificAttr>();
> +}
> +
> void
> FunctionDecl::setPreviousDeclaration(FunctionDecl *PrevDecl) {
> redeclarable_base::setPreviousDecl(PrevDecl);
>
> Modified: cfe/trunk/lib/Basic/Targets/X86.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/Targets/X86.cpp?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/Basic/Targets/X86.cpp (original)
> +++ cfe/trunk/lib/Basic/Targets/X86.cpp Fri Jul 20 07:13:28 2018
> @@ -1484,6 +1484,38 @@ unsigned X86TargetInfo::multiVersionSort
> return getFeaturePriority(getFeature(Name)) << 1;
> }
>
> +bool X86TargetInfo::validateCPUSpecificCPUDispatch(StringRef Name) const {
> + return llvm::StringSwitch<bool>(Name)
> +#define CPU_SPECIFIC(NAME, MANGLING, FEATURES) .Case(NAME, true)
> +#define CPU_SPECIFIC_ALIAS(NEW_NAME, NAME) .Case(NEW_NAME, true)
> +#include "clang/Basic/X86Target.def"
> + .Default(false);
> +}
> +
> +static StringRef CPUSpecificCPUDispatchNameDealias(StringRef Name) {
> + return llvm::StringSwitch<StringRef>(Name)
> +#define CPU_SPECIFIC_ALIAS(NEW_NAME, NAME) .Case(NEW_NAME, NAME)
> +#include "clang/Basic/X86Target.def"
> + .Default(Name);
> +}
> +
> +char X86TargetInfo::CPUSpecificManglingCharacter(StringRef Name) const {
> + return llvm::StringSwitch<char>(CPUSpecificCPUDispatchNameDealias(Name))
> +#define CPU_SPECIFIC(NAME, MANGLING, FEATURES) .Case(NAME, MANGLING)
> +#include "clang/Basic/X86Target.def"
> + .Default(0);
> +}
> +
> +void X86TargetInfo::getCPUSpecificCPUDispatchFeatures(
> + StringRef Name, llvm::SmallVectorImpl<StringRef> &Features) const {
> + StringRef WholeList =
> +
> llvm::StringSwitch<StringRef>(CPUSpecificCPUDispatchNameDealias(Name))
> +#define CPU_SPECIFIC(NAME, MANGLING, FEATURES) .Case(NAME, FEATURES)
> +#include "clang/Basic/X86Target.def"
> + .Default("");
> + WholeList.split(Features, ',', /*MaxSplit=*/-1, /*KeepEmpty=*/false);
> +}
> +
> std::string X86TargetInfo::getCPUKindCanonicalName(CPUKind Kind) const {
> switch (Kind) {
> case CK_Generic:
>
> Modified: cfe/trunk/lib/Basic/Targets/X86.h
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/Targets/X86.h?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/Basic/Targets/X86.h (original)
> +++ cfe/trunk/lib/Basic/Targets/X86.h Fri Jul 20 07:13:28 2018
> @@ -150,6 +150,14 @@ public:
>
> bool validateCpuIs(StringRef Name) const override;
>
> + bool validateCPUSpecificCPUDispatch(StringRef Name) const override;
> +
> + char CPUSpecificManglingCharacter(StringRef Name) const override;
> +
> + void getCPUSpecificCPUDispatchFeatures(
> + StringRef Name,
> + llvm::SmallVectorImpl<StringRef> &Features) const override;
> +
> bool validateAsmConstraint(const char *&Name,
> TargetInfo::ConstraintInfo &info) const
> override;
>
>
> Modified: cfe/trunk/lib/CodeGen/CGBuiltin.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGBuiltin.cpp?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/CodeGen/CGBuiltin.cpp (original)
> +++ cfe/trunk/lib/CodeGen/CGBuiltin.cpp Fri Jul 20 07:13:28 2018
> @@ -8904,11 +8904,10 @@ Value *CodeGenFunction::EmitX86CpuSuppor
> return EmitX86CpuSupports(FeatureStr);
> }
>
> -Value *CodeGenFunction::EmitX86CpuSupports(ArrayRef<StringRef>
> FeatureStrs) {
> +uint32_t
> +CodeGenFunction::GetX86CpuSupportsMask(ArrayRef<StringRef> FeatureStrs) {
> // Processor features and mapping to processor feature value.
> -
> uint32_t FeaturesMask = 0;
> -
> for (const StringRef &FeatureStr : FeatureStrs) {
> unsigned Feature =
> StringSwitch<unsigned>(FeatureStr)
> @@ -8917,7 +8916,14 @@ Value *CodeGenFunction::EmitX86CpuSuppor
> ;
> FeaturesMask |= (1U << Feature);
> }
> + return FeaturesMask;
> +}
> +
> +Value *CodeGenFunction::EmitX86CpuSupports(ArrayRef<StringRef>
> FeatureStrs) {
> + return EmitX86CpuSupports(GetX86CpuSupportsMask(FeatureStrs));
> +}
>
> +llvm::Value *CodeGenFunction::EmitX86CpuSupports(uint32_t FeaturesMask) {
> // Matching the struct layout from the compiler-rt/libgcc structure
> that is
> // filled in:
> // unsigned int __cpu_vendor;
>
> Modified: cfe/trunk/lib/CodeGen/CodeGenFunction.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenFunction.cpp?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/CodeGen/CodeGenFunction.cpp (original)
> +++ cfe/trunk/lib/CodeGen/CodeGenFunction.cpp Fri Jul 20 07:13:28 2018
> @@ -2323,7 +2323,8 @@ void CodeGenFunction::checkTargetFeature
> << TargetDecl->getDeclName()
> << CGM.getContext().BuiltinInfo.getRequiredFeatures(BuiltinID);
>
> - } else if (TargetDecl->hasAttr<TargetAttr>()) {
> + } else if (TargetDecl->hasAttr<TargetAttr>() ||
> + TargetDecl->hasAttr<CPUSpecificAttr>()) {
> // Get the required features for the callee.
>
> const TargetAttr *TD = TargetDecl->getAttr<TargetAttr>();
> @@ -2358,8 +2359,8 @@ void CodeGenFunction::EmitSanitizerStatR
> CGM.getSanStats().create(IRB, SSK);
> }
>
> -llvm::Value *
> -CodeGenFunction::FormResolverCondition(const MultiVersionResolverOption
> &RO) {
> +llvm::Value *CodeGenFunction::FormResolverCondition(
> + const TargetMultiVersionResolverOption &RO) {
> llvm::Value *TrueCondition = nullptr;
> if (!RO.ParsedAttribute.Architecture.empty())
> TrueCondition = EmitX86CpuIs(RO.ParsedAttribute.Architecture);
> @@ -2377,8 +2378,9 @@ CodeGenFunction::FormResolverCondition(c
> return TrueCondition;
> }
>
> -void CodeGenFunction::EmitMultiVersionResolver(
> - llvm::Function *Resolver, ArrayRef<MultiVersionResolverOption>
> Options) {
> +void CodeGenFunction::EmitTargetMultiVersionResolver(
> + llvm::Function *Resolver,
> + ArrayRef<TargetMultiVersionResolverOption> Options) {
> assert((getContext().getTargetInfo().getTriple().getArch() ==
> llvm::Triple::x86 ||
> getContext().getTargetInfo().getTriple().getArch() ==
> @@ -2391,7 +2393,7 @@ void CodeGenFunction::EmitMultiVersionRe
> EmitX86CpuInit();
>
> llvm::Function *DefaultFunc = nullptr;
> - for (const MultiVersionResolverOption &RO : Options) {
> + for (const TargetMultiVersionResolverOption &RO : Options) {
> Builder.SetInsertPoint(CurBlock);
> llvm::Value *TrueCondition = FormResolverCondition(RO);
>
> @@ -2412,6 +2414,44 @@ void CodeGenFunction::EmitMultiVersionRe
> Builder.CreateRet(DefaultFunc);
> }
>
> +void CodeGenFunction::EmitCPUDispatchMultiVersionResolver(
> + llvm::Function *Resolver,
> + ArrayRef<CPUDispatchMultiVersionResolverOption> Options) {
> + assert((getContext().getTargetInfo().getTriple().getArch() ==
> + llvm::Triple::x86 ||
> + getContext().getTargetInfo().getTriple().getArch() ==
> + llvm::Triple::x86_64) &&
> + "Only implemented for x86 targets");
> +
> + // Main function's basic block.
> + llvm::BasicBlock *CurBlock = createBasicBlock("resolver_entry",
> Resolver);
> + Builder.SetInsertPoint(CurBlock);
> + EmitX86CpuInit();
> +
> + for (const CPUDispatchMultiVersionResolverOption &RO : Options) {
> + Builder.SetInsertPoint(CurBlock);
> +
> + // "generic" case should catch-all.
> + if (RO.FeatureMask == 0) {
> + Builder.CreateRet(RO.Function);
> + return;
> + }
> + llvm::BasicBlock *RetBlock = createBasicBlock("resolver_return",
> Resolver);
> + llvm::IRBuilder<> RetBuilder(RetBlock);
> + RetBuilder.CreateRet(RO.Function);
> + CurBlock = createBasicBlock("resolver_else", Resolver);
> + llvm::Value *TrueCondition = EmitX86CpuSupports(RO.FeatureMask);
> + Builder.CreateCondBr(TrueCondition, RetBlock, CurBlock);
> + }
> +
> + Builder.SetInsertPoint(CurBlock);
> + llvm::CallInst *TrapCall = EmitTrapCall(llvm::Intrinsic::trap);
> + TrapCall->setDoesNotReturn();
> + TrapCall->setDoesNotThrow();
> + Builder.CreateUnreachable();
> + Builder.ClearInsertionPoint();
> +}
> +
> llvm::DebugLoc CodeGenFunction::SourceLocToDebugLoc(SourceLocation
> Location) {
> if (CGDebugInfo *DI = getDebugInfo())
> return DI->SourceLocToDebugLoc(Location);
>
> Modified: cfe/trunk/lib/CodeGen/CodeGenFunction.h
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenFunction.h?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/CodeGen/CodeGenFunction.h (original)
> +++ cfe/trunk/lib/CodeGen/CodeGenFunction.h Fri Jul 20 07:13:28 2018
> @@ -4113,12 +4113,13 @@ public:
>
> void EmitSanitizerStatReport(llvm::SanitizerStatKind SSK);
>
> - struct MultiVersionResolverOption {
> + struct TargetMultiVersionResolverOption {
> llvm::Function *Function;
> TargetAttr::ParsedTargetAttr ParsedAttribute;
> unsigned Priority;
> - MultiVersionResolverOption(const TargetInfo &TargInfo, llvm::Function
> *F,
> - const clang::TargetAttr::ParsedTargetAttr
> &PT)
> + TargetMultiVersionResolverOption(
> + const TargetInfo &TargInfo, llvm::Function *F,
> + const clang::TargetAttr::ParsedTargetAttr &PT)
> : Function(F), ParsedAttribute(PT), Priority(0u) {
> for (StringRef Feat : PT.Features)
> Priority = std::max(Priority,
> @@ -4129,12 +4130,30 @@ public:
>
> TargInfo.multiVersionSortPriority(PT.Architecture));
> }
>
> - bool operator>(const MultiVersionResolverOption &Other) const {
> + bool operator>(const TargetMultiVersionResolverOption &Other) const {
> return Priority > Other.Priority;
> }
> };
> - void EmitMultiVersionResolver(llvm::Function *Resolver,
> - ArrayRef<MultiVersionResolverOption>
> Options);
> + void EmitTargetMultiVersionResolver(
> + llvm::Function *Resolver,
> + ArrayRef<TargetMultiVersionResolverOption> Options);
> +
> + struct CPUDispatchMultiVersionResolverOption {
> + llvm::Function *Function;
> + // Note: EmitX86CPUSupports only has 32 bits available, so we store
> the mask
> + // as 32 bits here. When 64-bit support is added to
> __builtin_cpu_supports,
> + // this can be extended to 64 bits.
> + uint32_t FeatureMask;
> + CPUDispatchMultiVersionResolverOption(llvm::Function *F, uint64_t
> Mask)
> + : Function(F), FeatureMask(static_cast<uint32_t>(Mask)) {}
> + bool operator>(const CPUDispatchMultiVersionResolverOption &Other)
> const {
> + return FeatureMask > Other.FeatureMask;
> + }
> + };
> + void EmitCPUDispatchMultiVersionResolver(
> + llvm::Function *Resolver,
> + ArrayRef<CPUDispatchMultiVersionResolverOption> Options);
> + static uint32_t GetX86CpuSupportsMask(ArrayRef<StringRef> FeatureStrs);
>
> private:
> QualType getVarArgType(const Expr *Arg);
> @@ -4151,8 +4170,10 @@ private:
> llvm::Value *EmitX86CpuIs(StringRef CPUStr);
> llvm::Value *EmitX86CpuSupports(const CallExpr *E);
> llvm::Value *EmitX86CpuSupports(ArrayRef<StringRef> FeatureStrs);
> + llvm::Value *EmitX86CpuSupports(uint32_t Mask);
> llvm::Value *EmitX86CpuInit();
> - llvm::Value *FormResolverCondition(const MultiVersionResolverOption
> &RO);
> + llvm::Value *
> + FormResolverCondition(const TargetMultiVersionResolverOption &RO);
> };
>
> /// Helper class with most of the code for saving a value for a
>
> Modified: cfe/trunk/lib/CodeGen/CodeGenModule.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenModule.cpp?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/CodeGen/CodeGenModule.cpp (original)
> +++ cfe/trunk/lib/CodeGen/CodeGenModule.cpp Fri Jul 20 07:13:28 2018
> @@ -861,22 +861,38 @@ void CodeGenModule::setTLSMode(llvm::Glo
> GV->setThreadLocalMode(TLM);
> }
>
> +static std::string getCPUSpecificMangling(const CodeGenModule &CGM,
> + StringRef Name) {
> + const TargetInfo &Target = CGM.getTarget();
> + return (Twine('.') +
> Twine(Target.CPUSpecificManglingCharacter(Name))).str();
> +}
> +
> +static void AppendCPUSpecificCPUDispatchMangling(const CodeGenModule &CGM,
> + const CPUSpecificAttr
> *Attr,
> + raw_ostream &Out) {
> + // cpu_specific gets the current name, dispatch gets the resolver.
> + if (Attr)
> + Out << getCPUSpecificMangling(CGM, Attr->getCurCPUName()->getName());
> + else
> + Out << ".resolver";
> +}
> +
> static void AppendTargetMangling(const CodeGenModule &CGM,
> const TargetAttr *Attr, raw_ostream
> &Out) {
> if (Attr->isDefaultVersion())
> return;
>
> Out << '.';
> - const auto &Target = CGM.getTarget();
> + const TargetInfo &Target = CGM.getTarget();
> TargetAttr::ParsedTargetAttr Info =
> Attr->parse([&Target](StringRef LHS, StringRef RHS) {
> - // Multiversioning doesn't allow "no-${feature}", so
> we can
> - // only have "+" prefixes here.
> - assert(LHS.startswith("+") && RHS.startswith("+") &&
> - "Features should always have a prefix.");
> - return Target.multiVersionSortPriority(LHS.substr(1))
> >
> - Target.multiVersionSortPriority(RHS.substr(1));
> - });
> + // Multiversioning doesn't allow "no-${feature}", so we can
> + // only have "+" prefixes here.
> + assert(LHS.startswith("+") && RHS.startswith("+") &&
> + "Features should always have a prefix.");
> + return Target.multiVersionSortPriority(LHS.substr(1)) >
> + Target.multiVersionSortPriority(RHS.substr(1));
> + });
>
> bool IsFirst = true;
>
> @@ -895,7 +911,7 @@ static void AppendTargetMangling(const C
>
> static std::string getMangledNameImpl(const CodeGenModule &CGM,
> GlobalDecl GD,
> const NamedDecl *ND,
> - bool OmitTargetMangling = false) {
> + bool OmitMultiVersionMangling =
> false) {
> SmallString<256> Buffer;
> llvm::raw_svector_ostream Out(Buffer);
> MangleContext &MC = CGM.getCXXABI().getMangleContext();
> @@ -922,8 +938,14 @@ static std::string getMangledNameImpl(co
> }
>
> if (const auto *FD = dyn_cast<FunctionDecl>(ND))
> - if (FD->isMultiVersion() && !OmitTargetMangling)
> - AppendTargetMangling(CGM, FD->getAttr<TargetAttr>(), Out);
> + if (FD->isMultiVersion() && !OmitMultiVersionMangling) {
> + if (FD->isCPUDispatchMultiVersion() ||
> FD->isCPUSpecificMultiVersion())
> + AppendCPUSpecificCPUDispatchMangling(
> + CGM, FD->getAttr<CPUSpecificAttr>(), Out);
> + else
> + AppendTargetMangling(CGM, FD->getAttr<TargetAttr>(), Out);
> + }
> +
> return Out.str();
> }
>
> @@ -936,7 +958,7 @@ void CodeGenModule::UpdateMultiVersionNa
> // allows us to lookup the version that was emitted when this wasn't a
> // multiversion function.
> std::string NonTargetName =
> - getMangledNameImpl(*this, GD, FD, /*OmitTargetMangling=*/true);
> + getMangledNameImpl(*this, GD, FD,
> /*OmitMultiVersionMangling=*/true);
> GlobalDecl OtherGD;
> if (lookupRepresentativeDecl(NonTargetName, OtherGD)) {
> assert(OtherGD.getCanonicalDecl()
> @@ -979,11 +1001,30 @@ StringRef CodeGenModule::getMangledName(
> }
> }
>
> + const auto *FD = dyn_cast<FunctionDecl>(GD.getDecl());
> + // Since CPUSpecific can require multiple emits per decl, store the
> manglings
> + // separately.
> + if (FD &&
> + (FD->isCPUDispatchMultiVersion() ||
> FD->isCPUSpecificMultiVersion())) {
> + const auto *SD = FD->getAttr<CPUSpecificAttr>();
> +
> + std::pair<GlobalDecl, unsigned> SpecCanonicalGD{
> + CanonicalGD,
> + SD ? SD->ActiveArgIndex : std::numeric_limits<unsigned>::max()};
> +
> + auto FoundName = CPUSpecificMangledDeclNames.find(SpecCanonicalGD);
> + if (FoundName != CPUSpecificMangledDeclNames.end())
> + return FoundName->second;
> +
> + auto Result = CPUSpecificManglings.insert(
> + std::make_pair(getMangledNameImpl(*this, GD, FD),
> SpecCanonicalGD));
> + return CPUSpecificMangledDeclNames[SpecCanonicalGD] =
> Result.first->first();
> + }
> +
> auto FoundName = MangledDeclNames.find(CanonicalGD);
> if (FoundName != MangledDeclNames.end())
> return FoundName->second;
>
> -
> // Keep the first result in the case of a mangling collision.
> const auto *ND = cast<NamedDecl>(GD.getDecl());
> auto Result =
> @@ -1321,8 +1362,9 @@ bool CodeGenModule::GetCPUAndFeaturesAtt
> const auto *FD = dyn_cast_or_null<FunctionDecl>(D);
> FD = FD ? FD->getMostRecentDecl() : FD;
> const auto *TD = FD ? FD->getAttr<TargetAttr>() : nullptr;
> + const auto *SD = FD ? FD->getAttr<CPUSpecificAttr>() : nullptr;
> bool AddedAttr = false;
> - if (TD) {
> + if (TD || SD) {
> llvm::StringMap<bool> FeatureMap;
> getFunctionFeatureMap(FeatureMap, FD);
>
> @@ -1334,10 +1376,12 @@ bool CodeGenModule::GetCPUAndFeaturesAtt
> // While we populated the feature map above, we still need to
> // get and parse the target attribute so we can get the cpu for
> // the function.
> - TargetAttr::ParsedTargetAttr ParsedAttr = TD->parse();
> - if (ParsedAttr.Architecture != "" &&
> - getTarget().isValidCPUName(ParsedAttr.Architecture))
> - TargetCPU = ParsedAttr.Architecture;
> + if (TD) {
> + TargetAttr::ParsedTargetAttr ParsedAttr = TD->parse();
> + if (ParsedAttr.Architecture != "" &&
> + getTarget().isValidCPUName(ParsedAttr.Architecture))
> + TargetCPU = ParsedAttr.Architecture;
> + }
> } else {
> // Otherwise just add the existing target cpu and target features to
> the
> // function.
> @@ -2037,6 +2081,10 @@ void CodeGenModule::EmitGlobal(GlobalDec
> if (Global->hasAttr<IFuncAttr>())
> return emitIFuncDefinition(GD);
>
> + // If this is a cpu_dispatch multiversion function, emit the resolver.
> + if (Global->hasAttr<CPUDispatchAttr>())
> + return emitCPUDispatchDefinition(GD);
> +
> // If this is CUDA, be selective about which declarations we emit.
> if (LangOpts.CUDA) {
> if (LangOpts.CUDAIsDevice) {
> @@ -2355,7 +2403,7 @@ static void ReplaceUsesOfNonProtoTypeWit
>
> void CodeGenModule::emitMultiVersionFunctions() {
> for (GlobalDecl GD : MultiVersionFuncs) {
> - SmallVector<CodeGenFunction::MultiVersionResolverOption, 10> Options;
> + SmallVector<CodeGenFunction::TargetMultiVersionResolverOption, 10>
> Options;
> const FunctionDecl *FD = cast<FunctionDecl>(GD.getDecl());
> getContext().forEachMultiversionedFunctionVersion(
> FD, [this, &GD, &Options](const FunctionDecl *CurFD) {
> @@ -2387,28 +2435,75 @@ void CodeGenModule::emitMultiVersionFunc
> getModule().getOrInsertComdat(ResolverFunc->getName()));
> std::stable_sort(
> Options.begin(), Options.end(),
> - std::greater<CodeGenFunction::MultiVersionResolverOption>());
> +
> std::greater<CodeGenFunction::TargetMultiVersionResolverOption>());
> CodeGenFunction CGF(*this);
> - CGF.EmitMultiVersionResolver(ResolverFunc, Options);
> + CGF.EmitTargetMultiVersionResolver(ResolverFunc, Options);
> }
> }
>
> +void CodeGenModule::emitCPUDispatchDefinition(GlobalDecl GD) {
> + const auto *FD = cast<FunctionDecl>(GD.getDecl());
> + assert(FD && "Not a FunctionDecl?");
> + const auto *DD = FD->getAttr<CPUDispatchAttr>();
> + assert(DD && "Not a cpu_dispatch Function?");
> + llvm::Type *DeclTy = getTypes().ConvertTypeForMem(FD->getType());
> +
> + StringRef ResolverName = getMangledName(GD);
> + llvm::Type *ResolverType = llvm::FunctionType::get(
> + llvm::PointerType::get(DeclTy,
> +
> Context.getTargetAddressSpace(FD->getType())),
> + false);
> + auto *ResolverFunc = cast<llvm::Function>(
> + GetOrCreateLLVMFunction(ResolverName, ResolverType, GlobalDecl{},
> + /*ForVTable=*/false));
> +
> + SmallVector<CodeGenFunction::CPUDispatchMultiVersionResolverOption, 10>
> + Options;
> + const TargetInfo &Target = getTarget();
> + for (const IdentifierInfo *II : DD->cpus()) {
> + // Get the name of the target function so we can look it up/create it.
> + std::string MangledName = getMangledNameImpl(*this, GD, FD, true) +
> + getCPUSpecificMangling(*this,
> II->getName());
> + llvm::Constant *Func = GetOrCreateLLVMFunction(
> + MangledName, DeclTy, GD, /*ForVTable=*/false, /*DontDefer=*/false,
> + /*IsThunk=*/false, llvm::AttributeList(), ForDefinition);
> + llvm::SmallVector<StringRef, 32> Features;
> + Target.getCPUSpecificCPUDispatchFeatures(II->getName(), Features);
> + llvm::transform(Features, Features.begin(),
> + [](StringRef Str) { return Str.substr(1); });
> + Features.erase(std::remove_if(
> + Features.begin(), Features.end(), [&Target](StringRef Feat) {
> + return !Target.validateCpuSupports(Feat);
> + }), Features.end());
> + Options.emplace_back(cast<llvm::Function>(Func),
> +
> CodeGenFunction::GetX86CpuSupportsMask(Features));
> + }
> +
> + llvm::sort(
> + Options.begin(), Options.end(),
> +
> std::greater<CodeGenFunction::CPUDispatchMultiVersionResolverOption>());
> + CodeGenFunction CGF(*this);
> + CGF.EmitCPUDispatchMultiVersionResolver(ResolverFunc, Options);
> +}
> +
> /// If an ifunc for the specified mangled name is not in the module,
> create and
> /// return an llvm IFunc Function with the specified type.
> llvm::Constant *
> CodeGenModule::GetOrCreateMultiVersionIFunc(GlobalDecl GD, llvm::Type
> *DeclTy,
> - StringRef MangledName,
> const FunctionDecl *FD) {
> - std::string IFuncName = (MangledName + ".ifunc").str();
> + std::string MangledName =
> + getMangledNameImpl(*this, GD, FD,
> /*OmitMultiVersionMangling=*/true);
> + std::string IFuncName = MangledName + ".ifunc";
> if (llvm::GlobalValue *IFuncGV = GetGlobalValue(IFuncName))
> return IFuncGV;
>
> // Since this is the first time we've created this IFunc, make sure
> // that we put this multiversioned function into the list to be
> - // replaced later.
> - MultiVersionFuncs.push_back(GD);
> + // replaced later if necessary (target multiversioning only).
> + if (!FD->isCPUDispatchMultiVersion() &&
> !FD->isCPUSpecificMultiVersion())
> + MultiVersionFuncs.push_back(GD);
>
> - std::string ResolverName = (MangledName + ".resolver").str();
> + std::string ResolverName = MangledName + ".resolver";
> llvm::Type *ResolverType = llvm::FunctionType::get(
> llvm::PointerType::get(DeclTy,
>
> Context.getTargetAddressSpace(FD->getType())),
> @@ -2455,10 +2550,12 @@ llvm::Constant *CodeGenModule::GetOrCrea
> addDeferredDeclToEmit(GDDef);
> }
>
> - if (FD->isMultiVersion() &&
> FD->getAttr<TargetAttr>()->isDefaultVersion()) {
> - UpdateMultiVersionNames(GD, FD);
> + if (FD->isMultiVersion()) {
> + const auto *TA = FD->getAttr<TargetAttr>();
> + if (TA && TA->isDefaultVersion())
> + UpdateMultiVersionNames(GD, FD);
> if (!IsForDefinition)
> - return GetOrCreateMultiVersionIFunc(GD, Ty, MangledName, FD);
> + return GetOrCreateMultiVersionIFunc(GD, Ty, FD);
> }
> }
>
> @@ -3727,6 +3824,15 @@ void CodeGenModule::EmitGlobalFunctionDe
> AddGlobalDtor(Fn, DA->getPriority());
> if (D->hasAttr<AnnotateAttr>())
> AddGlobalAnnotations(D, Fn);
> +
> + if (D->isCPUSpecificMultiVersion()) {
> + auto *Spec = D->getAttr<CPUSpecificAttr>();
> + // If there is another specific version we need to emit, do so here.
> + if (Spec->ActiveArgIndex + 1 < Spec->cpus_size()) {
> + ++Spec->ActiveArgIndex;
> + EmitGlobalFunctionDefinition(GD, nullptr);
> + }
> + }
> }
>
> void CodeGenModule::EmitAliasDefinition(GlobalDecl GD) {
> @@ -5107,6 +5213,12 @@ void CodeGenModule::getFunctionFeatureMa
> // the attribute.
> Target.initFeatureMap(FeatureMap, getDiags(), TargetCPU,
> ParsedAttr.Features);
> + } else if (const auto *SD = FD->getAttr<CPUSpecificAttr>()) {
> + llvm::SmallVector<StringRef, 32> FeaturesTmp;
> +
> Target.getCPUSpecificCPUDispatchFeatures(SD->getCurCPUName()->getName(),
> + FeaturesTmp);
> + std::vector<std::string> Features(FeaturesTmp.begin(),
> FeaturesTmp.end());
> + Target.initFeatureMap(FeatureMap, getDiags(), TargetCPU, Features);
> } else {
> Target.initFeatureMap(FeatureMap, getDiags(), TargetCPU,
> Target.getTargetOpts().Features);
>
> Modified: cfe/trunk/lib/CodeGen/CodeGenModule.h
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenModule.h?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/CodeGen/CodeGenModule.h (original)
> +++ cfe/trunk/lib/CodeGen/CodeGenModule.h Fri Jul 20 07:13:28 2018
> @@ -366,6 +366,13 @@ private:
> llvm::MapVector<GlobalDecl, StringRef> MangledDeclNames;
> llvm::StringMap<GlobalDecl, llvm::BumpPtrAllocator> Manglings;
>
> + // An ordered map of canonical GlobalDecls paired with the cpu-index for
> + // cpu-specific name manglings.
> + llvm::MapVector<std::pair<GlobalDecl, unsigned>, StringRef>
> + CPUSpecificMangledDeclNames;
> + llvm::StringMap<std::pair<GlobalDecl, unsigned>, llvm::BumpPtrAllocator>
> + CPUSpecificManglings;
> +
> /// Global annotations.
> std::vector<llvm::Constant*> Annotations;
>
> @@ -1283,7 +1290,6 @@ private:
>
> llvm::Constant *GetOrCreateMultiVersionIFunc(GlobalDecl GD,
> llvm::Type *DeclTy,
> - StringRef MangledName,
> const FunctionDecl *FD);
> void UpdateMultiVersionNames(GlobalDecl GD, const FunctionDecl *FD);
>
> @@ -1307,6 +1313,7 @@ private:
> void EmitGlobalVarDefinition(const VarDecl *D, bool IsTentative =
> false);
> void EmitAliasDefinition(GlobalDecl GD);
> void emitIFuncDefinition(GlobalDecl GD);
> + void emitCPUDispatchDefinition(GlobalDecl GD);
> void EmitObjCPropertyImplementations(const ObjCImplementationDecl *D);
> void EmitObjCIvarInitializations(ObjCImplementationDecl *D);
>
>
> Modified: cfe/trunk/lib/Parse/ParseDecl.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/ParseDecl.cpp?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/Parse/ParseDecl.cpp (original)
> +++ cfe/trunk/lib/Parse/ParseDecl.cpp Fri Jul 20 07:13:28 2018
> @@ -215,6 +215,15 @@ static bool attributeHasIdentifierArg(co
> #undef CLANG_ATTR_IDENTIFIER_ARG_LIST
> }
>
> +/// Determine whether the given attribute has a variadic identifier
> argument.
> +static bool attributeHasVariadicIdentifierArg(const IdentifierInfo &II) {
> +#define CLANG_ATTR_VARIADIC_IDENTIFIER_ARG_LIST
> + return llvm::StringSwitch<bool>(normalizeAttrName(II.getName()))
> +#include "clang/Parse/AttrParserStringSwitches.inc"
> + .Default(false);
> +#undef CLANG_ATTR_VARIADIC_IDENTIFIER_ARG_LIST
> +}
> +
> /// Determine whether the given attribute parses a type argument.
> static bool attributeIsTypeArgAttr(const IdentifierInfo &II) {
> #define CLANG_ATTR_TYPE_ARG_LIST
> @@ -282,7 +291,8 @@ unsigned Parser::ParseAttributeArgsCommo
> ArgsVector ArgExprs;
> if (Tok.is(tok::identifier)) {
> // If this attribute wants an 'identifier' argument, make it so.
> - bool IsIdentifierArg = attributeHasIdentifierArg(*AttrName);
> + bool IsIdentifierArg = attributeHasIdentifierArg(*AttrName) ||
> + attributeHasVariadicIdentifierArg(*AttrName);
> ParsedAttr::Kind AttrKind =
> ParsedAttr::getKind(AttrName, ScopeName, Syntax);
>
> @@ -305,19 +315,25 @@ unsigned Parser::ParseAttributeArgsCommo
>
> // Parse the non-empty comma-separated list of expressions.
> do {
> - bool Uneval = attributeParsedArgsUnevaluated(*AttrName);
> - EnterExpressionEvaluationContext Unevaluated(
> - Actions,
> - Uneval ? Sema::ExpressionEvaluationContext::Unevaluated
> - : Sema::ExpressionEvaluationContext::ConstantEvaluated);
> -
> - ExprResult ArgExpr(
> - Actions.CorrectDelayedTyposInExpr(ParseAssignmentExpression()));
> - if (ArgExpr.isInvalid()) {
> - SkipUntil(tok::r_paren, StopAtSemi);
> - return 0;
> + ExprResult ArgExpr;
> + if (Tok.is(tok::identifier) &&
> + attributeHasVariadicIdentifierArg(*AttrName)) {
> + ArgExprs.push_back(ParseIdentifierLoc());
> + } else {
> + bool Uneval = attributeParsedArgsUnevaluated(*AttrName);
> + EnterExpressionEvaluationContext Unevaluated(
> + Actions,
> + Uneval ? Sema::ExpressionEvaluationContext::Unevaluated
> + :
> Sema::ExpressionEvaluationContext::ConstantEvaluated);
> +
> + ExprResult ArgExpr(
> +
> Actions.CorrectDelayedTyposInExpr(ParseAssignmentExpression()));
> + if (ArgExpr.isInvalid()) {
> + SkipUntil(tok::r_paren, StopAtSemi);
> + return 0;
> + }
> + ArgExprs.push_back(ArgExpr.get());
> }
> - ArgExprs.push_back(ArgExpr.get());
> // Eat the comma, move to the next argument
> } while (TryConsumeToken(tok::comma));
> }
>
> Modified: cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp (original)
> +++ cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp Fri Jul 20 07:13:28 2018
> @@ -658,6 +658,11 @@ static void CheckFallThroughForBody(Sema
> else
> S.Diag(Loc, DiagID);
> };
> +
> + // cpu_dispatch functions permit empty function bodies for ICC
> compatibility.
> + if (D->getAsFunction() &&
> D->getAsFunction()->isCPUDispatchMultiVersion())
> + return;
> +
> // Either in a function body compound statement, or a
> function-try-block.
> switch (CheckFallThrough(AC)) {
> case UnknownFallThrough:
>
> Modified: cfe/trunk/lib/Sema/Sema.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/Sema.cpp?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/Sema/Sema.cpp (original)
> +++ cfe/trunk/lib/Sema/Sema.cpp Fri Jul 20 07:13:28 2018
> @@ -1585,6 +1585,7 @@ bool Sema::tryExprAsCall(Expr &E, QualTy
> }
>
> bool Ambiguous = false;
> + bool IsMV = false;
>
> if (Overloads) {
> for (OverloadExpr::decls_iterator it = Overloads->decls_begin(),
> @@ -1598,11 +1599,16 @@ bool Sema::tryExprAsCall(Expr &E, QualTy
> if (const FunctionDecl *OverloadDecl
> = dyn_cast<FunctionDecl>((*it)->getUnderlyingDecl())) {
> if (OverloadDecl->getMinRequiredArguments() == 0) {
> - if (!ZeroArgCallReturnTy.isNull() && !Ambiguous) {
> + if (!ZeroArgCallReturnTy.isNull() && !Ambiguous &&
> + (!IsMV || !(OverloadDecl->isCPUDispatchMultiVersion() ||
> + OverloadDecl->isCPUSpecificMultiVersion()))) {
> ZeroArgCallReturnTy = QualType();
> Ambiguous = true;
> - } else
> + } else {
> ZeroArgCallReturnTy = OverloadDecl->getReturnType();
> + IsMV = OverloadDecl->isCPUDispatchMultiVersion() ||
> + OverloadDecl->isCPUSpecificMultiVersion();
> + }
> }
> }
> }
> @@ -1683,7 +1689,7 @@ static void noteOverloads(Sema &S, const
> NamedDecl *Fn = (*It)->getUnderlyingDecl();
> // Don't print overloads for non-default multiversioned functions.
> if (const auto *FD = Fn->getAsFunction()) {
> - if (FD->isMultiVersion() &&
> + if (FD->isMultiVersion() && FD->hasAttr<TargetAttr>() &&
> !FD->getAttr<TargetAttr>()->isDefaultVersion())
> continue;
> }
> @@ -1725,6 +1731,21 @@ static bool IsCallableWithAppend(Expr *E
> !isa<CXXOperatorCallExpr>(E));
> }
>
> +static bool IsCPUDispatchCPUSpecificMultiVersion(const Expr *E) {
> + if (const auto *UO = dyn_cast<UnaryOperator>(E))
> + E = UO->getSubExpr();
> +
> + if (const auto *ULE = dyn_cast<UnresolvedLookupExpr>(E)) {
> + if (ULE->getNumDecls() == 0)
> + return false;
> +
> + const NamedDecl *ND = *ULE->decls_begin();
> + if (const auto *FD = dyn_cast<FunctionDecl>(ND))
> + return FD->isCPUDispatchMultiVersion() ||
> FD->isCPUSpecificMultiVersion();
> + }
> + return false;
> +}
> +
> bool Sema::tryToRecoverWithCall(ExprResult &E, const PartialDiagnostic
> &PD,
> bool ForceComplain,
> bool (*IsPlausibleResult)(QualType)) {
> @@ -1741,12 +1762,13 @@ bool Sema::tryToRecoverWithCall(ExprResu
> // so we can emit a fixit and carry on pretending that E was
> // actually a CallExpr.
> SourceLocation ParenInsertionLoc =
> getLocForEndOfToken(Range.getEnd());
> - Diag(Loc, PD)
> - << /*zero-arg*/ 1 << Range
> - << (IsCallableWithAppend(E.get())
> - ? FixItHint::CreateInsertion(ParenInsertionLoc, "()")
> - : FixItHint());
> - notePlausibleOverloads(*this, Loc, Overloads, IsPlausibleResult);
> + bool IsMV = IsCPUDispatchCPUSpecificMultiVersion(E.get());
> + Diag(Loc, PD) << /*zero-arg*/ 1 << IsMV << Range
> + << (IsCallableWithAppend(E.get())
> + ? FixItHint::CreateInsertion(ParenInsertionLoc,
> "()")
> + : FixItHint());
> + if (!IsMV)
> + notePlausibleOverloads(*this, Loc, Overloads, IsPlausibleResult);
>
> // FIXME: Try this before emitting the fixit, and suppress diagnostics
> // while doing so.
> @@ -1757,8 +1779,10 @@ bool Sema::tryToRecoverWithCall(ExprResu
>
> if (!ForceComplain) return false;
>
> - Diag(Loc, PD) << /*not zero-arg*/ 0 << Range;
> - notePlausibleOverloads(*this, Loc, Overloads, IsPlausibleResult);
> + bool IsMV = IsCPUDispatchCPUSpecificMultiVersion(E.get());
> + Diag(Loc, PD) << /*not zero-arg*/ 0 << IsMV << Range;
> + if (!IsMV)
> + notePlausibleOverloads(*this, Loc, Overloads, IsPlausibleResult);
> E = ExprError();
> return true;
> }
>
> Modified: cfe/trunk/lib/Sema/SemaDecl.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/SemaDecl.cpp?rev=337552&r1=337551&r2=337552&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/Sema/SemaDecl.cpp (original)
> +++ cfe/trunk/lib/Sema/SemaDecl.cpp Fri Jul 20 07:13:28 2018
> @@ -9275,6 +9275,20 @@ bool Sema::shouldLinkDependentDeclWithPr
> D->getFriendObjectKind() != Decl::FOK_None);
> }
>
> +namespace MultiVersioning {
> +enum Type { None, Target, CPUSpecific, CPUDispatch};
> +} // MultiVersionType
> +
> +static MultiVersioning::Type
> +getMultiVersionType(const FunctionDecl *FD) {
> + if (FD->hasAttr<TargetAttr>())
> + return MultiVersioning::Target;
> + if (FD->hasAttr<CPUDispatchAttr>())
> + return MultiVersioning::CPUDispatch;
> + if (FD->hasAttr<CPUSpecificAttr>())
> + return MultiVersioning::CPUSpecific;
> + return MultiVersioning::None;
> +}
> /// Check the target attribute of the function for MultiVersion
> /// validity.
> ///
> @@ -9313,7 +9327,8 @@ static bool CheckMultiVersionValue(Sema
>
> static bool CheckMultiVersionAdditionalRules(Sema &S, const FunctionDecl
> *OldFD,
> const FunctionDecl *NewFD,
> - bool CausesMV) {
> + bool CausesMV,
> + MultiVersioning::Type
> MVType) {
> enum DoesntSupport {
> FuncTemplates = 0,
> VirtFuncs = 1,
> @@ -9321,7 +9336,8 @@ static bool CheckMultiVersionAdditionalR
> Constructors = 3,
> Destructors = 4,
> DeletedFuncs = 5,
> - DefaultedFuncs = 6
> + DefaultedFuncs = 6,
> + ConstexprFuncs = 7,
> };
> enum Different {
> CallingConv = 0,
> @@ -9332,46 +9348,73 @@ static bool CheckMultiVersionAdditionalR
> Linkage = 5
> };
>
> + bool IsCPUSpecificCPUDispatchMVType =
> + MVType == MultiVersioning::CPUDispatch ||
> + MVType == MultiVersioning::CPUSpecific;
> +
> + if (OldFD && !OldFD->getType()->getAs<FunctionProtoType>()) {
> + S.Diag(OldFD->getLocation(), diag::err_multiversion_noproto);
> + S.Diag(NewFD->getLocation(), diag::note_multiversioning_caused_here);
> + return true;
> + }
> +
> + if (!NewFD->getType()->getAs<FunctionProtoType>())
> + return S.Diag(NewFD->getLocation(), diag::err_multiversion_noproto);
> +
> + if (!S.getASTContext().getTargetInfo().supportsMultiVersioning()) {
> + S.Diag(NewFD->getLocation(), diag::err_multiversion_not_supported);
> + if (OldFD)
> + S.Diag(OldFD->getLocation(), diag::note_previous_declaration);
> + return true;
> + }
> +
> // For now, disallow all other attributes. These should be opt-in, but
> // an analysis of all of them is a future FIXME.
> if (CausesMV && OldFD &&
> std::distance(OldFD->attr_begin(), OldFD->attr_end()) != 1) {
> - S.Diag(OldFD->getLocation(), diag::err_multiversion_no_other_attrs);
> + S.Diag(OldFD->getLocation(), diag::err_multiversion_no_other_attrs)
> + << IsCPUSpecificCPUDispatchMVType;
> S.Diag(NewFD->getLocation(), diag::note_multiversioning_caused_here);
> return true;
> }
>
> if (std::distance(NewFD->attr_begin(), NewFD->attr_end()) != 1)
> - return S.Diag(NewFD->getLocation(),
> diag::err_multiversion_no_other_attrs);
> + return S.Diag(NewFD->getLocation(),
> diag::err_multiversion_no_other_attrs)
> + << IsCPUSpecificCPUDispatchMVType;
>
> if (NewFD->getTemplatedKind() == FunctionDecl::TK_FunctionTemplate)
> return S.Diag(NewFD->getLocation(),
> diag::err_multiversion_doesnt_support)
> - << FuncTemplates;
> + << IsCPUSpecificCPUDispatchMVType << FuncTemplates;
>
> if (const auto *NewCXXFD = dyn_cast<CXXMethodDecl>(NewFD)) {
> if (NewCXXFD->isVirtual())
> return S.Diag(NewCXXFD->getLocation(),
> diag::err_multiversion_doesnt_support)
> - << VirtFuncs;
> + << IsCPUSpecificCPUDispatchMVType << VirtFuncs;
>
> if (const auto *NewCXXCtor = dyn_cast<CXXConstructorDecl>(NewFD))
> return S.Diag(NewCXXCtor->getLocation(),
> diag::err_multiversion_doesnt_support)
> - << Constructors;
> + << IsCPUSpecificCPUDispatchMVType << Constructors;
>
> if (const auto *NewCXXDtor = dyn_cast<CXXDestructorDecl>(NewFD))
> return S.Diag(NewCXXDtor->getLocation(),
> diag::err_multiversion_doesnt_support)
> - << Destructors;
> + << IsCPUSpecificCPUDispatchMVType << Destructors;
> }
>
> if (NewFD->isDeleted())
> return S.Diag(NewFD->getLocation(),
> diag::err_multiversion_doesnt_support)
> - << DeletedFuncs;
> + << IsCPUSpecificCPUDispatchMVType << DeletedFuncs;
>
> if (NewFD->isDefaulted())
> return S.Diag(NewFD->getLocation(),
> diag::err_multiversion_doesnt_support)
> - << DefaultedFuncs;
> + << IsCPUSpecificCPUDispatchMVType << DefaultedFuncs;
> +
> + if (NewFD->isConstexpr() && (MVType == MultiVersioning::CPUDispatch ||
> + MVType == MultiVersioning::CPUSpecific))
> + return S.Diag(NewFD->getLocation(),
> diag::err_multiversion_doesnt_support)
> + << IsCPUSpecificCPUDispatchMVType << ConstexprFuncs;
>
> QualType NewQType =
> S.getASTContext().getCanonicalType(NewFD->getType());
> const auto *NewType = cast<FunctionType>(NewQType);
> @@ -9379,7 +9422,7 @@ static bool CheckMultiVersionAdditionalR
>
> if (NewReturnType->isUndeducedType())
> return S.Diag(NewFD->getLocation(),
> diag::err_multiversion_doesnt_support)
> - << DeducedReturn;
> + << IsCPUSpecificCPUDispatchMVType << DeducedReturn;
>
> // Only allow transition to MultiVersion if it hasn't been used.
> if (OldFD && CausesMV && OldFD->isUsed(false))
> @@ -9426,138 +9469,133 @@ static bool CheckMultiVersionAdditionalR
> return false;
> }
>
> -/// Check the validity of a mulitversion function declaration.
> -/// Also sets the multiversion'ness' of the function itself.
> +/// Check the validity of a multiversion function declaration that is the
> +/// first of its kind. Also sets the multiversion'ness' of the function
> itself.
> ///
> /// This sets NewFD->isInvalidDecl() to true if there was an error.
> ///
> /// Returns true if there was an error, false otherwise.
> -static bool CheckMultiVersionFunction(Sema &S, FunctionDecl *NewFD,
> - bool &Redeclaration, NamedDecl
> *&OldDecl,
> - bool &MergeTypeWithPrevious,
> - LookupResult &Previous) {
> - const auto *NewTA = NewFD->getAttr<TargetAttr>();
> - if (NewFD->isMain()) {
> - if (NewTA && NewTA->isDefaultVersion()) {
> - S.Diag(NewFD->getLocation(),
> diag::err_multiversion_not_allowed_on_main);
> - NewFD->setInvalidDecl();
> - return true;
> - }
> +static bool CheckMultiVersionFirstFunction(Sema &S, FunctionDecl *FD,
> + MultiVersioning::Type MVType,
> + const TargetAttr *TA,
> + const CPUDispatchAttr *CPUDisp,
> + const CPUSpecificAttr
> *CPUSpec) {
> + assert(MVType != MultiVersioning::None &&
> + "Function lacks multiversion attribute");
> +
> + // Target only causes MV if it is default, otherwise this is a normal
> + // function.
> + if (MVType == MultiVersioning::Target && !TA->isDefaultVersion())
> return false;
> - }
>
> - // If there is no matching previous decl, only 'default' can
> - // cause MultiVersioning.
> - if (!OldDecl) {
> - if (NewTA && NewTA->isDefaultVersion()) {
> - if (!NewFD->getType()->getAs<FunctionProtoType>()) {
> - S.Diag(NewFD->getLocation(), diag::err_multiversion_noproto);
> - NewFD->setInvalidDecl();
> - return true;
> - }
> - if (CheckMultiVersionAdditionalRules(S, nullptr, NewFD, true)) {
> - NewFD->setInvalidDecl();
> - return true;
> - }
> - if (!S.getASTContext().getTargetInfo().supportsMultiVersioning()) {
> - S.Diag(NewFD->getLocation(),
> diag::err_multiversion_not_supported);
> - NewFD->setInvalidDecl();
> - return true;
> - }
> + if (MVType == MultiVersioning::Target && CheckMultiVersionValue(S, FD))
> {
> + FD->setInvalidDecl();
> + return true;
> + }
>
> - NewFD->setIsMultiVersion();
> - }
> - return false;
> + if (CheckMultiVersionAdditionalRules(S, nullptr, FD, true, MVType)) {
> + FD->setInvalidDecl();
> + return true;
> }
>
> - if (OldDecl->getDeclContext()->getRedeclContext() !=
> - NewFD->getDeclContext()->getRedeclContext())
> - return false;
> + FD->setIsMultiVersion();
> + return false;
> +}
>
> - FunctionDecl *OldFD = OldDecl->getAsFunction();
> - // Unresolved 'using' statements (the other way OldDecl can be not a
> function)
> - // likely cannot cause a problem here.
> - if (!OldFD)
> - return false;
> +static bool CheckTargetCausesMultiVersioning(
> + Sema &S, FunctionDecl *OldFD, FunctionDecl *NewFD, const TargetAttr
> *NewTA,
> + bool &Redeclaration, NamedDecl *&OldDecl, bool &MergeTypeWithPrevious,
> + LookupResult &Previous) {
> + const auto *OldTA = OldFD->getAttr<TargetAttr>();
> + TargetAttr::ParsedTargetAttr NewParsed = NewTA->parse();
> + // Sort order doesn't matter, it just needs to be consistent.
> + llvm::sort(NewParsed.Features.begin(), NewParsed.Features.end());
>
> - if (!OldFD->isMultiVersion() && !NewTA)
> + // If the old decl is NOT MultiVersioned yet, and we don't cause that
> + // to change, this is a simple redeclaration.
> + if (!OldTA || OldTA->getFeaturesStr() == NewTA->getFeaturesStr())
> return false;
>
> - if (OldFD->isMultiVersion() && !NewTA) {
> - S.Diag(NewFD->getLocation(), diag::err_target_required_in_redecl);
> + // Otherwise, this decl causes MultiVersioning.
> + if (!S.getASTContext().getTargetInfo().supportsMultiVersioning()) {
> + S.Diag(NewFD->getLocation(), diag::err_multiversion_not_supported);
> + S.Diag(OldFD->getLocation(), diag::note_previous_declaration);
> NewFD->setInvalidDecl();
> return true;
> }
>
> - TargetAttr::ParsedTargetAttr NewParsed = NewTA->parse();
> - // Sort order doesn't matter, it just needs to be consistent.
> - llvm::sort(NewParsed.Features.begin(), NewParsed.Features.end());
> + if (CheckMultiVersionAdditionalRules(S, OldFD, NewFD, true,
> + MultiVersioning::Target)) {
> + NewFD->setInvalidDecl();
> + return true;
> + }
>
> - const auto *OldTA = OldFD->getAttr<TargetAttr>();
> - if (!OldFD->isMultiVersion()) {
> - // If the old decl is NOT MultiVersioned yet, and we don't cause that
> - // to change, this is a simple redeclaration.
> - if (!OldTA || OldTA->getFeaturesStr() == NewTA->getFeaturesStr())
> - return false;
> + if (CheckMultiVersionValue(S, NewFD)) {
> + NewFD->setInvalidDecl();
> + return true;
> + }
>
> - // Otherwise, this decl causes MultiVersioning.
> - if (!S.getASTContext().getTargetInfo().supportsMultiVersioning()) {
> - S.Diag(NewFD->getLocation(), diag::err_multiversion_not_supported);
> - S.Diag(OldFD->getLocation(), diag::note_previous_declaration);
> - NewFD->setInvalidDecl();
> - return true;
> - }
> + if (CheckMultiVersionValue(S, OldFD)) {
> + S.Diag(NewFD->getLocation(), diag::note_multiversioning_caused_here);
> + NewFD->setInvalidDecl();
> + return true;
> + }
>
> - if (!OldFD->getType()->getAs<FunctionProtoType>()) {
> - S.Diag(OldFD->getLocation(), diag::err_multiversion_noproto);
> - S.Diag(NewFD->getLocation(),
> diag::note_multiversioning_caused_here);
> - NewFD->setInvalidDecl();
> - return true;
> - }
> + TargetAttr::ParsedTargetAttr OldParsed =
> + OldTA->parse(std::less<std::string>());
>
> - if (CheckMultiVersionValue(S, NewFD)) {
> - NewFD->setInvalidDecl();
> - return true;
> - }
> + if (OldParsed == NewParsed) {
> + S.Diag(NewFD->getLocation(), diag::err_multiversion_
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20180907/9b3a7661/attachment-0001.html>
More information about the cfe-commits
mailing list