<div dir="ltr"><div dir="ltr"><div dir="ltr">(Noticed while reviewing <a href="https://reviews.llvm.org/D51812">https://reviews.llvm.org/D51812</a>)</div><div dir="ltr"><br></div><div>Should these attributes be disallowed on lambdas? Currently we allow them, but that seems questionable since the call operator of a lambda can't be overloaded. Eg:</div><div><br></div><div>auto x = []() __attribute__((cpu_dispatch(atom))) {};</div><div>void f() { x(); }<br></div><div><br></div><div>... generates an ifuc calling a resolver that can never be defined (and likewise for cpu_specific).</div></div></div><br><div class="gmail_quote"><div dir="ltr">On Fri, 20 Jul 2018 at 07:18, Erich Keane via cfe-commits <<a href="mailto:cfe-commits@lists.llvm.org">cfe-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: erichkeane<br>
Date: Fri Jul 20 07:13:28 2018<br>
New Revision: 337552<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=337552&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=337552&view=rev</a><br>
Log:<br>
Implement cpu_dispatch/cpu_specific Multiversioning<br>
<br>
As documented here: <a href="https://software.intel.com/en-us/node/682969" rel="noreferrer" target="_blank">https://software.intel.com/en-us/node/682969</a> and<br>
<a href="https://software.intel.com/en-us/node/523346" rel="noreferrer" target="_blank">https://software.intel.com/en-us/node/523346</a>. cpu_dispatch multiversioning<br>
is an ICC feature that provides for function multiversioning.<br>
<br>
This feature is implemented with two attributes: First, cpu_specific,<br>
which specifies the individual function versions. Second, cpu_dispatch,<br>
which specifies the location of the resolver function and the list of<br>
resolvable functions.<br>
<br>
This is valuable since it provides a mechanism where the resolver's TU<br>
can be specified in one location, and the individual implementions<br>
each in their own translation units.<br>
<br>
The goal of this patch is to be source-compatible with ICC, so this<br>
implementation diverges from the ICC implementation in a few ways:<br>
1- Linux x86/64 only: This implementation uses ifuncs in order to<br>
properly dispatch functions. This is is a valuable performance benefit<br>
over the ICC implementation. A future patch will be provided to enable<br>
this feature on Windows, but it will obviously more closely fit ICC's<br>
implementation.<br>
2- CPU Identification functions: ICC uses a set of custom functions to identify<br>
the feature list of the host processor. This patch uses the cpu_supports<br>
functionality in order to better align with 'target' multiversioning.<br>
1- cpu_dispatch function def/decl: ICC's cpu_dispatch requires that the function<br>
marked cpu_dispatch be an empty definition. This patch supports that as well,<br>
however declarations are also permitted, since the linker will solve the<br>
issue of multiple emissions.<br>
<br>
Differential Revision: <a href="https://reviews.llvm.org/D47474" rel="noreferrer" target="_blank">https://reviews.llvm.org/D47474</a><br>
<br>
Added:<br>
cfe/trunk/test/CodeGen/attr-cpuspecific.c<br>
cfe/trunk/test/Sema/attr-cpuspecific.c<br>
cfe/trunk/test/SemaCXX/attr-cpuspecific.cpp<br>
Modified:<br>
cfe/trunk/include/clang/AST/Decl.h<br>
cfe/trunk/include/clang/Basic/Attr.td<br>
cfe/trunk/include/clang/Basic/AttrDocs.td<br>
cfe/trunk/include/clang/Basic/DiagnosticGroups.td<br>
cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td<br>
cfe/trunk/include/clang/Basic/TargetInfo.h<br>
cfe/trunk/include/clang/Basic/X86Target.def<br>
cfe/trunk/lib/AST/Decl.cpp<br>
cfe/trunk/lib/Basic/Targets/X86.cpp<br>
cfe/trunk/lib/Basic/Targets/X86.h<br>
cfe/trunk/lib/CodeGen/CGBuiltin.cpp<br>
cfe/trunk/lib/CodeGen/CodeGenFunction.cpp<br>
cfe/trunk/lib/CodeGen/CodeGenFunction.h<br>
cfe/trunk/lib/CodeGen/CodeGenModule.cpp<br>
cfe/trunk/lib/CodeGen/CodeGenModule.h<br>
cfe/trunk/lib/Parse/ParseDecl.cpp<br>
cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp<br>
cfe/trunk/lib/Sema/Sema.cpp<br>
cfe/trunk/lib/Sema/SemaDecl.cpp<br>
cfe/trunk/lib/Sema/SemaDeclAttr.cpp<br>
cfe/trunk/lib/Sema/SemaExpr.cpp<br>
cfe/trunk/lib/Sema/SemaOverload.cpp<br>
cfe/trunk/test/Misc/pragma-attribute-supported-attributes-list.test<br>
cfe/trunk/utils/TableGen/ClangAttrEmitter.cpp<br>
<br>
Modified: cfe/trunk/include/clang/AST/Decl.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/AST/Decl.h?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/AST/Decl.h?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/include/clang/AST/Decl.h (original)<br>
+++ cfe/trunk/include/clang/AST/Decl.h Fri Jul 20 07:13:28 2018<br>
@@ -2209,6 +2209,13 @@ public:<br>
getCanonicalDecl()->IsMultiVersion = V;<br>
}<br>
<br>
+ /// True if this function is a multiversioned dispatch function as a part of<br>
+ /// the cpu_specific/cpu_dispatch functionality.<br>
+ bool isCPUDispatchMultiVersion() const;<br>
+ /// True if this function is a multiversioned processor specific function as a<br>
+ /// part of the cpu_specific/cpu_dispatch functionality.<br>
+ bool isCPUSpecificMultiVersion() const;<br>
+<br>
void setPreviousDeclaration(FunctionDecl * PrevDecl);<br>
<br>
FunctionDecl *getCanonicalDecl() override;<br>
<br>
Modified: cfe/trunk/include/clang/Basic/Attr.td<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/Attr.td?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/Attr.td?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/include/clang/Basic/Attr.td (original)<br>
+++ cfe/trunk/include/clang/Basic/Attr.td Fri Jul 20 07:13:28 2018<br>
@@ -168,6 +168,7 @@ class UnsignedArgument<string name, bit<br>
class VariadicUnsignedArgument<string name> : Argument<name, 1>;<br>
class VariadicExprArgument<string name> : Argument<name, 1>;<br>
class VariadicStringArgument<string name> : Argument<name, 1>;<br>
+class VariadicIdentifierArgument<string name> : Argument<name, 1>;<br>
<br>
// Like VariadicUnsignedArgument except values are ParamIdx.<br>
class VariadicParamIdxArgument<string name> : Argument<name, 1>;<br>
@@ -845,6 +846,27 @@ def Constructor : InheritableAttr {<br>
let Documentation = [Undocumented];<br>
}<br>
<br>
+def CPUSpecific : InheritableAttr {<br>
+ let Spellings = [Clang<"cpu_specific">];<br>
+ let Args = [VariadicIdentifierArgument<"Cpus">];<br>
+ let Subjects = SubjectList<[Function]>;<br>
+ let Documentation = [CPUSpecificCPUDispatchDocs];<br>
+ let AdditionalMembers = [{<br>
+ unsigned ActiveArgIndex = 0;<br>
+<br>
+ IdentifierInfo *getCurCPUName() const {<br>
+ return *(cpus_begin() + ActiveArgIndex);<br>
+ }<br>
+ }];<br>
+}<br>
+<br>
+def CPUDispatch : InheritableAttr {<br>
+ let Spellings = [Clang<"cpu_dispatch">];<br>
+ let Args = [VariadicIdentifierArgument<"Cpus">];<br>
+ let Subjects = SubjectList<[Function]>;<br>
+ let Documentation = [CPUSpecificCPUDispatchDocs];<br>
+}<br>
+<br>
// CUDA attributes are spelled __attribute__((attr)) or __declspec(__attr__),<br>
// and they do not receive a [[]] spelling.<br>
def CUDAConstant : InheritableAttr {<br>
<br>
Modified: cfe/trunk/include/clang/Basic/AttrDocs.td<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/AttrDocs.td?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/AttrDocs.td?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/include/clang/Basic/AttrDocs.td (original)<br>
+++ cfe/trunk/include/clang/Basic/AttrDocs.td Fri Jul 20 07:13:28 2018<br>
@@ -191,6 +191,65 @@ in generation of more efficient code.<br>
}];<br>
}<br>
<br>
+def CPUSpecificCPUDispatchDocs : Documentation {<br>
+ let Category = DocCatFunction;<br>
+ let Content = [{<br>
+The ``cpu_specific`` and ``cpu_dispatch`` attributes are used to define and<br>
+resolve multiversioned functions. This form of multiversioning provides a<br>
+mechanism for declaring versions across translation units and manually<br>
+specifying the resolved function list. A specified CPU defines a set of minimum<br>
+features that are required for the function to be called. The result of this is<br>
+that future processors execute the most restrictive version of the function the<br>
+new processor can execute.<br>
+<br>
+Function versions are defined with ``cpu_specific``, which takes one or more CPU<br>
+names as a parameter. For example:<br>
+<br>
+.. code-block:: c<br>
+<br>
+ // Declares and defines the ivybridge version of single_cpu.<br>
+ __attribute__((cpu_specific(ivybridge)))<br>
+ void single_cpu(void){}<br>
+<br>
+ // Declares and defines the atom version of single_cpu.<br>
+ __attribute__((cpu_specific(atom)))<br>
+ void single_cpu(void){}<br>
+<br>
+ // Declares and defines both the ivybridge and atom version of multi_cpu.<br>
+ __attribute__((cpu_specific(ivybridge, atom)))<br>
+ void multi_cpu(void){}<br>
+<br>
+A dispatching (or resolving) function can be declared anywhere in a project's<br>
+source code with ``cpu_dispatch``. This attribute takes one or more CPU names<br>
+as a parameter (like ``cpu_specific``). Functions marked with ``cpu_dispatch``<br>
+are not expected to be defined, only declared. If such a marked function has a<br>
+definition, any side effects of the function are ignored; trivial function<br>
+bodies are permissible for ICC compatibility.<br>
+<br>
+.. code-block:: c<br>
+<br>
+ // Creates a resolver for single_cpu above.<br>
+ __attribute__((cpu_dispatch(ivybridge, atom)))<br>
+ void single_cpu(void){}<br>
+<br>
+ // Creates a resolver for multi_cpu, but adds a 3rd version defined in another<br>
+ // translation unit.<br>
+ __attribute__((cpu_dispatch(ivybridge, atom, sandybridge)))<br>
+ void multi_cpu(void){}<br>
+<br>
+Note that it is possible to have a resolving function that dispatches based on<br>
+more or fewer options than are present in the program. Specifying fewer will<br>
+result in the omitted options not being considered during resolution. Specifying<br>
+a version for resolution that isn't defined in the program will result in a<br>
+linking failure.<br>
+<br>
+It is also possible to specify a CPU name of ``generic`` which will be resolved<br>
+if the executing processor doesn't satisfy the features required in the CPU<br>
+name. The behavior of a program executing on a processor that doesn't satisfy<br>
+any option of a multiversioned function is undefined.<br>
+ }];<br>
+}<br>
+<br>
def C11NoReturnDocs : Documentation {<br>
let Category = DocCatFunction;<br>
let Content = [{<br>
<br>
Modified: cfe/trunk/include/clang/Basic/DiagnosticGroups.td<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticGroups.td?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticGroups.td?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/include/clang/Basic/DiagnosticGroups.td (original)<br>
+++ cfe/trunk/include/clang/Basic/DiagnosticGroups.td Fri Jul 20 07:13:28 2018<br>
@@ -1022,3 +1022,7 @@ def SpirCompat : DiagGroup<"spir-compat"<br>
<br>
// Warning for the experimental-isel options.<br>
def ExperimentalISel : DiagGroup<"experimental-isel">;<br>
+<br>
+// A warning group specifically for warnings related to function<br>
+// multiversioning.<br>
+def FunctionMultiVersioning : DiagGroup<"function-multiversion">;<br>
<br>
Modified: cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td (original)<br>
+++ cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td Fri Jul 20 07:13:28 2018<br>
@@ -608,6 +608,8 @@ def err_builtin_redeclare : Error<"canno<br>
def err_arm_invalid_specialreg : Error<"invalid special register for builtin">;<br>
def err_invalid_cpu_supports : Error<"invalid cpu feature string for builtin">;<br>
def err_invalid_cpu_is : Error<"invalid cpu name for builtin">;<br>
+def err_invalid_cpu_specific_dispatch_value : Error<<br>
+"invalid option '%0' for %select{cpu_specific|cpu_dispatch}1">;<br>
def err_builtin_needs_feature : Error<"%0 needs target feature %1">;<br>
def err_function_needs_feature<br>
: Error<"always_inline function %1 requires target feature '%2', but would "<br>
@@ -3788,8 +3790,8 @@ def err_ovl_no_viable_subscript :<br>
def err_ovl_no_oper :<br>
Error<"type %0 does not provide a %select{subscript|call}1 operator">;<br>
def err_ovl_unresolvable : Error<<br>
- "reference to overloaded function could not be resolved; "<br>
- "did you mean to call it%select{| with no arguments}0?">;<br>
+ "reference to %select{overloaded|multiversioned}1 function could not be "<br>
+ "resolved; did you mean to call it%select{| with no arguments}0?">;<br>
def err_bound_member_function : Error<<br>
"reference to non-static member function must be called"<br>
"%select{|; did you mean to call it with no arguments?}0">;<br>
@@ -9355,9 +9357,9 @@ def warn_shadow_field :<br>
InGroup<ShadowField>, DefaultIgnore;<br>
def note_shadow_field : Note<"declared here">;<br>
<br>
-def err_target_required_in_redecl : Error<<br>
- "function declaration is missing 'target' attribute in a multiversioned "<br>
- "function">;<br>
+def err_multiversion_required_in_redecl : Error<<br>
+ "function declaration is missing %select{'target'|'cpu_specific' or "<br>
+ "'cpu_dispatch'}0 attribute in a multiversioned function">;<br>
def note_multiversioning_caused_here : Note<<br>
"function multiversioning caused by this declaration">;<br>
def err_multiversion_after_used : Error<<br>
@@ -9371,20 +9373,33 @@ def err_multiversion_duplicate : Error<<br>
def err_multiversion_noproto : Error<<br>
"multiversioned function must have a prototype">;<br>
def err_multiversion_no_other_attrs : Error<<br>
- "attribute 'target' multiversioning cannot be combined with other "<br>
- "attributes">;<br>
+ "attribute '%select{target|cpu_specific|cpu_dispatch}0' multiversioning cannot be combined"<br>
+ " with other attributes">;<br>
def err_multiversion_diff : Error<<br>
"multiversioned function declaration has a different %select{calling convention"<br>
"|return type|constexpr specification|inline specification|storage class|"<br>
"linkage}0">;<br>
def err_multiversion_doesnt_support : Error<<br>
- "multiversioned functions do not yet support %select{function templates|"<br>
- "virtual functions|deduced return types|constructors|destructors|"<br>
- "deleted functions|defaulted functions}0">;<br>
+ "attribute '%select{target|cpu_specific|cpu_dispatch}0' multiversioned functions do not "<br>
+ "yet support %select{function templates|virtual functions|"<br>
+ "deduced return types|constructors|destructors|deleted functions|"<br>
+ "defaulted functions|constexpr functions}1">;<br>
def err_multiversion_not_allowed_on_main : Error<<br>
"'main' cannot be a multiversioned function">;<br>
def err_multiversion_not_supported : Error<<br>
"function multiversioning is not supported on the current target">;<br>
+def err_multiversion_types_mixed : Error<<br>
+ "multiversioning attributes cannot be combined">;<br>
+def err_cpu_dispatch_mismatch : Error<<br>
+ "'cpu_dispatch' function redeclared with different CPUs">;<br>
+def err_cpu_specific_multiple_defs : Error<<br>
+ "multiple 'cpu_specific' functions cannot specify the same CPU: %0">;<br>
+def warn_multiversion_duplicate_entries : Warning<<br>
+ "CPU list contains duplicate entries; attribute ignored">,<br>
+ InGroup<FunctionMultiVersioning>;<br>
+def warn_dispatch_body_ignored : Warning<<br>
+ "body of cpu_dispatch function will be ignored">,<br>
+ InGroup<FunctionMultiVersioning>;<br>
<br>
// three-way comparison operator diagnostics<br>
def err_implied_comparison_category_type_not_found : Error<<br>
<br>
Modified: cfe/trunk/include/clang/Basic/TargetInfo.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/TargetInfo.h?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/TargetInfo.h?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/include/clang/Basic/TargetInfo.h (original)<br>
+++ cfe/trunk/include/clang/Basic/TargetInfo.h Fri Jul 20 07:13:28 2018<br>
@@ -1092,6 +1092,27 @@ public:<br>
// argument.<br>
virtual bool validateCpuIs(StringRef Name) const { return false; }<br>
<br>
+ // Validate a cpu_dispatch/cpu_specific CPU option, which is a different list<br>
+ // from cpu_is, since it checks via features rather than CPUs directly.<br>
+ virtual bool validateCPUSpecificCPUDispatch(StringRef Name) const {<br>
+ return false;<br>
+ }<br>
+<br>
+ // Get the character to be added for mangling purposes for cpu_specific.<br>
+ virtual char CPUSpecificManglingCharacter(StringRef Name) const {<br>
+ llvm_unreachable(<br>
+ "cpu_specific Multiversioning not implemented on this target");<br>
+ }<br>
+<br>
+ // Get a list of the features that make up the CPU option for<br>
+ // cpu_specific/cpu_dispatch so that it can be passed to llvm as optimization<br>
+ // options.<br>
+ virtual void getCPUSpecificCPUDispatchFeatures(<br>
+ StringRef Name, llvm::SmallVectorImpl<StringRef> &Features) const {<br>
+ llvm_unreachable(<br>
+ "cpu_specific Multiversioning not implemented on this target");<br>
+ }<br>
+<br>
// Returns maximal number of args passed in registers.<br>
unsigned getRegParmMax() const {<br>
assert(RegParmMax < 7 && "RegParmMax value is larger than AST can handle");<br>
<br>
Modified: cfe/trunk/include/clang/Basic/X86Target.def<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/X86Target.def?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/X86Target.def?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/include/clang/Basic/X86Target.def (original)<br>
+++ cfe/trunk/include/clang/Basic/X86Target.def Fri Jul 20 07:13:28 2018<br>
@@ -29,6 +29,14 @@<br>
#define FEATURE(ENUM)<br>
#endif<br>
<br>
+#ifndef CPU_SPECIFIC<br>
+#define CPU_SPECIFIC(NAME, MANGLING, FEATURES)<br>
+#endif<br>
+<br>
+#ifndef CPU_SPECIFIC_ALIAS<br>
+#define CPU_SPECIFIC_ALIAS(NEW_NAME, NAME)<br>
+#endif<br>
+<br>
#define PROC_64_BIT true<br>
#define PROC_32_BIT false<br>
<br>
@@ -276,6 +284,45 @@ FEATURE(FEATURE_AVX5124VNNIW)<br>
FEATURE(FEATURE_AVX5124FMAPS)<br>
FEATURE(FEATURE_AVX512VPOPCNTDQ)<br>
<br>
+<br>
+// FIXME: When commented out features are supported in LLVM, enable them here.<br>
+CPU_SPECIFIC("generic", 'A', "")<br>
+CPU_SPECIFIC("pentium", 'B', "")<br>
+CPU_SPECIFIC("pentium_pro", 'C', "+cmov")<br>
+CPU_SPECIFIC("pentium_mmx", 'D', "+mmx")<br>
+CPU_SPECIFIC("pentium_ii", 'E', "+cmov,+mmx")<br>
+CPU_SPECIFIC("pentium_iii", 'H', "+cmov,+mmx,+sse")<br>
+CPU_SPECIFIC("pentium_iii_no_xmm_regs", 'H',"+cmov,+sse")<br>
+CPU_SPECIFIC("pentium_4", 'J', "+cmov,+mmx,+sse,+sse2")<br>
+CPU_SPECIFIC("pentium_m", 'K', "+cmov,+mmx,+sse,+sse2")<br>
+CPU_SPECIFIC("pentium_4_sse3", 'L', "+cmov,+mmx,+sse,+sse2,+sse3")<br>
+CPU_SPECIFIC("core_2_duo_ssse3", 'M', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3")<br>
+CPU_SPECIFIC("core_2_duo_sse4_1", 'N', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1")<br>
+CPU_SPECIFIC("atom", 'O', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+movbe")<br>
+CPU_SPECIFIC("atom_sse4_2", 'c', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+popcnt")<br>
+CPU_SPECIFIC("core_i7_sse4_2", 'P', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+popcnt")<br>
+CPU_SPECIFIC("core_aes_pclmulqdq", 'Q', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+popcnt")<br>
+CPU_SPECIFIC("atom_sse4_2_movbe", 'd', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt")<br>
+CPU_SPECIFIC("goldmont", 'i', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt")<br>
+CPU_SPECIFIC("sandybridge", 'R', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+popcnt,+avx")<br>
+CPU_SPECIFIC_ALIAS("core_2nd_gen_avx", "sandybridge")<br>
+CPU_SPECIFIC("ivybridge", 'S', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+popcnt,+f16c,+avx")<br>
+CPU_SPECIFIC_ALIAS("core_3rd_gen_avx", "ivybridge")<br>
+CPU_SPECIFIC("haswell", 'V', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2")<br>
+CPU_SPECIFIC_ALIAS("core_4th_gen_avx", "haswell")<br>
+CPU_SPECIFIC("core_4th_gen_avx_tsx", 'W', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2")<br>
+CPU_SPECIFIC("broadwell", 'X', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2,+adx")<br>
+CPU_SPECIFIC_ALIAS("core_5th_gen_avx", "broadwell")<br>
+CPU_SPECIFIC("core_5th_gen_avx_tsx", 'Y', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2,+adx")<br>
+CPU_SPECIFIC("knl", 'Z', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2,+avx512f,+adx,+avx512er,+avx512pf,+avx512cd")<br>
+CPU_SPECIFIC_ALIAS("mic_avx512", "knl")<br>
+CPU_SPECIFIC("skylake", 'b', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2,+adx,+mpx")<br>
+CPU_SPECIFIC( "skylake_avx512", 'a', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2,+avx512dq,+avx512f,+adx,+avx512cd,+avx512bw,+avx512vl,+clwb")<br>
+CPU_SPECIFIC("cannonlake", 'e', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2,+avx512dq,+avx512f,+adx,+avx512ifma,+avx512cd,+avx512bw,+avx512vl,+avx512vbmi")<br>
+CPU_SPECIFIC("knm", 'j', "+cmov,+mmx,+sse,+sse2,+sse3,+ssse3,+sse4.1,+sse4.2,+movbe,+popcnt,+f16c,+avx,+fma,+bmi,+lzcnt,+avx2,+avx512f,+adx,+avx512er,+avx512pf,+avx512cd,+avx5124fmaps,+avx5124vnniw,+avx512vpopcntdq")<br>
+<br>
+#undef CPU_SPECIFIC_ALIAS<br>
+#undef CPU_SPECIFIC<br>
#undef PROC_64_BIT<br>
#undef PROC_32_BIT<br>
#undef FEATURE<br>
<br>
Modified: cfe/trunk/lib/AST/Decl.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/AST/Decl.cpp?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/AST/Decl.cpp?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/AST/Decl.cpp (original)<br>
+++ cfe/trunk/lib/AST/Decl.cpp Fri Jul 20 07:13:28 2018<br>
@@ -2873,6 +2873,14 @@ bool FunctionDecl::isNoReturn() const {<br>
return false;<br>
}<br>
<br>
+bool FunctionDecl::isCPUDispatchMultiVersion() const {<br>
+ return isMultiVersion() && hasAttr<CPUDispatchAttr>();<br>
+}<br>
+<br>
+bool FunctionDecl::isCPUSpecificMultiVersion() const {<br>
+ return isMultiVersion() && hasAttr<CPUSpecificAttr>();<br>
+}<br>
+<br>
void<br>
FunctionDecl::setPreviousDeclaration(FunctionDecl *PrevDecl) {<br>
redeclarable_base::setPreviousDecl(PrevDecl);<br>
<br>
Modified: cfe/trunk/lib/Basic/Targets/X86.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/Targets/X86.cpp?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/Targets/X86.cpp?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/Basic/Targets/X86.cpp (original)<br>
+++ cfe/trunk/lib/Basic/Targets/X86.cpp Fri Jul 20 07:13:28 2018<br>
@@ -1484,6 +1484,38 @@ unsigned X86TargetInfo::multiVersionSort<br>
return getFeaturePriority(getFeature(Name)) << 1;<br>
}<br>
<br>
+bool X86TargetInfo::validateCPUSpecificCPUDispatch(StringRef Name) const {<br>
+ return llvm::StringSwitch<bool>(Name)<br>
+#define CPU_SPECIFIC(NAME, MANGLING, FEATURES) .Case(NAME, true)<br>
+#define CPU_SPECIFIC_ALIAS(NEW_NAME, NAME) .Case(NEW_NAME, true)<br>
+#include "clang/Basic/X86Target.def"<br>
+ .Default(false);<br>
+}<br>
+<br>
+static StringRef CPUSpecificCPUDispatchNameDealias(StringRef Name) {<br>
+ return llvm::StringSwitch<StringRef>(Name)<br>
+#define CPU_SPECIFIC_ALIAS(NEW_NAME, NAME) .Case(NEW_NAME, NAME)<br>
+#include "clang/Basic/X86Target.def"<br>
+ .Default(Name);<br>
+}<br>
+<br>
+char X86TargetInfo::CPUSpecificManglingCharacter(StringRef Name) const {<br>
+ return llvm::StringSwitch<char>(CPUSpecificCPUDispatchNameDealias(Name))<br>
+#define CPU_SPECIFIC(NAME, MANGLING, FEATURES) .Case(NAME, MANGLING)<br>
+#include "clang/Basic/X86Target.def"<br>
+ .Default(0);<br>
+}<br>
+<br>
+void X86TargetInfo::getCPUSpecificCPUDispatchFeatures(<br>
+ StringRef Name, llvm::SmallVectorImpl<StringRef> &Features) const {<br>
+ StringRef WholeList =<br>
+ llvm::StringSwitch<StringRef>(CPUSpecificCPUDispatchNameDealias(Name))<br>
+#define CPU_SPECIFIC(NAME, MANGLING, FEATURES) .Case(NAME, FEATURES)<br>
+#include "clang/Basic/X86Target.def"<br>
+ .Default("");<br>
+ WholeList.split(Features, ',', /*MaxSplit=*/-1, /*KeepEmpty=*/false);<br>
+}<br>
+<br>
std::string X86TargetInfo::getCPUKindCanonicalName(CPUKind Kind) const {<br>
switch (Kind) {<br>
case CK_Generic:<br>
<br>
Modified: cfe/trunk/lib/Basic/Targets/X86.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/Targets/X86.h?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/Targets/X86.h?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/Basic/Targets/X86.h (original)<br>
+++ cfe/trunk/lib/Basic/Targets/X86.h Fri Jul 20 07:13:28 2018<br>
@@ -150,6 +150,14 @@ public:<br>
<br>
bool validateCpuIs(StringRef Name) const override;<br>
<br>
+ bool validateCPUSpecificCPUDispatch(StringRef Name) const override;<br>
+<br>
+ char CPUSpecificManglingCharacter(StringRef Name) const override;<br>
+<br>
+ void getCPUSpecificCPUDispatchFeatures(<br>
+ StringRef Name,<br>
+ llvm::SmallVectorImpl<StringRef> &Features) const override;<br>
+<br>
bool validateAsmConstraint(const char *&Name,<br>
TargetInfo::ConstraintInfo &info) const override;<br>
<br>
<br>
Modified: cfe/trunk/lib/CodeGen/CGBuiltin.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGBuiltin.cpp?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGBuiltin.cpp?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/CodeGen/CGBuiltin.cpp (original)<br>
+++ cfe/trunk/lib/CodeGen/CGBuiltin.cpp Fri Jul 20 07:13:28 2018<br>
@@ -8904,11 +8904,10 @@ Value *CodeGenFunction::EmitX86CpuSuppor<br>
return EmitX86CpuSupports(FeatureStr);<br>
}<br>
<br>
-Value *CodeGenFunction::EmitX86CpuSupports(ArrayRef<StringRef> FeatureStrs) {<br>
+uint32_t<br>
+CodeGenFunction::GetX86CpuSupportsMask(ArrayRef<StringRef> FeatureStrs) {<br>
// Processor features and mapping to processor feature value.<br>
-<br>
uint32_t FeaturesMask = 0;<br>
-<br>
for (const StringRef &FeatureStr : FeatureStrs) {<br>
unsigned Feature =<br>
StringSwitch<unsigned>(FeatureStr)<br>
@@ -8917,7 +8916,14 @@ Value *CodeGenFunction::EmitX86CpuSuppor<br>
;<br>
FeaturesMask |= (1U << Feature);<br>
}<br>
+ return FeaturesMask;<br>
+}<br>
+<br>
+Value *CodeGenFunction::EmitX86CpuSupports(ArrayRef<StringRef> FeatureStrs) {<br>
+ return EmitX86CpuSupports(GetX86CpuSupportsMask(FeatureStrs));<br>
+}<br>
<br>
+llvm::Value *CodeGenFunction::EmitX86CpuSupports(uint32_t FeaturesMask) {<br>
// Matching the struct layout from the compiler-rt/libgcc structure that is<br>
// filled in:<br>
// unsigned int __cpu_vendor;<br>
<br>
Modified: cfe/trunk/lib/CodeGen/CodeGenFunction.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenFunction.cpp?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenFunction.cpp?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/CodeGen/CodeGenFunction.cpp (original)<br>
+++ cfe/trunk/lib/CodeGen/CodeGenFunction.cpp Fri Jul 20 07:13:28 2018<br>
@@ -2323,7 +2323,8 @@ void CodeGenFunction::checkTargetFeature<br>
<< TargetDecl->getDeclName()<br>
<< CGM.getContext().BuiltinInfo.getRequiredFeatures(BuiltinID);<br>
<br>
- } else if (TargetDecl->hasAttr<TargetAttr>()) {<br>
+ } else if (TargetDecl->hasAttr<TargetAttr>() ||<br>
+ TargetDecl->hasAttr<CPUSpecificAttr>()) {<br>
// Get the required features for the callee.<br>
<br>
const TargetAttr *TD = TargetDecl->getAttr<TargetAttr>();<br>
@@ -2358,8 +2359,8 @@ void CodeGenFunction::EmitSanitizerStatR<br>
CGM.getSanStats().create(IRB, SSK);<br>
}<br>
<br>
-llvm::Value *<br>
-CodeGenFunction::FormResolverCondition(const MultiVersionResolverOption &RO) {<br>
+llvm::Value *CodeGenFunction::FormResolverCondition(<br>
+ const TargetMultiVersionResolverOption &RO) {<br>
llvm::Value *TrueCondition = nullptr;<br>
if (!RO.ParsedAttribute.Architecture.empty())<br>
TrueCondition = EmitX86CpuIs(RO.ParsedAttribute.Architecture);<br>
@@ -2377,8 +2378,9 @@ CodeGenFunction::FormResolverCondition(c<br>
return TrueCondition;<br>
}<br>
<br>
-void CodeGenFunction::EmitMultiVersionResolver(<br>
- llvm::Function *Resolver, ArrayRef<MultiVersionResolverOption> Options) {<br>
+void CodeGenFunction::EmitTargetMultiVersionResolver(<br>
+ llvm::Function *Resolver,<br>
+ ArrayRef<TargetMultiVersionResolverOption> Options) {<br>
assert((getContext().getTargetInfo().getTriple().getArch() ==<br>
llvm::Triple::x86 ||<br>
getContext().getTargetInfo().getTriple().getArch() ==<br>
@@ -2391,7 +2393,7 @@ void CodeGenFunction::EmitMultiVersionRe<br>
EmitX86CpuInit();<br>
<br>
llvm::Function *DefaultFunc = nullptr;<br>
- for (const MultiVersionResolverOption &RO : Options) {<br>
+ for (const TargetMultiVersionResolverOption &RO : Options) {<br>
Builder.SetInsertPoint(CurBlock);<br>
llvm::Value *TrueCondition = FormResolverCondition(RO);<br>
<br>
@@ -2412,6 +2414,44 @@ void CodeGenFunction::EmitMultiVersionRe<br>
Builder.CreateRet(DefaultFunc);<br>
}<br>
<br>
+void CodeGenFunction::EmitCPUDispatchMultiVersionResolver(<br>
+ llvm::Function *Resolver,<br>
+ ArrayRef<CPUDispatchMultiVersionResolverOption> Options) {<br>
+ assert((getContext().getTargetInfo().getTriple().getArch() ==<br>
+ llvm::Triple::x86 ||<br>
+ getContext().getTargetInfo().getTriple().getArch() ==<br>
+ llvm::Triple::x86_64) &&<br>
+ "Only implemented for x86 targets");<br>
+<br>
+ // Main function's basic block.<br>
+ llvm::BasicBlock *CurBlock = createBasicBlock("resolver_entry", Resolver);<br>
+ Builder.SetInsertPoint(CurBlock);<br>
+ EmitX86CpuInit();<br>
+<br>
+ for (const CPUDispatchMultiVersionResolverOption &RO : Options) {<br>
+ Builder.SetInsertPoint(CurBlock);<br>
+<br>
+ // "generic" case should catch-all.<br>
+ if (RO.FeatureMask == 0) {<br>
+ Builder.CreateRet(RO.Function);<br>
+ return;<br>
+ }<br>
+ llvm::BasicBlock *RetBlock = createBasicBlock("resolver_return", Resolver);<br>
+ llvm::IRBuilder<> RetBuilder(RetBlock);<br>
+ RetBuilder.CreateRet(RO.Function);<br>
+ CurBlock = createBasicBlock("resolver_else", Resolver);<br>
+ llvm::Value *TrueCondition = EmitX86CpuSupports(RO.FeatureMask);<br>
+ Builder.CreateCondBr(TrueCondition, RetBlock, CurBlock);<br>
+ }<br>
+<br>
+ Builder.SetInsertPoint(CurBlock);<br>
+ llvm::CallInst *TrapCall = EmitTrapCall(llvm::Intrinsic::trap);<br>
+ TrapCall->setDoesNotReturn();<br>
+ TrapCall->setDoesNotThrow();<br>
+ Builder.CreateUnreachable();<br>
+ Builder.ClearInsertionPoint();<br>
+}<br>
+<br>
llvm::DebugLoc CodeGenFunction::SourceLocToDebugLoc(SourceLocation Location) {<br>
if (CGDebugInfo *DI = getDebugInfo())<br>
return DI->SourceLocToDebugLoc(Location);<br>
<br>
Modified: cfe/trunk/lib/CodeGen/CodeGenFunction.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenFunction.h?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenFunction.h?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/CodeGen/CodeGenFunction.h (original)<br>
+++ cfe/trunk/lib/CodeGen/CodeGenFunction.h Fri Jul 20 07:13:28 2018<br>
@@ -4113,12 +4113,13 @@ public:<br>
<br>
void EmitSanitizerStatReport(llvm::SanitizerStatKind SSK);<br>
<br>
- struct MultiVersionResolverOption {<br>
+ struct TargetMultiVersionResolverOption {<br>
llvm::Function *Function;<br>
TargetAttr::ParsedTargetAttr ParsedAttribute;<br>
unsigned Priority;<br>
- MultiVersionResolverOption(const TargetInfo &TargInfo, llvm::Function *F,<br>
- const clang::TargetAttr::ParsedTargetAttr &PT)<br>
+ TargetMultiVersionResolverOption(<br>
+ const TargetInfo &TargInfo, llvm::Function *F,<br>
+ const clang::TargetAttr::ParsedTargetAttr &PT)<br>
: Function(F), ParsedAttribute(PT), Priority(0u) {<br>
for (StringRef Feat : PT.Features)<br>
Priority = std::max(Priority,<br>
@@ -4129,12 +4130,30 @@ public:<br>
TargInfo.multiVersionSortPriority(PT.Architecture));<br>
}<br>
<br>
- bool operator>(const MultiVersionResolverOption &Other) const {<br>
+ bool operator>(const TargetMultiVersionResolverOption &Other) const {<br>
return Priority > Other.Priority;<br>
}<br>
};<br>
- void EmitMultiVersionResolver(llvm::Function *Resolver,<br>
- ArrayRef<MultiVersionResolverOption> Options);<br>
+ void EmitTargetMultiVersionResolver(<br>
+ llvm::Function *Resolver,<br>
+ ArrayRef<TargetMultiVersionResolverOption> Options);<br>
+<br>
+ struct CPUDispatchMultiVersionResolverOption {<br>
+ llvm::Function *Function;<br>
+ // Note: EmitX86CPUSupports only has 32 bits available, so we store the mask<br>
+ // as 32 bits here. When 64-bit support is added to __builtin_cpu_supports,<br>
+ // this can be extended to 64 bits.<br>
+ uint32_t FeatureMask;<br>
+ CPUDispatchMultiVersionResolverOption(llvm::Function *F, uint64_t Mask)<br>
+ : Function(F), FeatureMask(static_cast<uint32_t>(Mask)) {}<br>
+ bool operator>(const CPUDispatchMultiVersionResolverOption &Other) const {<br>
+ return FeatureMask > Other.FeatureMask;<br>
+ }<br>
+ };<br>
+ void EmitCPUDispatchMultiVersionResolver(<br>
+ llvm::Function *Resolver,<br>
+ ArrayRef<CPUDispatchMultiVersionResolverOption> Options);<br>
+ static uint32_t GetX86CpuSupportsMask(ArrayRef<StringRef> FeatureStrs);<br>
<br>
private:<br>
QualType getVarArgType(const Expr *Arg);<br>
@@ -4151,8 +4170,10 @@ private:<br>
llvm::Value *EmitX86CpuIs(StringRef CPUStr);<br>
llvm::Value *EmitX86CpuSupports(const CallExpr *E);<br>
llvm::Value *EmitX86CpuSupports(ArrayRef<StringRef> FeatureStrs);<br>
+ llvm::Value *EmitX86CpuSupports(uint32_t Mask);<br>
llvm::Value *EmitX86CpuInit();<br>
- llvm::Value *FormResolverCondition(const MultiVersionResolverOption &RO);<br>
+ llvm::Value *<br>
+ FormResolverCondition(const TargetMultiVersionResolverOption &RO);<br>
};<br>
<br>
/// Helper class with most of the code for saving a value for a<br>
<br>
Modified: cfe/trunk/lib/CodeGen/CodeGenModule.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenModule.cpp?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenModule.cpp?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/CodeGen/CodeGenModule.cpp (original)<br>
+++ cfe/trunk/lib/CodeGen/CodeGenModule.cpp Fri Jul 20 07:13:28 2018<br>
@@ -861,22 +861,38 @@ void CodeGenModule::setTLSMode(llvm::Glo<br>
GV->setThreadLocalMode(TLM);<br>
}<br>
<br>
+static std::string getCPUSpecificMangling(const CodeGenModule &CGM,<br>
+ StringRef Name) {<br>
+ const TargetInfo &Target = CGM.getTarget();<br>
+ return (Twine('.') + Twine(Target.CPUSpecificManglingCharacter(Name))).str();<br>
+}<br>
+<br>
+static void AppendCPUSpecificCPUDispatchMangling(const CodeGenModule &CGM,<br>
+ const CPUSpecificAttr *Attr,<br>
+ raw_ostream &Out) {<br>
+ // cpu_specific gets the current name, dispatch gets the resolver.<br>
+ if (Attr)<br>
+ Out << getCPUSpecificMangling(CGM, Attr->getCurCPUName()->getName());<br>
+ else<br>
+ Out << ".resolver";<br>
+}<br>
+<br>
static void AppendTargetMangling(const CodeGenModule &CGM,<br>
const TargetAttr *Attr, raw_ostream &Out) {<br>
if (Attr->isDefaultVersion())<br>
return;<br>
<br>
Out << '.';<br>
- const auto &Target = CGM.getTarget();<br>
+ const TargetInfo &Target = CGM.getTarget();<br>
TargetAttr::ParsedTargetAttr Info =<br>
Attr->parse([&Target](StringRef LHS, StringRef RHS) {<br>
- // Multiversioning doesn't allow "no-${feature}", so we can<br>
- // only have "+" prefixes here.<br>
- assert(LHS.startswith("+") && RHS.startswith("+") &&<br>
- "Features should always have a prefix.");<br>
- return Target.multiVersionSortPriority(LHS.substr(1)) ><br>
- Target.multiVersionSortPriority(RHS.substr(1));<br>
- });<br>
+ // Multiversioning doesn't allow "no-${feature}", so we can<br>
+ // only have "+" prefixes here.<br>
+ assert(LHS.startswith("+") && RHS.startswith("+") &&<br>
+ "Features should always have a prefix.");<br>
+ return Target.multiVersionSortPriority(LHS.substr(1)) ><br>
+ Target.multiVersionSortPriority(RHS.substr(1));<br>
+ });<br>
<br>
bool IsFirst = true;<br>
<br>
@@ -895,7 +911,7 @@ static void AppendTargetMangling(const C<br>
<br>
static std::string getMangledNameImpl(const CodeGenModule &CGM, GlobalDecl GD,<br>
const NamedDecl *ND,<br>
- bool OmitTargetMangling = false) {<br>
+ bool OmitMultiVersionMangling = false) {<br>
SmallString<256> Buffer;<br>
llvm::raw_svector_ostream Out(Buffer);<br>
MangleContext &MC = CGM.getCXXABI().getMangleContext();<br>
@@ -922,8 +938,14 @@ static std::string getMangledNameImpl(co<br>
}<br>
<br>
if (const auto *FD = dyn_cast<FunctionDecl>(ND))<br>
- if (FD->isMultiVersion() && !OmitTargetMangling)<br>
- AppendTargetMangling(CGM, FD->getAttr<TargetAttr>(), Out);<br>
+ if (FD->isMultiVersion() && !OmitMultiVersionMangling) {<br>
+ if (FD->isCPUDispatchMultiVersion() || FD->isCPUSpecificMultiVersion())<br>
+ AppendCPUSpecificCPUDispatchMangling(<br>
+ CGM, FD->getAttr<CPUSpecificAttr>(), Out);<br>
+ else<br>
+ AppendTargetMangling(CGM, FD->getAttr<TargetAttr>(), Out);<br>
+ }<br>
+<br>
return Out.str();<br>
}<br>
<br>
@@ -936,7 +958,7 @@ void CodeGenModule::UpdateMultiVersionNa<br>
// allows us to lookup the version that was emitted when this wasn't a<br>
// multiversion function.<br>
std::string NonTargetName =<br>
- getMangledNameImpl(*this, GD, FD, /*OmitTargetMangling=*/true);<br>
+ getMangledNameImpl(*this, GD, FD, /*OmitMultiVersionMangling=*/true);<br>
GlobalDecl OtherGD;<br>
if (lookupRepresentativeDecl(NonTargetName, OtherGD)) {<br>
assert(OtherGD.getCanonicalDecl()<br>
@@ -979,11 +1001,30 @@ StringRef CodeGenModule::getMangledName(<br>
}<br>
}<br>
<br>
+ const auto *FD = dyn_cast<FunctionDecl>(GD.getDecl());<br>
+ // Since CPUSpecific can require multiple emits per decl, store the manglings<br>
+ // separately.<br>
+ if (FD &&<br>
+ (FD->isCPUDispatchMultiVersion() || FD->isCPUSpecificMultiVersion())) {<br>
+ const auto *SD = FD->getAttr<CPUSpecificAttr>();<br>
+<br>
+ std::pair<GlobalDecl, unsigned> SpecCanonicalGD{<br>
+ CanonicalGD,<br>
+ SD ? SD->ActiveArgIndex : std::numeric_limits<unsigned>::max()};<br>
+<br>
+ auto FoundName = CPUSpecificMangledDeclNames.find(SpecCanonicalGD);<br>
+ if (FoundName != CPUSpecificMangledDeclNames.end())<br>
+ return FoundName->second;<br>
+<br>
+ auto Result = CPUSpecificManglings.insert(<br>
+ std::make_pair(getMangledNameImpl(*this, GD, FD), SpecCanonicalGD));<br>
+ return CPUSpecificMangledDeclNames[SpecCanonicalGD] = Result.first->first();<br>
+ }<br>
+<br>
auto FoundName = MangledDeclNames.find(CanonicalGD);<br>
if (FoundName != MangledDeclNames.end())<br>
return FoundName->second;<br>
<br>
-<br>
// Keep the first result in the case of a mangling collision.<br>
const auto *ND = cast<NamedDecl>(GD.getDecl());<br>
auto Result =<br>
@@ -1321,8 +1362,9 @@ bool CodeGenModule::GetCPUAndFeaturesAtt<br>
const auto *FD = dyn_cast_or_null<FunctionDecl>(D);<br>
FD = FD ? FD->getMostRecentDecl() : FD;<br>
const auto *TD = FD ? FD->getAttr<TargetAttr>() : nullptr;<br>
+ const auto *SD = FD ? FD->getAttr<CPUSpecificAttr>() : nullptr;<br>
bool AddedAttr = false;<br>
- if (TD) {<br>
+ if (TD || SD) {<br>
llvm::StringMap<bool> FeatureMap;<br>
getFunctionFeatureMap(FeatureMap, FD);<br>
<br>
@@ -1334,10 +1376,12 @@ bool CodeGenModule::GetCPUAndFeaturesAtt<br>
// While we populated the feature map above, we still need to<br>
// get and parse the target attribute so we can get the cpu for<br>
// the function.<br>
- TargetAttr::ParsedTargetAttr ParsedAttr = TD->parse();<br>
- if (ParsedAttr.Architecture != "" &&<br>
- getTarget().isValidCPUName(ParsedAttr.Architecture))<br>
- TargetCPU = ParsedAttr.Architecture;<br>
+ if (TD) {<br>
+ TargetAttr::ParsedTargetAttr ParsedAttr = TD->parse();<br>
+ if (ParsedAttr.Architecture != "" &&<br>
+ getTarget().isValidCPUName(ParsedAttr.Architecture))<br>
+ TargetCPU = ParsedAttr.Architecture;<br>
+ }<br>
} else {<br>
// Otherwise just add the existing target cpu and target features to the<br>
// function.<br>
@@ -2037,6 +2081,10 @@ void CodeGenModule::EmitGlobal(GlobalDec<br>
if (Global->hasAttr<IFuncAttr>())<br>
return emitIFuncDefinition(GD);<br>
<br>
+ // If this is a cpu_dispatch multiversion function, emit the resolver.<br>
+ if (Global->hasAttr<CPUDispatchAttr>())<br>
+ return emitCPUDispatchDefinition(GD);<br>
+<br>
// If this is CUDA, be selective about which declarations we emit.<br>
if (LangOpts.CUDA) {<br>
if (LangOpts.CUDAIsDevice) {<br>
@@ -2355,7 +2403,7 @@ static void ReplaceUsesOfNonProtoTypeWit<br>
<br>
void CodeGenModule::emitMultiVersionFunctions() {<br>
for (GlobalDecl GD : MultiVersionFuncs) {<br>
- SmallVector<CodeGenFunction::MultiVersionResolverOption, 10> Options;<br>
+ SmallVector<CodeGenFunction::TargetMultiVersionResolverOption, 10> Options;<br>
const FunctionDecl *FD = cast<FunctionDecl>(GD.getDecl());<br>
getContext().forEachMultiversionedFunctionVersion(<br>
FD, [this, &GD, &Options](const FunctionDecl *CurFD) {<br>
@@ -2387,28 +2435,75 @@ void CodeGenModule::emitMultiVersionFunc<br>
getModule().getOrInsertComdat(ResolverFunc->getName()));<br>
std::stable_sort(<br>
Options.begin(), Options.end(),<br>
- std::greater<CodeGenFunction::MultiVersionResolverOption>());<br>
+ std::greater<CodeGenFunction::TargetMultiVersionResolverOption>());<br>
CodeGenFunction CGF(*this);<br>
- CGF.EmitMultiVersionResolver(ResolverFunc, Options);<br>
+ CGF.EmitTargetMultiVersionResolver(ResolverFunc, Options);<br>
}<br>
}<br>
<br>
+void CodeGenModule::emitCPUDispatchDefinition(GlobalDecl GD) {<br>
+ const auto *FD = cast<FunctionDecl>(GD.getDecl());<br>
+ assert(FD && "Not a FunctionDecl?");<br>
+ const auto *DD = FD->getAttr<CPUDispatchAttr>();<br>
+ assert(DD && "Not a cpu_dispatch Function?");<br>
+ llvm::Type *DeclTy = getTypes().ConvertTypeForMem(FD->getType());<br>
+<br>
+ StringRef ResolverName = getMangledName(GD);<br>
+ llvm::Type *ResolverType = llvm::FunctionType::get(<br>
+ llvm::PointerType::get(DeclTy,<br>
+ Context.getTargetAddressSpace(FD->getType())),<br>
+ false);<br>
+ auto *ResolverFunc = cast<llvm::Function>(<br>
+ GetOrCreateLLVMFunction(ResolverName, ResolverType, GlobalDecl{},<br>
+ /*ForVTable=*/false));<br>
+<br>
+ SmallVector<CodeGenFunction::CPUDispatchMultiVersionResolverOption, 10><br>
+ Options;<br>
+ const TargetInfo &Target = getTarget();<br>
+ for (const IdentifierInfo *II : DD->cpus()) {<br>
+ // Get the name of the target function so we can look it up/create it.<br>
+ std::string MangledName = getMangledNameImpl(*this, GD, FD, true) +<br>
+ getCPUSpecificMangling(*this, II->getName());<br>
+ llvm::Constant *Func = GetOrCreateLLVMFunction(<br>
+ MangledName, DeclTy, GD, /*ForVTable=*/false, /*DontDefer=*/false,<br>
+ /*IsThunk=*/false, llvm::AttributeList(), ForDefinition);<br>
+ llvm::SmallVector<StringRef, 32> Features;<br>
+ Target.getCPUSpecificCPUDispatchFeatures(II->getName(), Features);<br>
+ llvm::transform(Features, Features.begin(),<br>
+ [](StringRef Str) { return Str.substr(1); });<br>
+ Features.erase(std::remove_if(<br>
+ Features.begin(), Features.end(), [&Target](StringRef Feat) {<br>
+ return !Target.validateCpuSupports(Feat);<br>
+ }), Features.end());<br>
+ Options.emplace_back(cast<llvm::Function>(Func),<br>
+ CodeGenFunction::GetX86CpuSupportsMask(Features));<br>
+ }<br>
+<br>
+ llvm::sort(<br>
+ Options.begin(), Options.end(),<br>
+ std::greater<CodeGenFunction::CPUDispatchMultiVersionResolverOption>());<br>
+ CodeGenFunction CGF(*this);<br>
+ CGF.EmitCPUDispatchMultiVersionResolver(ResolverFunc, Options);<br>
+}<br>
+<br>
/// If an ifunc for the specified mangled name is not in the module, create and<br>
/// return an llvm IFunc Function with the specified type.<br>
llvm::Constant *<br>
CodeGenModule::GetOrCreateMultiVersionIFunc(GlobalDecl GD, llvm::Type *DeclTy,<br>
- StringRef MangledName,<br>
const FunctionDecl *FD) {<br>
- std::string IFuncName = (MangledName + ".ifunc").str();<br>
+ std::string MangledName =<br>
+ getMangledNameImpl(*this, GD, FD, /*OmitMultiVersionMangling=*/true);<br>
+ std::string IFuncName = MangledName + ".ifunc";<br>
if (llvm::GlobalValue *IFuncGV = GetGlobalValue(IFuncName))<br>
return IFuncGV;<br>
<br>
// Since this is the first time we've created this IFunc, make sure<br>
// that we put this multiversioned function into the list to be<br>
- // replaced later.<br>
- MultiVersionFuncs.push_back(GD);<br>
+ // replaced later if necessary (target multiversioning only).<br>
+ if (!FD->isCPUDispatchMultiVersion() && !FD->isCPUSpecificMultiVersion())<br>
+ MultiVersionFuncs.push_back(GD);<br>
<br>
- std::string ResolverName = (MangledName + ".resolver").str();<br>
+ std::string ResolverName = MangledName + ".resolver";<br>
llvm::Type *ResolverType = llvm::FunctionType::get(<br>
llvm::PointerType::get(DeclTy,<br>
Context.getTargetAddressSpace(FD->getType())),<br>
@@ -2455,10 +2550,12 @@ llvm::Constant *CodeGenModule::GetOrCrea<br>
addDeferredDeclToEmit(GDDef);<br>
}<br>
<br>
- if (FD->isMultiVersion() && FD->getAttr<TargetAttr>()->isDefaultVersion()) {<br>
- UpdateMultiVersionNames(GD, FD);<br>
+ if (FD->isMultiVersion()) {<br>
+ const auto *TA = FD->getAttr<TargetAttr>();<br>
+ if (TA && TA->isDefaultVersion())<br>
+ UpdateMultiVersionNames(GD, FD);<br>
if (!IsForDefinition)<br>
- return GetOrCreateMultiVersionIFunc(GD, Ty, MangledName, FD);<br>
+ return GetOrCreateMultiVersionIFunc(GD, Ty, FD);<br>
}<br>
}<br>
<br>
@@ -3727,6 +3824,15 @@ void CodeGenModule::EmitGlobalFunctionDe<br>
AddGlobalDtor(Fn, DA->getPriority());<br>
if (D->hasAttr<AnnotateAttr>())<br>
AddGlobalAnnotations(D, Fn);<br>
+<br>
+ if (D->isCPUSpecificMultiVersion()) {<br>
+ auto *Spec = D->getAttr<CPUSpecificAttr>();<br>
+ // If there is another specific version we need to emit, do so here.<br>
+ if (Spec->ActiveArgIndex + 1 < Spec->cpus_size()) {<br>
+ ++Spec->ActiveArgIndex;<br>
+ EmitGlobalFunctionDefinition(GD, nullptr);<br>
+ }<br>
+ }<br>
}<br>
<br>
void CodeGenModule::EmitAliasDefinition(GlobalDecl GD) {<br>
@@ -5107,6 +5213,12 @@ void CodeGenModule::getFunctionFeatureMa<br>
// the attribute.<br>
Target.initFeatureMap(FeatureMap, getDiags(), TargetCPU,<br>
ParsedAttr.Features);<br>
+ } else if (const auto *SD = FD->getAttr<CPUSpecificAttr>()) {<br>
+ llvm::SmallVector<StringRef, 32> FeaturesTmp;<br>
+ Target.getCPUSpecificCPUDispatchFeatures(SD->getCurCPUName()->getName(),<br>
+ FeaturesTmp);<br>
+ std::vector<std::string> Features(FeaturesTmp.begin(), FeaturesTmp.end());<br>
+ Target.initFeatureMap(FeatureMap, getDiags(), TargetCPU, Features);<br>
} else {<br>
Target.initFeatureMap(FeatureMap, getDiags(), TargetCPU,<br>
Target.getTargetOpts().Features);<br>
<br>
Modified: cfe/trunk/lib/CodeGen/CodeGenModule.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenModule.h?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenModule.h?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/CodeGen/CodeGenModule.h (original)<br>
+++ cfe/trunk/lib/CodeGen/CodeGenModule.h Fri Jul 20 07:13:28 2018<br>
@@ -366,6 +366,13 @@ private:<br>
llvm::MapVector<GlobalDecl, StringRef> MangledDeclNames;<br>
llvm::StringMap<GlobalDecl, llvm::BumpPtrAllocator> Manglings;<br>
<br>
+ // An ordered map of canonical GlobalDecls paired with the cpu-index for<br>
+ // cpu-specific name manglings.<br>
+ llvm::MapVector<std::pair<GlobalDecl, unsigned>, StringRef><br>
+ CPUSpecificMangledDeclNames;<br>
+ llvm::StringMap<std::pair<GlobalDecl, unsigned>, llvm::BumpPtrAllocator><br>
+ CPUSpecificManglings;<br>
+<br>
/// Global annotations.<br>
std::vector<llvm::Constant*> Annotations;<br>
<br>
@@ -1283,7 +1290,6 @@ private:<br>
<br>
llvm::Constant *GetOrCreateMultiVersionIFunc(GlobalDecl GD,<br>
llvm::Type *DeclTy,<br>
- StringRef MangledName,<br>
const FunctionDecl *FD);<br>
void UpdateMultiVersionNames(GlobalDecl GD, const FunctionDecl *FD);<br>
<br>
@@ -1307,6 +1313,7 @@ private:<br>
void EmitGlobalVarDefinition(const VarDecl *D, bool IsTentative = false);<br>
void EmitAliasDefinition(GlobalDecl GD);<br>
void emitIFuncDefinition(GlobalDecl GD);<br>
+ void emitCPUDispatchDefinition(GlobalDecl GD);<br>
void EmitObjCPropertyImplementations(const ObjCImplementationDecl *D);<br>
void EmitObjCIvarInitializations(ObjCImplementationDecl *D);<br>
<br>
<br>
Modified: cfe/trunk/lib/Parse/ParseDecl.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/ParseDecl.cpp?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/ParseDecl.cpp?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/Parse/ParseDecl.cpp (original)<br>
+++ cfe/trunk/lib/Parse/ParseDecl.cpp Fri Jul 20 07:13:28 2018<br>
@@ -215,6 +215,15 @@ static bool attributeHasIdentifierArg(co<br>
#undef CLANG_ATTR_IDENTIFIER_ARG_LIST<br>
}<br>
<br>
+/// Determine whether the given attribute has a variadic identifier argument.<br>
+static bool attributeHasVariadicIdentifierArg(const IdentifierInfo &II) {<br>
+#define CLANG_ATTR_VARIADIC_IDENTIFIER_ARG_LIST<br>
+ return llvm::StringSwitch<bool>(normalizeAttrName(II.getName()))<br>
+#include "clang/Parse/AttrParserStringSwitches.inc"<br>
+ .Default(false);<br>
+#undef CLANG_ATTR_VARIADIC_IDENTIFIER_ARG_LIST<br>
+}<br>
+<br>
/// Determine whether the given attribute parses a type argument.<br>
static bool attributeIsTypeArgAttr(const IdentifierInfo &II) {<br>
#define CLANG_ATTR_TYPE_ARG_LIST<br>
@@ -282,7 +291,8 @@ unsigned Parser::ParseAttributeArgsCommo<br>
ArgsVector ArgExprs;<br>
if (Tok.is(tok::identifier)) {<br>
// If this attribute wants an 'identifier' argument, make it so.<br>
- bool IsIdentifierArg = attributeHasIdentifierArg(*AttrName);<br>
+ bool IsIdentifierArg = attributeHasIdentifierArg(*AttrName) ||<br>
+ attributeHasVariadicIdentifierArg(*AttrName);<br>
ParsedAttr::Kind AttrKind =<br>
ParsedAttr::getKind(AttrName, ScopeName, Syntax);<br>
<br>
@@ -305,19 +315,25 @@ unsigned Parser::ParseAttributeArgsCommo<br>
<br>
// Parse the non-empty comma-separated list of expressions.<br>
do {<br>
- bool Uneval = attributeParsedArgsUnevaluated(*AttrName);<br>
- EnterExpressionEvaluationContext Unevaluated(<br>
- Actions,<br>
- Uneval ? Sema::ExpressionEvaluationContext::Unevaluated<br>
- : Sema::ExpressionEvaluationContext::ConstantEvaluated);<br>
-<br>
- ExprResult ArgExpr(<br>
- Actions.CorrectDelayedTyposInExpr(ParseAssignmentExpression()));<br>
- if (ArgExpr.isInvalid()) {<br>
- SkipUntil(tok::r_paren, StopAtSemi);<br>
- return 0;<br>
+ ExprResult ArgExpr;<br>
+ if (Tok.is(tok::identifier) &&<br>
+ attributeHasVariadicIdentifierArg(*AttrName)) {<br>
+ ArgExprs.push_back(ParseIdentifierLoc());<br>
+ } else {<br>
+ bool Uneval = attributeParsedArgsUnevaluated(*AttrName);<br>
+ EnterExpressionEvaluationContext Unevaluated(<br>
+ Actions,<br>
+ Uneval ? Sema::ExpressionEvaluationContext::Unevaluated<br>
+ : Sema::ExpressionEvaluationContext::ConstantEvaluated);<br>
+<br>
+ ExprResult ArgExpr(<br>
+ Actions.CorrectDelayedTyposInExpr(ParseAssignmentExpression()));<br>
+ if (ArgExpr.isInvalid()) {<br>
+ SkipUntil(tok::r_paren, StopAtSemi);<br>
+ return 0;<br>
+ }<br>
+ ArgExprs.push_back(ArgExpr.get());<br>
}<br>
- ArgExprs.push_back(ArgExpr.get());<br>
// Eat the comma, move to the next argument<br>
} while (TryConsumeToken(tok::comma));<br>
}<br>
<br>
Modified: cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp (original)<br>
+++ cfe/trunk/lib/Sema/AnalysisBasedWarnings.cpp Fri Jul 20 07:13:28 2018<br>
@@ -658,6 +658,11 @@ static void CheckFallThroughForBody(Sema<br>
else<br>
S.Diag(Loc, DiagID);<br>
};<br>
+<br>
+ // cpu_dispatch functions permit empty function bodies for ICC compatibility.<br>
+ if (D->getAsFunction() && D->getAsFunction()->isCPUDispatchMultiVersion())<br>
+ return;<br>
+<br>
// Either in a function body compound statement, or a function-try-block.<br>
switch (CheckFallThrough(AC)) {<br>
case UnknownFallThrough:<br>
<br>
Modified: cfe/trunk/lib/Sema/Sema.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/Sema.cpp?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/Sema.cpp?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/Sema/Sema.cpp (original)<br>
+++ cfe/trunk/lib/Sema/Sema.cpp Fri Jul 20 07:13:28 2018<br>
@@ -1585,6 +1585,7 @@ bool Sema::tryExprAsCall(Expr &E, QualTy<br>
}<br>
<br>
bool Ambiguous = false;<br>
+ bool IsMV = false;<br>
<br>
if (Overloads) {<br>
for (OverloadExpr::decls_iterator it = Overloads->decls_begin(),<br>
@@ -1598,11 +1599,16 @@ bool Sema::tryExprAsCall(Expr &E, QualTy<br>
if (const FunctionDecl *OverloadDecl<br>
= dyn_cast<FunctionDecl>((*it)->getUnderlyingDecl())) {<br>
if (OverloadDecl->getMinRequiredArguments() == 0) {<br>
- if (!ZeroArgCallReturnTy.isNull() && !Ambiguous) {<br>
+ if (!ZeroArgCallReturnTy.isNull() && !Ambiguous &&<br>
+ (!IsMV || !(OverloadDecl->isCPUDispatchMultiVersion() ||<br>
+ OverloadDecl->isCPUSpecificMultiVersion()))) {<br>
ZeroArgCallReturnTy = QualType();<br>
Ambiguous = true;<br>
- } else<br>
+ } else {<br>
ZeroArgCallReturnTy = OverloadDecl->getReturnType();<br>
+ IsMV = OverloadDecl->isCPUDispatchMultiVersion() ||<br>
+ OverloadDecl->isCPUSpecificMultiVersion();<br>
+ }<br>
}<br>
}<br>
}<br>
@@ -1683,7 +1689,7 @@ static void noteOverloads(Sema &S, const<br>
NamedDecl *Fn = (*It)->getUnderlyingDecl();<br>
// Don't print overloads for non-default multiversioned functions.<br>
if (const auto *FD = Fn->getAsFunction()) {<br>
- if (FD->isMultiVersion() &&<br>
+ if (FD->isMultiVersion() && FD->hasAttr<TargetAttr>() &&<br>
!FD->getAttr<TargetAttr>()->isDefaultVersion())<br>
continue;<br>
}<br>
@@ -1725,6 +1731,21 @@ static bool IsCallableWithAppend(Expr *E<br>
!isa<CXXOperatorCallExpr>(E));<br>
}<br>
<br>
+static bool IsCPUDispatchCPUSpecificMultiVersion(const Expr *E) {<br>
+ if (const auto *UO = dyn_cast<UnaryOperator>(E))<br>
+ E = UO->getSubExpr();<br>
+<br>
+ if (const auto *ULE = dyn_cast<UnresolvedLookupExpr>(E)) {<br>
+ if (ULE->getNumDecls() == 0)<br>
+ return false;<br>
+<br>
+ const NamedDecl *ND = *ULE->decls_begin();<br>
+ if (const auto *FD = dyn_cast<FunctionDecl>(ND))<br>
+ return FD->isCPUDispatchMultiVersion() || FD->isCPUSpecificMultiVersion();<br>
+ }<br>
+ return false;<br>
+}<br>
+<br>
bool Sema::tryToRecoverWithCall(ExprResult &E, const PartialDiagnostic &PD,<br>
bool ForceComplain,<br>
bool (*IsPlausibleResult)(QualType)) {<br>
@@ -1741,12 +1762,13 @@ bool Sema::tryToRecoverWithCall(ExprResu<br>
// so we can emit a fixit and carry on pretending that E was<br>
// actually a CallExpr.<br>
SourceLocation ParenInsertionLoc = getLocForEndOfToken(Range.getEnd());<br>
- Diag(Loc, PD)<br>
- << /*zero-arg*/ 1 << Range<br>
- << (IsCallableWithAppend(E.get())<br>
- ? FixItHint::CreateInsertion(ParenInsertionLoc, "()")<br>
- : FixItHint());<br>
- notePlausibleOverloads(*this, Loc, Overloads, IsPlausibleResult);<br>
+ bool IsMV = IsCPUDispatchCPUSpecificMultiVersion(E.get());<br>
+ Diag(Loc, PD) << /*zero-arg*/ 1 << IsMV << Range<br>
+ << (IsCallableWithAppend(E.get())<br>
+ ? FixItHint::CreateInsertion(ParenInsertionLoc, "()")<br>
+ : FixItHint());<br>
+ if (!IsMV)<br>
+ notePlausibleOverloads(*this, Loc, Overloads, IsPlausibleResult);<br>
<br>
// FIXME: Try this before emitting the fixit, and suppress diagnostics<br>
// while doing so.<br>
@@ -1757,8 +1779,10 @@ bool Sema::tryToRecoverWithCall(ExprResu<br>
<br>
if (!ForceComplain) return false;<br>
<br>
- Diag(Loc, PD) << /*not zero-arg*/ 0 << Range;<br>
- notePlausibleOverloads(*this, Loc, Overloads, IsPlausibleResult);<br>
+ bool IsMV = IsCPUDispatchCPUSpecificMultiVersion(E.get());<br>
+ Diag(Loc, PD) << /*not zero-arg*/ 0 << IsMV << Range;<br>
+ if (!IsMV)<br>
+ notePlausibleOverloads(*this, Loc, Overloads, IsPlausibleResult);<br>
E = ExprError();<br>
return true;<br>
}<br>
<br>
Modified: cfe/trunk/lib/Sema/SemaDecl.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/SemaDecl.cpp?rev=337552&r1=337551&r2=337552&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/SemaDecl.cpp?rev=337552&r1=337551&r2=337552&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/Sema/SemaDecl.cpp (original)<br>
+++ cfe/trunk/lib/Sema/SemaDecl.cpp Fri Jul 20 07:13:28 2018<br>
@@ -9275,6 +9275,20 @@ bool Sema::shouldLinkDependentDeclWithPr<br>
D->getFriendObjectKind() != Decl::FOK_None);<br>
}<br>
<br>
+namespace MultiVersioning {<br>
+enum Type { None, Target, CPUSpecific, CPUDispatch};<br>
+} // MultiVersionType<br>
+<br>
+static MultiVersioning::Type<br>
+getMultiVersionType(const FunctionDecl *FD) {<br>
+ if (FD->hasAttr<TargetAttr>())<br>
+ return MultiVersioning::Target;<br>
+ if (FD->hasAttr<CPUDispatchAttr>())<br>
+ return MultiVersioning::CPUDispatch;<br>
+ if (FD->hasAttr<CPUSpecificAttr>())<br>
+ return MultiVersioning::CPUSpecific;<br>
+ return MultiVersioning::None;<br>
+}<br>
/// Check the target attribute of the function for MultiVersion<br>
/// validity.<br>
///<br>
@@ -9313,7 +9327,8 @@ static bool CheckMultiVersionValue(Sema<br>
<br>
static bool CheckMultiVersionAdditionalRules(Sema &S, const FunctionDecl *OldFD,<br>
const FunctionDecl *NewFD,<br>
- bool CausesMV) {<br>
+ bool CausesMV,<br>
+ MultiVersioning::Type MVType) {<br>
enum DoesntSupport {<br>
FuncTemplates = 0,<br>
VirtFuncs = 1,<br>
@@ -9321,7 +9336,8 @@ static bool CheckMultiVersionAdditionalR<br>
Constructors = 3,<br>
Destructors = 4,<br>
DeletedFuncs = 5,<br>
- DefaultedFuncs = 6<br>
+ DefaultedFuncs = 6,<br>
+ ConstexprFuncs = 7,<br>
};<br>
enum Different {<br>
CallingConv = 0,<br>
@@ -9332,46 +9348,73 @@ static bool CheckMultiVersionAdditionalR<br>
Linkage = 5<br>
};<br>
<br>
+ bool IsCPUSpecificCPUDispatchMVType =<br>
+ MVType == MultiVersioning::CPUDispatch ||<br>
+ MVType == MultiVersioning::CPUSpecific;<br>
+<br>
+ if (OldFD && !OldFD->getType()->getAs<FunctionProtoType>()) {<br>
+ S.Diag(OldFD->getLocation(), diag::err_multiversion_noproto);<br>
+ S.Diag(NewFD->getLocation(), diag::note_multiversioning_caused_here);<br>
+ return true;<br>
+ }<br>
+<br>
+ if (!NewFD->getType()->getAs<FunctionProtoType>())<br>
+ return S.Diag(NewFD->getLocation(), diag::err_multiversion_noproto);<br>
+<br>
+ if (!S.getASTContext().getTargetInfo().supportsMultiVersioning()) {<br>
+ S.Diag(NewFD->getLocation(), diag::err_multiversion_not_supported);<br>
+ if (OldFD)<br>
+ S.Diag(OldFD->getLocation(), diag::note_previous_declaration);<br>
+ return true;<br>
+ }<br>
+<br>
// For now, disallow all other attributes. These should be opt-in, but<br>
// an analysis of all of them is a future FIXME.<br>
if (CausesMV && OldFD &&<br>
std::distance(OldFD->attr_begin(), OldFD->attr_end()) != 1) {<br>
- S.Diag(OldFD->getLocation(), diag::err_multiversion_no_other_attrs);<br>
+ S.Diag(OldFD->getLocation(), diag::err_multiversion_no_other_attrs)<br>
+ << IsCPUSpecificCPUDispatchMVType;<br>
S.Diag(NewFD->getLocation(), diag::note_multiversioning_caused_here);<br>
return true;<br>
}<br>
<br>
if (std::distance(NewFD->attr_begin(), NewFD->attr_end()) != 1)<br>
- return S.Diag(NewFD->getLocation(), diag::err_multiversion_no_other_attrs);<br>
+ return S.Diag(NewFD->getLocation(), diag::err_multiversion_no_other_attrs)<br>
+ << IsCPUSpecificCPUDispatchMVType;<br>
<br>
if (NewFD->getTemplatedKind() == FunctionDecl::TK_FunctionTemplate)<br>
return S.Diag(NewFD->getLocation(), diag::err_multiversion_doesnt_support)<br>
- << FuncTemplates;<br>
+ << IsCPUSpecificCPUDispatchMVType << FuncTemplates;<br>
<br>
if (const auto *NewCXXFD = dyn_cast<CXXMethodDecl>(NewFD)) {<br>
if (NewCXXFD->isVirtual())<br>
return S.Diag(NewCXXFD->getLocation(),<br>
diag::err_multiversion_doesnt_support)<br>
- << VirtFuncs;<br>
+ << IsCPUSpecificCPUDispatchMVType << VirtFuncs;<br>
<br>
if (const auto *NewCXXCtor = dyn_cast<CXXConstructorDecl>(NewFD))<br>
return S.Diag(NewCXXCtor->getLocation(),<br>
diag::err_multiversion_doesnt_support)<br>
- << Constructors;<br>
+ << IsCPUSpecificCPUDispatchMVType << Constructors;<br>
<br>
if (const auto *NewCXXDtor = dyn_cast<CXXDestructorDecl>(NewFD))<br>
return S.Diag(NewCXXDtor->getLocation(),<br>
diag::err_multiversion_doesnt_support)<br>
- << Destructors;<br>
+ << IsCPUSpecificCPUDispatchMVType << Destructors;<br>
}<br>
<br>
if (NewFD->isDeleted())<br>
return S.Diag(NewFD->getLocation(), diag::err_multiversion_doesnt_support)<br>
- << DeletedFuncs;<br>
+ << IsCPUSpecificCPUDispatchMVType << DeletedFuncs;<br>
<br>
if (NewFD->isDefaulted())<br>
return S.Diag(NewFD->getLocation(), diag::err_multiversion_doesnt_support)<br>
- << DefaultedFuncs;<br>
+ << IsCPUSpecificCPUDispatchMVType << DefaultedFuncs;<br>
+<br>
+ if (NewFD->isConstexpr() && (MVType == MultiVersioning::CPUDispatch ||<br>
+ MVType == MultiVersioning::CPUSpecific))<br>
+ return S.Diag(NewFD->getLocation(), diag::err_multiversion_doesnt_support)<br>
+ << IsCPUSpecificCPUDispatchMVType << ConstexprFuncs;<br>
<br>
QualType NewQType = S.getASTContext().getCanonicalType(NewFD->getType());<br>
const auto *NewType = cast<FunctionType>(NewQType);<br>
@@ -9379,7 +9422,7 @@ static bool CheckMultiVersionAdditionalR<br>
<br>
if (NewReturnType->isUndeducedType())<br>
return S.Diag(NewFD->getLocation(), diag::err_multiversion_doesnt_support)<br>
- << DeducedReturn;<br>
+ << IsCPUSpecificCPUDispatchMVType << DeducedReturn;<br>
<br>
// Only allow transition to MultiVersion if it hasn't been used.<br>
if (OldFD && CausesMV && OldFD->isUsed(false))<br>
@@ -9426,138 +9469,133 @@ static bool CheckMultiVersionAdditionalR<br>
return false;<br>
}<br>
<br>
-/// Check the validity of a mulitversion function declaration.<br>
-/// Also sets the multiversion'ness' of the function itself.<br>
+/// Check the validity of a multiversion function declaration that is the<br>
+/// first of its kind. Also sets the multiversion'ness' of the function itself.<br>
///<br>
/// This sets NewFD->isInvalidDecl() to true if there was an error.<br>
///<br>
/// Returns true if there was an error, false otherwise.<br>
-static bool CheckMultiVersionFunction(Sema &S, FunctionDecl *NewFD,<br>
- bool &Redeclaration, NamedDecl *&OldDecl,<br>
- bool &MergeTypeWithPrevious,<br>
- LookupResult &Previous) {<br>
- const auto *NewTA = NewFD->getAttr<TargetAttr>();<br>
- if (NewFD->isMain()) {<br>
- if (NewTA && NewTA->isDefaultVersion()) {<br>
- S.Diag(NewFD->getLocation(), diag::err_multiversion_not_allowed_on_main);<br>
- NewFD->setInvalidDecl();<br>
- return true;<br>
- }<br>
+static bool CheckMultiVersionFirstFunction(Sema &S, FunctionDecl *FD,<br>
+ MultiVersioning::Type MVType,<br>
+ const TargetAttr *TA,<br>
+ const CPUDispatchAttr *CPUDisp,<br>
+ const CPUSpecificAttr *CPUSpec) {<br>
+ assert(MVType != MultiVersioning::None &&<br>
+ "Function lacks multiversion attribute");<br>
+<br>
+ // Target only causes MV if it is default, otherwise this is a normal<br>
+ // function.<br>
+ if (MVType == MultiVersioning::Target && !TA->isDefaultVersion())<br>
return false;<br>
- }<br>
<br>
- // If there is no matching previous decl, only 'default' can<br>
- // cause MultiVersioning.<br>
- if (!OldDecl) {<br>
- if (NewTA && NewTA->isDefaultVersion()) {<br>
- if (!NewFD->getType()->getAs<FunctionProtoType>()) {<br>
- S.Diag(NewFD->getLocation(), diag::err_multiversion_noproto);<br>
- NewFD->setInvalidDecl();<br>
- return true;<br>
- }<br>
- if (CheckMultiVersionAdditionalRules(S, nullptr, NewFD, true)) {<br>
- NewFD->setInvalidDecl();<br>
- return true;<br>
- }<br>
- if (!S.getASTContext().getTargetInfo().supportsMultiVersioning()) {<br>
- S.Diag(NewFD->getLocation(), diag::err_multiversion_not_supported);<br>
- NewFD->setInvalidDecl();<br>
- return true;<br>
- }<br>
+ if (MVType == MultiVersioning::Target && CheckMultiVersionValue(S, FD)) {<br>
+ FD->setInvalidDecl();<br>
+ return true;<br>
+ }<br>
<br>
- NewFD->setIsMultiVersion();<br>
- }<br>
- return false;<br>
+ if (CheckMultiVersionAdditionalRules(S, nullptr, FD, true, MVType)) {<br>
+ FD->setInvalidDecl();<br>
+ return true;<br>
}<br>
<br>
- if (OldDecl->getDeclContext()->getRedeclContext() !=<br>
- NewFD->getDeclContext()->getRedeclContext())<br>
- return false;<br>
+ FD->setIsMultiVersion();<br>
+ return false;<br>
+}<br>
<br>
- FunctionDecl *OldFD = OldDecl->getAsFunction();<br>
- // Unresolved 'using' statements (the other way OldDecl can be not a function)<br>
- // likely cannot cause a problem here.<br>
- if (!OldFD)<br>
- return false;<br>
+static bool CheckTargetCausesMultiVersioning(<br>
+ Sema &S, FunctionDecl *OldFD, FunctionDecl *NewFD, const TargetAttr *NewTA,<br>
+ bool &Redeclaration, NamedDecl *&OldDecl, bool &MergeTypeWithPrevious,<br>
+ LookupResult &Previous) {<br>
+ const auto *OldTA = OldFD->getAttr<TargetAttr>();<br>
+ TargetAttr::ParsedTargetAttr NewParsed = NewTA->parse();<br>
+ // Sort order doesn't matter, it just needs to be consistent.<br>
+ llvm::sort(NewParsed.Features.begin(), NewParsed.Features.end());<br>
<br>
- if (!OldFD->isMultiVersion() && !NewTA)<br>
+ // If the old decl is NOT MultiVersioned yet, and we don't cause that<br>
+ // to change, this is a simple redeclaration.<br>
+ if (!OldTA || OldTA->getFeaturesStr() == NewTA->getFeaturesStr())<br>
return false;<br>
<br>
- if (OldFD->isMultiVersion() && !NewTA) {<br>
- S.Diag(NewFD->getLocation(), diag::err_target_required_in_redecl);<br>
+ // Otherwise, this decl causes MultiVersioning.<br>
+ if (!S.getASTContext().getTargetInfo().supportsMultiVersioning()) {<br>
+ S.Diag(NewFD->getLocation(), diag::err_multiversion_not_supported);<br>
+ S.Diag(OldFD->getLocation(), diag::note_previous_declaration);<br>
NewFD->setInvalidDecl();<br>
return true;<br>
}<br>
<br>
- TargetAttr::ParsedTargetAttr NewParsed = NewTA->parse();<br>
- // Sort order doesn't matter, it just needs to be consistent.<br>
- llvm::sort(NewParsed.Features.begin(), NewParsed.Features.end());<br>
+ if (CheckMultiVersionAdditionalRules(S, OldFD, NewFD, true,<br>
+ MultiVersioning::Target)) {<br>
+ NewFD->setInvalidDecl();<br>
+ return true;<br>
+ }<br>
<br>
- const auto *OldTA = OldFD->getAttr<TargetAttr>();<br>
- if (!OldFD->isMultiVersion()) {<br>
- // If the old decl is NOT MultiVersioned yet, and we don't cause that<br>
- // to change, this is a simple redeclaration.<br>
- if (!OldTA || OldTA->getFeaturesStr() == NewTA->getFeaturesStr())<br>
- return false;<br>
+ if (CheckMultiVersionValue(S, NewFD)) {<br>
+ NewFD->setInvalidDecl();<br>
+ return true;<br>
+ }<br>
<br>
- // Otherwise, this decl causes MultiVersioning.<br>
- if (!S.getASTContext().getTargetInfo().supportsMultiVersioning()) {<br>
- S.Diag(NewFD->getLocation(), diag::err_multiversion_not_supported);<br>
- S.Diag(OldFD->getLocation(), diag::note_previous_declaration);<br>
- NewFD->setInvalidDecl();<br>
- return true;<br>
- }<br>
+ if (CheckMultiVersionValue(S, OldFD)) {<br>
+ S.Diag(NewFD->getLocation(), diag::note_multiversioning_caused_here);<br>
+ NewFD->setInvalidDecl();<br>
+ return true;<br>
+ }<br>
<br>
- if (!OldFD->getType()->getAs<FunctionProtoType>()) {<br>
- S.Diag(OldFD->getLocation(), diag::err_multiversion_noproto);<br>
- S.Diag(NewFD->getLocation(), diag::note_multiversioning_caused_here);<br>
- NewFD->setInvalidDecl();<br>
- return true;<br>
- }<br>
+ TargetAttr::ParsedTargetAttr OldParsed =<br>
+ OldTA->parse(std::less<std::string>());<br>
<br>
- if (CheckMultiVersionValue(S, NewFD)) {<br>
- NewFD->setInvalidDecl();<br>
- return true;<br>
- }<br>
+ if (OldParsed == NewParsed) {<br>
+ S.Diag(NewFD->getLocation(), diag::err_multiversion_</blockquote></div>