[llvm] 10b781f - Mark type test intrinsics as speculatable to fix inline cost
Teresa Johnson via llvm-commits
llvm-commits at lists.llvm.org
Fri Apr 23 10:02:46 PDT 2021
Author: Teresa Johnson
Date: 2021-04-23T10:02:31-07:00
New Revision: 10b781fb033ee8d8a02e1050976b3832ad50542a
URL: https://github.com/llvm/llvm-project/commit/10b781fb033ee8d8a02e1050976b3832ad50542a
DIFF: https://github.com/llvm/llvm-project/commit/10b781fb033ee8d8a02e1050976b3832ad50542a.diff
LOG: Mark type test intrinsics as speculatable to fix inline cost
There is already code in InlineCost.cpp to identify and ignore ephemeral
values (llvm.assume intrinsics and other side-effect free instructions
only feeding the assumes). However, because llvm.type.test intrinsics
were not marked speculatable, they and any instructions specifically
feeding the type test (typically a bitcast) were being counted towards
the instruction cost when inlining. This was causing profile matching
issues in some cases when enabling -fwhole-program-vtables for whole
program devirtualization.
According to the language reference, the speculatable attribute means:
"the function does not have any effects besides calculating its result
and does not have undefined behavior". I see no reason why type tests
cannot be marked with this attribute.
There are 2 test changes:
llvm/test/Transforms/Inline/ephemeral.ll: I added a type test intrinsic
here to verify the fix. Also, I found the test was not actually testing
what it originally intended. Many of the existing instructions were
optimized away by -Oz, and the cost of inlining was negative due to the
benefit of removing the call. So I changed the test to simply invoke the
inline pass and check the number of instructions computed by InlineCost.
I also fixed an instruction that was not actually used anywhere.
llvm/test/Transforms/SimplifyCFG/no-md-sink.ll needed to be made more
robust to code changes that reordered the metadata.
Differential Revision: https://reviews.llvm.org/D101180
Added:
Modified:
llvm/include/llvm/IR/Intrinsics.td
llvm/test/Transforms/Inline/ephemeral.ll
llvm/test/Transforms/SimplifyCFG/no-md-sink.ll
Removed:
################################################################################
diff --git a/llvm/include/llvm/IR/Intrinsics.td b/llvm/include/llvm/IR/Intrinsics.td
index 62b5c0fc2a524..63f8169df8549 100644
--- a/llvm/include/llvm/IR/Intrinsics.td
+++ b/llvm/include/llvm/IR/Intrinsics.td
@@ -1461,7 +1461,7 @@ def int_masked_compressstore:
// Test whether a pointer is associated with a type metadata identifier.
def int_type_test : DefaultAttrsIntrinsic<[llvm_i1_ty], [llvm_ptr_ty, llvm_metadata_ty],
- [IntrNoMem, IntrWillReturn]>;
+ [IntrNoMem, IntrWillReturn, IntrSpeculatable]>;
// Safely loads a function pointer from a virtual table pointer using type metadata.
def int_type_checked_load : DefaultAttrsIntrinsic<[llvm_ptr_ty, llvm_i1_ty],
diff --git a/llvm/test/Transforms/Inline/ephemeral.ll b/llvm/test/Transforms/Inline/ephemeral.ll
index 6261d4b39eab5..7573ec81e89ec 100644
--- a/llvm/test/Transforms/Inline/ephemeral.ll
+++ b/llvm/test/Transforms/Inline/ephemeral.ll
@@ -1,30 +1,35 @@
-; RUN: opt -S -Oz %s | FileCheck %s
+; RUN: opt -S -inline %s -debug-only=inline-cost 2>&1 | FileCheck %s
+; Only the load and ret should be included in the instruction count, not
+; the instructions feeding the assume.
+; CHECK: NumInstructions: 2
@a = global i32 4
-define i32 @inner() {
+define i32 @inner(i8* %y) {
%a1 = load volatile i32, i32* @a
- ; Here are enough instructions to prevent inlining, but because they are used
- ; only by the @llvm.assume intrinsic, they're free (and, thus, inlining will
- ; still happen).
+ ; Because these instructions are used only by the @llvm.assume intrinsic,
+ ; they're free and should not be included in the instruction count when
+ ; computing the inline cost.
%a2 = mul i32 %a1, %a1
- %a3 = sub i32 %a1, 5
+ %a3 = sub i32 %a1, %a2
%a4 = udiv i32 %a3, -13
%a5 = mul i32 %a4, %a4
%a6 = add i32 %a5, %a5
%ca = icmp sgt i32 %a6, -7
- tail call void @llvm.assume(i1 %ca)
+ %r = call i1 @llvm.type.test(i8* %y, metadata !0)
+ %ca2 = icmp eq i1 %ca, %r
+ tail call void @llvm.assume(i1 %ca2)
ret i32 %a1
}
-; @inner() should be inlined for -Oz.
-; CHECK-NOT: call i1 @inner
-define i32 @outer() optsize {
- %r = call i32 @inner()
+define i32 @outer(i8* %y) optsize {
+ %r = call i32 @inner(i8* %y)
ret i32 %r
}
declare void @llvm.assume(i1) nounwind
+declare i1 @llvm.type.test(i8*, metadata) nounwind readnone
+!0 = !{i32 0, !"typeid1"}
diff --git a/llvm/test/Transforms/SimplifyCFG/no-md-sink.ll b/llvm/test/Transforms/SimplifyCFG/no-md-sink.ll
index 8c8e3fd871cab..ff2c01db4448f 100644
--- a/llvm/test/Transforms/SimplifyCFG/no-md-sink.ll
+++ b/llvm/test/Transforms/SimplifyCFG/no-md-sink.ll
@@ -47,6 +47,6 @@ if.end:
; CHECK-LABEL: test2
; CHECK: %[[S:[a-z0-9.]+]] = select i1 %flag, i8* %y, i8* %z
-; CHECK: %[[R:[a-z0-9.]+]] = call i1 @llvm.type.test(i8* %[[S]], metadata !0)
+; CHECK: %[[R:[a-z0-9.]+]] = call i1 @llvm.type.test(i8* %[[S]], metadata ![[MD:[0-9]+]]
; CHECK: ret i1 %[[R]]
-
+; CHECK: ![[MD]] = !{i32 0, !"typeid1"}
More information about the llvm-commits
mailing list