[clang] 9d9c25f - [C++20] [Coroutines] Don't mark await_suspend as noinline if it is specified as always_inline already

[Chuanqi Xu via cfe-commits]

Author: Chuanqi Xu
Date: 2023-08-28T11:43:33+08:00
New Revision: 9d9c25f81456aace2bec4b58498a420e650007d9

URL: https://github.com/llvm/llvm-project/commit/9d9c25f81456aace2bec4b58498a420e650007d9
DIFF: https://github.com/llvm/llvm-project/commit/9d9c25f81456aace2bec4b58498a420e650007d9.diff

LOG: [C++20] [Coroutines] Don't mark await_suspend as noinline if it is specified as always_inline already

Address https://github.com/llvm/llvm-project/issues/64933 and partially
https://github.com/llvm/llvm-project/issues/64945.

After c467245, we will add a noinline attribute to the await_suspend
member function of an awaiter if the awaiter has any non static member
functions.

Obviously, this decision will bring some performance regressions. And
people may complain about this while the long term solution may not be
available soon. In such cases, it is better to provide a solution for
the users who met the regression surprisingly.

Also it is natural to not prevent the inlining if the function is marked
as always_inline by the users already.

Added: 
    

Modified: 
    clang/lib/CodeGen/CGCall.cpp
    clang/test/CodeGenCoroutines/coro-awaiter-noinline-suspend.cpp

Removed: 
    


################################################################################
diff  --git a/clang/lib/CodeGen/CGCall.cpp b/clang/lib/CodeGen/CGCall.cpp
index 68d77f76666f68..8acb03cd5cfc4d 100644
--- a/clang/lib/CodeGen/CGCall.cpp
+++ b/clang/lib/CodeGen/CGCall.cpp
@@ -5548,7 +5548,11 @@ RValue CodeGenFunction::EmitCall(const CGFunctionInfo &CallInfo,
   // Then it is much easier to perform the safety analysis in the middle end.
   // If it is safe to inline the call to awaitSuspend, we can replace it in the
   // CoroEarly pass. Otherwise we could replace it in the CoroSplit pass.
-  if (inSuspendBlock() && mayCoroHandleEscape())
+  //
+  // If the `await_suspend()` function is marked as `always_inline` explicitly,
+  // we should give the user the right to control the codegen.
+  if (inSuspendBlock() && mayCoroHandleEscape() &&
+      !TargetDecl->hasAttr<AlwaysInlineAttr>())
     Attrs = Attrs.addFnAttribute(getLLVMContext(), llvm::Attribute::NoInline);
 
   // Disable inlining inside SEH __try blocks.

diff  --git a/clang/test/CodeGenCoroutines/coro-awaiter-noinline-suspend.cpp b/clang/test/CodeGenCoroutines/coro-awaiter-noinline-suspend.cpp
index f935e256d9db99..fe66d67e6e3557 100644
--- a/clang/test/CodeGenCoroutines/coro-awaiter-noinline-suspend.cpp
+++ b/clang/test/CodeGenCoroutines/coro-awaiter-noinline-suspend.cpp
@@ -67,6 +67,17 @@ StatefulAwaiter& operator co_await(Awaitable2) {
   return GlobalAwaiter;
 }
 
+struct AlwaysInlineStatefulAwaiter {
+    void* value;
+    bool await_ready() const noexcept { return false; }
+
+    template <typename PromiseType>
+    __attribute__((always_inline))
+    void await_suspend(std::coroutine_handle<PromiseType> h) noexcept {}
+
+    void await_resume() noexcept {}
+};
+
 Task testing() {
     co_await std::suspend_always{};
     co_await StatefulAwaiter{};
@@ -85,6 +96,8 @@ Task testing() {
 
     co_await Awaitable{};
     co_await Awaitable2{};
+
+    co_await AlwaysInlineStatefulAwaiter{};
 }
 
 // CHECK-LABEL: @_Z7testingv
@@ -135,6 +148,10 @@ Task testing() {
 // CHECK: call token @llvm.coro.save
 // CHECK: call void @_ZN15StatefulAwaiter13await_suspendIN4Task12promise_typeEEEvSt16coroutine_handleIT_E{{.*}}#[[NOINLINE_ATTR]]
 
+// Check `co_await AlwaysInlineStatefulAwaiter{};` to make sure user can force the await_suspend function to get inlined.
+// CHECK: call token @llvm.coro.save
+// CHECK: call void @_ZN27AlwaysInlineStatefulAwaiter13await_suspendIN4Task12promise_typeEEEvSt16coroutine_handleIT_E{{.*}}#[[NORMAL_ATTR]]
+
 // Check `co_await __promise__.final_suspend();`. We don't emit an blocker here since it is
 // empty.
 // CHECK: call token @llvm.coro.save