[clang] [HIP] Document func ptr and virtual func (PR #68126)

[Yaxun Liu via cfe-commits]

https://github.com/yxsamliu updated https://github.com/llvm/llvm-project/pull/68126

>From cb0a36f061ee7c31eb63655ce0ecea00e1a94dec Mon Sep 17 00:00:00 2001
From: "Yaxun (Sam) Liu" <yaxun.liu at amd.com>
Date: Tue, 3 Oct 2023 12:21:10 -0400
Subject: [PATCH 1/2] [HIP] Document func ptr and virtual func

Document clang support for function pointers and virtual functions with HIP
---
 clang/docs/HIPSupport.rst | 62 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)

diff --git a/clang/docs/HIPSupport.rst b/clang/docs/HIPSupport.rst
index 8b4649733a9c777..63c16c0051153a0 100644
--- a/clang/docs/HIPSupport.rst
+++ b/clang/docs/HIPSupport.rst
@@ -176,3 +176,65 @@ Predefined Macros
    * - ``HIP_API_PER_THREAD_DEFAULT_STREAM``
      - Alias to ``__HIP_API_PER_THREAD_DEFAULT_STREAM__``. Deprecated.
 
+Function Pointers Support in Clang with HIP
+===========================================
+
+Function pointers' support varies with the usage mode in Clang with HIP. The following table provides an overview of the support status across different use-cases and modes.
+
+.. list-table:: Function Pointers Support Overview
+   :widths: 25 25 25
+   :header-rows: 1
+
+   * - Use Case
+     - ``-fno-gpu-rdc`` Mode (default)
+     - ``-fgpu-rdc`` Mode
+   * - Defined and used in the same TU
+     - Supported
+     - Supported
+   * - Defined in one TU and used in another TU
+     - Not Supported
+     - Supported
+
+In the ``-fno-gpu-rdc`` mode, the compiler calculates the resource usage of kernels based only on functions present within the same Translation Unit (TU). This mode does not support the use of function pointers defined in a different TU due to the possibility of incorrect resource usage calculations, leading to undefined behavior.
+
+On the other hand, the ``-fgpu-rdc`` mode allows the definition and use of function pointers across different TUs, as resource usage calculations can accommodate functions from disparate TUs.
+
+Virtual Function Support in Clang with HIP
+==========================================
+
+In Clang with HIP, support for calling virtual functions of an object in device or host code is contingent on where the object is constructed.
+
+- **Constructed in Device Code**: Virtual functions of an object can be called in device code if the object is constructed in device code.
+- **Constructed in Host Code**: Virtual functions of an object can be called in host code if the object is constructed in host code.
+
+In other scenarios, calling virtual functions is not allowed.
+
+Explanation
+-----------
+
+An object constructed on the device side contains a pointer to the virtual function table on the device side, which is not accessible in host code, and vice versa. Thus, trying to invoke virtual functions from a context different from where the object was constructed will be disallowed because the appropriate virtual table cannot be accessed.
+
+Example Usage
+-------------
+
+.. code-block:: c++
+
+   class Base {
+   public:
+      __device__ virtual void virtualFunction() {
+         // Base virtual function implementation
+      }
+   };
+
+   class Derived : public Base {
+   public:
+      __device__ void virtualFunction() override {
+         // Derived virtual function implementation
+      }
+   };
+
+   __global__ void kernel() {
+      Derived obj;
+      Base* basePtr = &obj;
+      basePtr->virtualFunction(); // Allowed since obj is constructed in device code
+   }

>From 1e9ab73e4aa0d39322ea16ba59e5feabf16ca778 Mon Sep 17 00:00:00 2001
From: "Yaxun (Sam) Liu" <yaxun.liu at amd.com>
Date: Sat, 7 Oct 2023 10:37:54 -0400
Subject: [PATCH 2/2] Add documentation about rdc mode

and possible way to improve virtual function support
---
 clang/docs/HIPSupport.rst | 44 ++++++++++++++++++++++++++++++++-------
 1 file changed, 37 insertions(+), 7 deletions(-)

diff --git a/clang/docs/HIPSupport.rst b/clang/docs/HIPSupport.rst
index 63c16c0051153a0..7aa3741749e70c1 100644
--- a/clang/docs/HIPSupport.rst
+++ b/clang/docs/HIPSupport.rst
@@ -176,8 +176,36 @@ Predefined Macros
    * - ``HIP_API_PER_THREAD_DEFAULT_STREAM``
      - Alias to ``__HIP_API_PER_THREAD_DEFAULT_STREAM__``. Deprecated.
 
-Function Pointers Support in Clang with HIP
-===========================================
+Compilation Modes
+=================
+
+Each HIP source file contains intertwined device and host code. Depending on the chosen compilation mode by the compiler options ``-fno-gpu-rdc`` and ``-fgpu-rdc``, these portions of code are compiled differently.
+
+Device Code Compilation
+-----------------------
+
+**``-fno-gpu-rdc`` Mode (default)**:
+
+- Compiles to a self-contained, fully linked offloading device binary for each offloading device architecture.
+- Device code within a Translation Unit (TU) cannot call functions located in another TU.
+
+**``-fgpu-rdc`` Mode**:
+
+- Compiles to a bitcode for each GPU architecture.
+- For each offloading device architecture, the bitcode from different TUs are linked together to create a single offloading device binary.
+- Device code in one TU can call functions located in another TU.
+
+Host Code Compilation
+---------------------
+
+**Both Modes**:
+
+- Compiles to a relocatable object for each TU.
+- These relocatable objects are then linked together.
+- Host code within a TU can call host functions and launch kernels from another TU.
+
+Function Pointers Support
+=========================
 
 Function pointers' support varies with the usage mode in Clang with HIP. The following table provides an overview of the support status across different use-cases and modes.
 
@@ -195,16 +223,16 @@ Function pointers' support varies with the usage mode in Clang with HIP. The fol
      - Not Supported
      - Supported
 
-In the ``-fno-gpu-rdc`` mode, the compiler calculates the resource usage of kernels based only on functions present within the same Translation Unit (TU). This mode does not support the use of function pointers defined in a different TU due to the possibility of incorrect resource usage calculations, leading to undefined behavior.
+In the ``-fno-gpu-rdc`` mode, the compiler calculates the resource usage of kernels based only on functions present within the same TU. This mode does not support the use of function pointers defined in a different TU due to the possibility of incorrect resource usage calculations, leading to undefined behavior.
 
 On the other hand, the ``-fgpu-rdc`` mode allows the definition and use of function pointers across different TUs, as resource usage calculations can accommodate functions from disparate TUs.
 
-Virtual Function Support in Clang with HIP
-==========================================
+Virtual Function Support
+========================
 
 In Clang with HIP, support for calling virtual functions of an object in device or host code is contingent on where the object is constructed.
 
-- **Constructed in Device Code**: Virtual functions of an object can be called in device code if the object is constructed in device code.
+- **Constructed in Device Code**: Virtual functions of an object can be called in device code on a specific offloading device if the object is constructed in device code on an offloading device with the same architecture.
 - **Constructed in Host Code**: Virtual functions of an object can be called in host code if the object is constructed in host code.
 
 In other scenarios, calling virtual functions is not allowed.
@@ -212,7 +240,9 @@ In other scenarios, calling virtual functions is not allowed.
 Explanation
 -----------
 
-An object constructed on the device side contains a pointer to the virtual function table on the device side, which is not accessible in host code, and vice versa. Thus, trying to invoke virtual functions from a context different from where the object was constructed will be disallowed because the appropriate virtual table cannot be accessed.
+An object constructed on the device side contains a pointer to the virtual function table on the device side, which is not accessible in host code, and vice versa. Thus, trying to invoke virtual functions from a context different from where the object was constructed will be disallowed because the appropriate virtual table cannot be accessed. The virtual function tables for offloading devices with different architecures are different, therefore trying to invoke virtual functions from an offloading device with a different architecture than where the object is constructed is also disallowed.
+
+A possible way to alleviate the current limitation of virtual function support in HIP is through the use of a "composite vtable". This involves creating a vtable that combines those from the host and all offloading device architectures, storing it in memory accessible by both. A dedicated registration function is introduced to populate this composite vtable. This function is invoked during global initialization to ensure the vtable is ready before any virtual function calls are made. For every virtual function call, irrespective of context, the system refers to this composite vtable to determine the correct function execution.
 
 Example Usage
 -------------