[all-commits] [llvm/llvm-project] 8e4f9b: [libc] Add initial support for an RPC mechanism fo...

[Joseph Huber via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 8e4f9b1fcbfd5e747b0df9a2de511b43bfa13348
      https://github.com/llvm/llvm-project/commit/8e4f9b1fcbfd5e747b0df9a2de511b43bfa13348
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2023-03-17 (Fri, 17 Mar 2023)

  Changed paths:
    M libc/src/__support/CMakeLists.txt
    M libc/src/__support/OSUtil/CMakeLists.txt
    M libc/src/__support/OSUtil/gpu/CMakeLists.txt
    A libc/src/__support/OSUtil/gpu/io.cpp
    A libc/src/__support/OSUtil/gpu/io.h
    M libc/src/__support/OSUtil/gpu/quick_exit.cpp
    M libc/src/__support/OSUtil/io.h
    A libc/src/__support/RPC/CMakeLists.txt
    A libc/src/__support/RPC/rpc.h
    A libc/src/__support/RPC/rpc_client.cpp
    A libc/src/__support/RPC/rpc_client.h
    M libc/startup/gpu/amdgpu/CMakeLists.txt
    M libc/startup/gpu/amdgpu/start.cpp
    M libc/utils/gpu/loader/amdgpu/CMakeLists.txt
    M libc/utils/gpu/loader/amdgpu/Loader.cpp

  Log Message:
  -----------
  [libc] Add initial support for an RPC mechanism for the GPU

This patch adds initial support for an RPC client / server architecture.
The GPU is unable to perform several system utilities on its own, so in
order to implement features like printing or memory allocation we need
to be able to communicate with the executing process. This is done via a
buffer of "sharable" memory. That is, a buffer with a unified pointer
that both the client and server can use to communicate.

The implementation here is based off of Jon Chesterfields minimal RPC
example in his work. We use an `inbox` and `outbox` to communicate
between if there is an RPC request and to signify when work is done.
We use a fixed-size buffer for the communication channel. This is fixed
size so that we can ensure that there is enough space for all
compute-units on the GPU to issue work to any of the ports. Right now
the implementation is single threaded so there is only a single buffer
that is not shared.

This implementation still has several features missing to be complete.
Such as multi-threaded support and asynchrnonous calls.

Depends on D145912

Reviewed By: sivachandra

Differential Revision: https://reviews.llvm.org/D145913


  Commit: 39e91098b58bfbef973bc759803fbfcc6064b42b
      https://github.com/llvm/llvm-project/commit/39e91098b58bfbef973bc759803fbfcc6064b42b
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2023-03-17 (Fri, 17 Mar 2023)

  Changed paths:
    M libc/CMakeLists.txt
    M libc/cmake/modules/LLVMLibCTestRules.cmake
    M libc/startup/gpu/amdgpu/CMakeLists.txt
    M libc/test/CMakeLists.txt
    M libc/test/IntegrationTest/CMakeLists.txt
    A libc/test/integration/startup/gpu/CMakeLists.txt
    A libc/test/integration/startup/gpu/args_test.cpp
    M libc/utils/gpu/loader/CMakeLists.txt

  Log Message:
  -----------
  [libc] Enable integration tests targeting the GPU

This patch enables integration tests running on the GPU. This uses the
RPC interface implemented in D145913 to compile the necessary
dependencies for the integration test object. We can then use this to
compile the objects for the GPU directly and execute them using the AMD
HSA loader combined with its RPC server. For example, the compiler is
performing the following actions to execute the integration tests.

```
$ clang++ --target=amdgcn-amd-amdhsa -mcpu=gfx1030 -nostdlib -flto -ffreestanding \
    crt1.o io.o quick_exit.o test.o rpc_client.o args_test.o -o image
$ ./amdhsa_loader image 1 2 5
args_test.cpp:24: Expected 'my_streq(argv[3], "3")' to be true, but is false
```

This currently only works with a single threaded client implementation
running on AMDGPU. Further work will implement multiple clients for AMD
and the ability to run on NVPTX as well.

Depends on D145913

Reviewed By: sivachandra, JonChesterfield

Differential Revision: https://reviews.llvm.org/D146256


Compare: https://github.com/llvm/llvm-project/compare/8c040d0f4941...39e91098b58b