[clang] ee31ad0 - [clang-offload-bundler][docs][NFC] Add archive unbundling documentation

[Saiyedul Islam via cfe-commits]

Author: Saiyedul Islam
Date: 2021-09-21T19:24:44+05:30
New Revision: ee31ad0ab5f7d443b0bd582582a3524dcf7f13f0

URL: https://github.com/llvm/llvm-project/commit/ee31ad0ab5f7d443b0bd582582a3524dcf7f13f0
DIFF: https://github.com/llvm/llvm-project/commit/ee31ad0ab5f7d443b0bd582582a3524dcf7f13f0.diff

LOG: [clang-offload-bundler][docs][NFC] Add archive unbundling documentation

Add documentation of unbundling of heterogeneous device archives to
create device specific archives, as introduced by D93525. Also, add
documentation for supported text file formats.

Reviewed By: yaxunl

Differential Revision: https://reviews.llvm.org/D110083

Added: 
    

Modified: 
    clang/docs/ClangOffloadBundler.rst

Removed: 
    


################################################################################
diff  --git a/clang/docs/ClangOffloadBundler.rst b/clang/docs/ClangOffloadBundler.rst
index a0e446f766eea..312c45602c991 100644
--- a/clang/docs/ClangOffloadBundler.rst
+++ b/clang/docs/ClangOffloadBundler.rst
@@ -30,9 +30,62 @@ includes an ``init`` function that will use the runtime corresponding to the
 offload kind (see :ref:`clang-offload-kind-table`) to load the offload code
 objects appropriate to the devices present when the host program is executed.
 
+Supported File Formats
+======================
+Several text and binary file formats are supported for bundling/unbundling. See
+:ref:`supported-file-formats-table` for a list of currently supported formats.
+
+  .. table:: Supported File Formats
+     :name: supported-file-formats-table
+
+     +--------------------+----------------+-------------+
+     | File Format        | File Extension | Text/Binary |
+     +====================+================+=============+
+     | CPP output         |        i       |     Text    |
+     +--------------------+----------------+-------------+
+     | C++ CPP output     |       ii       |     Text    |
+     +--------------------+----------------+-------------+
+     | CUDA/HIP output    |       cui      |     Text    |
+     +--------------------+----------------+-------------+
+     | Dependency         |        d       |     Text    |
+     +--------------------+----------------+-------------+
+     | LLVM               |       ll       |     Text    |
+     +--------------------+----------------+-------------+
+     | LLVM Bitcode       |       bc       |    Binary   |
+     +--------------------+----------------+-------------+
+     | Assembler          |        s       |     Text    |
+     +--------------------+----------------+-------------+
+     | Object             |        o       |    Binary   |
+     +--------------------+----------------+-------------+
+     | Archive of objects |        a       |    Binary   |
+     +--------------------+----------------+-------------+
+     | Precompiled header |       gch      |    Binary   |
+     +--------------------+----------------+-------------+
+     | Clang AST file     |       ast      |    Binary   |
+     +--------------------+----------------+-------------+
+
+.. _clang-bundled-code-object-layout-text:
+
+Bundled Text File Layout
+========================
+
+The format of the bundled files is currently very simple: text formats are
+concatenated with comments that have a magic string and bundle entry ID in
+between.
+
+::
+
+  "Comment OFFLOAD_BUNDLER_MAGIC_STR__START__ 1st Bundle Entry ID"
+  Bundle 1
+  "Comment OFFLOAD_BUNDLER_MAGIC_STR__END__ 1st Bundle Entry ID"
+  ...
+  "Comment OFFLOAD_BUNDLER_MAGIC_STR__START__ Nth Bundle Entry ID"
+  Bundle N
+  "Comment OFFLOAD_BUNDLER_MAGIC_STR__END__ 1st Bundle Entry ID"
+
 .. _clang-bundled-code-object-layout:
 
-Bundled Code Object Layout
+Bundled Binary File Layout
 ==========================
 
 The layout of a bundled code object is defined by the following table:
@@ -209,3 +262,35 @@ Target specific information is available for the following:
   supported.
 
 Most other targets do not support target IDs.
+
+Archive Unbundling
+==================
+
+Unbundling of heterogeneous device archive is done to create device specific
+archives. Heterogeneous Device Archive is in a format compatible with GNU ar
+utility and contains a collection of bundled device binaries where each bundle
+file will contain device binaries for a host and one or more targets. The
+output device specific archive is in a format compatible with GNU ar utility
+and contains a collection of device binaries for a specific target.
+
+  Heterogeneous Device Archive, HDA = {F1.X, F2.X, ..., FN.Y}
+  where, Fi = Bundle{Host-DeviceBinary, T1-DeviceBinary, T2-DeviceBinary, ...,
+                     Tm-DeviceBinary},
+         Ti = {Target i, qualified using Bundle Entry ID},
+         X/Y = \*.bc for AMDGPU and \*.cubin for NVPTX
+
+  Device Specific Archive, DSA(Tk) = {F1-Tk-DeviceBinary.X, F2-Tk-DeviceBinary.X, ...
+                                      FN-Tk-DeviceBinary.Y}
+  where, Fi-Tj-DeviceBinary.X represents device binary of i-th bundled device
+  binary file for target Tj.
+
+clang-offload-bundler extracts compatible device binaries for a given target
+from the bundled device binaries in a heterogeneous device archive and creates
+a target specific device archive without bundling.
+
+clang-offlocad-bundler determines whether a device binary is compatible with a
+target by comparing bundle ID's. Two bundle ID's are considered compatible if:
+
+  * Their offload kind are the same
+  * Their target triple are the same
+  * Their GPUArch are the same