[PATCH] D93525: [OpenMP] Add unbundling of archives containing bundled object files into device specific archives

[Saiyedul Islam via Phabricator via cfe-commits]

saiislam added a comment.

In D93525#2495937 <https://reviews.llvm.org/D93525#2495937>, @t-tye wrote:

> In D93525#2495374 <https://reviews.llvm.org/D93525#2495374>, @saiislam wrote:
>
>> In D93525#2493752 <https://reviews.llvm.org/D93525#2493752>, @t-tye wrote:
>>
>>> In D93525#2493024 <https://reviews.llvm.org/D93525#2493024>, @yaxunl wrote:
>>>
>>>> can you document this in ClangOffloadBundler.rst ? I think we need a clear description about how clang-offload-bundler knows which file in the .a file belongs to which target.
>>>
>>> How does the .a relate to bundled code objects? Does the .a have a number of bundled code objects? If so wouldn't the identity of code objects be defined by the existing bundled code object ABI already documented? If the .a is a set of non-bundled code objects then defining how they are identified is not part of the clang-offload-bundler documentation as there are no bundled code objects involved. It would seem that the documentation belongs with the OpenMP runtime/compiler that is choosing to use .a files in this manner.
>>
>> Bundles (created using clang-offload-bundler) are passed to llvm-ar to create an archive of bundled objects (*.a file). An archive can have bundles for multiple device types. So, yes, the identity of code objects is defined by the existing bundled code object ABI.
>> This patch reads such an archive and produces a device-specific archive for each of the target devices given as input. Each device-specific archive contains all the code objects corresponding to that particular device and are written as per llvm archive format.
>>
>> Here is a snippet of relevant lit run lines:
>>
>>   // RUN: %clang -O0 -target %itanium_abi_triple %s -c -o %t.o
>>   
>>   // RUN: echo 'Content of device file 1' > %t.tgt1
>>   // RUN: clang-offload-bundler -type=o -targets=host-%itanium_abi_triple,openmp-amdgcn-amd-amdhsa-gfx900 -inputs=%t.o,%t.tgt1 -outputs=%t.abundle1.o
>>    
>>   // RUN: echo 'Content of device file 2' > %t.tgt2
>>   // RUN: clang-offload-bundler -type=o -targets=host-%itanium_abi_triple,openmp-amdgcn-amd-amdhsa-gfx900 -inputs=%t.o,%t.tgt2 -outputs=%t.abundle2.o
>>    
>>   // RUN: llvm-ar cr %t.lib.a %t.abundle1.o %t.abundle2.o
>>   
>>   This patch ==>
>>   // RUN: clang-offload-bundler -unbundle -type=a -targets=openmp-amdgcn-amd-amdhsa-gfx900 -inputs=%t.lib.a -outputs=%t.devicelib.a
>>   
>>   %t.devicelib.a will contain all devices objects corresponding to gfx900
>>
>> Though my interest originates from OpenMP side, Device-specific Archive Libraries created like this can be used by other offloading languages like HIP, CUDA, and OpenCL. Pelase refer D81109 <https://reviews.llvm.org/D81109> for the an earlier patch in the series of patches which will enable this.
>
> The naming of code objects in a bundled code object includes the processor name and the settings for target features (see https://clang.llvm.org/docs/ClangOffloadBundler.html#target-id and https://llvm.org/docs/AMDGPUUsage.html#target-id). The compatibility of code objects considers both target processor matching and target feature compatibility. Target features can have three settings: on, off and any. The compatibility is that each feature that is on/off must exactly match, but any will match either on or off.
>
> So when unbundling an archive how is the desired code object being requested? How is it handling the target features? For example, if code objects that will be compatible with a feature being on is required, then matching code objects in the archive would be those that have that feature either on or any.

At the moment this patch defines compatibility as exact string match of bundler entry ID. So, it doesn't support target ID concept fully. But, following example work.
Supporting target ID requires little more work and discussion.

  // RUN: clang-offload-bundler -type=o -targets=host-%itanium_abi_triple,openmp-amdgcn-amd-amdhsa--gfx908 -inputs=%t.o,%t.tgt1 -outputs=%t.abundle1.o
  // RUN: clang-offload-bundler -type=o -targets=host-%itanium_abi_triple,openmp-amdgcn-amd-amdhsa--gfx908:sramecc+:xnack+,openmp-amdgcn-amd-amdhsa--gfx908:sramecc-:xnack+ -inputs=%t.o,%t.tgt1,%t.tgt2 -outputs=%t.targetIDbundle.o
  // RUN: llvm-ar cr %t.targetIDlib.a %t.abundle1.o %t.targetIDbundle.o
  // RUN: clang-offload-bundler -unbundle -type=a -targets=openmp-amdgcn-amd-amdhsa--gfx908:sramecc+:xnack+ -inputs=%t.targetIDlib.a -outputs=%t.devicelibt-sramecc+.a
  // RUN: llvm-ar t %t.devicelibt-sramecc+.a | FileCheck %s -check-prefix=SRAMECCplus
  // SRAMECCplus: targetIDbundle.bc
  // SRAMECCplus-NOT: abundle1.bc


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D93525/new/

https://reviews.llvm.org/D93525