[Openmp-dev] [RFC] Clarify the absence of API stability for the *device runtime* (aka. libomptarget-nvptx-sm_XX.bc)

[Johannes Doerfert via Openmp-dev]

On 7/9/20 5:15 PM, Hal Finkel via Openmp-dev wrote:
>
> On 7/9/20 5:00 PM, Michael Kruse wrote:
>> Am Do., 9. Juli 2020 um 15:57 Uhr schrieb Jonas Hahnfeld 
>> <hahnjo at hahnjo.de>:
>>> I asked the maintainer to install libomptarget and the plugin because
>>> otherwise there are link errors when compiling an OpenMP application
>>> with target directives. That's worse from a users perspective.
>> Isn't subtly miscompiling because a runtime function semantics changed
>> between versions worse than a link-time error?
>>
>>
>>> Really, this thread was about the device runtime which I still think
>>> would be nice to keep working with one released version of Clang. Based
>>> on the other comments in this thread, that's not going to happen -
>>> acknowledged.
>>> The host runtimes on the other side are linked dynamically and IMO it's
>>> one of the strengths of libomp to mix-and-match versions. Unless
>>> there's a compelling reason, I'll make another plea to keep
>>> compatibility.
>> I wonder whether we can cleanly separate libomptarget ABI stability
>> from device-side runtime ABI.
>
>
> I think that part of the issue here is that the device-side runtime 
> has two ABIs. One used for interaction with the plugin (how to receive 
> kernel parameters, etc.). A second used to communicate with the 
> application code on the device. I believe that the intent of this 
> thread was only to talk about the second one. My understanding is 
> these two ABIs can be considered separately. Johannes, is that correct.


Correct.

Furthermore, the one we want to change now is neither backward nor 
forward compatible, as I mentioned in my initial email. At least 
assuming the compilation mode via `libomptarget-nvptx-<version>.a` is is 
something we actually do not support because even if it would work it 
should never be done.



>
>  -Hal
>
>
>>
>> In the current clang driver, when the runtime .bc is found it is
>> linked into the executable using the -mlink-builtin-bitcode flag to
>> -cc1. If not found, it emits a warning:
>>> No library '%0' found in the default clang lib directory or in 
>>> LIBRARY_PATH. Expect degraded performance due to no inlining of 
>>> runtime functions on target devices.
>> I am not sure what happens when not found. I assume it requires the
>> libomptarget-nvptx.a (containing the device-side runtime) to be linked
>> statically instead and the cuda runtime will link the device-side
>> functions (i.e. without inlining as by the warning).
>>
>> The question is whether libomptarget-nvptx.a is considered part of the
>> distribution (i.e. installed next to libomptarget.so as `ninja
>> install` currently does). Maybe only libomptarget-nvptx.a needs to be
>> versioned (e.g. by appending a suffix such as
>> libomptarget-nvptx-<version>.a) to ensure that when linking, the
>> device code matching the version of the clang driver is linked
>> (theoretically, the driver could be used to link object files from
>> other versions of clang, but I think that risk is negligible).
>>
>> As by a comment in the CMakeLists.txt, "Dynamic linking is not yet
>> supported by the CUDA toolchain on the device.", which inhibits device
>> RTL in the libomptarget.so itself. Also, the proposal by Greg Rodgers
>> for fat static libraries presented at LLVM-CTH
>> (https://hps.vi4io.org/events/2020/llvm) only considers static
>> linking.
>>
>> Michael
>