[Openmp-dev] [RFC] Clarify the absence of API stability for the *device runtime* (aka. libomptarget-nvptx-sm_XX.bc)

[Hal Finkel via Openmp-dev]

On 7/9/20 5:33 PM, Johannes Doerfert wrote:
>
>
> On 7/9/20 5:21 PM, Hal Finkel via Openmp-dev wrote:
>>
>> On 7/9/20 5:17 PM, Michael Kruse wrote:
>>> Correct.
>>>
>>> I inferred from your response that we have no such guarantee yet, we 
>>> just haven't broken it yet.
>>>
>>> However, I think that change will break ABI of libomptarget-nvptx.a
>>
>>
>> I think that this is an important point. I interpreted this thread in 
>> the context of the API between the runtime and the device-side 
>> application code. Not the ABI between the plugin and the device-side 
>> runtime. I suspect these two are separable, but we should definitely 
>> clarify this.
>>
>
> Neither should be considered stable at this point. We kept 
> libomptarget stable while we recently added functions but I would not 
> assume it will stay that way. As I mentioned before, for the device 
> runtime there is basically no supported way today in which we could 
> even guarantee stability.
>

I think that we should separate these concerns.

The device-side runtime library is a static library (in IR form), and 
multiple different versions can co-exist within one application (in 
device code contained in different shared libraries). It seems 
completely reasonable to consider that version-locked to the compiler 
that compiled the code. It's an internal interface between two parts of 
a translation unit compiled by the same compiler.

libomptarget is a shared library. libomptarget.rtl.cuda.so is a shared 
library. Unless we take special care in the naming and linking, managing 
of global state, etc. I think that we need to consider these to have 
some kind of ABI stability (because you only get to have one in each 
process). That doesn't mean that we can't ever decide to break that ABI, 
but we would likely decide not to do so silently. This implies that the 
device-side runtime should maintain ABI compatibility with 
libomptarget.rtl.cuda.so unless we make a non-silent breaking change. 
Just having kernels in a shared library suddenly stop working correctly 
when a newer version of libomptarget.rtl.cuda.so is loaded is probably 
not something we can considerately do at this point.

  -Hal


>
>>  -Hal
>>
>>
>>>
>>> Just want to clarify what ABI stability guarantees we have.
>>>
>>> Michael
>>>
>>>
>>> Le jeu. 9 juil. 2020 à 17:08, Hal Finkel <hfinkel at anl.gov 
>>> <mailto:hfinkel at anl.gov>> a écrit :
>>>
>>>
>>>     On 7/9/20 3:39 PM, Michael Kruse wrote:
>>>     > Am Do., 9. Juli 2020 um 14:08 Uhr schrieb Hal Finkel
>>>     <hfinkel at anl.gov <mailto:hfinkel at anl.gov>>:
>>>     >> Also, if we wanted to change clang so that it linked 
>>> version-locked
>>>     >> versions of these libraries, -lomptarget-11 or whatever, that,
>>>     in my
>>>     >> opinion, would also be a reasonable choice to discuss.
>>>     >>
>>>     >> One thing worth capturing is the extent to which these things 
>>> are
>>>     >> connected. There is a relationship between libomptarget and its
>>>     plugins,
>>>     >> and the plugins and the device-side runtimes. There is an ABI
>>>     boundary
>>>     >> there somewhere. If we change nothing else, we might need to
>>>     consider
>>>     >> ABI stability of this part of the device-side interface.
>>>     > The official distribution apt.llvm.org <http://apt.llvm.org>
>>>     contains libomptarget.so for
>>>     > LLVM 7 to 10, but each into separate directories under
>>>     > /usr/lib/llvm-<version>/libomptarget.so. The prebuilt binaries 
>>> under
>>>     > https://releases.llvm.org/download.html
>>> <https://releases.llvm.org/download.html> puts it directly under
>>>     > <prefix>/libomptarget.so. If libomptarget is version-locked, 
>>> users
>>>     > need to be careful about pointing to the right LD_LIBRARY_PATH.
>>>     > However, I could not find target device plugins in the 
>>> distributions
>>>     > (such as lib/libomptarget-nvptx-sm_60.bc when built on a machine
>>>     with
>>>     > CUDA). The official ubuntu repository doesn't contain
>>>     libomptarget at
>>>     > all. Arch Linux contains at least the x86_64 rtl
>>>     > (https://www.archlinux.org/packages/extra/x86_64/openmp/files/
>>> <https://www.archlinux.org/packages/extra/x86_64/openmp/files/>)
>>>     > without any versioning resolution.
>>>     >
>>>     > Should make it explicit what the compatibility guarantees for
>>>     > libomptarget are, maybe even discourage OS distributions to
>>>     > pre-package libomptarget into ldconfig default paths? At least
>>>     on Arch
>>>     > Linux updating the openmp package will break previously
>>>     > compiled-with-offloading binaries.
>>>
>>>
>>>     You mean that it will break them *if* we make an ABI-breaking
>>>     change in
>>>     libomptarget. Changing the device-side runtime doesn't necessarily
>>>     imply
>>>     that. Nevertheless, certainly good to know.
>>>
>>>       -Hal
>>>
>>>
>>>     >
>>>     > Michael
>>>
>>>     --     Hal Finkel
>>>     Lead, Compiler Technology and Programming Languages
>>>     Leadership Computing Facility
>>>     Argonne National Laboratory
>>>
>>>
>>> _______________________________________________
>>> Openmp-dev mailing list
>>> Openmp-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev

-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/openmp-dev/attachments/20200709/04a16403/attachment.html>