[llvm] [Offload] Have olMemFree accept a platform as a param (PR #157478)

[Ross Brunton via llvm-commits]

RossBrunton wrote:

@jhuber6 @pbalcer To summarize the discussion in the meeting:

We need the ability to, given a pointer to somewhere in memory, figure out which platform allocated it so that we can dispatch to the appropriate platform for olMemFree/olGetMemInfo and friends.

There are (at least) two possible approaches:
1. We store a list of allocations in liboffload, along with their size, device and maybe some other information. This is an ordered list, and we can do a binary search to find which memory allocation a pointer is in. When we allocate, we check to see if the new allocation overlaps another one. If it does, we make a new allocation and throw away the old one. This ensures that every address is associated with at most one device/platform.
2. We require the liboffload user to track the device/platform, and pass that as a parameter to MemFree and GetMemInfo. This means that liboffload doesn't need to keep its own list of allocations (they will de-facto be managed by the user, which may have been tracking it anyway). GetInfo queries themselves can be done in a platform-specific way which will hopefully be more performant.

Performance wise, option 1 requires a binary search on every allocation, free and info query. Hopefully collisions are rare enough that they don't have an impact on performance. Option 2's performance will depend on how the user keeps track of platforms, and could be faster if they can access the platform handle in constant time.

Ergonomics wise, option 1 is much simpler, since the user doesn't have to track metadata about their allocations.

I don't remember if this got mentioned in the meeting, but I think there was an idea for having the runtime query each platform in turn to see if they owned the pointer. I don't think this will work because unused platforms might be in some uninitialized or sleeping state. So sending information queries might provoke the driver into going and waking up a GPU or something. And that's assuming that the platform driver is not bugged in such a way that it "claims" memory it hasn't allocated.

https://github.com/llvm/llvm-project/pull/157478