[PATCH] D12607: [ExecutionEngine] Add to the C API possibility to create custom SectionMemoryManager

[Lang Hames via llvm-commits]

lhames added a comment.

Hi Jauhien,

> I'll work on the support for symbol resolution methods then. But before it I want to figure out one thing that I do not understand. This thing is how APIs that I've added need SectionMemoryManager implementation to be stable.

> 

> Three facts that I rely on (memory management API, symbol resolution API, existence of some MM) will be exposed by execution engine API anyway. If some of MM methods will change implementation, client that uses it correctly will not see it, as it sees only quite common interface. Even SectionMemoryManager itself can be removed and replaced by something that implements necessary functionality. Could you, please, point me to the exact place where this patch (API, not realization) demands SectionMemoryManager to be stable?


The problem is the fall-back to SectionMemoryManager for unimplemented callbacks, which seems to be a central aim for this patch.

As an example: If an API takes a callback called 'allocateCodeSection' it should document some contract on it. E.g. 'allocateCodeSection should allocate memory for text sections, the memory will be aligned to at least Align bytes (but could be higher). Memory allocated via this method must be marked executable when memory is finalized, etc.'. Once that's documented people can reason about the guarantees provided by the contract.

In this patch, the contract is 'If you do not provide an allocateCodeSection method you will get whatever SectionMemoryManager does', but there's no contract for SectionMemoryManager's behavior. Since the C API is stable, people are going to assume that the implied contract is "Whatever SectionMemoryManager does Today", and they may come to rely on implementation details of SectionMemoryManager that aren't actually guaranteed to be stable.

If we were to provide a C API for obtaining a default memory manager implementation, it would have to have a very restricted contract* (i.e. the default implementation would only be guaranteed to meet the same bare-minimum requirements that we place on user-supplied memory managers). The question is whether a default implementation with so few guarantees would be useful to clients. I can imagine it being useful as a stop-gap solution for people bringing up a new JIT, but anyone building a robust solution would want to roll their own versions of all methods. (As a straw man example of the problem: the lack of contract means we might choose to slow every method down by 1000000x and nobody would have the right to complain, because the contract makes no guarantees on performance).

If all you're looking for is a bare-bones default memory manager implementation I think this patch could serve as a base for that, but I have to stress: This would *not* be a matter of exposing access to SectionMemoryManager through the C API, even if that's how it happened to be implemented today. Tomorrow we might replace SectionMemoryManager with something that still met the restricted contact, but behaved totally different to SectionMemoryManager.

So - the question is whether you're just looking for a default memory manager, or for SectionMemoryManager in particular. The former is doable, the latter is not.

- There's nothing stopping us from providing more guarantees on a default memory manager in time, but the more guarantees we provide the higher the maintenance burden going forward, so the amount of consideration and commitment needed rises a lot.


http://reviews.llvm.org/D12607