[LLVMdev] Making it possible to clear the LLVMContext

[Mathias Fröhlich]

Hi,

On Tuesday, June 24, 2014 15:18:51 Marcello Maggioni wrote:
> I'm trying to develop a way to reliably clean the LLVMContext in order to
> make it possible to use it multiple times.
> 
> LLVMContext itself is an almost empty object delegating almost all its
> content to LLVMContextImpl.
> This makes it very clean ideally, because clearing the LLVMContext would be
> as easy as deleting the LLVMContextImpl and creating a new one.
> 
> The problem is that for some reason which I'm not aware of LLVMContextImpl
> is actually exposed as a public pointer in the LLVMContext interface,making
> it publicly available to objects that use it directly (this seems to happen
> quite a lot in the codebase).
> 
> In LLVMContext the LLVMContextImpl is contained in a pImpl pointer that is
> const (the pointer itself can't be changed) and I guess this is some kind
> of protection against object replacing the LLVMContextImpl directly, which
> stops us from just deleting it + getting a new one.
> In addition to that, being pImpl public, there is no guarantee that objects
> don't rely on pImpl remaining always the same pointer.
> 
> This makes it more difficult to clear LLVMContext.

I can't comment on your solution.
But again the open source OpenGL driver stack is suffering from problems connected to
LLVMContext.

So our observation is, if we use several LLVMContexts within the program execution
lifetime, LLVM accumulates memory that is only freed on destruction of the
managed singletons. It turns out that in SDNode::getValueTypeList(EVT) the non simple
value types are cached and keyed by a llvm::Type value which in the end refers to a
LLVMContext reference value. Which in turn leads to the llvm library hogging memory
in this EVT std::set for each new LLVMContext.
For the OpenGL drivers using the global LLVMContext singleton leads to serious
threadsafeness problems. One driver is just non OpenGL conformant
non thread safe because of that, some others just tolerate LLVM hogging
memory because of that. Even worse for the ones just tolerating hogging
memory, I could potentially think of possible ABA like problems with a
new LLVMContext or llvm::Type being malloced at the same address than a previous already deleted
one but not knowing about the cached EVT's which appear to belong to this context.

That said I think you need to be careful since some of the llvm static caches refer to
LLVMContext pointers and at least the one I mentioned above is not cleared if either
a whole LLVMContext is cleared nor it is cleared by what you want to achieve.

Also for my interest:
Does anybody know of other statically cached object instances that are tied to an
LLVMContext but not cleaned up when the referring LLVMContext is destroyed?

Greetings and thanks

Mathias