[LLVMdev] Making it possible to clear the LLVMContext

[Philip Reames]

Simply using a new context would certainly be easier, but idealistically 
we should be able to support long lived contexts.

Are the things we're "caching" truly just a cache?  If so, we could pick 
a maximum size for the cache and evict old items.  My worry is that we 
may rely on some objects being in the "cache" for things like pointer 
equality.  If so, we may need a more complicated weak/strong ref 
management scheme.

Philip

On 06/24/2014 11:10 AM, Eric Christopher wrote:
> Right. Would make more sense to just start a new context past a
> certain limit (assuming that you're done with the existing cached
> information from the modules).
>
> -eric
>
> On Tue, Jun 24, 2014 at 11:07 AM, Marcello Maggioni <hayarms at gmail.com> wrote:
>> Something like, keeping the compiler alive for a many compilation sessions,
>> using the same LLVMContext, but without the problem of accumulating data in
>> the LLVMContext that would slowly fill up the memory.
>>
>> This as much I can enter into details that I can :/
>>
>> Probably this is also a quite common use case scenario too.
>>
>> Marcello
>>
>>
>> 2014-06-24 18:52 GMT+01:00 Eric Christopher <echristo at gmail.com>:
>>
>>> Right, this is likely going to have at least some of the same problems
>>> that Bill ran into when he tried to update a TargetMachine by
>>> placement newing a new one into it.
>>>
>>> Can you elaborate a bit more about what you're trying to do? (I
>>> realize this may be difficult for reasons.)
>>>
>>> -eric
>>>
>>> On Tue, Jun 24, 2014 at 10:44 AM, Marcello Maggioni <hayarms at gmail.com>
>>> wrote:
>>>> Hello,
>>>>
>>>> the need here is to have a single LLVMContext used for multiple
>>>> compilations.
>>>>
>>>> You make a good point about that by the way. If there are outstanding
>>>> users
>>>> cleaning the context under their seats might still pose a risk to them,
>>>> and
>>>> in that case deleting + newing a new LLVMContextImpl might actually not
>>>> be
>>>> very different.
>>>>
>>>> Marcello
>>>>
>>>> 2014-06-24 17:14 GMT+01:00 David Blaikie <dblaikie at gmail.com>:
>>>>
>>>>> What're the situation in which you need to clear it? If there are
>>>>> outstanding users of it (given that you mention clients possibly
>>>>> holding references to the pimpl, it sounds like you might have
>>>>> outstanding users) then wouldn't they be at risk of breaking if you
>>>>> mutate the LLVMContext underneath them?
>>>>>
>>>>> & if you don't have outstanding users, is there any particular benefit
>>>>> to resetting the LLVMContext compared to just making a new one?
>>>>>
>>>>> On Tue, Jun 24, 2014 at 7:18 AM, Marcello Maggioni <hayarms at gmail.com>
>>>>> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I'm trying to develop a way to reliably clean the LLVMContext in
>>>>>> order
>>>>>> to
>>>>>> make it possible to use it multiple times.
>>>>>>
>>>>>> LLVMContext itself is an almost empty object delegating almost all
>>>>>> its
>>>>>> content to LLVMContextImpl.
>>>>>> This makes it very clean ideally, because clearing the LLVMContext
>>>>>> would
>>>>>> be
>>>>>> as easy as deleting the LLVMContextImpl and creating a new one.
>>>>>>
>>>>>> The problem is that for some reason which I'm not aware of
>>>>>> LLVMContextImpl
>>>>>> is actually exposed as a public pointer in the LLVMContext
>>>>>> interface,making
>>>>>> it publicly available to objects that use it directly (this seems to
>>>>>> happen
>>>>>> quite a lot in the codebase).
>>>>>>
>>>>>> In LLVMContext the LLVMContextImpl is contained in a pImpl pointer
>>>>>> that
>>>>>> is
>>>>>> const (the pointer itself can't be changed) and I guess this is some
>>>>>> kind of
>>>>>> protection against object replacing the LLVMContextImpl directly,
>>>>>> which
>>>>>> stops us from just deleting it + getting a new one.
>>>>>> In addition to that, being pImpl public, there is no guarantee that
>>>>>> objects
>>>>>> don't rely on pImpl remaining always the same pointer.
>>>>>>
>>>>>> This makes it more difficult to clear LLVMContext.
>>>>>>
>>>>>> An approach I thought of could be adding a clear() method to
>>>>>> LLVMContext
>>>>>> that:
>>>>>>   - Calls directly the destructor of LLVMContextImpl on the pImpl
>>>>>> object
>>>>>>   - Uses a placement new to reinitialize the object.
>>>>>>   - Recreates the fixed metadata kinds like the LLVMContext
>>>>>> constructor
>>>>>> does
>>>>>>
>>>>>> I'm attaching a patch that show this approach in this mail.
>>>>>>
>>>>>> I would like to know a general idea about what people think about
>>>>>> this
>>>>>> and
>>>>>> see what people think would be the best approach would be.
>>>>>>
>>>>>> Thanks,
>>>>>> Marcello
>>>>>>
>>>>>> _______________________________________________
>>>>>> LLVM Developers mailing list
>>>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>
>>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev