[LLVMdev] Thread-safe cloning

[Nick Lewycky]

On 5 November 2013 08:40, Andrew Clinton <andrew at sidefx.com> wrote:

>  Sorry to resurrect an old thread, but I finally got around to testing
> this approach (round tripping through bitcode in memory) and it works
> beautifully - and isn't that much slower than cloning.
>
> I have noticed however that the copy process isn't thread-safe. The
> problem is that in Function, there is lazy initialization code for
> arguments:
>
>   void CheckLazyArguments() const {
>     if (hasLazyArguments())
>       BuildLazyArguments();
>   }
>   void BuildLazyArguments() const;
>

If you're interested, I think we shouldn't make this lazy at all, I don't
think it buys us anything. I once filed llvm.org/PR16536 with what I think
we should do, though it's a fundamental change that is may be more work
than you want to take on.

Nick

I've worked around this by calling Function::getArgumentList() outside the
> threaded code to harden it before the threaded copies.  Are there other
> lazy data structures that need to be solidified before threading?  Should I
> be playing it safe and put a thread lock around the entire copy procedure?
>
> In case you are interested, here is the algorithm I'm using to copy a
> Module to a same (or different) LLVMContext:
>
>         if (&context == &other.myContext)
>         {
>             // If the context is shared, we can use cloning
>             ValueToValueMapTy   map;
>             myModule = CloneModule(other.myModule, map);
>         }
>         else
>         {
>             // Otherwise, round trip the module to a stream and then back
>             // into the new context.  This approach allows for duplication
>             // and optimization to proceed in parallel for different
>             // modules.
>             std::string         str;
>             raw_string_ostream  stream(str);
>             WriteBitcodeToFile(other.myModule, stream);
>
>             StringRef                   ref(stream.str());
>             UT_ScopedPtr<MemoryBuffer>
> buf(MemoryBuffer::getMemBuffer(ref));
>             myModule = ParseBitcodeFile(buf.get(), myContext);
>
>             UT_ASSERT(myModule);
>
>         }
>
>
> On 06/18/2013 01:29 PM, Reid Kleckner wrote:
>
> You could probably round trip it through bitcode in memory.  I think all
> of the IR cloning functionality assumes that only one context is being
> used.  Even if the serialization isn't efficient as a clone could be, it
> should give you very high confidence that everything Just Works.  :)
>
>
> On Tue, Jun 18, 2013 at 1:16 PM, Andrew Clinton <andrew at sidefx.com> wrote:
>
>> I have a Module/LLVMContext that I'd like to clone and manipulate in
>> different threads (each thread may perform different translation /
>> optimization, so they need unique copies).  Currently this process has to
>> be locked, since each clone of the Module still refers to the same
>> LLVMContext.  Is there a way to clone both the Module and LLVMContext so
>> that the copies can be manipulated independently in different threads?
>>
>> Andrew
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131105/3510a1e3/attachment.html>