[LLVMdev] Thread-safe cloning

[Andrew Clinton]

Sorry to resurrect an old thread, but I finally got around to testing 
this approach (round tripping through bitcode in memory) and it works 
beautifully - and isn't that much slower than cloning.

I have noticed however that the copy process isn't thread-safe. The 
problem is that in Function, there is lazy initialization code for 
arguments:

   void CheckLazyArguments() const {
     if (hasLazyArguments())
       BuildLazyArguments();
   }
   void BuildLazyArguments() const;

I've worked around this by calling Function::getArgumentList() outside 
the threaded code to harden it before the threaded copies.  Are there 
other lazy data structures that need to be solidified before threading?  
Should I be playing it safe and put a thread lock around the entire copy 
procedure?

In case you are interested, here is the algorithm I'm using to copy a 
Module to a same (or different) LLVMContext:

         if (&context == &other.myContext)
         {
             // If the context is shared, we can use cloning
             ValueToValueMapTy   map;
             myModule = CloneModule(other.myModule, map);
         }
         else
         {
             // Otherwise, round trip the module to a stream and then back
             // into the new context.  This approach allows for duplication
             // and optimization to proceed in parallel for different
             // modules.
             std::string         str;
             raw_string_ostream  stream(str);
             WriteBitcodeToFile(other.myModule, stream);

             StringRef                   ref(stream.str());
             UT_ScopedPtr<MemoryBuffer>  
buf(MemoryBuffer::getMemBuffer(ref));
             myModule = ParseBitcodeFile(buf.get(), myContext);

             UT_ASSERT(myModule);
         }


On 06/18/2013 01:29 PM, Reid Kleckner wrote:
> You could probably round trip it through bitcode in memory.  I think 
> all of the IR cloning functionality assumes that only one context is 
> being used.  Even if the serialization isn't efficient as a clone 
> could be, it should give you very high confidence that everything Just 
> Works.  :)
>
>
> On Tue, Jun 18, 2013 at 1:16 PM, Andrew Clinton <andrew at sidefx.com 
> <mailto:andrew at sidefx.com>> wrote:
>
>     I have a Module/LLVMContext that I'd like to clone and manipulate
>     in different threads (each thread may perform different
>     translation / optimization, so they need unique copies).
>      Currently this process has to be locked, since each clone of the
>     Module still refers to the same LLVMContext.  Is there a way to
>     clone both the Module and LLVMContext so that the copies can be
>     manipulated independently in different threads?
>
>     Andrew
>     _______________________________________________
>     LLVM Developers mailing list
>     LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu>
>     http://llvm.cs.uiuc.edu
>     http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131105/2ce93cd2/attachment.html>