[LLVMdev] Available code-generation parallism

[Chris Lattner]

On Nov 2, 2008, at 2:20 PM, Jonathan Brandmeyer wrote:
> I am interested in making my LLVM front-end multi-threaded in a way
> similar to the GCC compiler server proposal and was wondering about  
> the
> extent that the LLVM passes support it.

Do you have a link for this?  I'm not familiar with any parallelism  
proposed by that project.  My understanding was that it was mostly  
about sharing across invocations of the compiler.

> Expression-at-a-time parallel construction:
> If function definitions are built purely depth-first, such that the
> parent pointers are not provided as they are created, what will break?
> I noted that the function and module verifiers aren't complaining, at
> least not yet.  Is there a generic "fixup upward-pointing parent
> pointers" pass that can be run afterwords?  If not, do I need to
> implement and perform that pass?  I suspect that emitting code for
> individual expressions in parallel will probably end up being too
> fine-grained, which leads me to...

Are you talking about building your AST or about building LLVM IR.   
The rules for constructing your AST are pretty much defined by you.   
The rules for constructing LLVM IR are a bit more tricky.  The most  
significant issue right now is that certain objects in LLVM IR are  
uniqued (like constants) and these have use/def chains.  Since use/def  
chain updating is not atomic or locked, this means that you can't  
create llvm ir on multiple threads.  This is something that I'm very  
much interested in solving someday, but no one is working on it at  
this time (that I'm aware of).


> Function-at-a-time parallel construction:
> Which (if any) LLVM objects support the object-level thread safety
> guarantee?  If I construct two separate function pass managers in
> separate threads and use them to optimize and emit object code for
> separate llvm::Function definitions in the program, will this work?
> Same question for llvm::Modules.

Unfortunately, for the above reason... basically none.  The LLVM code  
generators are actually very close to being able to run in parallel.   
The major issue is that they run a few llvm IR level passes first (LSR  
and codegen prepare) that hack on LLVM IR before the code generators  
run.  Because of this, they inherit the limitations of LLVM IR  
passes.  Very long term, I'd really like to make the code generator  
not affect the LLVM IR being put into them, but this is not likely to  
happen anytime in the near future.

If you're interested in this, tackling the use/def atomicity issues  
would be a great place to start.

-Chris