[LLVMdev] [LLVM Dev] [Discussion] Function-based parallel LLVM backend code generation

[Wan, Xiaofei]

Hi, community:

For the sake of our business need, I want to enable "Function-based parallel code generation" to boost up the compilation of single module, please see the details of the design and provide your feedbacks on below aspects, thanks!
1. Is this idea the proper solution for my requirement
2. This new feature will be enabled by llc -thd=N and has no impact on original llc when -thd=1
3. Can this new feature of llc be accepted by community and merged into LLVM code tree

Patches
The patch is divided into four separated parts, the all-in-one patch could be found here:
http://llvm-reviews.chandlerc.com/D1152

Design
https://docs.google.com/document/d/1QSkP6AumMCAVpgzwympD5pI3btPJt4SRgjY-vhyfySg/edit?usp=sharing


Background
1. Our business need to compile C/C++ source files into LLVM IR and link them into a big BC file; the big BC file is then compiled into binary code on different arch/target devices.
2. Backend code generation is a time-consuming activity happened on target device which makes it an important user experience.
3. Make -j or file based parallelism can't help here since there is only one big BC file; function-based parallel LLVM backend code generation is a good solution to improve compilation time which will fully utilize multi-cores.

Overall design strategy and goal
1. Generate totally same binary as what single thread output
2. No impacts on single thread performance & conformance
3. Little impacts on LLVM code infrastructure

Current status and test result
1. Parallel llc can generate same code as single thread by "objdump -d", it could pass 10 hours stress test for all performance benchmark
2. Parallel llc can introduce ~2.9X performance gain on XEON sever for 4 threads


Thanks
Wan Xiaofei
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Parallel.CG.7z
Type: application/octet-stream
Size: 24682 bytes
Desc: Parallel.CG.7z
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130716/9fa272e0/attachment.obj>