[Patch][Review Request][llvm-link] Fix One Performance Bug

[Wan, Xiaofei]

Background:
Our business need to link many BC files together to generate a final big BC file; we noticed that the linking time increase significantly(more than linearly) with the increase of the BC file number, our biggest case (with ~8000 BC files ) takes > 1 hour time on XEON server which is unbearable to users.

Profiling and Analysis:

a.       From profiling data, it was observed that TypeFinder() consumes 98% of total time; TypeFinder() is not a key activity in llvm-link, it is just used to filter out all "Named Structure Types" in a module

b.       In current llvm-link, when linking all BC files, the BC file is linked one by one; for example, link bc1, bc2, bc3, bc4, bc5, bcN, the workflow is that, link bc1 & bc2 to bc12, then link with bc3 to bc123, then link with bc4 to bc1234, ..., finally bc12345...N.

The "Named Structure Types" in destination module is calculated in each linking; with the size increase of destination module, TypeFinder() will consume more and more time, this explains why the linking time increase is more than linearly

Solution & Fix & Result:

a.       "Named Structure Types" can be maintained in an incremental way in destination module, when linking a new module, just need to add new "Named Structure Types" into destination module to keep it most up-to-date

b.       The fix is very small, just add a collection in module to keep all "Named Structure Types" in destination module for link. See attachment for details.

c.       After this fix:

For our biggest case, the linking time decrease from 70 minutes to 35 seconds.

For Xalan in SPEC2006, the linking time decrease from 30 seconds to 2 second

Please review, thanks!

--
Best Regards
Wan Xiaofei (xiaofei.wan at intel.com)
Intel Corporation, Shanghai, China
MCG, Android Runtime & Compiler Team
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20130428/ee4fa37e/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Fix.TypeFinder.in.Linker.patch
Type: application/octet-stream
Size: 5235 bytes
Desc: Fix.TypeFinder.in.Linker.patch
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20130428/ee4fa37e/attachment.obj>