[Patch][Review Request][llvm-link] Fix One Performance Bug

[Du Toit, Stefanus]

Hi Xiaofei,

That's a nice speedup :)

I don't think it makes sense to add bufNamesStructTypesSet to Module. Can you keep this in the linker code instead, and maintain an mapping from Module* to the corresponding set there?

Thanks,

Stefanus

--
Stefanus Du Toit <stefanus.du.toit at intel.com>
Intel Waterloo
Phone: 519-591-1738


From: <Wan>, Xiaofei <xiaofei.wan at intel.com<mailto:xiaofei.wan at intel.com>>
Date: Sunday, 28 April, 2013 1:43 AM
To: "llvm-commits at cs.uiuc.edu<mailto:llvm-commits at cs.uiuc.edu>" <llvm-commits at cs.uiuc.edu<mailto:llvm-commits at cs.uiuc.edu>>
Subject: [Patch][Review Request][llvm-link] Fix One Performance Bug

Background:
Our business need to link many BC files together to generate a final big BC file; we noticed that the linking time increase significantly(more than linearly) with the increase of the BC file number, our biggest case (with ~8000 BC files ) takes > 1 hour time on XEON server which is unbearable to users.

Profiling and Analysis:

a.       From profiling data, it was observed that TypeFinder() consumes 98% of total time; TypeFinder() is not a key activity in llvm-link, it is just used to filter out all “Named Structure Types” in a module

b.       In current llvm-link, when linking all BC files, the BC file is linked one by one; for example, link bc1, bc2, bc3, bc4, bc5, bcN, the workflow is that, link bc1 & bc2 to bc12, then link with bc3 to bc123, then link with bc4 to bc1234, …, finally bc12345…N.

The “Named Structure Types” in destination module is calculated in each linking; with the size increase of destination module, TypeFinder() will consume more and more time, this explains why the linking time increase is more than linearly

Solution & Fix & Result:

a.       “Named Structure Types” can be maintained in an incremental way in destination module, when linking a new module, just need to add new “Named Structure Types” into destination module to keep it most up-to-date

b.       The fix is very small, just add a collection in module to keep all “Named Structure Types” in destination module for link. See attachment for details.

c.       After this fix:

For our biggest case, the linking time decrease from 70 minutes to 35 seconds.

ForXalan in SPEC2006, the linking time decrease from 30 seconds to 2 second

Please review, thanks!

--
Best Regards
Wan Xiaofei (xiaofei.wan at intel.com<mailto:xiaofei.wan at intel.com>)
Intel Corporation, Shanghai, China
MCG, Android Runtime & Compiler Team