[Patch][Review Request][llvm-link] Fix One Performance Bug
Du Toit, Stefanus
stefanus.du.toit at intel.com
Sun Apr 28 18:27:44 PDT 2013
Hi Xiaofei,
That's a nice speedup :)
I don't think it makes sense to add bufNamesStructTypesSet to Module. Can you keep this in the linker code instead, and maintain an mapping from Module* to the corresponding set there?
Thanks,
Stefanus
--
Stefanus Du Toit <stefanus.du.toit at intel.com>
Intel Waterloo
Phone: 519-591-1738
From: <Wan>, Xiaofei <xiaofei.wan at intel.com<mailto:xiaofei.wan at intel.com>>
Date: Sunday, 28 April, 2013 1:43 AM
To: "llvm-commits at cs.uiuc.edu<mailto:llvm-commits at cs.uiuc.edu>" <llvm-commits at cs.uiuc.edu<mailto:llvm-commits at cs.uiuc.edu>>
Subject: [Patch][Review Request][llvm-link] Fix One Performance Bug
Background:
Our business need to link many BC files together to generate a final big BC file; we noticed that the linking time increase significantly(more than linearly) with the increase of the BC file number, our biggest case (with ~8000 BC files ) takes > 1 hour time on XEON server which is unbearable to users.
Profiling and Analysis:
a. From profiling data, it was observed that TypeFinder() consumes 98% of total time; TypeFinder() is not a key activity in llvm-link, it is just used to filter out all “Named Structure Types” in a module
b. In current llvm-link, when linking all BC files, the BC file is linked one by one; for example, link bc1, bc2, bc3, bc4, bc5, bcN, the workflow is that, link bc1 & bc2 to bc12, then link with bc3 to bc123, then link with bc4 to bc1234, …, finally bc12345…N.
The “Named Structure Types” in destination module is calculated in each linking; with the size increase of destination module, TypeFinder() will consume more and more time, this explains why the linking time increase is more than linearly
Solution & Fix & Result:
a. “Named Structure Types” can be maintained in an incremental way in destination module, when linking a new module, just need to add new “Named Structure Types” into destination module to keep it most up-to-date
b. The fix is very small, just add a collection in module to keep all “Named Structure Types” in destination module for link. See attachment for details.
c. After this fix:
For our biggest case, the linking time decrease from 70 minutes to 35 seconds.
ForXalan in SPEC2006, the linking time decrease from 30 seconds to 2 second
Please review, thanks!
--
Best Regards
Wan Xiaofei (xiaofei.wan at intel.com<mailto:xiaofei.wan at intel.com>)
Intel Corporation, Shanghai, China
MCG, Android Runtime & Compiler Team
More information about the llvm-commits
mailing list