[Patch][Review Request][llvm-link] Fix One Performance Bug

[Du Toit, Stefanus]

On 2013-04-28 9:27 PM, "Du Toit, Stefanus" <stefanus.du.toit at intel.com>
wrote:

>I don't think it makes sense to add bufNamesStructTypesSet to Module. Can
>you keep this in the linker code instead, and maintain an mapping from
>Module* to the corresponding set there?

To be more specific, I think the allocation of this data structure belongs
in llvm-link itself.

Since that code already has a single destination module ("Composite"),
that would also be a good place to allocate this data structure. You could
then pass a pointer to it as an optional parameter to ModuleLinker's
constructor.

That way, you avoid having to do any special book-keeping (e.g. a map from
Module* to the set as I suggested in my first message) and don't have to
worry about what would happen if the Module got deleted in that case, etc.

Stefanus

>From: <Wan>, Xiaofei <xiaofei.wan at intel.com<mailto:xiaofei.wan at intel.com>>
>Date: Sunday, 28 April, 2013 1:43 AM
>To: "llvm-commits at cs.uiuc.edu<mailto:llvm-commits at cs.uiuc.edu>"
><llvm-commits at cs.uiuc.edu<mailto:llvm-commits at cs.uiuc.edu>>
>Subject: [Patch][Review Request][llvm-link] Fix One Performance Bug
>
>Background:
>Our business need to link many BC files together to generate a final big
>BC file; we noticed that the linking time increase significantly(more
>than linearly) with the increase of the BC file number, our biggest case
>(with ~8000 BC files ) takes > 1 hour time on XEON server which is
>unbearable to users.
>
>Profiling and Analysis:
>
>a.       From profiling data, it was observed that TypeFinder() consumes
>98% of total time; TypeFinder() is not a key activity in llvm-link, it is
>just used to filter out all ³Named Structure Types² in a module
>
>b.       In current llvm-link, when linking all BC files, the BC file is
>linked one by one; for example, link bc1, bc2, bc3, bc4, bc5, bcN, the
>workflow is that, link bc1 & bc2 to bc12, then link with bc3 to bc123,
>then link with bc4 to bc1234, Š, finally bc12345ŠN.
>
>The ³Named Structure Types² in destination module is calculated in each
>linking; with the size increase of destination module, TypeFinder() will
>consume more and more time, this explains why the linking time increase
>is more than linearly
>
>Solution & Fix & Result:
>
>a.       ³Named Structure Types² can be maintained in an incremental way
>in destination module, when linking a new module, just need to add new
>³Named Structure Types² into destination module to keep it most up-to-date
>
>b.       The fix is very small, just add a collection in module to keep
>all ³Named Structure Types² in destination module for link. See
>attachment for details.
>
>c.       After this fix:
>
>For our biggest case, the linking time decrease from 70 minutes to 35
>seconds.
>
>ForXalan in SPEC2006, the linking time decrease from 30 seconds to 2
>second
>
>Please review, thanks!
>
>--
>Best Regards
>Wan Xiaofei (xiaofei.wan at intel.com<mailto:xiaofei.wan at intel.com>)
>Intel Corporation, Shanghai, China
>MCG, Android Runtime & Compiler Team
>
>_______________________________________________
>llvm-commits mailing list
>llvm-commits at cs.uiuc.edu
>http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits