[Patch][Review Request][llvm-link] Fix One Performance Bug

[Wan, Xiaofei]

Stefanus:

Thanks for your suggestion, it is better to limit bufNamesStructTypesSet within llvm-link, this is what I planned to do at the beginning; I just worry about it will affect other component where ModuleLinker is used

Thanks
Wan Xiaofei

-----Original Message-----
From: Du Toit, Stefanus 
Sent: Monday, April 29, 2013 9:44 AM
To: Du Toit, Stefanus; Wan, Xiaofei; llvm-commits at cs.uiuc.edu
Subject: Re: [Patch][Review Request][llvm-link] Fix One Performance Bug

On 2013-04-28 9:27 PM, "Du Toit, Stefanus" <stefanus.du.toit at intel.com>
wrote:

>I don't think it makes sense to add bufNamesStructTypesSet to Module. 
>Can you keep this in the linker code instead, and maintain an mapping 
>from
>Module* to the corresponding set there?

To be more specific, I think the allocation of this data structure belongs in llvm-link itself.

Since that code already has a single destination module ("Composite"), that would also be a good place to allocate this data structure. You could then pass a pointer to it as an optional parameter to ModuleLinker's constructor.

That way, you avoid having to do any special book-keeping (e.g. a map from
Module* to the set as I suggested in my first message) and don't have to worry about what would happen if the Module got deleted in that case, etc.

Stefanus

>From: <Wan>, Xiaofei 
><xiaofei.wan at intel.com<mailto:xiaofei.wan at intel.com>>
>Date: Sunday, 28 April, 2013 1:43 AM
>To: "llvm-commits at cs.uiuc.edu<mailto:llvm-commits at cs.uiuc.edu>"
><llvm-commits at cs.uiuc.edu<mailto:llvm-commits at cs.uiuc.edu>>
>Subject: [Patch][Review Request][llvm-link] Fix One Performance Bug
>
>Background:
>Our business need to link many BC files together to generate a final 
>big BC file; we noticed that the linking time increase 
>significantly(more than linearly) with the increase of the BC file 
>number, our biggest case (with ~8000 BC files ) takes > 1 hour time on 
>XEON server which is unbearable to users.
>
>Profiling and Analysis:
>
>a.       From profiling data, it was observed that TypeFinder() consumes
>98% of total time; TypeFinder() is not a key activity in llvm-link, it 
>is just used to filter out all ³Named Structure Types² in a module
>
>b.       In current llvm-link, when linking all BC files, the BC file is
>linked one by one; for example, link bc1, bc2, bc3, bc4, bc5, bcN, the 
>workflow is that, link bc1 & bc2 to bc12, then link with bc3 to bc123, 
>then link with bc4 to bc1234, Š, finally bc12345ŠN.
>
>The ³Named Structure Types² in destination module is calculated in each 
>linking; with the size increase of destination module, TypeFinder() 
>will consume more and more time, this explains why the linking time 
>increase is more than linearly
>
>Solution & Fix & Result:
>
>a.       ³Named Structure Types² can be maintained in an incremental way
>in destination module, when linking a new module, just need to add new 
>³Named Structure Types² into destination module to keep it most 
>up-to-date
>
>b.       The fix is very small, just add a collection in module to keep
>all ³Named Structure Types² in destination module for link. See 
>attachment for details.
>
>c.       After this fix:
>
>For our biggest case, the linking time decrease from 70 minutes to 35 
>seconds.
>
>ForXalan in SPEC2006, the linking time decrease from 30 seconds to 2 
>second
>
>Please review, thanks!
>
>--
>Best Regards
>Wan Xiaofei (xiaofei.wan at intel.com<mailto:xiaofei.wan at intel.com>)
>Intel Corporation, Shanghai, China
>MCG, Android Runtime & Compiler Team
>
>_______________________________________________
>llvm-commits mailing list
>llvm-commits at cs.uiuc.edu
>http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits