[PATCH] D27689: Module: hash the pcm content and use it as SIGNATURE for implicit modules.

[Manman Ren via Phabricator via cfe-commits]

manmanren added a comment.

Bruno helped on collecting performance data for this patch:
$ cat cocoa-test.h
#import <Cocoa/Cocoa.h>

$ rm -rf tmp; clang -x objective-c -isysroot `xcrun --sdk macosx --show-sdk-path` -fblocks -fmodules -fmodules-cache-path=tmp cocoa-test.h -fsyntax-only

clang-release (10 runs)
-----------------------

name      avg     min     med     max     SD      SD%    total
user     2.0023  1.9804  1.9994  2.0408  0.0149    0.74 20.0225
 sys     0.3601  0.3547  0.3505  0.4454  0.0287    7.98  3.6007
wall     2.5432  2.4291  2.4264  3.3604  0.2737   10.76 25.4324

clang-modhash-release (10 runs)
-------------------------------

name      avg     min     med     max     SD      SD%    total
user     2.1101  2.0935  2.1095  2.1274  0.0112    0.53 21.1012
 sys     0.3485  0.3462  0.3556  0.3476  0.0052    1.50  3.4854
wall     2.5614  2.5163  2.5605  2.6657  0.0419    1.63 25.6138

This ~5% slowdown for the total time of compiling all the modules (30+ modules).

A similar test for one object file in selfhost clang:

[TemplateName.cpp.o] clang-release (10 runs)
--------------------------------------------

name      avg     min     med     max     SD      SD%    total
user     7.4332  7.3716  7.4369  7.4976  0.0444    0.60 74.3318
 sys     0.5646  0.5512  0.5688  0.5570  0.0161    2.86  5.6464
wall     8.3357  8.1675  8.2991  8.2570  0.3240    3.89 83.3568

[TemplateName.cpp.o] clang-modhash-release (10 runs)
----------------------------------------------------

name      avg     min     med     max     SD      SD%    total
user     7.6911  7.6202  7.6979  7.7779  0.0418    0.54 76.9114
 sys     0.5531  0.5437  0.5520  0.5622  0.0105    1.89  5.5315
wall     8.4598  8.3868  8.4839  8.5497  0.0455    0.54 84.5977

~3.5% slowdown for total of modules:

$ ls module.cache/1UKP65OUKXURH/
Clang_AST-1XMT5DOVQWGLJ.pcm                   LLVM_C-2WKTHJHA6SDF0.pcm                      LLVM_Pass-HNKP05GYNHFQ.pcm                    _Builtin_stddef_max_align_t-2T552E4NSAIDI.pcm
Clang_Basic-1XMT5DOVQWGLJ.pcm                 LLVM_Config_ABI_Breaking-2FT1AFJTZ51QZ.pcm    LLVM_Support_DataTypes-2FT1AFJTZ51QZ.pcm      modules.idx
Darwin-399Z2LWDSMSTV.pcm                      LLVM_IR-HNKP05GYNHFQ.pcm                      LLVM_Utils-HNKP05GYNHFQ.pcm                   std-1337JMP4UGPIP.pcm

Note that the cache was wiped out right before every tests instance.
--------------------------------------------------------------------

Thanks Bruno. I am going to guard the hashing with a CC1 option (something like fmodules-hash-content) and force that flag to be enabled when the frontend implicitly builds a module. We will try to improve the hashing performance later (either improving SHA1 or using MD5 instead of SHA1).

Manman


https://reviews.llvm.org/D27689