[LLVMdev] [cfe-dev] [llvmdev] Clang 3.6 and trunk, high RSS usage compared to GCC (12.5GB vs. 0.5GB)

[David Abdurachmanov]

Hi,

I reduced -O1 passed to the following:

opt -lazy-value-info -correlated-propagation -reassociate -loops -lcssa 
-loop-rotate -licm -loop-unswitch -scalar-evolution -loop-deletion 
-function_tti -loop-unroll -memdep -memcpyopt -sccp  -lazy-value-info 
-correlated-propagation -memdep -dse -adce a.bc > /dev/null

Seems that removing or re-aranging the passes removes the problem.

david

On Jan 28, 2015, at 6:43 PM, David Abdurachmanov wrote:

> We also looked into it.
> 
> This seems to be coming from LLVM, not Clang code. We did a heap profile
> with IgProf [1] in a middle of compilation.
> 
> In our case seems that all allocations are from here:
> 
> https://github.com/llvm-mirror/llvm/blob/f04ce0e65747b16e6f321c0fdd38b6e1dc3271a3/lib/Analysis/LazyValueInfo.cpp#L519
> 
> lookup(Val)[BB] is constantly allocating std::map<AssertingVH<BasicBlock>, 
> LVILatticeVal> (104 bytes), that slowly eats the memory.
> 
> Current impression is that cache is exploding.
> 
> [1] http://igprof.org/
> 
> On Jan 28, 2015, at 11:11 AM, Kevin Funk wrote:
> 
>> On Tuesday 27 January 2015 15:19:33 David Abdurachmanov wrote:
>>> Hi,
>>> 
>>> I found that after moving to Clang pre-3.6 (git
>>> 65d8b4c4998b3a0c20934ea72ede72ef4838a004) and trunk (git
>>> 718825a8666acd9ceaab70fc7868332f20e2758f) our internal build machines
>>> started going offline in Jenkins. Clang after 3.5 release is consuming
>>> extreme amounts of memory in some cases.
>>> 
>>> I have uploaded [1] one of affected files.
>>> 
>>> $ g++ -std=c++11 -c -O1 -fPIC vpp_generated.ii -o vpp_generated.o
>>> 
>>> vmpeak: 582432 KB
>>> rspeak: 504500 KB = ~ 0.5GB
>>> 
>>> $ clang++ -std=c++11 -c -O1 -fPIC vpp_generated.ii -o vpp_generated.o
>>> 
>>> vmpeak: 12992076 KB
>>> rspeak: 12820184 KB = ~12.5GB
>>> 
>>> Disabling optimzer (-O0) resolves the issue, and RSS usage drops to ~300MB.
>>> 
>>> I decided to write here directly instead of creating yet another bug report,
>>> which usually don't get any feedback/comments.
>>> 
>>> clang version 3.7.0 (git 718825a8666acd9ceaab70fc7868332f20e2758f)
>>> Target: x86_64-unknown-linux-gnu
>>> Thread model: posix
>>> 
>>> Compiled:
>>> 
>>> ../configure --prefix=<..> --enable-optimized --with-binutils-include=<..>
>>> --disable-terminfo --enable-bindings=none CC=gcc CXX=g++ 'CPP=gcc -E'
>>> 'CXXCPP=g++ -E'
>>> 
>>> Cheers,
>>> david
>>> - - -
>>> [1] http://davidlt.web.cern.ch/davidlt/vault/vpp_generated.ii.xz
>> 
>> I'd like to add that I was having similar issues with Clang 3.5, while 
>> compiling test-code inside the QtCreator code base, with optimizer enabled.
>> 
>> I'm not sure if it's known or if I should file a bug report for that.
>> 
>> Relevant issue: Attempting to compile the tst_dumpers.cpp [1] target, with 
>> optimizing flags enabled (-O2, iirc). => Clang runs OOM (with more than 5 GB 
>> allocated)
>> Fix: Don't pass -On -- https://codereview.qt-project.org/#/c/103556/
>> 
>> Note: tst_Dumpers::dumper() is a 5 KLOC function, which *might* be an issue :)
>> 
>> [1] https://qt.gitorious.org/qt-creator/qt-creator/source/487b05dba8b6e8f548ec3cd451965fdb6df71e4d:tests/auto/debugger/tst_dumpers.cpp
>> 
>> -- 
>> Kevin Funk | kfunk at kde.org | http://kfunk.org
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150129/1b4564bf/attachment.sig>