[LLVMdev] [cfe-dev] [llvmdev] Clang 3.6 and trunk, high RSS usage compared to GCC (12.5GB vs. 0.5GB)

[David Abdurachmanov]

Hi,

It seems that we always disabled optimizer on some files for Clang and
I dropped this particular patch from our code base.

I tried to bisect Clang and went as far as 3.3 LLVM/Clang release with
the same memory consumption.

I moved to Fedora 21 based VM. GCC 4.9.2 used <500MB, and Clang 3.3 till
trunk above 10GB until VM hanged.

We didn't use --disable-assertions. Clang still has a number of issue
with our code base (incl. crashing/asserting). pre-3.6 Clang improves
(no more aborting), but some issues remain. For those I filled bug reports.
Yet to test the current 3.6 RC1 with our code base to see if it regressed
or not.

david

On Jan 27, 2015, at 4:35 PM, Milian Wolff wrote:

> On Tuesday 27 January 2015 15:19:33 David Abdurachmanov wrote:
>> Hi,
>> 
>> I found that after moving to Clang pre-3.6 (git
>> 65d8b4c4998b3a0c20934ea72ede72ef4838a004) and trunk (git
>> 718825a8666acd9ceaab70fc7868332f20e2758f) our internal build machines
>> started going offline in Jenkins. Clang after 3.5 release is consuming
>> extreme amounts of memory in some cases.
>> 
>> I have uploaded [1] one of affected files.
>> 
>> $ g++ -std=c++11 -c -O1 -fPIC vpp_generated.ii -o vpp_generated.o
>> 
>> vmpeak: 582432 KB
>> rspeak: 504500 KB = ~ 0.5GB
>> 
>> $ clang++ -std=c++11 -c -O1 -fPIC vpp_generated.ii -o vpp_generated.o
>> 
>> vmpeak: 12992076 KB
>> rspeak: 12820184 KB = ~12.5GB
>> 
>> Disabling optimzer (-O0) resolves the issue, and RSS usage drops to ~300MB.
>> 
>> I decided to write here directly instead of creating yet another bug report,
>> which usually don't get any feedback/comments.
>> 
>> clang version 3.7.0 (git 718825a8666acd9ceaab70fc7868332f20e2758f)
>> Target: x86_64-unknown-linux-gnu
>> Thread model: posix
>> 
>> Compiled:
>> 
>> ../configure --prefix=<..> --enable-optimized --with-binutils-include=<..>
>> --disable-terminfo --enable-bindings=none CC=gcc CXX=g++ 'CPP=gcc -E'
>> 'CXXCPP=g++ -E'
>> 
>> Cheers,
>> david
>> - - -
>> [1] http://davidlt.web.cern.ch/davidlt/vault/vpp_generated.ii.xz
> 
> I could reproduce this locally with clang 3.5.1 from the ArchLinux packages. I 
> ran it through my heaptrack [1] tool until ~630MB peak memory where consumed, 
> which should give enough of an insight already. The full report can be found 
> at [2]. Here is an excerpt:
> 
> 7137759 calls with 475.81MB peak consumption from:
>    0x7f83d81ff4ef
>      in /usr/bin/../lib/libLLVM-3.5.so
>    llvm::LazyValueInfo::getConstant(llvm::Value*, llvm::BasicBlock*)
>      in /usr/bin/../lib/libLLVM-3.5.so
>    0x7f83d80ba72f
>      in /usr/bin/../lib/libLLVM-3.5.so
>    llvm::FPPassManager::runOnFunction(llvm::Function&)
>      in /usr/bin/../lib/libLLVM-3.5.so
>    0x7f83d78df9c3
>      in /usr/bin/../lib/libLLVM-3.5.so
>    llvm::legacy::PassManagerImpl::run(llvm::Module&)
>      in /usr/bin/../lib/libLLVM-3.5.so
>    clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::CodeGenOptions 
> const&, clang::TargetOptions const&, clang::LangOptions const&, 
> llvm::StringRef, llvm::Module*, clang::BackendAction, llvm::raw_ostream*)
>      in /usr/bin/clang
>    0x8735a1
>      in /usr/bin/clang
>    clang::ParseAST(clang::Sema&, bool, bool)
>      in /usr/bin/clang
>    clang::CodeGenAction::ExecuteAction()
>      in /usr/bin/clang
>    clang::FrontendAction::Execute()
>      in /usr/bin/clang
>    clang::CompilerInstance::ExecuteAction(clang::FrontendAction&)
>      in /usr/bin/clang
>    clang::ExecuteCompilerInvocation(clang::CompilerInstance*)
>      in /usr/bin/clang
>    cc1_main(char const**, char const**, char const*, void*)
>      in /usr/bin/clang
>    main
>      in /usr/bin/clang
> 
> HTH
> 
> [1]: http://milianw.de/blog/heaptrack-a-heap-memory-profiler-for-linux
> [2]: https://userpage.physik.fu-berlin.de/~milianw/heaptrack.clang.report.txt.gz
> 
> 
> -- 
> Milian Wolff
> mail at milianw.de
> http://milianw.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150128/b889a144/attachment.sig>