[cfe-dev] [llvm-dev] Disk space and RAM requirements in docs

[Alex Bradbury via cfe-dev]

On Tue, 30 Apr 2019 at 07:38, Petr Penzin via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> Hi,
>
> Have anybody recently built LLVM in Debug mode within space requirements from the Getting Started doc?
>
> https://llvm.org/docs/GettingStarted.html#hardware
>
> > An LLVM-only build will need about 1-3 GB of space. A full build of LLVM and Clang will need around 15-20 GB of disk space.
>
> From my experience this numbers looks drastically low. On FreeBSD my recent builds consumed more than 70 GB and on Linux - more than 60. I am building LLVM, Clang, and LLD with all default targets and default settings. On FreeBSD I am also building Polly. Also, building `clang-check` target adds around to 20 GB to the build directory size.
>
> Apologies for the long snippet, but here is top 20 lines from `du -h | sort -rh` on my FreeBSD directory (full log is attached):
>
>  91G    build
>  40G    build/bin
>  25G    build/tools
>  25G    build/lib
>  24G    build/tools/clang
>  15G    build/tools/clang/unittests
> 5.0G    build/tools/clang/lib
> 3.9G    build/lib/Target
> 3.1G    build/tools/clang/unittests/Tooling
> 3.1G    build/tools/clang/test
> 2.8G    build/tools/clang/test/Tooling/Output/clang-check-mac-libcxx-fixed-compilation-db.cpp.tmp/mock-libcxx/bin
> 2.8G    build/tools/clang/test/Tooling/Output/clang-check-mac-libcxx-fixed-compilation-db.cpp.tmp/mock-libcxx
> 2.8G    build/tools/clang/test/Tooling/Output/clang-check-mac-libcxx-fixed-compilation-db.cpp.tmp
> 2.8G    build/tools/clang/test/Tooling/Output
> 2.8G    build/tools/clang/test/Tooling
> 1.9G    build/tools/clang/unittests/Frontend
> 1.6G    build/tools/clang/unittests/Driver
> 1.5G    build/tools/clang/unittests/ASTMatchers
> 1.4G    build/tools/clang/lib/StaticAnalyzer
> 1.3G    build/lib/CodeGen
>
> Note that Clang's `unittest` and `test` directory together occupy more than 18 GB (which is about the size manual assigns for the entire build), with most of the space in `test` taken up by output of tooling tests (is that for MacOS, by the way?). Also, looks like five of the unittest directories for clang components are taking more than a gigabyte each (with Tooling using more than 3 GB).
>
> Is this expected or am I doing something wrong? If it is expected, should the manual be fixed?
>
> There are no RAM requirements in the doc, from my observation linker process  on Linux or FreeBSD needs about 10 GB at the moment (and this has been growing for a while). Should this be also reflected in hardware requirements as well?

I think it would be worth strongly recommending the use of LLD or gold
to link. I've helped out a number of students/newcomers getting LLVM
to build for the first time and encountering issues due to binutils ld
OOMing is a common issue.

> And lastly is there a way to reduce the space consumption by Clang unittests and Tooling tests aside of disabling those?

Setting -DBUILD_SHARED_LIBS=True will reduce space requirements by
using shared libraries rather than static libraries. It will impact
the time taken to run LLVM and Clang tests though, due to greater
startup time.

Best,

Alex