[LLVMdev] Quirks and errors with cmake with non-standard prefixes

[Emanuele Cestari]

Hi everyone,
before starting I'm giving you a generic description of the environment I'm currently using :
  - Ubuntu 13.10 amd64
  - Python 2, libxml2, libffi and libedit installed under a non-standard prefix
  - I used both cmake-2.8.12.2-Linux-i386 and cmake-3.0.20140416-g2bb14-Linux-i386 in this process and they give the same results

Please note that I have no major issues when building clang/llvm/lldb with libraries that are available in standard prefixes or libraries that are simply provided by the OS/package manager itself; there are some subtle errors that prevent me from using libraries in custom directories when configuring and building.
Clang compiles and runs just fine in the trivial case but I think that the use of standard prefixes is hiding some flaws in the building scripts/configuration.


My copy of the llvm repositories is being layed out like so :

  - the llvm svn repository is the <root> dir
  - clang under <root>/tools
  - clang-tools-extra under <root>/tools/clang/tools
  - compiler-rt under <root>/projects
  - libcxx under <root>/projects
 
Most of the time I build clang without lldb, but when I need lldb, I put it under <root>/tools


As a side note I invite you to compile clang in a clean environment, just like I did, and try to compile clang from sources with libraries in a non-standard prefix and binaries (  like svn and cmake ) in custom paths; you will probably experience this kind of troubles.


I experienced the following issues that prevents me from compiling clang/llvm :

Issue #1
when building with all the 5 repos + lldb I noticed that if I set the cache entries 


[ code ]
LIBXML2_INCLUDE_DIR:PATH="/opt/lib64/libxml2/2.9.1/include/libxml2"
LIBXML2_LIBRARIES:PATH="/opt/lib64/libxml2/2.9.1/lib"
LIBXML2_XMLLINT_EXECUTABLE:FILEPATH="/opt/lib64/libxml2/2.9.1/bin/xmllint"
[ /code ]

the cmake test for libxml2 is ok but the target c-index-test it's being dropped even if cmake itself says that the target can only link to a "real" library

[ code ]
WARNING: Target "c-index-test" requests linking to directory "/opt/lib64/libxml2/2.9.1/lib".  Targets may link only to libraries.  CMake is dropping the item.

[ /code ]

So there is clearly something wrong with the test for libxml2 and it's probably something inside the lldb repository since this project requires libxml2 explicitly. Of course with

[ code ]
LIBXML2_INCLUDE_DIR:PATH="/opt/lib64/libxml2/2.9.1/include/libxml2"
LIBXML2_LIBRARIES:PATH="/opt/lib64/libxml2/2.9.1/lib/libxml2.so"
LIBXML2_XMLLINT_EXECUTABLE:FILEPATH="/opt/lib64/libxml2/2.9.1/bin/xmllint"

[ /code ]

Everything is ok and cmake says "the test is passed and c-index-test is being kept" but this is to prove that the "cmake test" for this library serves no purpose at all.

Issue #2
when building with all the 5 repos + lldb I noticed that the linking phase is weird, it doesn't take into account the linker-related cache entries that I'm giving to cmake, basically anything in the form CMAKE_*_LINKER_FLAGS, and it doesn't even use the specific cache entries ( specific to the library ) to form the correct set of flags that you have to pass to ld or gcc, even when cmake gets something like LIBXML2_LIBRARIES set to the correct value, it doesn't make any good use of it. Another example concerns passing the correct values for the -L and -l flags to the aformentioned CMAKE_*_LINKER_FLAGS cache entries because this serves no purpose and the building fails to find the library to link to the final executable.  Why not simplifying everything and just set 1 cache entry for the <root> dir of the library that needs to be found ? You can easily find the headers under <root>/include or the libs under <root>/lib .
To be fair this isn't always the case, for example with libxml2 the headers are under <root>/include/libxml2 and not <root>/include, but my idea is that you shouldn't set so many variables for just 1 library, it's redundant and leads to confusion most of the time, and I invite you to try for yourself.

A minor fix will also be to force the use of some "gcc signed" cache entries since I don't know how many still link with ld under linux when linking with gcc it's probably a more modern approach, so something like "GCC_LINKER_FLAGS" will probably fit better, but I think that the linking phase as it is now it's not predictable and consistent in its own behaviour as it should be . And in the end it doesn't even work.


Issue #3
This is again another example of some weird combination of a "cmake test" that succeeds that leads to a failed step.
If you consider the file CMakeLists.txt that it's in the llvm root, at the line 349 you have some code that uses the PYTHON_EXECUTABLE

[ code ]
...
  COMMAND ${PYTHON_EXECUTABLE} ${LLVMBUILDTOOL}
            --native-target "${LLVM_NATIVE_ARCH}"
            --enable-targets "${LLVM_TARGETS_TO_BUILD}"
            --enable-optional-components "${LLVMOPTIONALCOMPONENTS}"

...

[ /code ]

This test fails under a customized python installation, furthermore the deduction of the right value for PYTHON_EXECUTABLE is subject to the FindPythonInterp builtin cmake module which is basically not documented and if you use cmake you have no clue and no guarantees about what steps and processes are activated by cmake the "right" Python installation. For example there are expansion rules that have effect on the value that the user is giving to the related cmake cache entry ? The Python interpreter it's just a valid file with a valid "pythonish" name ? Cmake is testing this hypothetically valid python executable with some module-specific tests or it takes for granted that something that looks like a binary will work ? And bla bla, plus other dissertations about the fact that this module gives no insights and informations.
You are probably thinking that you can inspect the source code for the right cmake module to know what cmake is really doing but this way of thinking doesn't really take into account the fact that something really critical as the location of the right python executable ( since llvm/clang is using python a lot and relying heavily on it ) it's not an invariance, it's not even something with a consistent behaviour .

As I mentioned there is also an extra problem with this approach, python, and this is even more popular for many custom python installations, often relies on some special environemnt variables like PYTHONHOME and PYTHONPATH ( https://docs.python.org/2/using/cmdline.html for more ), you can't just use the full path to the python executable and feed some scripts and flags to it, it's not going to work out that well.

Also many GNU/Linux distributions provide a default python installation, so if you have 1 system-wide python installation plus another custom one that you are going to use as a developer library, you end up with having at least 2 python installations at the same time, and this leads us to the problem that exporting PYTHONHOME or PYTHONPATH it's likely to cause more harm than good.
There should be a way to pass and use customized PYTHONPATH,PYTHONHOME right inside the scripts that are using the PYTHON_EXECUTABLE otherwise this is not going to work at all and the result of the script will be 100% unreliable .

Issue #4

The check done for the libffi doesn't work, I extracted the relevant bits for the cmake log


[ code ]
Determining if the ffi_call exist failed with the following output:
Change Dir: /tmp/llvmSvn/build/build1/CMakeFiles/CMakeTmp

Run Build Command:/usr/bin/make "cmTryCompileExec3438544690/fast"
/usr/bin/make -f CMakeFiles/cmTryCompileExec3438544690.dir/build.make CMakeFiles/cmTryCompileExec3438544690.dir/build
make[1]: Entering directory `/tmp/llvmSvn/build/build1/CMakeFiles/CMakeTmp'
/opt/redist/bin32/cmake/cmake-2.8.12.2-Linux-i386/bin/cmake -E cmake_progress_report /tmp/llvmSvn/build/build1/CMakeFiles/CMakeTmp/CMakeFiles 1
Building C object CMakeFiles/cmTryCompileExec3438544690.dir/CheckSymbolExists.c.o
/usr/bin/cc   -I/tmp/llvmSvn/build/build1/CMakeFiles/CMakeTmp/\"/opt/lib64/libffi/3.0.13/include/ffi.h\"    -o CMakeFiles/cmTryCompileExec3438544690.dir/CheckSymbolExists.c.o   -c /tmp/llvmSvn/build/build1/CMakeFiles/CMakeTmp/CheckSymbolExists.c
/tmp/llvmSvn/build/build1/CMakeFiles/CMakeTmp/CheckSymbolExists.c:2:17: fatal error: ffi.h: No such file or directory
 #include <ffi.h>
                 ^
compilation terminated.
make[1]: *** [CMakeFiles/cmTryCompileExec3438544690.dir/CheckSymbolExists.c.o] Error 1
make[1]: Leaving directory `/tmp/llvmSvn/build/build1/CMakeFiles/CMakeTmp'
make: *** [cmTryCompileExec3438544690/fast] Error 2

File /tmp/llvmSvn/build/build1/CMakeFiles/CMakeTmp/CheckSymbolExists.c:

[ /code ]


note the values given to the -I flag, once again the rules used for the expansion of the values given to cmake entries are not clear, and this test is doomed to fail no matter what I'm going to feed as an input.
Plus if I go where the building cache is at, and I type 


[ code ]
cmake -LAH | grep FFI

[ /code ]


I can see that the FFI related cache entries are correctly expanded to

[ code ]
FFI_INCLUDE_DIR:PATH="/opt/lib64/libffi/3.0.13/include"
FFI_INCLUDE_PATH:PATH="/opt/lib64/libffi/3.0.13/include/ffi.h"
FFI_LIBRARY_DIR:PATH="/opt/lib64/libffi/3.0.13/lib"
FFI_LIBRARY_PATH:FILEPATH="/opt/lib64/libffi/3.0.13/lib/libffi.so"

[ /code ]

Issue #5
the above example shows how much redundancy there is and that a simple value leads to an error, there are 4 variables set for basically the same value and all 4 could be easily derived from 1 single value, the root path for the ffi library.

Issue #6
What are the required modules that my Python 2 installation needs to have ? I can't find any doc about that, but I can't use clang/llvm without either, this is a real issue that I'm having with the documentation.


Generally speaking there is a lot of redundancy in the cmake entries and in what the checks and the tests are trying to execute, this leads to errors that make it impossible to compile clang under a custom environment that is just using python under a custom prefix and 2-3 customized libraries.

I also don't think that the checks and tests used in the cmake modules are valid, or in better terms, they can't provided any good invariant, ask yourself what a find<something> cmake function really does and you will probably end up sharing my view.

http://www.cmake.org/Wiki/CMake_2.8.11_Docs
At this page you can easily find the "most verbose" description of the FindPythonInterp module that the authors of cmake are providing at the time I'm writing this, considering both the online and offline cmake documentation, it just makes this and other modules useless and unreliable, it doesn't even tells you what are the cache entries influencing the behaviour of that module, this kind of documentation it's not even useful for the users, imagine for someone who needs a reliable tool.

In the end I think that you should just get a clean 64 bit linux based environment and compile Python, libxml2, libedit and libffi from the source, and try to solve the problem that I'm experiencing, because I think that the building system as it is now doesn't work, it's not reliable and if it works and when it works, it's because there are other implicit steps and system-wide configurations that are hiding some quirks and bugs.


This are my 2 cent, I like llvm and clang, I use libcxx and libcxxabi under linux as much as possible to make it easier to code with the latest C++ standard, but since I began to customize my environment just a little bit I noticed this kind of disproportions between gcc and clang, and I don't think that a compiler should adopt a building system like this one; a compiler it's a tool that is way too important to leave some space to some undefined and "unclear" steps as the ones that  I'm experiencing right now, even more true if this tools includes an external dependency or external project/s that play/s a big role in it/them like Python 2 in this case does.

If you know how to fix even 1 of this issues or if I'm wrong please let me know.

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140419/3d8d7f5d/attachment.html>