[llvm-dev] Regression tests

Nemanja Ivanovic via llvm-dev llvm-dev at lists.llvm.org
Wed Jan 3 08:00:17 PST 2018 

Re-adding the dev list. It's best to always keep the discussions on the
list.
Don't worry about the length of the message, you need to communicate the
information that is important to describe your problem.

So I am not the best person to answer this question since I don't
build/test libcxx or libcxxabi, but I'm sure there are plenty of people on
the list that are quite knowledgeable in this regard.
This appears to me like some kind of an issue with how you configured
things with CMake. But I don't know what is required for these libraries.

On the question of whether it makes sense that lit uses gcc for these
tests, I think that makes sense. After all, you're trying to test the
libraries in these tests, so you want to build the tests with a stable
[system] compiler. Tests for clang will use the clang you just built. Just
like opt/llc tests will use the opt/llc you just built respectively.

Sorry I couldn't be of more help and I hope someone chimes in with the
correct solution soon.

On Wed, Jan 3, 2018 at 4:48 PM, Son Tuan VU <sontuan.vu119 at gmail.com> wrote:

> Hello again,
>
> After digging a little deeper to this, I guess this is the reason:
>
> lit.py: /dsk/l1/misc/vusontuan/llvm/projects/libcxxabi/test/lit.cfg:58:
> note: Using configuration variant: libcxxabi
> lit.py: /dsk/l1/misc/vusontuan/llvm/projects/libcxx/utils/libcxx/test/config.py:297:
> note: inferred use_system_cxx_lib as: None
> lit.py: /dsk/l1/misc/vusontuan/llvm/projects/libcxx/utils/libcxx/test/config.py:303:
> note: inferred with_availability as: False
> lit.py: /dsk/l1/misc/vusontuan/llvm/projects/libcxx/utils/libcxx/test/config.py:335:
> note: inferred use_clang_verify as: False
> lit.py: /dsk/l1/misc/vusontuan/llvm/projects/libcxx/utils/libcxx/test/config.py:524:
> note: inferred language dialect as: c++11
> lit.py: /dsk/l1/misc/vusontuan/llvm/projects/libcxx/utils/libcxx/test/config.py:426:
> note: inferred long_tests as: True
> lit.py: /dsk/l1/misc/vusontuan/llvm/projects/libcxx/utils/libcxx/test/config.py:156:
> note: *Using compiler: /usr/lib64/ccache/c++*
>
> Is it normal that lit is using gcc to run the regression tests? I guess
> not, but this also means that other (successful) tests case do not make
> sense, since all that they have tested is gcc. However, can you tell me how
> can I change this? I've looked at *config.py* but not sure how to replace
> c++ by clang :(
>
> Thank you again for your help!
>
>
> Son Tuan Vu
>
> 2018-01-03 10:41 GMT+01:00 Son Tuan VU <sontuan.vu119 at gmail.com>:
>
>> Hello Nemanja,
>>
>> Thank you for your reply. This was my first message in llvm-dev mailing
>> list so I was afraid that a too long message would bother everyone. Here's
>> an example of failed test (I am really sorry because it's a bit too
>> long...):
>>
>> ********************
>> FAIL: libc++abi :: catch_multi_level_pointer.pass.cpp (42355 of 43597)
>> ******************** TEST 'libc++abi :: catch_multi_level_pointer.pass.cpp'
>> FAILED ********************
>> Command: ['/usr/lib64/ccache/c++', '-o', '/dsk/l1/misc/vusontuan/llvm/b
>> uild/projects/libcxxabi/test/Output/catch_multi_level_pointer.pass.cpp.o',
>> '-x', 'c++', '/dsk/l1/misc/vusontuan/llvm/projects/libcxxabi/test/catch_multi_level_pointer.pass.cpp',
>> '-c', '-v', '-D_LIBCPP_DISABLE_AVAILABILITY', '-DLIBCXXABI_NO_TIMER',
>> '-funwind-tables', '-std=c++11', '-nostdinc++',
>> '-I/dsk/l1/misc/vusontuan/llvm/projects/libcxx/include',
>> '-I/dsk/l1/misc/vusontuan/llvm/projects/libcxxabi/include',
>> '-D__STDC_FORMAT_MACROS', '-D__STDC_LIMIT_MACROS',
>> '-D__STDC_CONSTANT_MACROS', '-I/dsk/l1/misc/vusontuan/llvm
>> /projects/libcxx/test/support', '-c', '&&', '/usr/lib64/ccache/c++',
>> '-o', '/dsk/l1/misc/vusontuan/llvm/build/projects/libcxxabi/test/O
>> utput/catch_multi_level_pointer.pass.cpp.exe',
>> '/dsk/l1/misc/vusontuan/llvm/build/projects/libcxxabi/test/O
>> utput/catch_multi_level_pointer.pass.cpp.o', '-v',
>> '-D_LIBCPP_DISABLE_AVAILABILITY', '-L/dsk/l1/misc/vusontuan/llvm/build/./lib',
>> '-Wl,-rpath,/dsk/l1/misc/vusontuan/llvm/build/./lib', '-nodefaultlibs',
>> '-lc++', '-lc++abi', '-lm', '-lgcc_s', '-lgcc', '-lpthread', '-lc',
>> '-lgcc_s', '-lgcc']
>> Exit Code: 1
>> Standard Error:
>> --
>> Utilisation des specs internes.
>> COLLECT_GCC=/usr/bin/c++
>> Target: x86_64-redhat-linux
>> Configuré avec: ../configure --prefix=/usr --mandir=/usr/share/man
>> --infodir=/usr/share/info --with-bugurl=http://bugzilla.
>> redhat.com/bugzilla --enable-bootstrap --enable-shared
>> --enable-threads=posix --enable-checking=release --with-system-zlib
>> --enable-__cxa_atexit --disable-libunwind-exceptions
>> --enable-gnu-unique-object --enable-linker-build-id
>> --with-linker-hash-style=gnu --enable-languages=c,c++,objc,
>> obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-initfini-array
>> --disable-libgcj --with-isl=/builddir/build/BUI
>> LD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/isl-install
>> --with-cloog=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x8
>> 6_64-redhat-linux/cloog-install --enable-gnu-indirect-function
>> --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
>> Modèle de thread: posix
>> gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC)
>> COLLECT_GCC_OPTIONS='-v' '-funwind-tables' '-std=c++11' '-D'
>> '_LIBCPP_DISABLE_AVAILABILITY' '-D' 'LIBCXXABI_NO_TIMER' '-nostdinc++' '-I'
>> '/dsk/l1/misc/vusontuan/llvm/projects/libcxx/include' '-I'
>> '/dsk/l1/misc/vusontuan/llvm/projects/libcxxabi/include' '-D'
>> '__STDC_FORMAT_MACROS' '-D' '__STDC_LIMIT_MACROS' '-D'
>> '__STDC_CONSTANT_MACROS' '-I' '/dsk/l1/misc/vusontuan/llvm/projects/libcxx/test/support'
>> '-c' '-o' '/dsk/l1/misc/vusontuan/llvm/build/projects/libcxxabi/test/O
>> utput/catch_multi_level_pointer.pass.cpp.o' '-shared-libgcc'
>> '-mtune=generic' '-march=x86-64'
>>  /usr/libexec/gcc/x86_64-redhat-linux/4.8.5/cc1plus -quiet -nostdinc++
>> -v -I /dsk/l1/misc/vusontuan/llvm/projects/libcxx/include -I
>> /dsk/l1/misc/vusontuan/llvm/projects/libcxxabi/include -I
>> /dsk/l1/misc/vusontuan/llvm/projects/libcxx/test/support -D_GNU_SOURCE
>> -D _LIBCPP_DISABLE_AVAILABILITY -D LIBCXXABI_NO_TIMER -D
>> __STDC_FORMAT_MACROS -D __STDC_LIMIT_MACROS -D __STDC_CONSTANT_MACROS
>> /dsk/l1/misc/vusontuan/llvm/projects/libcxxabi/test/catch_multi_level_pointer.pass.cpp
>> -quiet -dumpbase catch_multi_level_pointer.pass.cpp -mtune=generic
>> -march=x86-64 -auxbase-strip /dsk/l1/misc/vusontuan/llvm/bu
>> ild/projects/libcxxabi/test/Output/catch_multi_level_pointer.pass.cpp.o
>> -std=c++11 -version -funwind-tables -o /tmp/lit_tmp_eb4yOu/cc2UDAyE.s
>> GNU C++ (GCC) version 4.8.5 20150623 (Red Hat 4.8.5-11)
>> (x86_64-redhat-linux)
>>         compiled by GNU C version 4.8.5 20150623 (Red Hat 4.8.5-11), GMP
>> version 6.0.0, MPFR version 3.1.1, MPC version 1.0.1
>> heuristiques GGC: --param ggc-min-expand=100 --param
>> ggc-min-heapsize=131072
>> le répertoire « /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include-fixed »
>> est ignoré car inexistant
>> le répertoire « /usr/lib/gcc/x86_64-redhat-lin
>> ux/4.8.5/../../../../x86_64-redhat-linux/include » est ignoré car
>> inexistant
>> la recherche pour #include "..." débute ici :
>> la recherche pour #include <...> débute ici:
>>  /dsk/l1/misc/vusontuan/llvm/projects/libcxx/include
>>  /dsk/l1/misc/vusontuan/llvm/projects/libcxxabi/include
>>  /dsk/l1/misc/vusontuan/llvm/projects/libcxx/test/support
>>  /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include
>>  /usr/local/include
>>  /usr/include
>> Fin de la liste de recherche.
>> GNU C++ (GCC) version 4.8.5 20150623 (Red Hat 4.8.5-11)
>> (x86_64-redhat-linux)
>>         compiled by GNU C version 4.8.5 20150623 (Red Hat 4.8.5-11), GMP
>> version 6.0.0, MPFR version 3.1.1, MPC version 1.0.1
>> heuristiques GGC: --param ggc-min-expand=100 --param
>> ggc-min-heapsize=131072
>> Compiler executable checksum: 10b61985772f737334cdfc042f8e6c1c
>> COLLECT_GCC_OPTIONS='-v' '-funwind-tables' '-std=c++11' '-D'
>> '_LIBCPP_DISABLE_AVAILABILITY' '-D' 'LIBCXXABI_NO_TIMER' '-nostdinc++' '-I'
>> '/dsk/l1/misc/vusontuan/llvm/projects/libcxx/include' '-I'
>> '/dsk/l1/misc/vusontuan/llvm/projects/libcxxabi/include' '-D'
>> '__STDC_FORMAT_MACROS' '-D' '__STDC_LIMIT_MACROS' '-D'
>> '__STDC_CONSTANT_MACROS' '-I' '/dsk/l1/misc/vusontuan/llvm/projects/libcxx/test/support'
>> '-c' '-o' '/dsk/l1/misc/vusontuan/llvm/build/projects/libcxxabi/test/O
>> utput/catch_multi_level_pointer.pass.cpp.o' '-shared-libgcc'
>> '-mtune=generic' '-march=x86-64'
>>  as -v -I /dsk/l1/misc/vusontuan/llvm/projects/libcxx/include -I
>> /dsk/l1/misc/vusontuan/llvm/projects/libcxxabi/include -I
>> /dsk/l1/misc/vusontuan/llvm/projects/libcxx/test/support --64 -o
>> /dsk/l1/misc/vusontuan/llvm/build/projects/libcxxabi/test/Ou
>> tput/catch_multi_level_pointer.pass.cpp.o /tmp/lit_tmp_eb4yOu/cc2UDAyE.s
>> Version de l'assembleur GNU 2.25.1 (x86_64-redhat-linux) utilisant la
>> version BFD version 2.25.1-22.base.el7
>> COMPILER_PATH=/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/:/
>> usr/libexec/gcc/x86_64-redhat-linux/4.8.5/:/usr/libexec/gcc/
>> x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/
>> :/usr/lib/gcc/x86_64-redhat-linux/
>> LIBRARY_PATH=/usr/lib/gcc/x86_64-redhat-linux/4.8.5/:/usr/li
>> b/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/:/lib/../
>> lib64/:/usr/lib/../lib64/:/usr/lib/gcc/x86_64-redhat-linux/
>> 4.8.5/../../../:/lib/:/usr/lib/
>> COLLECT_GCC_OPTIONS='-v' '-funwind-tables' '-std=c++11' '-D'
>> '_LIBCPP_DISABLE_AVAILABILITY' '-D' 'LIBCXXABI_NO_TIMER' '-nostdinc++' '-I'
>> '/dsk/l1/misc/vusontuan/llvm/projects/libcxx/include' '-I'
>> '/dsk/l1/misc/vusontuan/llvm/projects/libcxxabi/include' '-D'
>> '__STDC_FORMAT_MACROS' '-D' '__STDC_LIMIT_MACROS' '-D'
>> '__STDC_CONSTANT_MACROS' '-I' '/dsk/l1/misc/vusontuan/llvm/projects/libcxx/test/support'
>> '-c' '-o' '/dsk/l1/misc/vusontuan/llvm/build/projects/libcxxabi/test/O
>> utput/catch_multi_level_pointer.pass.cpp.o' '-shared-libgcc'
>> '-mtune=generic' '-march=x86-64'
>> Utilisation des specs internes.
>> COLLECT_GCC=/usr/bin/c++
>> COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.
>> 8.5/lto-wrapper
>> Target: x86_64-redhat-linux
>> Configuré avec: ../configure --prefix=/usr --mandir=/usr/share/man
>> --infodir=/usr/share/info --with-bugurl=http://bugzilla.
>> redhat.com/bugzilla --enable-bootstrap --enable-shared
>> --enable-threads=posix --enable-checking=release --with-system-zlib
>> --enable-__cxa_atexit --disable-libunwind-exceptions
>> --enable-gnu-unique-object --enable-linker-build-id
>> --with-linker-hash-style=gnu --enable-languages=c,c++,objc,
>> obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-initfini-array
>> --disable-libgcj --with-isl=/builddir/build/BUI
>> LD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/isl-install
>> --with-cloog=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x8
>> 6_64-redhat-linux/cloog-install --enable-gnu-indirect-function
>> --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
>> Modèle de thread: posix
>> gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC)
>> COMPILER_PATH=/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/:/
>> usr/libexec/gcc/x86_64-redhat-linux/4.8.5/:/usr/libexec/gcc/
>> x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/
>> :/usr/lib/gcc/x86_64-redhat-linux/
>> LIBRARY_PATH=/usr/lib/gcc/x86_64-redhat-linux/4.8.5/:/usr/li
>> b/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/:/lib/../
>> lib64/:/usr/lib/../lib64/:/usr/lib/gcc/x86_64-redhat-linux/
>> 4.8.5/../../../:/lib/:/usr/lib/
>> COLLECT_GCC_OPTIONS='-o' '/dsk/l1/misc/vusontuan/llvm/b
>> uild/projects/libcxxabi/test/Output/catch_multi_level_pointer.pass.cpp.exe'
>> '-v' '-D' '_LIBCPP_DISABLE_AVAILABILITY' '-L/dsk/l1/misc/vusontuan/llvm/build/./lib'
>> '-nodefaultlibs' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
>>  /usr/libexec/gcc/x86_64-redhat-linux/4.8.5/collect2 --build-id
>> --no-add-needed --eh-frame-hdr --hash-style=gnu -m elf_x86_64
>> -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o
>> /dsk/l1/misc/vusontuan/llvm/build/projects/libcxxabi/test/Ou
>> tput/catch_multi_level_pointer.pass.cpp.exe
>> /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crt1.o
>> /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crti.o
>> /usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtbegin.o
>> -L/dsk/l1/misc/vusontuan/llvm/build/./lib -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5
>> -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64
>> -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../..
>> /dsk/l1/misc/vusontuan/llvm/build/projects/libcxxabi/test/Ou
>> tput/catch_multi_level_pointer.pass.cpp.o -rpath
>> /dsk/l1/misc/vusontuan/llvm/build/./lib -lc++ -lc++abi -lm -lgcc_s -lgcc
>> -lpthread -lc -lgcc_s -lgcc /usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtend.o
>> /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crtn.o
>> /dsk/l1/misc/vusontuan/llvm/build/projects/libcxxabi/test/Ou
>> tput/catch_multi_level_pointer.pass.cpp.o: dans la fonction «
>> std::__1::ostreambuf_iterator<char, std::__1::char_traits<char> >
>> std::__1::__pad_and_output<char, std::__1::char_traits<char>
>> >(std::__1::ostreambuf_iterator<char, std::__1::char_traits<char> >,
>> char const*, char const*, char const*, std::__1::ios_base&, char) »:
>> catch_multi_level_pointer.pass.cpp:(.text._ZNSt3__116__pad_
>> and_outputIcNS_11char_traitsIcEEEENS_19ostreambuf_iteratorIT
>> _T0_EES6_PKS4_S8_S8_RNS_8ios_baseES4_[_ZNSt3__116__pad_and_
>> outputIcNS_11char_traitsIcEEEENS_19ostreambuf_iteratorIT_T0_
>> EES6_PKS4_S8_S8_RNS_8ios_baseES4_]+0x13f): référence indéfinie vers «
>> std::__1::basic_string<char, std::__1::char_traits<char>,
>> std::__1::allocator<char> >::basic_string(unsigned long, char) »
>> /usr/bin/ld: /dsk/l1/misc/vusontuan/llvm/build/projects/libcxxabi/test/Ou
>> tput/catch_multi_level_pointer.pass.cpp.exe: symbole caché «
>> _ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEC1Emc »
>> n'est pas défini
>> /usr/bin/ld: édition de lien finale en échec: Mauvaise valeur
>> collect2: erreur: ld a retourné 1 code d'état d'exécution
>> --
>>
>> Can you tell me what's going wrong here please?
>>
>> Thank you for your help, and Happy new year btw!
>>
>> Son Tuan Vu
>>
>> Son Tuan Vu
>>
>> 2018-01-02 17:56 GMT+01:00 Nemanja Ivanovic <nemanja.i.ibm at gmail.com>:
>>
>>> Hi Son Tuan,
>>>
>>> have you been able to resolve the unexpected failures? I imagine this is
>>> something unique to your environment since the build bots don't have those
>>> failures. In any case, if you still have issues with unmodified LLVM
>>> causing lit failures, you should provide more information as to what is
>>> failing and what the error messages are, etc.
>>>
>>> On Tue, Dec 26, 2017 at 4:24 PM, Son Tuan VU via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> Hello,
>>>>
>>>> I have just built LLVM with CMake by following these instructions
>>>> https://llvm.org/docs/GettingStarted.html#getti
>>>> ng-started-quickly-a-summary and I am trying to run the regression
>>>> tests using "make check-all" but there are some unexpected failures in
>>>> libcxx and libcxxabi. Here's the test log:
>>>>
>>>>   Expected Passes    : 40745
>>>>   Expected Failures  : 258
>>>>   Unsupported Tests  : 1594
>>>>   Unexpected Failures: 1000
>>>>
>>>>  Are these failures really "unexpected", because I haven't made any
>>>> changes to LLVM?
>>>>
>>>> Thank you for your help!
>>>>
>>>>
>>>> Son Tuan Vu
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180103/3af6ad81/attachment.html>

More information about the llvm-dev mailing list