[cfe-dev] [3.8 Release] RC1 has been tagged

[Dimitry Andric via cfe-dev]

On 20 Jan 2016, at 18:23, Hans Wennborg <hans at chromium.org> wrote:
> 
> On Wed, Jan 20, 2016 at 5:25 AM, Dimitry Andric <dimitry at andric.com> wrote:
>> Unfortunately I'm having lots of trouble with rc1 at this point:
>> * libcxxabi can't build, because it requires unwind.h, which we do not yet have on FreeBSD 10.x (Ed Maste is working on it for 11.x, but that is not ready for general consumption).
>> * The test-release.sh script has no option to disable only libcxxabi, you can only disable libcxx, libcxxabi and libunwind together (maybe this can be improved)
> 
> Yes, I'd be happy to take a patch for this, or I suppose you could
> just hack it out locally when building.

It's not terribly important right now, as libcxx isn't succeeding its tests anyway.  I can locally hack it in, but I hope I won't forget it later. :)


>> * Last time I hand-built libcxx, it still had a lot of test failures in the locale parts, but I haven't had time to investigate.
> 
> Did we have that for 3.7 too?

With 3.7, we used autoconf builds for FreeBSD, and that skipped libcxx altogether.  I did a few builds by hand, and I remember I saw similar failures.  This is just something that nobody (from FreeBSD) has seriously looked at, and I never had the time for it.  There are only so much hours in a day...

For example, recently with trunk r256945, I saw these:

Failing Tests (39):
    libc++ :: std/depr/depr.c.headers/stddef_h.pass.cpp
    libc++ :: std/depr/depr.c.headers/wchar_h.pass.cpp
    libc++ :: std/language.support/support.types/max_align_t.pass.cpp
    libc++ :: std/localization/locale.categories/category.collate/locale.collate.byname/compare.pass.cpp
    libc++ :: std/localization/locale.categories/category.collate/locale.collate.byname/transform.pass.cpp
    libc++ :: std/localization/locale.categories/category.ctype/locale.ctype.byname/tolower_1.pass.cpp
    libc++ :: std/localization/locale.categories/category.ctype/locale.ctype.byname/tolower_many.pass.cpp
    libc++ :: std/localization/locale.categories/category.ctype/locale.ctype.byname/toupper_1.pass.cpp
    libc++ :: std/localization/locale.categories/category.ctype/locale.ctype.byname/toupper_many.pass.cpp
    libc++ :: std/localization/locale.categories/category.monetary/locale.money.get/locale.money.get.members/get_long_double_fr_FR.pass.cpp
    libc++ :: std/localization/locale.categories/category.monetary/locale.money.get/locale.money.get.members/get_long_double_ru_RU.pass.cpp
    libc++ :: std/localization/locale.categories/category.monetary/locale.money.get/locale.money.get.members/get_long_double_zh_CN.pass.cpp
    libc++ :: std/localization/locale.categories/category.monetary/locale.money.put/locale.money.put.members/put_long_double_fr_FR.pass.cpp
    libc++ :: std/localization/locale.categories/category.monetary/locale.money.put/locale.money.put.members/put_long_double_ru_RU.pass.cpp
    libc++ :: std/localization/locale.categories/category.monetary/locale.money.put/locale.money.put.members/put_long_double_zh_CN.pass.cpp
    libc++ :: std/localization/locale.categories/category.monetary/locale.moneypunct.byname/curr_symbol.pass.cpp
    libc++ :: std/localization/locale.categories/category.monetary/locale.moneypunct.byname/grouping.pass.cpp
    libc++ :: std/localization/locale.categories/category.monetary/locale.moneypunct.byname/neg_format.pass.cpp
    libc++ :: std/localization/locale.categories/category.monetary/locale.moneypunct.byname/pos_format.pass.cpp
    libc++ :: std/localization/locale.categories/category.monetary/locale.moneypunct.byname/thousands_sep.pass.cpp
    libc++ :: std/localization/locale.categories/category.time/locale.time.get.byname/get_date.pass.cpp
    libc++ :: std/localization/locale.categories/category.time/locale.time.get.byname/get_date_wide.pass.cpp
    libc++ :: std/localization/locale.categories/category.time/locale.time.get.byname/get_one.pass.cpp
    libc++ :: std/localization/locale.categories/category.time/locale.time.get.byname/get_one_wide.pass.cpp
    libc++ :: std/localization/locale.categories/category.time/locale.time.put.byname/put1.pass.cpp
    libc++ :: std/localization/locale.categories/facet.numpunct/locale.numpunct.byname/grouping.pass.cpp
    libc++ :: std/localization/locale.categories/facet.numpunct/locale.numpunct.byname/thousands_sep.pass.cpp
    libc++ :: std/re/re.alg/re.alg.match/basic.pass.cpp
    libc++ :: std/re/re.alg/re.alg.match/ecma.pass.cpp
    libc++ :: std/re/re.alg/re.alg.match/extended.pass.cpp
    libc++ :: std/re/re.alg/re.alg.search/awk.pass.cpp
    libc++ :: std/re/re.alg/re.alg.search/basic.pass.cpp
    libc++ :: std/re/re.alg/re.alg.search/ecma.pass.cpp
    libc++ :: std/re/re.alg/re.alg.search/extended.pass.cpp
    libc++ :: std/re/re.traits/lookup_collatename.pass.cpp
    libc++ :: std/re/re.traits/transform_primary.pass.cpp
    libc++ :: std/re/re.traits/translate_nocase.pass.cpp
    libc++ :: std/strings/string.conversions/stof.pass.cpp
    libc++ :: std/utilities/meta/meta.trans/meta.trans.other/aligned_storage.pass.cpp

Individual failures were typically of the form:

FAIL: libc++ :: std/localization/locale.categories/category.monetary/locale.moneypunct.byname/thousands_sep.pass.cpp (27654 of 30312)
******************** TEST 'libc++ :: std/localization/locale.categories/category.monetary/locale.moneypunct.byname/thousands_sep.pass.cpp' FAILED ********************
Compiled With: ['/usr/bin/clang++', '-o', '/home/dim/obj/llvm-256945-trunk-freebsd11-i386-ninja-rel-1/projects/libcxx/test/std/localization/locale.categories/category.monetary/locale.moneypunct.byname/Output/thousands_sep.pass.cpp.o', '-x', 'c++', '/share/dim/
src/llvm/trunk/projects/libcxx/test/std/localization/locale.categories/category.monetary/locale.moneypunct.byname/thousands_sep.pass.cpp', '-c', '-v', '-std=c++1z', '-nostdinc++', '-I/share/dim/src/llvm/trunk/projects/libcxx/test/support', '-include', '/share/
dim/src/llvm/trunk/projects/libcxx/test/support/nasty_macros.hpp', '-I/share/dim/src/llvm/trunk/projects/libcxx/include', '&&', '/usr/bin/clang++', '-o', '/home/dim/obj/llvm-256945-trunk-freebsd11-i386-ninja-rel-1/projects/libcxx/test/std/localization/locale.c
ategories/category.monetary/locale.moneypunct.byname/Output/thousands_sep.pass.cpp.exe', '/home/dim/obj/llvm-256945-trunk-freebsd11-i386-ninja-rel-1/projects/libcxx/test/std/localization/locale.categories/category.monetary/locale.moneypunct.byname/Output/thous
ands_sep.pass.cpp.o', '-v', '-nodefaultlibs', '-L/home/dim/obj/llvm-256945-trunk-freebsd11-i386-ninja-rel-1/lib', '-Wl,-rpath,/home/dim/obj/llvm-256945-trunk-freebsd11-i386-ninja-rel-1/lib', '-lc++', '-lc', '-lm', '-lpthread', '-lgcc_s', '-lcxxrt']
Command: ['/home/dim/obj/llvm-256945-trunk-freebsd11-i386-ninja-rel-1/projects/libcxx/test/std/localization/locale.categories/category.monetary/locale.moneypunct.byname/Output/thousands_sep.pass.cpp.exe']
Exit Code: -6
Standard Error:
--
Assertion failed: (f.thousands_sep() == ' '), function main, file /share/dim/src/llvm/trunk/projects/libcxx/test/std/localization/locale.categories/category.monetary/locale.moneypunct.byname/thousands_sep.pass.cpp, line 103.
--

So there is probably quite some work to be done to get them all sorted out.  (And localization stuff tends to be a tricky area, in my experience.)


>> * OpenMP does not support i386-freebsd, so I have to disable it there
> 
> That's perfectly fine. Including OpenMP by default is new for this
> release, so if it causes any problems, just exclude it.

Okay.  That said, in the past, I have built OpenMP for x86_64, and after some twiddling, I remember that (most of) the tests ran correctly.  I will at least try to do builds for x86_64.


>> * Last but not least: the host compiler on FreeBSD 10.x is clang 3.4.1 (the last version that can build without C++11 support), and it crashes with a segfault during building of CGBlocks.cpp.  I'll need to find some way to work around this failure, since we cannot upgrade the compiler easily on FreeBSD 10.x.
> 
> This sounds like the biggest problem. Is there a PR for the crash? I
> suppose the alternatives are either to try not to tickle the crash in
> our source, or fixing the 3.4.1 compiler?

There is no PR, but I did a bisection, and the crash turns out to be fixed by r210467 ("[DAG] Expose NoSignedWrap, NoUnsignedWrap and Exact flags to SelectionDAG.") by Andrea DiBiagio.

I do not know exactly *why* this fixes it, but this particular revision is rather tricky to merge to a 3.4 tree, since it touches quite some parts of SelectionDAG, and there were lots of stylistic changes and cleanups in this area around (or before) that time.

I'm currently attempting to either modify the fix so it applies directly on top of a 3.4 tree, or adding the other changes needed to be able to apply it.  But it isn't trivial, unfortunately.


>> I also had to hack the test-release.sh script to fix a number of problems that I encountered during the 3.7.1 release, but haven't gotten to upstreaming them.  E.g. the way the source code is checked out with symlinks all over the place does not work, and I need to add a custom patch to clang-tools-extra to make the tests succeed, because there is a race condition in the Makefile.
> 
> :-( What's not working with the symlinks? Please send patches. I'm
> sorry for the trouble here.

As I should have done earlier (after 3.7.1 in fact :), I committed the fix for clang-tools-extra's test Makefile in r258327.  This is a trivial fix, so please merge it to release_38.

As to the symlinks, the test-release.sh script originally checks out the sources in parallel directories, e.g.:

llvm.src
cfe.src
compiler-rt.src

and so on.  Within llvm.src, symlinks are made to point to each of these.  For some reason, on FreeBSD, this causes .cpp files under llvm.src/tools/clang/tools/extra to not be able to find their include files, and my log files show the following kind of errors:

/home/dim/llvm-3.7.0/rc3/llvm.src/tools/clang/tools/extra/clang-apply-replacements/lib/Tooling/../../include/clang-apply-replacements/Tooling/ApplyReplacements.h:19:10: fatal error: 'clang/Tooling/Refactoring.h' file not found
#include "clang/Tooling/Refactoring.h"
         ^
1 error generated.

I remember trying lots of things to make this work, but failing.  Apparently there is some issue with following a double symlink path, e.g. llvm.src/tools/clang is a symlink to ../../cfe.src, while llvm.src/tools/clang/tools/extra (which really is under ../../cfe.src/tools) is a symlink to ../../clang-tools-extra.src.

The way I fixed this during the 3.7 test phase, is by changing test-release.sh so it exports directly into these locations:

# Exporting llvm 3.7.0-rc3 sources to llvm.src
# Exporting cfe 3.7.0-rc3 sources to llvm.src/tools/clang
# Exporting clang-tools-extra 3.7.0-rc3 sources to llvm.src/tools/clang/tools/extra
# Exporting compiler-rt 3.7.0-rc3 sources to llvm.src/projects/compiler-rt
# Exporting libcxx 3.7.0-rc3 sources to llvm.src/projects/libcxx
# Exporting libcxxabi 3.7.0-rc3 sources to llvm.src/projects/libcxxabi
# Exporting libunwind 3.7.0-rc3 sources to llvm.src/projects/libunwind
# Exporting test-suite 3.7.0-rc3 sources to llvm.src/projects/test-suite

This is probably not so handy if you want to create tarballs of the individual components, though.

-Dimitry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160120/04ecb2c3/attachment.sig>