[cfe-dev] Expanding the Clang Status page to report the status of several OSes and projects

[Ruben Van Boxem]

2011/8/1 Douglas Gregor <dgregor at apple.com>:
>
> On Aug 1, 2011, at 4:37 AM, Ruben Van Boxem wrote:
>
>> Hi,
>>
>> I just noticed that the table listing projects and Clang compatibility
>> was recently removed. This idea still has a lot of merit, and has been
>> breeding in my mind for several days.
>>
>> I would like to provide test result data for several popular and high
>> quality open source libraries that provide an extensive test suite so
>> the compiled result is strictly analysed. I would do this on Windows
>> (mingw-w64 CRT+ GCC 4.6 libstdc++ and linker), but I would like this
>> to be more than a single email to the mailing list.
>>
>> To this effect, I thought that perhaps expanding the first table that
>> was on http://clang.llvm.org/cxx_status.html could be expanded
>> significantly, in vertical (projects) and horizontal
>> (platforms=OS+arch(+crt in the case of Windows)).
>>
>> Although I believe that Clang is production ready on Mac OS, it is far
>> from my elementary trust in the Windows MinGW world. Therefore, I
>> think it is useful to keep track of wide-scale progress in a
>> centralized place, because I feel the bugtracker doesn't get enough
>> attention (at least the problems I am experiencing ;-)).
>
> The main problem with the Windows MinGW world is that there are far too few LLVM/Clang developers who actively work on MinGW. Mac OS, Linux, and FreeBSD (to name a few) have a number of LLVM/Clang developers who made Clang work well on that platform and have kept it that way. We need more interest from the MinGW world for that to happen there.

I understand completely. The GCC world has the same problem...
unfortunately for the users. Note that MinGW shouldn't differ all that
much from MSVC for Clang. The ABI of the resulting objects is (should
be) the same. I just hope that the *-*-mingw* triplets aren't being
forgotten in important parts of the object file creation process. I
hope that at least the developers working on Windows support are aware
of where MSVC and MinGW agree on things.

>
>> Concretely, I suggest a larger table with colors like the C++0x table,
>> with red = doesn't build, orange/yellow: builds, but tests fail due to
>> a problem in clang (which also means that the tests pass for GCC), and
>> green is that it builds and all tests pass without a hitch. Links
>> could be placed to bug reports about the issue, where detailed
>> information for interested developers could be placed.
>>
>> The libraries I suggest that should be used as a "quality mark" are
>> (more are of course welcome!):
>>
>> Qt: Huge GUI and core library framework. High quality code that is
>> extremely cross-platform. Has an internal test suite, but I haven't
>> yet figured out how to run it (I can't as the build fails on Windows)
>> wxWidgets: that other tadbit less huge GUI and core framework. Also
>> high quality code.
>> GSL: GNU Scientific library: comprehensive test suite.
>> FFTW: Known around the world, comprehensive test suite
>> Eigen: idem
>> Flac: idem
>> GMP, MPFR, MPC, PPL, CLooG: very fragile and comprehensive libraries.
>> Also all feature a nice test suite.
>> Perhaps all the open source graphics libraries (Tiff, png, jpeg, turbojpeg...)
>>
>> What do you guys think? I do not know HTML, so the table editing would
>> have to be almost trivial for me to get it done :s. I can test
>> regularly and see if certain revisions fix the problem (on Windows).
>
>
> The problem with putting these tables up on a web page is that they get out of date very, very quickly. So, for them to be useful, they really need to be updated automatically. Then best way to do that, in my mind, would be to set up a buildbot that tests these projects. Bring the buildbot online when one of the projects starts working on that platform, and keep it running: the buildbot will tell us when things are broken, so we won't regress.

I understand, that's probably the reason it was removed in the first
place. The buildbot is an awesome idea, and LLVM/Clang wouldn't be the
first Apple sponsored project to have every commit go through a
build+test project (yes, I'm looking at you, WebKit). This kind of
large-scale infrastructure would have to be LLVM-wide to be effective,
as Clang and LLVM development are intertwined.

Are there any plans for such a buildbot and continuous testing infrastructure?

Ruben