[LLVMdev] compiler-rt CMake build

Thu Feb 6 05:12:41 PST 2014

Hi Greg,

On Thu, Feb 6, 2014 at 12:55 AM, Greg Fitzgerald <garious at gmail.com> wrote:

> The trouble is that the sanitizer tests are not really compiler-rt
> tests.  The original compiler-rt tests are, but not the sanitizer
> tests.  The sanitizer tests are "compiler-rt tests via clang".  The
> sanitizer runtime is a C++ library requiring only a C++ compiler to
> build it (and to *unit* test it).  The ASan lit tests, however,
> require clang, because they are testing those C++ functions in the
> context of a particular C/C++ compiler's instrumentation.  I believe
> these tests should not be called "compiler-rt tests".  They are clang
> tests.

Well, not really. I agree that there is a significant difference between
plain compiler-rt and
sanitizers:
1) compiler-rt is more or less a regular C++ library, with the exception
that a compiler
sometimes creates a calls into compiler-rt. compiler-rt can be built with
any C compiler,
and can be fairly easy unit-tested.
2) sanitizer libraries are special in a way that they more heavily rely on
compiler instrumentation and
code transformations. sanitizers can be built with any C++ compiler, but
"unit-testing" is far
less important for them, and although we have regular unit tests (that
don't expect
instrumentation) for some sanitizers, we are really more concentrated on
testing
compiler+run-time together.

But our tests (call them "integration tests" or "toolchain tests") are not
Clang-specific - they
should work (and we'd love to make them work) with GCC version of ASan as
well. So, in
a perfect world, we would have a "sanitizer-tests" test suite, which we can
run with
any compiler we specify at configuration time, be it just-built Clang, or
GCC [1].

>  In fact, they are just-built-clang tests.  They don't, for
> example, verify the C++ library is useful to another compiler, another
> version of clang, or simply as a C++ library.  They are integration
> tests (tests clang+asan), as opposed to unit tests (tests only the
> asan runtime).
>
> I understand that the integration tests are more relevant to the
> sanitizers than they are clang, and so it make sense to me that they
> reside in the compiler-rt repo alongside the sanitizers.  But rather
> than build them in the compiler-rt build, I think the directory should
> be exported so that clang can optionally import it.  The clang build
> can then add those tests to the clang tests suite, and install the
> runtime in whatever location is most appropriate for clang.
>
> Why is this important now?  The CMake developers are trying to make
> the LLVM toolchain easy to build for package managers.  Compiler-rt's
> dependency on the just-built-clang makes that difficult.  Without that
> dependency, Compiler-rt could be built and tested against only an LLVM
> package.

Why does compiler-rt needs to be tested against LLVM package? All
compiler-rt runtimes can be built and unit-tested with any C/C++ host
compiler,
if they don't depend on just-built-Clang. Or do you mean LLVM testing
utilities (llvm-lit, FileCheck) here? If yes, maybe it makes sense to move
*them*
to another package as well?

>  And likewise, a Clang package would depend on Compiler-rt
> (and do further testing on compiler-rt with those lit tests).

That's a bit weird - what if compiler-rt is not supported on a platform,
where Clang is supported?

>  But
> with the dependency on the just-built-clang, there needs to be an
> additional top-level package, let's call it ClangAndFriends.
> ClangAndFriends would depend on Clang and Compiler-rt.  Compiler-rt
> would depend on the Clang package as well.  Once both are built,
> ClangAndFriends would copy the Clang install directory and then copy
> the Compiler-rt install directory into it.

Can we instead make a package SanitizerTests (or whatever)? It won't even
need to have any dependencies - instead we should make it configurable -
SanitizerTests should take as a parameter a path to the compiler, and use
it to run the lit-tests and GTest-style unit tests that require
instrumentation.

>  This extra package and
> extra copy would be unnecessary if the Compiler-rt build dropped its
> dependency on the just-built-clang.  After testing the sanitizers, the
> Clang package would copy the runtime into its own install directory.
>
> So what next?  Wait for Brad King's LLVM patches to be integrated.
> Then we tweak the compiler-rt build to optionally build against the
> LLVM install directory (instead of in the context of the LLVM source
> directory).

Please note that it makes a lot of sense to built compiler-rt (and
sanitizers) with just-built
Clang. In fact, even though we should support building it with another
compilers (gcc, MSVC),
using just-built-Clang should be a default scenario, like it is in
configure+make build.
In this way we'll be able to build compiler-rt libs for platforms that
just-build
Clang can target, and host compiler probably can't...

>  Next, we use CMake to export the Sanitizer lit tests.
> Lastly, Clang optionally imports the test suite and adds the sanitizer
> runtime to its install directory.  What do you think?  Squeaky clean?
>
> Thanks,
> Greg
>

[1] Except that we frequently add new features to sanitizers and modify
compiler<->runtime interface, so the test-suite is kind of tied to the ToT
Clang.
But, at the moment we merge Sanitizer runtime into GCC tree at revison X,
we can cut off Sanitizer testsuite at revision X as well, and expect it to
work.

>
>
> On Wed, Feb 5, 2014 at 12:55 AM, Alexey Samsonov <samsonov at google.com>
> wrote:
> >
> > On Tue, Feb 4, 2014 at 10:17 PM, Greg Fitzgerald <garious at gmail.com>
> wrote:
> >>
> >> Regarding Brad King's CMake patches for LLVM:
> >>
> >>   http://thread.gmane.org/gmane.comp.compilers.llvm.cvs/173517
> >>
> >> If compiler-rt could follow suit, the packaging for the toolchain
> >> becomes a nice, clean package-manager-friendly DAG.  Currently, the
> >> compiler-rt CMake build depends on the just-built-clang to run its
> >> test suite.  It's more intuitive to me that either:
> >>
> >>  1) compiler-rt's sanitizer tests would live inside the clang project
> >>  2) compiler-rt would export a test suite that clang could optionally
> >> import
> >>
> >> The former is simpler, but the latter allows the compiler-rt
> >> developers to retain control over those tests, which are typically
> >> updated alongside changes to compiler-rt's sanitizers.  Thoughts?
> >
> >
> > I'm pretty sure the compiler-rt tests should live in the compiler-rt
> > project. You're right:
> > these tests should generally be updated in sync with the runtimes. I'm
> fine
> > with creating
> > a large test suite comprising all individual tool-specific test suites we
> > have now. I'm not
> > sure how we can "export" this to clang, though, and why the current
> > situation is bad.
> >
> >
> >>
> >>
> >> Thanks,
> >> Greg
> >
> >
> >
> >
> > --
> > Alexey Samsonov, MSK
>

-- 
Alexey Samsonov, MSK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140206/4efb680b/attachment.html>