[PATCH] D51465: Revamp test-suite documentation

[Matthias Braun via Phabricator via llvm-commits]

MatzeB marked 2 inline comments as done.
MatzeB added inline comments.


================
Comment at: docs/TestSuiteGuide.md:34
+3. Create a build directory and use CMake to configure the suite. The
+   `CMAKE_C_COMPILER` option can be used to test a custom clang build.
+   The cache file provides a typical build configuration:
----------------
homerdin wrote:
> MatzeB wrote:
> > homerdin wrote:
> > > Do we want to mention that the test-suite should be able to build with other compilers?
> > Do you know whether the test-suite actually works with non-clang compilers? I can imagine it mostly works, but I also wouldn't be surprised if some clang specific flag or extension is used in few of the benchmarks...
> > 
> > I changed the wording to "custom compiler" but will not explicitely call out support for non-clang compilers for now.
> > 
> As you suspect it mostly works but gets some errors.  Also after fixing the errors you still get some misleading performance results.  Some tests use clang specific pragmas and one test just prints the reference output if you are not using clang.
> 
> The language led me to believe the test-suite is only for clang.  I had the impression the test-suite was not meant to be specific to clang (though its buggy on other compilers).
> 
> I'm good with the wording as is and agree with not explicitly calling out support for non-clang compilers.
I'd accept patches to the test-suite that disable clang specific benchmarks if someone cares enough to make them :)

(Should be possible to do in a similar way like `TEST_SUITE_BENCHMARKING_ONLY` disables programs unsuitable for performance testing)


Repository:
  rL LLVM

https://reviews.llvm.org/D51465