[all-commits] [llvm/llvm-project] ad7e12: [Compiler-rt] Distinguish between testing just bui...

[Dan Liew via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: ad7e12226f6b1ebf91511899016cdfb29aa8919e
      https://github.com/llvm/llvm-project/commit/ad7e12226f6b1ebf91511899016cdfb29aa8919e
  Author: Dan Liew <dliew at apple.com>
  Date:   2021-05-14 (Fri, 14 May 2021)

  Changed paths:
    M compiler-rt/test/CMakeLists.txt
    M compiler-rt/test/builtins/Unit/lit.site.cfg.py.in
    M compiler-rt/test/lit.common.cfg.py
    M compiler-rt/test/lit.common.configured.in

  Log Message:
  -----------
  [Compiler-rt] Distinguish between testing just built runtime libraries and the libraries shipped with the compiler.

The path to the runtime libraries used by the compiler under test
is normally identical to the path where just built libraries are
created. However, this is not necessarily the case when doing standalone
builds. This is because the external compiler used by tests may choose
to get its runtime libraries from somewhere else.

When doing standalone builds there are two types of testing we could be
doing:

* Test the just built runtime libraries.
* Test the runtime libraries shipped with the compile under test.

Both types of testing are valid but it confusingly turns out compiler-rt
actually did a mixture of these types of testing.

* The `test/builtins/Unit/` test suite always tested the just built runtime
  libraries.
* All other testsuites implicitly use whatever runtime library the
  compiler decides to link.

There is no way for us to infer which type of testing the developer
wants so this patch introduces a new
`COMPILER_RT_TEST_STANDALONE_BUILD_LIBS` CMake
option which explicitly declares which runtime libraries should be
tested. If it is `ON` then the just built libraries should be tested,
otherwise the libraries in the external compiler should be tested.

When testing starts the lit test suite queries the compiler used for
testing to see where it will get its runtime libraries from.  If these
paths are identical no action is taken (the common case). If the paths
are not identical then we check the value of
`COMPILER_RT_TEST_STANDALONE_BUILD_LIBS` (progated into the config as
`test_standalone_build_libs`) and check if the test suite supports testing in the
requested configuration.

* If we want to test just built libs and the test suite supports it
  (currently only `test/builtins/Unit`) then testing proceeds without any changes.
* If we want to test the just built libs and the test suite doesn't
  support it we emit a fatal error to prevent the developer from
  testing the wrong runtime libraries.
* If we are testing the compiler's built libs then we adjust
  `config.compiler_rt_libdir` to point at the compiler's runtime
  directory. This makes the `test/builtins/Unit` tests use the
  compiler's builtin library. No other changes are required because
  all other testsuites implicitly use the compiler's built libs.

To make the above work the
`test_suite_supports_overriding_runtime_lib_path` test suite config
option has been introduced so we can identify what each test suite
supports.

Note all of these checks **have to be performed** when lit runs.
We cannot run the checks at CMake generation time because
multi-configuration build systems prevent us from knowing what the
paths will be.

We could perhaps support `COMPILER_RT_TEST_STANDALONE_BUILD_LIBS` being
`ON` for most test suites (when the runtime library paths differs) in
the future by specifiying a custom compiler resource directory path.
Doing so is out of scope for this patch.

rdar://77182297

Differential Revision: https://reviews.llvm.org/D101681