[cfe-dev] Clang tests with new pass manager

[Petr Hosek via cfe-dev]

I've been experimenting with enabling the new PM by default in our
toolchain, using the -DENABLE_EXPERIMENTAL_NEW_PASS_MANAGER=ON option.
However, when I do that, I see a lot (498) of failing tests.

I've filed PR40724 for this issue and after a bit of investigating I also
found out why these tests are failing. It boils down to two issues:

1. Unlike the legacy pass manager, the new pass manager always makes the
target machine available to passes so we have to always instantiate it even
when we're only emitting bitcode, see
https://github.com/llvm/llvm-project/blob/master/clang/lib/CodeGen/BackendUtil.cpp#L952.
This breaks when some targets tested by Clang don't match targets built by
LLVM. In our case this is more prominent because we disable all targets
that we don't support in our toolchain, but even in the default
configuration you'll still see failures for experimental targets (e.g.
RISC-V).

2. Some Clang tests match the bitcode that's being emitted, but the emitted
sometimes differs between the legacy and new pass manager.

Has anyone already been thinking on how to handle these issues?

For the first one, the obvious solution is to mark all target-specific
tests with appropriate `REQUIRES: <target>-registered-target` conditions.
The downside is that we're going to loose coverage for some of the tests
that are being run today with the legacy pass manager even when the target
is disabled, but we're already seem to be using these conditions in many
Clang tests so maybe it's not a big problem.

For the second issue, we could set a new feature config in lit whenever new
PM is enabled (.e.g experimental-new-pm) and then either mark the broken
tests as unsupported under new PM or provide two different bitcode versions
to match against for legacy and new PM. It's probably easiest to start by
disabling the failing tests first and then try and update all of them to
make them work under new PM.

Do you have any opinion on this or other ideas/suggestions?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190213/5b4360ff/attachment.html>