[llvm-dev] Using clang/lld from the PATH when testing

[Fangrui Song via llvm-dev]

On 2021-05-06, James Henderson via llvm-dev wrote:
>Hi all,
>
>Summary: can we stop allowing clang/lld to be picked up from the PATH
>environment when running testing?
>
>Details:
>
>As part of my work on a cross-project testsuite (see
>https://reviews.llvm.org/D95339 and other related patches), I noticed that
>where lit has been configured using `use_clang()` or `use_lld()` the
>respective tools will be selected from the system PATH, if they aren't
>found in the build directory. If users are running check-* to run the
>relevant tests, this isn't an issue, as the tools will be built (and
>therefore used in preference).
>
>I personally, and I'm sure many others, routinely don't use these check-*
>targets, preferring to manually build and run the subset of tests that have
>the potential to be impacted by my change. Potentially this leaves room for
>human error (e.g. a misconfigured build bot or simply a user error when
>checking things) to not build clang (or lld) in a clean environment, and as
>such, the "wrong" tool might be used, causing spurious tests failures (or
>worse, spurious passes).
>
>An environment variable ('CLANG') can be specified to specify the clang
>executable to use, in preference to a built one or one on the PATH. I plan
>to make a similar change for LLD soon too. As such, do we need the "use the
>PATH version" behaviour anymore? Dropping it will allow simplifying code
>and reduce room for human error.

Is the plan to allow running the testsuite with Clang/LLD built from a
different commit?

If every executable needs an environment variable, I worry that there may be
too much customization.

>Thanks,
>
>James

>_______________________________________________
>LLVM Developers mailing list
>llvm-dev at lists.llvm.org
>https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev