[Lldb-commits] [PATCH] D54009: Refactor LLDB lit configuration files

Wed Nov 14 12:59:22 PST 2018

I think I have a pretty good handle on what the problems are.  I'm honestly
surprised the lit test suite ever worked, even before my patch that "broke"
it.  We were basically just picking up whatever the system PATH was and
just going with it, so a lot of the substitutions weren't actually
substitutions at all, they were just random executables found on path.
That's totally not how this is supposed to work at all.

The idea is to set up a hermetic environment where we we create
substitutions for everything we care about, those resolve to things in the
build tree (with suitable hooks for overriding the search path where that's
desired), and we actually delete the system path entirely so that it's
impossible to find anything on PATH.

So I'm currently working through getting all this working the way it's
supposed to, hopefully I can have something by tomorrow.

On Wed, Nov 14, 2018 at 9:21 AM Stella Stamenova <stilis at microsoft.com>
wrote:

> Simplifying (and making things more robust in the process) sounds great to
> me. I think the current iteration of how parameters are passed to the tests
> is quite complicated and unclear, so this will be a step in the right
> direction and if there’s a need for gcc later, we can take the time to
> design the feature properly so that it works across the lit tests and the
> test suite both.
>
>
>
> Thanks,
>
> -Stella
>
>
>
> *From:* Zachary Turner <zturner at google.com>
> *Sent:* Tuesday, November 13, 2018 4:30 PM
>
>
> *To:* Stella Stamenova <stilis at microsoft.com>
> *Cc:* reviews+D54009+public+0e164460da8f1d7f at reviews.llvm.org;
> pavel at labath.sk; chris.bieneman at me.com; dccitaliano at gmail.com;
> aleksandr.urakov at jetbrains.com; jdevlieghere at apple.com;
> abidh.haq at gmail.com; teemperor at gmail.com; ki.stfu at gmail.com;
> mgorny at gentoo.org; dan at su-root.co.uk; jfbastien at apple.com;
> lldb-commits at lists.llvm.org; llvm at inglorion.net
> *Subject:* Re: [PATCH] D54009: Refactor LLDB lit configuration files
>
>
>
> use_clang() already will fall back on searching the environment variable
> 'CLANG' to find a path to it.
>
>
>
>         self.config.clang = self.use_llvm_tool(
>
>             'clang', search_env='CLANG', required=required)
>
>
>
> But we could make this environment variable a parameter to use_clang() if
> we wanted to.  As long as we can agree that we don't need to worry about
> gcc -- at least for now -- then it should all simplify down quite a bit.
> And AFAICT, there's nobody using gcc with the lit tests right now, so it
> just adds unnecessary complexity.  And if and when we do have people using
> it, there is even more work to be done.
>
>
>
> If someone only wants a clang that isn't the just-built clang (for example
> a release version to make sure the tests run faster), all they need to do
> is set the environment variable 'CLANG' and it should be fine.
>
>
>
> Since the lit suite is still very new and developing, I'm not too
> concerned about regressing a feature (especially one with zero users),
> because the important thing to me is that it's designed right so that the
> feature can grow in organically and not be "forced" in with a subpar
> implementation.
>
>
>
> On Tue, Nov 13, 2018 at 4:23 PM Stella Stamenova <stilis at microsoft.com>
> wrote:
>
> The plan for the lit tests sounds reasonable to me. I would also remove
> LLDB_TEST_C/CXX_COMPILER entirely so that we can reduce confusion since
> they’re only used for the lit tests, right?
>
>
>
> My only concern is that I’ve been told that there are people who will
> build lldb with a different compiler than the tests – so the properties for
> LLDB_TEST_C/CXX_COMPILER might actually be used especially in cases where
> clang is not built alongside lldb.
>
>
>
> Thanks,
>
> -Stella
>
>
>
> *From:* Zachary Turner <zturner at google.com>
> *Sent:* Tuesday, November 13, 2018 4:16 PM
>
>
> *To:* Stella Stamenova <stilis at microsoft.com>
> *Cc:* reviews+D54009+public+0e164460da8f1d7f at reviews.llvm.org;
> pavel at labath.sk; chris.bieneman at me.com; dccitaliano at gmail.com;
> aleksandr.urakov at jetbrains.com; jdevlieghere at apple.com;
> abidh.haq at gmail.com; teemperor at gmail.com; ki.stfu at gmail.com;
> mgorny at gentoo.org; dan at su-root.co.uk; jfbastien at apple.com;
> lldb-commits at lists.llvm.org; llvm at inglorion.net
> *Subject:* Re: [PATCH] D54009: Refactor LLDB lit configuration files
>
>
>
> Ok so for dotest, it seems to be ignoring the config.cc and config.cxx
> entirely.  So we can theoretically do whatever we want with it, or change
> around the directory structure so that it's more like:
>
>
>
> lldb
>
> * lit
>
> * * Dotest
>
> * * Unit
>
> * * Tests
>
>
>
> and put the config.cc / config.cxx logic under Tests.  That's a large
> change though and probably not worth making such a large change right away.
>
>
>
> dotest tests manually construct the command line directly in CMake via
> this `LLDB_DOTEST_ARGS_PROPERTY` global property, and then in
> lldb/lit/Suite/lit.cfg we have this line:
>
>
>
> dotest_cmd = [config.dotest_path, '-q']
>
> dotest_cmd.extend(config.dotest_args_str.split(';'))
>
>
>
>
>
> So pretty much everythign the parent lit file has done is totally
> ignored.
>
>
>
> With that in mind, **for the lit tests only** I propose dropping support
> for non-clang compilers and ignoring LLDB_TEST_C_COMPILER and
> LLDB_TEST_CXX_COMPILER (you can still have a custom path to clang
> executable via an environment variable, which is consistent with how
> clang's test suite works).
>
>
>
> Note that when you run ninja check-lldb-lit you will now get messages that
> tell you the exact path to the clang executable, so you can see what the
> PATH resolution is doing.
>
>
>
> On Tue, Nov 13, 2018 at 4:02 PM Zachary Turner <zturner at google.com> wrote:
>
> On Tue, Nov 13, 2018 at 3:47 PM Stella Stamenova <stilis at microsoft.com>
> wrote:
>
> I am not sure if that’s the right solution for a couple of reasons:
>
>    1. As far as I can tell only clang calls use_clang (and only lld calls
>    use_lld), while the other projects such as lld and llvm rely on the
>    environment to be setup correctly
>    2. Lld also has tests that invoke clang-cl and they pass – while the
>    ones in LLDB do not, so the invocation of use_clang is not necessary for
>    the tests to pass (maybe?)
>    3. LLDB allows us to specify whether to use gcc or clang as well as
>    the path and it can also have a test compiler specified via
>    LLDB_USE_TEST_*_COMPILER, so we should first decide what scenarios we want
>    to support before trying to make this work and possibly making it even more
>    confusing and complicated
>
>
>
> Do you know what the answer for 3) is? What compilers are valid to specify
> for the lit/suite/unittests via the various parameters?
>
>
>
> For the unit tests, I don't think we ever specify a compiler, or we don't
> ever read the value.  Because a unit test shouldn't be compiling anything,
> it's a different kind of test.
>
>
>
> For the dotest suite, specifying the compiler is important and it can
> definitely be gcc, but I don't think this uses the same method of going
> through config.cc.  In fact, I'm not sure how it determines what compiler
> to use at the moment, as it's been a number of years since I've looked at
> the dotest suite.
>
>
>
> For the lit tests, I'm inclined to say we should keep things simple and
> only support clang for now, and add support for new compilers such as gcc
> if and when someone actually wants it.  Otherwise YAGNI.
>
>
>
> Definitely that time will come, but it doesn't make sense to support it
> immediately if nobody is using it today and nobody is planning to enable it
> immediately.
>
>
>
> So I'm tempted to say that perhaps we should just call
> llvm_config.use_clang() and llvm_config.use_lld() and ignore
> LLDB_TEST_COMPILER, which in my experience has only been a source of
> unnecessary complexity that never actually gets used in practice.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20181114/e1d38b1b/attachment-0001.html>