[lldb-dev] [RFC] Testsuite in lldb & possible future directions

[Zachary Turner via lldb-dev]

On Wed, Feb 7, 2018 at 2:38 AM Pavel Labath <labath at google.com> wrote:

> On 6 February 2018 at 18:53, Zachary Turner <zturner at google.com> wrote:
> > I'm not claiming that it's definitely caused by dotest and that moving
> away
> > from dotest is going to fix all the problems.  Rather, I'm claiming that
> > dotest has an unknown amount of flakiness (which may be 0, but may be
> > large), and the alternative has a known amount of flakiness (which is
> very
>
> Well, it may be unknown to you, but as someone who has managed a bot
> running tests for a long time, I can tell you that the it's pretty
> close to 0. Some test still fail sometimes, but the failure rate is
> approximately at the same level as failures caused by the bot not
> being able to reach the svn server to fetch the sources.

As someone who gave up on trying to set up a bot due to flakiness, I have a
different experience.



>
> That said, I'm still in favor of replacing the test runner with lit. I
> just think it needs to be done with a steady hand.
>
>
> >> So I believe we need more lightweight tests, and lldb-test can provide
> >> us with that. The main question for me (and that's something I don't
> >> really have an answer to) is how to make writing tests like that easy.
> >> E.g. for these "foreign" language plugins, the only way to make a
> >> self-contained regression test would be to check-in some dwarf which
> >> mimics what the compiler in question would produce. But doing that is
> >> extremely tedious as we don't have any tooling for that.
> >
> >
> >  Most of these other language plugins are being removed anyway.  Which
> > language plugins are going to still remain that aren't some flavor of
> c/c++?
>
> Well, right now we have another thread proposing the addition of a
> Rust plugin, and we will want to resurrect Java support sooner or
> later. Go/Ocaml <https://goto.google.com/Ocaml> folks may want to do the
> same, if doing that will not
> involve them inventing a whole test framework.
>
> So, I'm not sure where you were heading with that question..


Rust is based on llvm so we have the tools necessary for that.  The rest
are still maybe and someday so we can cross that bridge when (if) we come
to it
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20180207/300095c8/attachment.html>