[flang-dev] How to properly test a patch

[Mehdi AMINI via flang-dev]

On Fri, Sep 17, 2021 at 4:18 AM Andrzej Warzynski via flang-dev <
flang-dev at lists.llvm.org> wrote:

> Craig,
>
> There is quite a few Flang BuildBot workers available here:
> https://lab.llvm.org/buildbot/#/builders
> (type "flang" in the search box to filter the other ones out). Quite a
> few configurations are tested, but as Mehdi points out, these workers
> run post-merge.
>
> I'm not sure how to check the available configurations for pre-merge
> testing. I do know that there are both Linux and Windows bots running
> for every patch that you submit to Phabricator. A summary is usually
> available in the "Details" box at the top (alongside "Reviewers" etc).
>
> Most of us have had a patch reverted at some point - it's just really
> hard to cover the whole matrix. Your patch
> (https://reviews.llvm.org/D108059) landed with:
>  > This revision was landed with ongoing or failed builds.
> Perhaps pre-merge testing did hint something? I'm not sure whether it's
> possible to dig out the actual log, so we'll probably never know. And
> false negatives do happen occasionally.
>

The log is still accessible, in the comment feed you can look for the most
recent Harbormaster:

[image: Screen Shot 2021-09-17 at 12.29.57 PM.png]

In this case it only found a clang-tidy failure.


> I'm not aware of any efforts to improve the upstream testing of LLVM
> Flang just now, but every now and then some extra configuration is
> added. Recently, Michael Kruse added a Windows post-merge worker. That's
> been a huge improvement for us.
>
> Btw, most of the time, "shared library builds" are a better at capturing
> dependency/build errors and that's what I tend to use by default.
>

+1: the shared lib config is more sensitive to missing dependencies in
CMake.

-- 
Mehdi



>
> -Andrzej
>
> On 17/09/2021 00:08, Mehdi AMINI via flang-dev wrote:
> > Hi,
> >
> > It is really hard to cover the entire test matrix, and be bullet-proof:
> > next time you may get a revert because a sanitizer bot will fail
> > (ASAN/UBSAN/TSAN), or because you broke the IBM SystemZ bot or something
> > like that.
> > There is some testing on Phabricator revisions, but I believe it only
> > builds a single config on Linux and Windows.
> >
> > Some folks are looking into expanding the pre-merge testing flow, but
> > it'll take time!
> >
> > --
> > Mehdi
> >
> >
> >
> > On Thu, Sep 16, 2021 at 3:47 PM Rasmussen, Craig E. via flang-dev
> > <flang-dev at lists.llvm.org <mailto:flang-dev at lists.llvm.org>> wrote:
> >
> >     I just had a push reverted because linking failed with shared
> >     libraries.  I, unsurprisingly, had been testing on my local machine
> >     with static libraries.____
> >
> >     __ __
> >
> >     I’ve heard mention that someone was trying to create a buildbot that
> >     could be used for testing.  Perhaps I should have paid more
> >     attention to this thread, but perhaps some kind soul could fill me
> >     in on testing.____
> >
> >     __ __
> >
> >     Thanks,____
> >
> >     Craig____
> >
> >     __ __
> >
> >     _______________________________________________
> >     flang-dev mailing list
> >     flang-dev at lists.llvm.org <mailto:flang-dev at lists.llvm.org>
> >     https://lists.llvm.org/cgi-bin/mailman/listinfo/flang-dev
> >     <https://lists.llvm.org/cgi-bin/mailman/listinfo/flang-dev>
> >
> >
> > _______________________________________________
> > flang-dev mailing list
> > flang-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/flang-dev
> >
> _______________________________________________
> flang-dev mailing list
> flang-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/flang-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/flang-dev/attachments/20210917/71336f50/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen Shot 2021-09-17 at 12.29.57 PM.png
Type: image/png
Size: 23233 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/flang-dev/attachments/20210917/71336f50/attachment-0001.png>