[llvm-dev] Responsibilities of a buildbot owner
Omair Javaid via llvm-dev
llvm-dev at lists.llvm.org
Fri Jan 14 15:12:19 PST 2022
Hi Stella,
This is in reference to my email on lldb-dev about setting up a LLDB window
on Arm64 buildbot. We are currently working on setting up a Arm64 bot that
will run only unit-tests and shell-tests. However in future we are going to
be taking up LLDB on Windows Arm64 maintenance and hope to run a full
featured testsuite on our buildbots. Meanwhile, as python API support is a
very important LLDB feature, not running API tests will result in an
incremental pile of windows specific failures which will increase
engineering effort required for stabilising LLDB on windows. I have
suggested reducing the number of parallel API tests on windows to see if it
reduces the amount of noise generated by flaky tests.
https://reviews.llvm.org/D117363
In the case it doesnt work, I'll take up the ownership of Windows x64
buildbot as well and try to keep noise reduced similar to what I do for
LInux Arm/Arm64 LLDB bots.
Thanks!
Omair Javaid
www.linaro.org
On Fri, 14 Jan 2022 at 09:38, Stella Stamenova via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I had a chat with Jonas earlier today and one of the things that came out
> was that we actually have three separate suites of tests in lldb:
> - shell
> - unit
> - api
>
> The category that causes the most pain in general, including on the
> Windows lldb bot, is the API tests. The shell tests are very stable and so
> are all (but one) of the unit tests.
>
> Since, as Pavel pointed out, there's not a very active community for lldb
> on Windows, one thing we could do is run only the shell and unit test
> suites on the Windows buildbot and drop the API tests. This would allow us
> to prevent complete bit rot by providing relatively good coverage while at
> the same time removing the most unstable tests from the buildbot. Then we
> could dispense with having to disable individual API tests when they show
> instability on Windows.
>
> I drafted a patch that would do that (with the assumption that everyone
> would be on board):
> https://reviews.llvm.org/D117267
>
> Let me know if you disagree with this course of action or have any other
> concerns.
>
> Thanks,
> -Stella
>
> -----Original Message-----
> From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Stella
> Stamenova via llvm-dev
> Sent: Wednesday, January 12, 2022 9:07 AM
> To: Greg Clayton <clayborg at gmail.com>; Pavel Labath <pavel at labath.sk>
> Cc: Jim Ingham <jingham at apple.com>; llvm-dev <llvm-dev at lists.llvm.org>
> Subject: Re: [llvm-dev] [EXTERNAL] Re: Responsibilities of a buildbot owner
>
> > Can someone verify if we are testing with ProcessWindows or lldb-server
> on the build bot?
>
> Since I didn't set LLDB_USE_LLDB_SERVER on the buildbot itself and this is
> not in the zorg configuration, the buildbot is using ProcessWindows.
>
> I've never tried setting LLDB_USE_LLDB_SERVER to on when running the
> tests, so I am not sure what to expect from the results though. If I have
> time, I'll try it out locally this week to see what happens.
>
> -----Original Message-----
> From: Greg Clayton <clayborg at gmail.com>
> Sent: Tuesday, January 11, 2022 4:42 PM
> To: Pavel Labath <pavel at labath.sk>; Stella Stamenova <stilis at microsoft.com
> >
> Cc: Philip Reames <listmail at philipreames.com>; Jim Ingham <
> jingham at apple.com>; llvm-dev <llvm-dev at lists.llvm.org>
> Subject: Re: [llvm-dev] [EXTERNAL] Re: Responsibilities of a buildbot owner
>
> Does windows use lldb-server by default or does it use ProcessWindows?
> ProcessWindows is the native process debugger, and lldb-server is the way
> we want debugging to work. If we look at ProcessWindows.cpp:
>
> static bool ShouldUseLLDBServer() {
> llvm::StringRef use_lldb_server = ::getenv("LLDB_USE_LLDB_SERVER");
> return use_lldb_server.equals_insensitive("on") ||
> use_lldb_server.equals_insensitive("yes") ||
> use_lldb_server.equals_insensitive("1") ||
> use_lldb_server.equals_insensitive("true");
> }
>
> void ProcessWindows::Initialize() {
> if (!ShouldUseLLDBServer()) {
> static llvm::once_flag g_once_flag;
>
> llvm::call_once(g_once_flag, []() {
> PluginManager::RegisterPlugin(GetPluginNameStatic(),
> GetPluginDescriptionStatic(),
> CreateInstance);
> });
> }
> }
>
>
>
> We can see it is enabled if LLDB_USE_LLDB_SERVER is set the "on", "yes",
> "1", or "true". If this is not set then this is using the built in
> ProcessWindows.cpp native process plug-in which I believe was never fully
> fleshed out and had issues.
>
> Can someone verify if we are testing with ProcessWindows or lldb-server on
> the build bot?
>
>
> > On Jan 11, 2022, at 10:31 AM, Pavel Labath via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > On 11/01/2022 18:59, Stella Stamenova wrote:
> >> The windows lldb bot is running on a Hyper-V virtual machine, so it
> would make sense that if watchpoints don't work correctly in virtual
> environments they would be failing there. On the rare occasion I've had to
> run these tests locally, I have also seen them fail though, so that's not
> the only source of issues.
> >> Since I disabled the couple of tests yesterday, there's only one
> watchpoint test that is still failing randomly. One option would be to
> disable just this test and let the remaining few watchpoint tests continue
> to run on Windows (I prefer this option since some tests would continue to
> run). Alternatively, all the watchpoint tests can be skipped via the
> category flag, but in that case, I'd like us to undo the individual skips.
> >
> > For better or worse, you're currently the most (only?) interested person
> in keeping windows host support working, so I think you can manage the
> windows skips/fails in any way you see fit. The rest of us are mostly
> interested in having green builds. :)
> >
> > Hyper-V is _not_ among the virtualization systems I've tried using with
> lldb, so I cannot conclusively say anything about it (though I still have
> my doubts).
> >
> >> I did notice while going through the watchpoint tests to see what is
> still enabled on Windows, that the same watchpoint tests that are
> disabled/failing on Windows are disabled on multiple other platforms as
> well. The tests passing on Windows are also the ones that are not disabled
> on other platforms. A third option would be to add a separate category for
> the watchpoint tests that don't run correctly everywhere and use that to
> disable them instead. This would be a more generic way to disable the tests
> instead of adding multiple `skipIf` statements to each test.
> >
> > On non-x86 architectures, watchpoints tend to be available only on
> special (developer) hardware or similar (x86 is the outlier in having
> universal support), which is why these tests tend to accumulate various
> annotations. However, I don't think we need to solve this problem (how to
> skip the tests "nicely") here...
> >
> > pl
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.llvm.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fllvm-dev&data=04%7C01%7Cstilis%40microsoft.com%7Ceaf5b1164b4d47cd7d3908d9d5edf20a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637776040213456529%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=i4%2FHWKyjKWdXm5PE6dj339TNuFIs5xMNZr3yuFzMoVA%3D&reserved=0
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.llvm.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fllvm-dev&data=04%7C01%7Cstilis%40microsoft.com%7Ceaf5b1164b4d47cd7d3908d9d5edf20a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637776040213456529%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=i4%2FHWKyjKWdXm5PE6dj339TNuFIs5xMNZr3yuFzMoVA%3D&reserved=0
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20220115/8f525b80/attachment.html>
More information about the llvm-dev
mailing list