<div dir="ltr">Hi folks,<div><br></div><div>From my experience with running the mlir-nvidia band clangd buildbot workers and setting up pre-merge testing I can really recommend setting up the workers using Docker (or a similar technology):<br></div><div><ul><li>easy versioning/diffing/reverting of your changes</li><li>you can run two different images in production and staging</li><li>Easy to copy-and-past to run a different config</li><li>You can test your exact changes locally, before deploying them. This gives you much faster iteration times.</li></ul><div>Downsides:</div><ul><li>Need to get familiar with Docker</li><li>Need to workaround infrastructure issues if you have some special usages (e.g. GPU access)</li><li>Bumpy experience on Windows</li></ul><div>My Dockerfiles are in <a href="https://github.com/llvm/llvm-zorg/tree/main/buildbot/google/docker">llvm-zorg</a>, feel free to reuse these.</div><div><br></div><div><br></div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Best,<div>Christian</div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Dec 18, 2020 at 9:11 AM Serge Guelton via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Thanks Reid for taking this risky step. As someone who just lost one day upgrading/downgrading/messing up its laptop, I fully appreciate the risk you took, and I'm glad everything looks fine.<div><br></div><div>The two slaves I had issues whith were:</div><div><br></div><div> <a href="http://lab.llvm.org:8011/#/builders/127" target="_blank">http://lab.llvm.org:8011/#/builders/127</a> (now green)</div><div> <a href="http://lab.llvm.org:8011/#/workers/14" target="_blank">http://lab.llvm.org:8011/#/workers/14</a> (now green)</div><div><br></div><div>Thanks a bunch!</div><div><br></div><div>> As a long suffering maintainer </div><div><br></div><div>And extra thanks for the maintenance task too o/</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Dec 17, 2020 at 9:19 PM Reid Kleckner <<a href="mailto:rnk@google.com" target="_blank">rnk@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Nevermind. I went ahead and took the risk of installing Python 3 and putting it on PATH, and the buildbot service code seems to be working as is. Both bots have made it past cmake, so they're probably on track to recover.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Dec 17, 2020 at 11:38 AM Reid Kleckner <<a href="mailto:rnk@google.com" target="_blank">rnk@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Can we delay this? I maintain two Windows buildbots, and I'm about to go on holiday break, and I'd really rather not scramble to get them up to date:<div><a href="http://lab.llvm.org:8011/#/builders/clang-x64-windows-msvc" target="_blank">http://lab.llvm.org:8011/#/builders/clang-x64-windows-msvc</a><br></div><div><a href="http://lab.llvm.org:8011/#/builders/sanitizer-windows" target="_blank">http://lab.llvm.org:8011/#/builders/sanitizer-windows</a></div><div><br></div><div>Looking at them now, it seems you broke them five hours ago.</div><div><br></div><div>There is risk involved in simply installing Python 3, since it might necessitate upgrading the buildbot worker version, which I have successfully avoided doing for almost five years now.</div><div><br></div><div>This has not been a good month for Windows bots. As a long suffering maintainer, I'm asking for a reprieve.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Dec 14, 2020 at 10:30 PM Serge Guelton via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Folks,<div><br></div><div>unless someone has strong arguments against it, the transition to Python 3 will happen *this week*, on Thursday morning, european time.</div><div>The associated review is <a href="https://reviews.llvm.org/D93097" target="_blank">https://reviews.llvm.org/D93097</a>, feel free to drop in.</div><div>The impact should be low, Python3 is already picked by CMake as default, but a Python 2 fallback still exists, and it will be removed.</div><div><br></div><div>**If you're a bot owner** please make sure Python3 is available on your system ! Now, is the time :-)</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Oct 19, 2020 at 9:34 PM Jonas Devlieghere via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Sounds good. In LLDB we have a slightly different timeline for the test suite (outlined in <a href="http://lists.llvm.org/pipermail/lldb-dev/2020-August/016388.html" target="_blank">http://lists.llvm.org/pipermail/lldb-dev/2020-August/016388.html</a>) but everything else aligns with the plan here. I already removed the fallback to Python 2 in preparation for the 12.0 release. </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Oct 13, 2020 at 11:49 AM David Blaikie via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">*thumbs up*<br><br>Re: out of date buildbots: maybe stage the shebang changes in a time/way that's easy to rollback (or even intended to rollback after a small time window (few hours) off-peak) to flush out the buildbots. Then after that rollout, go over the buildbot failures and follow-up with buildbot owners to get them cleaned up before a more intended-to-stick rollout is done.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Oct 13, 2020 at 7:18 AM James Y Knight via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Yay, thanks for persevering with this migration!</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Oct 13, 2020 at 8:04 AM Serge Guelton via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Folks,<br>
<br>
Now that LLVM 11.0.0 has been released, it's time to prepare for the final step<br>
envisionned in the previous RFC named *[RFC] Python 2 / Python 3 status* [0],<br>
ie. requiring Python3.6 for LLVM 12.0.0, to be released in 2021.<br>
At least Fedora already only ships Python3 and we didn't have much bugs reported<br>
wrt. Python compatibility for the LLVM toolchain.<br>
<br>
Indeed, all Python scripts should now be at least compatible with both python2 (py2)<br>
and python3 (py3), some of them already are already py3 only.<br>
<br>
The build system still depends on Python in a few place, but it explicitly<br>
mentions Python3_EXECUTABLE, and the main dependency (llvm-build) is currently<br>
being removed in D89142.<br>
<br>
The shebangs have already been harmonized in D83857: some mention /usr/bin/env<br>
python, some mention /usr/bin/env python3, and none mention python2 anymore. It<br>
would be great to have all script use the same shebang, PEP394 [1] makes it<br>
(relatively) clear that in our case, explicitly requiring python3 is the way to<br>
go.<br>
<br>
So basically, next step would be to update the documentation, remove the Python2<br>
fallback in root CMakeLists.txt and update shebangs.<br>
<br>
Anyone seeing an issue with that approach? I'm still a bit worried about<br>
buildbots still running on py2 (?)<br>
<br>
Serge<br>
<br>
[0] <a href="http://lists.llvm.org/pipermail/llvm-dev/2020-January/138730.html" rel="noreferrer" target="_blank">http://lists.llvm.org/pipermail/llvm-dev/2020-January/138730.html</a><br>
[1] <a href="https://www.python.org/dev/peps/pep-0394/" rel="noreferrer" target="_blank">https://www.python.org/dev/peps/pep-0394/</a><br>
<br>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div>
</blockquote></div>
</blockquote></div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div>