[cfe-dev] Uncovering non-determinism in LLVM - An Update

[David Blaikie via cfe-dev]

On Tue, Aug 29, 2017 at 11:45 AM Grang, Mandeep Singh via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> Hi All,
>
> I wanted to share a couple of updates on the effort to uncover
> non-determinism in LLVM through reverse iteration.
>
> 1. Reverse iteration has now been enabled for DenseMap
> (https://reviews.llvm.org/D35043)
>
> 2. We have setup a nightly reverse iteration buildbot
> (http://lab.llvm.org:8011/builders/reverse-iteration).
> This builds all LLVM targets with reverse iteration ON and runs ninja
> check-all. Currently there are 14 unit test failures. Please feel free
> to fix these.
>
> Also currently, only I receive the nightly email notification for this
> buildbot run. My plan is to enable sending the nightly notifications to
> llvm-commits once all 14 failures have been resolved.
> Please let me know if the community wants the nightly notifications even
> with the failures.
> As a potential next step, I was thinking about bootstrapping this
> reverse iteration LLVM to compile itself. Not sure if it can uncover
> more bugs but maybe worth a shot.
>

To uncover bugs in this configuration, I believe you'd want/need a
stage2/stage3 comparison which might be a bit tricky/expensive*, something
like:

build clang twice (reverse and forward enabled) then build (in one mode,
doesn't matter which I think) clang or other release binaries (or even the
whole release) from each of those and compare them bit-for-bit, they should
be identical.

* If you want other developers to act on bugs found, the buildbot needs to
have a short blame list (this can be done on a slow buildbot by having
multiple slaves/builders running in parallel) but preferably also a short
cycle time (so failures are reported soon after they're created) -
otherwise expect to do a lot of triage yourself (& possibly leave the
emails only going to you - because they'll have too large blame
lists/revision ranges and people won't find them actionable) & then
probably following up on the specific commit you believe introduced the
problem and either fixing it yourself or replying on the commits list to
report it to the original contributor.


>
> All comments/suggestions welcome.
>
> Thanks,
> Mandeep
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170830/5ac300b1/attachment.html>