[llvm] [docs] Strengthen our quality standards and connect AI contribution policy to it (PR #154441)
Reid Kleckner via llvm-commits
llvm-commits at lists.llvm.org
Mon Aug 25 14:00:29 PDT 2025
================
@@ -293,57 +293,63 @@ warranted when performing a code review.
Quality
-------
-The minimum quality standards that any change must satisfy before being
-committed to the main development branch are:
-
-#. Code must adhere to the `LLVM Coding Standards <CodingStandards.html>`_.
-
-#. Code must compile cleanly (no errors, no warnings) on at least one platform.
+Sending patches, PRs, RFCs, comments, etc to LLVM, is not free -- it takes a lot
+of maintainer time and energy to review those contributions. Our quality bar can
+be summarized with the **golden rule** that a contribution should be worth more
+to the project than the time it takes to review it.
+
+However, this rule is held in balance with our goal to make the LLVM project
+welcoming and open to aspiring compiler engineers who are willing to invest time
+and effort to learn and grow, because growing our contributor base and
+potentially recruiting new maintainers helps sustain the project over the long
+term.
+
+Contributors who repeatedly make low-value contributions according to the
+standards of our maintainers will be asked to increase the quality of their
+contributions or cease contributing. As a community, we encourage maintainers to
+invest effort in reviewing work from new contributors. We label PRs from
+first-time contributors to set maintainer expectations on quality. However, we
+expect to see a growth pattern in the quality of a contributor's work over time.
+Maintainers can use their discretion and push back against burdensome
+contributions and ask contributors to meet their quality bar. In addition to the
+minimum guidelines outlined in thie section, maintainers can set their own
+quality standards using their considerable domain expertise.
+
+The following is a checklist of some of the minimum quality standards that any
+change must satisfy before being reviewed in detail or committed to the main
+development branch:
#. Bug fixes and new features should `include a testcase`_ so we know if the
fix/feature ever regresses in the future.
-#. Code must pass the ``llvm/test`` test suite.
+#. The code should compile cleanly on all supported platforms.
-#. The code must not cause regressions on a reasonable subset of llvm-test,
- where "reasonable" depends on the contributor's judgement and the scope of
- the change (more invasive changes require more testing). A reasonable subset
- might be something like "``llvm-test/MultiSource/Benchmarks``".
+#. Code must pass the relevant regression test suites on all supported
+ plaftorms.
+
+#. Code must adhere to the `LLVM Coding Standards <CodingStandards.html>`_.
#. Ensure that links in source code and test files point to publicly available
resources and are used primarily to add additional information rather than
to supply critical context. The surrounding comments should be sufficient
to provide the context behind such links.
-Additionally, the committer is responsible for addressing any problems found in
-the future that the change is responsible for. For example:
-
-* The code should compile cleanly on all supported platforms.
-
-* The changes should not cause any correctness regressions in the ``llvm-test``
- suite and must not cause any major performance regressions.
-
-* The change set should not cause performance or correctness regressions for the
- LLVM tools.
+Our quality policy goals are captured by this quote from `Working in Public: The
+Making and Maintenance of Open Source Software
+<https://press.stripe.com/working-in-public>`_ by Nadia Eghbal:
-* The changes should not cause performance or correctness regressions in code
- compiled by LLVM on all applicable targets.
-
-* You are expected to address any `GitHub Issues <https://github.com/llvm/llvm-project/issues>`_ that
- result from your change.
+ .. pull-quote::
-We prefer for this to be handled before submission but understand that it isn't
-possible to test all of this for every submission. Our build bots and nightly
-testing infrastructure normally finds these problems. A good rule of thumb is
-to check the nightly testers for regressions the day after your change. Build
-bots will directly email you if a group of commits that included yours caused a
-failure. You are expected to check the build bot messages to see if they are
-your fault and, if so, fix the breakage.
+ "When attention is being appropriated, producers need to weigh the costs and
+ benefits of the transaction. To assess whether the appropriation of attention
+ is net-positive, it’s useful to distinguish between *extractive* and
+ *non-extractive* contributions. Extractive contributions are those where the
+ marginal cost of reviewing and merging that contribution is greater than the
+ marginal benefit to the project’s producers. In the case of a code
+ contribution, it might be a pull request that’s too complex or unwieldy to
+ review, given the potential upside."
+ -- Nadia Eghbal
----------------
rnk wrote:
Given that the subjective part of the policy is probably more important than the rules-based checklist, I'll try to implement that.
https://github.com/llvm/llvm-project/pull/154441
More information about the llvm-commits
mailing list