[llvm] [docs] Add Included Suites to Test Suite Guide (PR #128937)

[Sam Elliott via llvm-commits]

https://github.com/lenary updated https://github.com/llvm/llvm-project/pull/128937

>From c36f6381008ffe5d4adf628668fc437c490ee978 Mon Sep 17 00:00:00 2001
From: Sam Elliott <quic_aelliott at quicinc.com>
Date: Wed, 26 Feb 2025 11:25:29 -0800
Subject: [PATCH 1/2] [docs] Add Included Suites to Test Suite Guide

This policy has never been written down, but was the intention when the
GCC C Torture Suite was added.

In the years since that suite was added, changes to these tests have
been landed, but we want to reject (or discourage) this going forwards,
and look into reverting those changes.
---
 llvm/docs/TestSuiteGuide.md | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/llvm/docs/TestSuiteGuide.md b/llvm/docs/TestSuiteGuide.md
index aa8332ed8b7e1..93a5bd88fa278 100644
--- a/llvm/docs/TestSuiteGuide.md
+++ b/llvm/docs/TestSuiteGuide.md
@@ -352,6 +352,27 @@ performance. You can find more information at
 of the LLVM project is hosted at [http://lnt.llvm.org](http://lnt.llvm.org).
 
 
+Included Test Suites
+--------------------
+
+The LLVM Test Suite repository includes a number of test suites that we have
+copied into the repository from external sources, and hand-written CMake files
+to hook into the LLVM Test Suite system. For example, these include GCC's C
+and Fortran Torture test suites.
+
+In some cases, these external tests have a different license to other code in
+the llvm-test-suite repository, which must be documented in `LICENSE.TXT` in the
+root of the test suite repository, under the heading "Software from third
+parties included in the LLVM Project".
+
+The only changes we can accept to the tests in these suites are where we bring
+in all the changes from the external sources. Users should not make individual
+downstream changes to these tests.
+
+We do accept changes to the CMake integration files for these tests, and where a
+test is wrong/broken, we should have the ability to exclude the test from our
+integration with the suite.
+
 External Suites
 ---------------
 

>From ba30024aec69a73825808734bc2df633546157dc Mon Sep 17 00:00:00 2001
From: Sam Elliott <quic_aelliott at quicinc.com>
Date: Fri, 4 Apr 2025 14:48:59 -0700
Subject: [PATCH 2/2] Nuance is good

---
 llvm/docs/TestSuiteGuide.md | 38 ++++++++++++++++++++++++++++---------
 1 file changed, 29 insertions(+), 9 deletions(-)

diff --git a/llvm/docs/TestSuiteGuide.md b/llvm/docs/TestSuiteGuide.md
index 93a5bd88fa278..c1f5b77e587b3 100644
--- a/llvm/docs/TestSuiteGuide.md
+++ b/llvm/docs/TestSuiteGuide.md
@@ -356,22 +356,42 @@ Included Test Suites
 --------------------
 
 The LLVM Test Suite repository includes a number of test suites that we have
-copied into the repository from external sources, and hand-written CMake files
-to hook into the LLVM Test Suite system. For example, these include GCC's C
-and Fortran Torture test suites.
+copied into the repository from external sources, and written our own CMake
+configuration to hook into the LLVM Test Suite system.
 
 In some cases, these external tests have a different license to other code in
 the llvm-test-suite repository, which must be documented in `LICENSE.TXT` in the
 root of the test suite repository, under the heading "Software from third
 parties included in the LLVM Project".
 
-The only changes we can accept to the tests in these suites are where we bring
-in all the changes from the external sources. Users should not make individual
-downstream changes to these tests.
+When there are problems with a test suite that require changes to the code,
+there are different approaches (given here in order of preference):
+
+1. Attempt to contribute the changes back to the external source and then update
+   the test suite to bring it in line with the external source again. This is
+   the best approach as we interact with other open source projects, but there
+   might be circumstances where it does not work (the external source may not
+   accept modifications, or may take a long time to accept them).
+
+2. We can update the CMake to either disable the test, or to provide additional
+   compiler options to work around the problem.
+
+3. We can commit changes to the tests' sources. This is acceptable for almost
+   all external sources in the LLVM Test Suite repository, but notable
+   exceptions are the GCC and GFortran test suites, which we do not accept
+   changes for.
+
+Even when deciding to follow options 2 and 3, the problem should be reported
+back to the external source if it is relevant to them, so they have the
+opportunity to fix it.
+
+When updating external sources, the PR should audit any changes to the tests'
+source code that we have made within the llvm-test-suite repository, and only
+reapply the changes that are still required. This is also a good moment to
+review any tests that have been disabled with CMake or that are configured to
+use additional compiler options.
+
 
-We do accept changes to the CMake integration files for these tests, and where a
-test is wrong/broken, we should have the ability to exclude the test from our
-integration with the suite.
 
 External Suites
 ---------------