[clang] [Doc] Update documentation for no-transitive-change (PR #96453)
Chuanqi Xu via cfe-commits
cfe-commits at lists.llvm.org
Wed Jun 26 20:24:42 PDT 2024
https://github.com/ChuanqiXu9 updated https://github.com/llvm/llvm-project/pull/96453
>From a035ae25314f3168f73108988f2bb7671e7d9e7f Mon Sep 17 00:00:00 2001
From: Chuanqi Xu <yedeng.yd at linux.alibaba.com>
Date: Mon, 24 Jun 2024 11:41:23 +0800
Subject: [PATCH 1/3] [Doc] Update documentation for no-transitive-change
---
clang/docs/ReleaseNotes.rst | 4 +
clang/docs/StandardCPlusPlusModules.rst | 128 ++++++++++++++++++++++++
2 files changed, 132 insertions(+)
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index 9c8f8c4a4fbaf..89e433870c9ff 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -229,6 +229,10 @@ C++20 Feature Support
will now work.
(#GH62925).
+- Clang refactored the BMI format to make it possible to support no transitive changes
+ mode for modules. See `StandardCPlusPlusModules <StandardCPlusPlusModules.html>`_ for
+ details.
+
C++23 Feature Support
^^^^^^^^^^^^^^^^^^^^^
diff --git a/clang/docs/StandardCPlusPlusModules.rst b/clang/docs/StandardCPlusPlusModules.rst
index 1c3c4d319c0e1..68854636e617d 100644
--- a/clang/docs/StandardCPlusPlusModules.rst
+++ b/clang/docs/StandardCPlusPlusModules.rst
@@ -652,6 +652,134 @@ in the future. The expected roadmap for Reduced BMIs as of Clang 19.x is:
comes, the term BMI will refer to the Reduced BMI and the Full BMI will only
be meaningful to build systems which elect to support two-phase compilation.
+Experimental No Transitive Change
+---------------------------------
+
+Starting with clang19.x, we introduced an experimental feature: the non-transitive
+change for modules, aimed at reducing unnecessary recompilations. For example,
+
+.. code-block:: c++
+
+ // m-partA.cppm
+ export module m:partA;
+
+ // m-partB.cppm
+ export module m:partB;
+ export int getB() { return 44; }
+
+ // m.cppm
+ export module m;
+ export import :partA;
+ export import :partB;
+
+ // useBOnly.cppm
+ export module useBOnly;
+ import m;
+ export int B() {
+ return getB();
+ }
+
+ // Use.cc
+ import useBOnly;
+ int get() {
+ return B();
+ }
+
+Now let's compile the project (For brevity, some commands are omitted.):
+
+.. code-block:: console
+
+ $ clang++ -std=c++20 m-partA.cppm --precompile -o m-partA.pcm
+ $ clang++ -std=c++20 m-partB.cppm --precompile -o m-partB.pcm
+ $ clang++ -std=c++20 m.cppm --precompile -o m.pcm -fprebuilt-module-path=.
+ $ clang++ -std=c++20 useBOnly.cppm --precompile -o useBOnly.pcm -fprebuilt-module-path=.
+ $ md5sum useBOnly.pcm
+ 07656bf4a6908626795729295f9608da useBOnly.pcm
+
+then let's change the interface of ``m-partA.cppm`` to:
+
+.. code-block:: c++
+
+ // m-partA.v1.cppm
+ export module m:partA;
+ export int getA() { return 43; }
+
+Let's compile the BMI for `useBOnly` again:
+
+.. code-block:: console
+
+ $ clang++ -std=c++20 m-partA.cppm --precompile -o m-partA.pcm
+ $ clang++ -std=c++20 m-partB.cppm --precompile -o m-partB.pcm
+ $ clang++ -std=c++20 m.cppm --precompile -o m.pcm -fprebuilt-module-path=.
+ $ clang++ -std=c++20 useBOnly.cppm --precompile -o useBOnly.pcm -fprebuilt-module-path=.
+ $ md5sum useBOnly.pcm
+ 07656bf4a6908626795729295f9608da useBOnly.pcm
+
+We observed that the contents of useBOnly.pcm remain unchanged.
+Consequently, if the build system bases recompilation decisions on directly imported modules only,
+it becomes possible to skip the recompilation of Use.cc.
+It should be fine because the altered interfaces do not affect Use.cc in any way.
+This concept is called as no transitive changes.
+
+When clang generates a BMI, it records the hash values of all potentially contributory BMIs
+into the currently written BMI. This ensures that build systems are not required to consider
+transitively imported modules when deciding on recompilations.
+
+The definition for potential contributory BMIs is implementation defined. We don't intend to
+display detailed rules for users. The contract is:
+
+1. It is a severe bug if a BMI remains unchanged erroneously following an observable change
+ that affects its users.
+2. It is an potential improvement opportunity if a BMI changes after an unobservable change
+ happens.
+
+We suggest build systems to support this feature as a configurable option for a long time.
+So that users can go back to the transitive change mode safely at any time.
+
+Interactions with Reduced BMI
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+With reduced BMI, the no transitive change feature can be more powerful if the change
+can be reduced. For example,
+
+.. code-block:: c++
+
+ // A.cppm
+ export module A;
+ export int a() { return 44; }
+
+ // B.cppm
+ export module B;
+ import A;
+ export int b() { return a(); }
+
+.. code-block:: console
+
+ $ clang++ -std=c++20 A.cppm -c -fmodule-output=A.pcm -fexperimental-modules-reduced-bmi -o A.o
+ $ clang++ -std=c++20 B.cppm -c -fmodule-output=B.pcm -fexperimental-modules-reduced-bmi -o B.o -fmodule-file=A=A.pcm
+ $md5sum B.pcm
+ 6c2bd452ca32ab418bf35cd141b060b9 B.pcm
+
+And let's change the implementation for ``A.cppm`` into:
+
+.. code-block:: c++
+
+ export module A;
+ int a_impl() { return 99; }
+ export int a() { return a_impl(); }
+
+and recompile the example:
+
+.. code-block:: console
+
+ $ clang++ -std=c++20 A.cppm -c -fmodule-output=A.pcm -fexperimental-modules-reduced-bmi -o A.o
+ $ clang++ -std=c++20 B.cppm -c -fmodule-output=B.pcm -fexperimental-modules-reduced-bmi -o B.o -fmodule-file=A=A.pcm
+ $md5sum B.pcm
+ 6c2bd452ca32ab418bf35cd141b060b9 B.pcm
+
+We should find the contents of ``B.pcm`` keeps the same. In such case, the build system is
+allowed to skip recompilations of TUs which solely and directly dependent on module B.
+
Performance Tips
----------------
>From 1856722bb5ce0e2f6f17efd135ff86277c346a5b Mon Sep 17 00:00:00 2001
From: Chuanqi Xu <yedeng.yd at linux.alibaba.com>
Date: Wed, 26 Jun 2024 10:39:14 +0800
Subject: [PATCH 2/3] Update
---
clang/docs/ReleaseNotes.rst | 11 +++--
clang/docs/StandardCPlusPlusModules.rst | 53 ++++++++++++++-----------
2 files changed, 37 insertions(+), 27 deletions(-)
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index 89e433870c9ff..0fbdd807640b4 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -157,6 +157,13 @@ here. Generic improvements to Clang as a whole or to its underlying
infrastructure are described first, followed by language-specific
sections with improvements to Clang's support for those languages.
+- Clang implemented improvements to BMI of C++20 Modules that can reduce
+ the number of rebuilds during incremental recompilation. We are seeking
+ feedback from Build System authors and other interested users, especially
+ when you feel Clang changes the BMI and missses an opportunity to avoid
+ recompilations or causes correctness issues. See StandardCPlusPlusModules
+ `StandardCPlusPlusModules <StandardCPlusPlusModules.html>`_ for more details.
+
- The ``\par`` documentation comment command now supports an optional
argument, which denotes the header of the paragraph started by
an instance of the ``\par`` command comment. The implementation
@@ -229,10 +236,6 @@ C++20 Feature Support
will now work.
(#GH62925).
-- Clang refactored the BMI format to make it possible to support no transitive changes
- mode for modules. See `StandardCPlusPlusModules <StandardCPlusPlusModules.html>`_ for
- details.
-
C++23 Feature Support
^^^^^^^^^^^^^^^^^^^^^
diff --git a/clang/docs/StandardCPlusPlusModules.rst b/clang/docs/StandardCPlusPlusModules.rst
index 68854636e617d..e509cc303f5ed 100644
--- a/clang/docs/StandardCPlusPlusModules.rst
+++ b/clang/docs/StandardCPlusPlusModules.rst
@@ -655,8 +655,16 @@ in the future. The expected roadmap for Reduced BMIs as of Clang 19.x is:
Experimental No Transitive Change
---------------------------------
-Starting with clang19.x, we introduced an experimental feature: the non-transitive
-change for modules, aimed at reducing unnecessary recompilations. For example,
+This section is primarily for build system vendors. For end compiler users,
+if you don't want to read it all, this is helpful to reduce recompilations
+We encourage build system vendors and end users try this out and bring feedbacks
+
+Before Clang 19, a change in BMI of any (transitive) dependency would case the
+outputs of the BMI to change. Starting with Clang 19, changes to non-direct
+dependencies should not directly affect the output BMI, unless they affect the
+results of the compilations. We expect that there are many more opportunities
+for this optimization than we currently have realized and would appreaciate
+feedback about missed optimization opportunities. For example,
.. code-block:: c++
@@ -685,7 +693,7 @@ change for modules, aimed at reducing unnecessary recompilations. For example,
return B();
}
-Now let's compile the project (For brevity, some commands are omitted.):
+To compile the project (for brevity, some commands are omitted.):
.. code-block:: console
@@ -696,7 +704,7 @@ Now let's compile the project (For brevity, some commands are omitted.):
$ md5sum useBOnly.pcm
07656bf4a6908626795729295f9608da useBOnly.pcm
-then let's change the interface of ``m-partA.cppm`` to:
+If the interface of ``m-partA.cppm`` is changed to:
.. code-block:: c++
@@ -704,7 +712,7 @@ then let's change the interface of ``m-partA.cppm`` to:
export module m:partA;
export int getA() { return 43; }
-Let's compile the BMI for `useBOnly` again:
+and the BMI for ``useBOnly`` is recompiled as in:
.. code-block:: console
@@ -715,32 +723,27 @@ Let's compile the BMI for `useBOnly` again:
$ md5sum useBOnly.pcm
07656bf4a6908626795729295f9608da useBOnly.pcm
-We observed that the contents of useBOnly.pcm remain unchanged.
-Consequently, if the build system bases recompilation decisions on directly imported modules only,
-it becomes possible to skip the recompilation of Use.cc.
-It should be fine because the altered interfaces do not affect Use.cc in any way.
-This concept is called as no transitive changes.
+then the contents of ``useBOnly.pcm`` remain unchanged.
+Consequently, if the build system only bases recompilation decisions on directly imported modules,
+it becomes possible to skip the recompilation of ``Use.cc``.
+It should be fine because the altered interfaces do not affect ``Use.cc`` in any way;
+there are no transitive changes.
When clang generates a BMI, it records the hash values of all potentially contributory BMIs
-into the currently written BMI. This ensures that build systems are not required to consider
-transitively imported modules when deciding on recompilations.
-
-The definition for potential contributory BMIs is implementation defined. We don't intend to
-display detailed rules for users. The contract is:
+for the BMI being produced. This ensures that build systems are not required to consider
+transitively imported modules when deciding whether to recompile.
-1. It is a severe bug if a BMI remains unchanged erroneously following an observable change
- that affects its users.
-2. It is an potential improvement opportunity if a BMI changes after an unobservable change
- happens.
+What is considered to be a potential contributory BMIs is currently unspecified.
+However, it is a severe bug for a BMI to remain unchanged following an observable change
+that affects its consumers.
-We suggest build systems to support this feature as a configurable option for a long time.
-So that users can go back to the transitive change mode safely at any time.
+We recommend that build systems support this feature as a configurable option so that users
+can go back to the transitive change mode safely at any time.
Interactions with Reduced BMI
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-With reduced BMI, the no transitive change feature can be more powerful if the change
-can be reduced. For example,
+With reduced BMI, the no transitive change feature can be more powerful. For example,
.. code-block:: c++
@@ -780,6 +783,10 @@ and recompile the example:
We should find the contents of ``B.pcm`` keeps the same. In such case, the build system is
allowed to skip recompilations of TUs which solely and directly dependent on module B.
+This only happens with reduced BMI. Since with reduced BMI, we won't record the function body
+of ``int b()`` in the BMI for ``B`` so that the module A doesn't contribute to the BMI of ``B``
+and we have less dependencies.
+
Performance Tips
----------------
>From 64a2037d1c591792af18891b1307a967e4da59b5 Mon Sep 17 00:00:00 2001
From: Chuanqi Xu <yedeng.yd at linux.alibaba.com>
Date: Thu, 27 Jun 2024 11:24:14 +0800
Subject: [PATCH 3/3] Update
---
clang/docs/StandardCPlusPlusModules.rst | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/clang/docs/StandardCPlusPlusModules.rst b/clang/docs/StandardCPlusPlusModules.rst
index e509cc303f5ed..8c1ea444ef993 100644
--- a/clang/docs/StandardCPlusPlusModules.rst
+++ b/clang/docs/StandardCPlusPlusModules.rst
@@ -652,14 +652,14 @@ in the future. The expected roadmap for Reduced BMIs as of Clang 19.x is:
comes, the term BMI will refer to the Reduced BMI and the Full BMI will only
be meaningful to build systems which elect to support two-phase compilation.
-Experimental No Transitive Change
----------------------------------
+Experimental Non Cascade Change
+-------------------------------
This section is primarily for build system vendors. For end compiler users,
if you don't want to read it all, this is helpful to reduce recompilations
We encourage build system vendors and end users try this out and bring feedbacks
-Before Clang 19, a change in BMI of any (transitive) dependency would case the
+Before Clang 19, a change in BMI of any (transitive) dependency would cause the
outputs of the BMI to change. Starting with Clang 19, changes to non-direct
dependencies should not directly affect the output BMI, unless they affect the
results of the compilations. We expect that there are many more opportunities
@@ -727,9 +727,9 @@ then the contents of ``useBOnly.pcm`` remain unchanged.
Consequently, if the build system only bases recompilation decisions on directly imported modules,
it becomes possible to skip the recompilation of ``Use.cc``.
It should be fine because the altered interfaces do not affect ``Use.cc`` in any way;
-there are no transitive changes.
+there are non cascade changes.
-When clang generates a BMI, it records the hash values of all potentially contributory BMIs
+When ``Clang`` generates a BMI, it records the hash values of all potentially contributory BMIs
for the BMI being produced. This ensures that build systems are not required to consider
transitively imported modules when deciding whether to recompile.
@@ -743,7 +743,7 @@ can go back to the transitive change mode safely at any time.
Interactions with Reduced BMI
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-With reduced BMI, the no transitive change feature can be more powerful. For example,
+With reduced BMI, the non cascade change feature can be more powerful. For example,
.. code-block:: c++
@@ -760,7 +760,7 @@ With reduced BMI, the no transitive change feature can be more powerful. For exa
$ clang++ -std=c++20 A.cppm -c -fmodule-output=A.pcm -fexperimental-modules-reduced-bmi -o A.o
$ clang++ -std=c++20 B.cppm -c -fmodule-output=B.pcm -fexperimental-modules-reduced-bmi -o B.o -fmodule-file=A=A.pcm
- $md5sum B.pcm
+ $ md5sum B.pcm
6c2bd452ca32ab418bf35cd141b060b9 B.pcm
And let's change the implementation for ``A.cppm`` into:
@@ -777,7 +777,7 @@ and recompile the example:
$ clang++ -std=c++20 A.cppm -c -fmodule-output=A.pcm -fexperimental-modules-reduced-bmi -o A.o
$ clang++ -std=c++20 B.cppm -c -fmodule-output=B.pcm -fexperimental-modules-reduced-bmi -o B.o -fmodule-file=A=A.pcm
- $md5sum B.pcm
+ $ md5sum B.pcm
6c2bd452ca32ab418bf35cd141b060b9 B.pcm
We should find the contents of ``B.pcm`` keeps the same. In such case, the build system is
More information about the cfe-commits
mailing list