[libc-commits] [libc] [libc][docs] Update docs to reflect new headergen (PR #102381)
via libc-commits
libc-commits at lists.llvm.org
Wed Aug 7 14:50:11 PDT 2024
llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-libc
Author: Michael Jones (michaelrj-google)
<details>
<summary>Changes</summary>
Since new headergen is now the default for building LLVM-libc, the docs
need to be updated to reflect that. While I was editing those docs, I
took a quick pass at updating other out-of-date pages.
---
Patch is 27.69 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/102381.diff
13 Files Affected:
- (modified) libc/docs/build_and_test.rst (-7)
- (modified) libc/docs/contributing.rst (+1-13)
- (removed) libc/docs/dev/api_test.rst (-25)
- (removed) libc/docs/dev/ground_truth_specification.rst (-11)
- (modified) libc/docs/dev/index.rst (-3)
- (removed) libc/docs/dev/mechanics_of_public_api.rst (-29)
- (modified) libc/docs/dev/source_tree_layout.rst (+17-7)
- (modified) libc/docs/full_cross_build.rst (+10-100)
- (modified) libc/docs/full_host_build.rst (+74-8)
- (modified) libc/docs/gpu/building.rst (+5-1)
- (modified) libc/docs/index.rst (+10-7)
- (modified) libc/docs/overlay_mode.rst (+20-16)
- (modified) libc/docs/porting.rst (-15)
``````````diff
diff --git a/libc/docs/build_and_test.rst b/libc/docs/build_and_test.rst
index 22b09b07d9612d..ccd8b5bbee4759 100644
--- a/libc/docs/build_and_test.rst
+++ b/libc/docs/build_and_test.rst
@@ -38,13 +38,6 @@ The libc can be built and tested in two different modes:
$> ninja libc-integration-tests
- #. API verification test - See :ref:`api_test` for more information about
- the API test. It can be run by the command:
-
- .. code-block:: sh
-
- $> ninja libc-api-test
-
Building with VSCode
====================
diff --git a/libc/docs/contributing.rst b/libc/docs/contributing.rst
index bd7d9d79be57d7..a674290cf6dc03 100644
--- a/libc/docs/contributing.rst
+++ b/libc/docs/contributing.rst
@@ -4,7 +4,7 @@
Contributing to the libc Project
================================
-LLVM's libc is being developed as part of the LLVM project so contributions
+LLVM-libc is being developed as part of the LLVM project so contributions
to the libc project should also follow the general LLVM
`contribution guidelines <https://llvm.org/docs/Contributing.html>`_. Below is
a list of open projects that one can start with:
@@ -31,24 +31,12 @@ a list of open projects that one can start with:
directory. So, a simple but mechanical project would be to move the parts
following the old styles to the new style.
-#. **Integrating with the rest of the LLVM project** - There are two parts to
- this project:
-
- #. One is about adding CMake facilities to optionally link the libc's overlay
- static archive (see :ref:`overlay_mode`) with other LLVM tools/executables.
- #. The other is about putting plumbing in place to release the overlay static
- archive (see :ref:`overlay_mode`) as part of the LLVM binary releases.
-
#. **Implement Linux syscall wrappers** - A large portion of the POSIX API can
be implemented as syscall wrappers on Linux. A good number have already been
implemented but many more are yet to be implemented. So, a project of medium
complexity would be to implement syscall wrappers which have not yet been
implemented.
-#. **Add a better random number generator** - The current random number
- generator has a very small range. This has to be improved or switched over
- to a fast random number generator with a large range.
-
#. **Update the clang-tidy lint rules and use them in the build and/or CI** -
Currently, the :ref:`clang_tidy_checks` have gone stale and are mostly unused
by the developers and on the CI builders. This project is about updating
diff --git a/libc/docs/dev/api_test.rst b/libc/docs/dev/api_test.rst
deleted file mode 100644
index 3191a32b7e3fb1..00000000000000
--- a/libc/docs/dev/api_test.rst
+++ /dev/null
@@ -1,25 +0,0 @@
-.. _api_test:
-
-========
-API Test
-========
-
-.. warning::
- This page is severely out of date. Much of the information it contains may be
- incorrect. Please only remove this warning once the page has been updated.
-
-The implementation of libc-project is unique because our public C header files
-are generated using information from ground truth captured in TableGen files.
-Unit tests only exercise the internal C++ implementations and don't ensure the
-headers were generated by the build system and that the generated header files
-contain the expected declarations and definitions. A simple solution is to have
-contributors write an integration test for each individual function as a C
-program; however, this would place a large burden on contributors and duplicates
-some effort from the unit tests.
-
-Instead we automate the generation of what we call as an API test. This API test
-ensures that public facing symbols are visible, that the header files are
-generated as expected, and that each libc function has the correct function
-prototype as specified by the standards. The API test cmake rules are located in
-``test/src/CMakeLists.txt``. The source file for the API test is generated in
-``<build directory>/projects/libc/test/src/public_api_test.cpp``
diff --git a/libc/docs/dev/ground_truth_specification.rst b/libc/docs/dev/ground_truth_specification.rst
deleted file mode 100644
index f2540b6f78e715..00000000000000
--- a/libc/docs/dev/ground_truth_specification.rst
+++ /dev/null
@@ -1,11 +0,0 @@
-The ground truth of standards
-=============================
-
-Like any modern libc, LLVM libc also supports a wide number of standards and
-extensions. To avoid developing headers, wrappers and sources in a disjointed
-fashion, LLVM libc employs ground truth files. These files live under the
-``spec`` directory and list ground truth corresponding the ISO C standard, the
-POSIX extension standard, etc. For example, the path to the ground truth file
-for the ISO C standard is ``spec/stdc.td``. Tools like the header generator
-(described in the header generation document), docs generator, etc. use the
-ground truth files to generate headers, docs etc.
diff --git a/libc/docs/dev/index.rst b/libc/docs/dev/index.rst
index 87712afcae2ac6..c16121feb3a45d 100644
--- a/libc/docs/dev/index.rst
+++ b/libc/docs/dev/index.rst
@@ -15,10 +15,7 @@ Navigate to the links below for information on the respective topics:
config_options
clang_tidy_checks
fuzzing
- ground_truth_specification
header_generation
implementation_standard
undefined_behavior
printf_behavior
- api_test
- mechanics_of_public_api
diff --git a/libc/docs/dev/mechanics_of_public_api.rst b/libc/docs/dev/mechanics_of_public_api.rst
deleted file mode 100644
index 257ab3d71bc17a..00000000000000
--- a/libc/docs/dev/mechanics_of_public_api.rst
+++ /dev/null
@@ -1,29 +0,0 @@
-The mechanics of the ``public_api`` command
-===========================================
-
-The build system, in combination with the header generation mechanism,
-facilitates the fine grained ability to pick and choose the public API one wants
-to expose on their platform. The public header files are always generated from
-the corresponding ``.h.def`` files. A header generation command ``%%public_api``
-is listed in these files. In the generated header file, the header generator
-replaces this command with the public API relevant for the target platform.
-
-Under the hood
---------------
-
-When the header generator sees the ``%%public_api`` command, it looks up the
-API config file for the platform in the path ``config/<platform>/api.td``.
-The API config file lists two kinds of items:
-
-1. The list of standards from which the public entities available on the platform
- are derived from.
-2. For each header file exposed on the platform, the list of public members
- provided in that header file.
-
-Note that, the header generator only learns the names of the public entities
-from the header config file (the 2nd item from above.) The exact manner in which
-the entities are to be declared is got from the standards (the 1st item from
-above.)
-
-See the ground truth document for more information on how the standards are
-formally listed in LLVM libc using LLVM table-gen files.
diff --git a/libc/docs/dev/source_tree_layout.rst b/libc/docs/dev/source_tree_layout.rst
index 0bcedc96a133c3..8b423a1712cc81 100644
--- a/libc/docs/dev/source_tree_layout.rst
+++ b/libc/docs/dev/source_tree_layout.rst
@@ -14,9 +14,10 @@ directories::
- docs
- examples
- fuzzing
+ - hdr
- include
- lib
- - spec
+ - newhdrgen
- src
- startup
- test
@@ -62,6 +63,14 @@ The directory structure within this directory mirrors the directory structure
of the top-level ``libc`` directory itself. For more details, see
:doc:`fuzzing`.
+The ``hdr`` directory
+---------------------
+
+This directory contains proxy headers which are included from the files in the
+src directory. These proxy headers either include our internal type or macro
+definitions, or the system's type or macro definitions, depending on if we are
+in fullbuild or overlay mode.
+
The ``include`` directory
-------------------------
@@ -80,13 +89,14 @@ The ``lib`` directory
This directory contains a ``CMakeLists.txt`` file listing the targets for the
public libraries ``libc.a``, ``libm.a`` etc.
-The ``spec`` directory
-----------------------
+The ``newhdrgen`` directory
+---------------------------
-This directory contains the specifications for the types, macros, and entrypoint
-functions. These definitions come from the various standards and extensions
-LLVM-libc supports, and they are used along with the ``*.h.def`` files and the
-config files to generate the headers for fullbuild mode.
+This directory contains the sources and specifications for the types, macros
+and entrypoint functions. These definitions are organized in the ``yaml``
+subdirectory and match the organization of the ``*.h.def`` files. This folder
+also contains the python sources for new headergen, which is what generates the
+headers.
The ``src`` directory
---------------------
diff --git a/libc/docs/full_cross_build.rst b/libc/docs/full_cross_build.rst
index 100e17a977e764..8570def4ecadd9 100644
--- a/libc/docs/full_cross_build.rst
+++ b/libc/docs/full_cross_build.rst
@@ -14,29 +14,22 @@ target system which is not the same as the system on which the libc is being
built. For example, you could be building for a bare metal aarch64 *target* on a
Linux x86_64 *host*.
-There are three main recipes to cross build the full libc. Each one serves a
+There are two main recipes to cross build the full libc. Each one serves a
different use case. Below is a short description of these recipes to help users
pick the recipe that best suites their needs and contexts.
* **Standalone cross build** - Using this recipe one can build the libc using a
compiler of their choice. One should use this recipe if their compiler can
build for the host as well as the target.
-* **Runtimes cross build** - In this recipe, one will have to first build the
- libc build tools for the host separately and then use those build tools to
- build the libc. Users can use the compiler of their choice to build the
- libc build tools as well as the libc. One should use this recipe if they
- have to use a host compiler to build the build tools for the host and then
- use a target compiler (which is different from the host compiler) to build
- the libc.
* **Bootstrap cross build** - In this recipe, one will build the ``clang``
compiler and the libc build tools for the host first, and then use them to
- build the libc for the target. Unlike with the runtimes build recipe, the
- user does not have explicitly build ``clang`` and other libc build tools.
+ build the libc for the target. Unlike with the standalone build recipe, the
+ user does not have explicitly build ``clang`` and other build tools.
They get built automatically before building the libc. One should use this
recipe if they intend use the built ``clang`` and the libc as part of their
toolchain for the target.
-The following sections present the three recipes in detail.
+The following sections present the two recipes in detail.
Standalone cross build
======================
@@ -61,9 +54,9 @@ Below is the CMake command to configure the standalone crossbuild of the libc.
$> cd build
$> C_COMPILER=<C compiler> # For example "clang"
$> CXX_COMPILER=<C++ compiler> # For example "clang++"
- $> cmake ../llvm \
+ $> cmake ../runtimes \
-G Ninja \
- -DLLVM_ENABLE_PROJECTS=libc \
+ -DLLVM_ENABLE_RUNTIMES=libc \
-DCMAKE_C_COMPILER=$C_COMPILER \
-DCMAKE_CXX_COMPILER=$CXX_COMPILER \
-DLLVM_LIBC_FULL_BUILD=ON \
@@ -72,8 +65,8 @@ Below is the CMake command to configure the standalone crossbuild of the libc.
We will go over the special options passed to the ``cmake`` command above.
-* **Enabled Projects** - Since we want to build the libc project, we list
- ``libc`` as the enabled project.
+* **Enabled Runtimes** - Since we want to build LLVM-libc, we list
+ ``libc`` as the enabled runtime.
* **The full build option** - Since we want to build the full libc, we pass
``-DLLVM_LIBC_FULL_BUILD=ON``.
* **The target triple** - This is the target triple of the target for which
@@ -94,88 +87,6 @@ The above ``ninja`` command will build the libc static archives ``libc.a`` and
``libm.a`` for the target specified with ``-DLIBC_TARGET_TRIPLE`` in the CMake
configure step.
-.. _runtimes_cross_build:
-
-Runtimes cross build
-====================
-
-The *runtimes cross build* is very similar to the standalone crossbuild but the
-user will have to first build the libc build tools for the host separately. One
-should use this recipe if they want to use a different host and target compiler.
-Note that the libc build tools MUST be in sync with the libc. That is, the
-libc build tools and the libc, both should be built from the same source
-revision. At the time of this writing, there is only one libc build tool that
-has to be built separately. It is done as follows:
-
-.. code-block:: sh
-
- $> cd llvm-project # The llvm-project checkout
- $> mkdir build-libc-tools # A different build directory for the build tools
- $> cd build-libc-tools
- $> HOST_C_COMPILER=<C compiler for the host> # For example "clang"
- $> HOST_CXX_COMPILER=<C++ compiler for the host> # For example "clang++"
- $> cmake ../llvm \
- -G Ninja \
- -DLLVM_ENABLE_PROJECTS=libc \
- -DCMAKE_C_COMPILER=$HOST_C_COMPILER \
- -DCMAKE_CXX_COMPILER=$HOST_CXX_COMPILER \
- -DLLVM_LIBC_FULL_BUILD=ON \
- -DCMAKE_BUILD_TYPE=Debug # User can choose to use "Release" build type
- $> ninja libc-hdrgen
-
-The above commands should build a binary named ``libc-hdrgen``. Copy this binary
-to a directory of your choice.
-
-CMake configure step
---------------------
-
-After copying the ``libc-hdrgen`` binary to say ``/path/to/libc-hdrgen``,
-configure the libc build using the following command:
-
-.. code-block:: sh
-
- $> cd llvm-project # The llvm-project checkout
- $> mkdir build
- $> cd build
- $> TARGET_C_COMPILER=<C compiler for the target>
- $> TARGET_CXX_COMPILER=<C++ compiler for the target>
- $> HDRGEN=</path/to/libc-hdrgen>
- $> TARGET_TRIPLE=<Your target triple>
- $> cmake ../runtimes \
- -G Ninja \
- -DLLVM_ENABLE_RUNTIMES=libc \
- -DCMAKE_C_COMPILER=$TARGET_C_COMPILER \
- -DCMAKE_CXX_COMPILER=$TARGET_CXX_COMPILER \
- -DLLVM_LIBC_FULL_BUILD=ON \
- -DLIBC_HDRGEN_EXE=$HDRGEN \
- -DLIBC_TARGET_TRIPLE=$TARGET_TRIPLE \
- -DCMAKE_BUILD_TYPE=Debug # User can choose to use "Release" build type
-
-Note the differences in the above cmake command versus the one used in the
-CMake configure step of the standalone build recipe:
-
-* Instead of listing ``libc`` in ``LLVM_ENABLED_PROJECTS``, we list it in
- ``LLVM_ENABLED_RUNTIMES``.
-* Instead of using ``llvm-project/llvm`` as the root CMake source directory,
- we use ``llvm-project/runtimes`` as the root CMake source directory.
-* The path to the ``libc-hdrgen`` binary built earlier is specified with
- ``-DLIBC_HDRGEN_EXE=/path/to/libc-hdrgen``.
-
-Build step
-----------
-
-The build step in the runtimes build recipe is exactly the same as that of
-the standalone build recipe:
-
-.. code-block:: sh
-
- $> ninja libc libm
-
-As with the standalone build recipe, the above ninja command will build the
-libc static archives for the target specified with ``-DLIBC_TARGET_TRIPLE`` in
-the CMake configure step.
-
-
Bootstrap cross build
=====================
@@ -203,8 +114,7 @@ CMake configure step
-DLLVM_RUNTIME_TARGETS=$TARGET_TRIPLE \
-DCMAKE_BUILD_TYPE=Debug
-Note how the above cmake command differs from the one used in the other two
-recipes:
+Note how the above cmake command differs from the one used in the other recipe:
* ``clang`` is listed in ``-DLLVM_ENABLE_PROJECTS`` and ``libc`` is
listed in ``-DLLVM_ENABLE_RUNTIMES``.
@@ -214,7 +124,7 @@ recipes:
Build step
----------
-The build step is similar to the other two recipes:
+The build step is similar to the other recipe:
.. code-block:: sh
diff --git a/libc/docs/full_host_build.rst b/libc/docs/full_host_build.rst
index 4fb3072590f322..cc05be9fb52afc 100644
--- a/libc/docs/full_host_build.rst
+++ b/libc/docs/full_host_build.rst
@@ -10,15 +10,83 @@ Full Host Build
In this document, we will present a recipe to build the full libc for the host.
When we say *build the libc for the host*, the goal is to build the libc for
-the same system on which the libc is being built. Also, we will take this
-opportunity to demonstrate how one can set up a *sysroot* (see the documentation
+the same system on which the libc is being built. First, we will explain how to
+build for developing LLVM-libc, then we will explain how to build LLVM-libc as
+part of a complete toolchain.
+
+Configure the build for development
+===================================
+
+
+Below is the list of commands for a simple recipe to build LLVM-libc for
+development. In this we've set the Ninja generator, set the build type to
+"Debug", and enabled the Scudo allocator. This build also enables generating the
+documentation and verbose cmake logging, which are useful development features.
+
+.. note::
+ if your build fails with an error saying the compiler can't find
+ ``<asm/unistd.h>`` or similar then you're probably missing the symlink from
+ ``/usr/include/asm`` to ``/usr/include/<HOST TRIPLE>/asm``. Installing the
+ ``gcc-multilib`` package creates this symlink, or you can do it manually with
+ this command:
+ ``sudo ln -s /usr/include/<HOST TRIPLE>/asm /usr/include/asm``
+ (your host triple will probably be similar to ``x86_64-linux-gnu``)
+
+.. code-block:: sh
+
+ $> cd llvm-project # The llvm-project checkout
+ $> mkdir build
+ $> cd build
+ $> cmake ../runtimes \
+ -G Ninja \
+ -DCMAKE_C_COMPILER=clang \
+ -DCMAKE_CXX_COMPILER=clang++ \
+ -DLLVM_ENABLE_RUNTIMES="libc;compiler-rt" \
+ -DLLVM_LIBC_FULL_BUILD=ON \
+ -DCMAKE_BUILD_TYPE=Debug \
+ -DLLVM_LIBC_INCLUDE_SCUDO=ON \
+ -DCOMPILER_RT_BUILD_SCUDO_STANDALONE_WITH_LLVM_LIBC=ON \
+ -DCOMPILER_RT_BUILD_GWP_ASAN=OFF \
+ -DCOMPILER_RT_SCUDO_STANDALONE_BUILD_SHARED=OFF \
+ -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
+ -DLLVM_ENABLE_SPHINX=ON -DLIBC_INCLUDE_DOCS=ON \
+ -DLIBC_CMAKE_VERBOSE_LOGGING=ON
+
+Build and test
+==============
+
+After configuring the build with the above ``cmake`` command, one can build test
+libc with the following command:
+
+.. code-block:: sh
+
+ $> ninja libc check-libc
+
+To build the docs run this command:
+
+
+.. code-block:: sh
+
+ $> ninja docs-libc-html
+
+To run a specific test, use the following:
+
+.. code-block:: sh
+
+ $> ninja libc.test.src.<HEADER>.<FUNCTION>_test.__unit__
+ $> ninja libc.test.src.ctype.isalpha_test.__unit__ # EXAMPLE
+
+Configure the complete toolchain build
+======================================
+
+For a complete toolchain we recommend creating a *sysroot* (see the documentation
of the ``--sysroot`` option here:
`<https://gcc.gnu.org/onlinedocs/gcc/Directory-Options.html>`_) which includes
not only the components of LLVM's libc, but also a full LLVM only toolchain
consisting of the `clang <https://clang.llvm.org/>`_ compiler, the
`lld <https://lld.llvm.org/>`_ linker and the
-`compiler-rt <https://compiler-rt.llvm.org/>`_ runtime libraries. LLVM's libc is
-not yet complete enough to allow using and linking a C++ application against
+`compiler-rt <https://compiler-rt.llvm.org/>`_ runtime libraries. LLVM-libc is
+not quite complete enough to allow using and linking a C++ application against
a C++ standard library (like libc++). Hence, we do not include
`libc++ <https://libcxx.llvm.org/>`_ in the sysroot.
@@ -26,9 +94,6 @@ a C++ standard library (like libc++). Hence, we do not include
`libc++ <https://libcxx.llvm.org/>`_, libcxx-abi and libunwind in the
LLVM only toolchain and use them to build and link C++ applications.
-Configure the full libc build
-===============================
-
Below is the list of commands for a simple recipe to build and install the
libc components along with other components of an LLVM only toolchain. In this
we've set the Ninja generator, enabled a full compiler suite, set the build
@@ -43,6 +108,7 @@ to use the freshly built lld and compiler-rt....
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/102381
More information about the libc-commits
mailing list