[clang] [llvm] Claude/implement customizable functions 017 pyt x2 dy ha66 ax vd laeae5 (PR #168420)
via llvm-commits
llvm-commits at lists.llvm.org
Mon Nov 17 10:22:37 PST 2025
llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-clang
Author: Greg von Winckel (gregvw)
<details>
<summary>Changes</summary>
---
Patch is 70.35 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/168420.diff
4 Files Affected:
- (added) CUSTOMIZABLE_FUNCTIONS_DEV.md (+392)
- (added) clang/docs/CustomizableFunctionsDesign.md (+1037)
- (added) clang/docs/CustomizableFunctionsTestPlan.md (+744)
- (added) custom-functions-dev.sh (+418)
``````````diff
diff --git a/CUSTOMIZABLE_FUNCTIONS_DEV.md b/CUSTOMIZABLE_FUNCTIONS_DEV.md
new file mode 100644
index 0000000000000..088d403cf0960
--- /dev/null
+++ b/CUSTOMIZABLE_FUNCTIONS_DEV.md
@@ -0,0 +1,392 @@
+# Customizable Functions Feature Development Guide
+
+This guide covers developing and testing the Customizable Functions feature in Clang.
+
+## Quick Start
+
+### First Time Setup
+
+```bash
+# 1. Configure the build (debug mode, optimized for development)
+./custom-functions-dev.sh configure debug
+
+# 2. Build clang
+./custom-functions-dev.sh build
+
+# 3. Verify the build
+./custom-functions-dev.sh info
+```
+
+### Development Workflow
+
+```bash
+# Make changes to source files...
+
+# Build only what changed
+./custom-functions-dev.sh build
+
+# Run customizable functions tests
+./custom-functions-dev.sh test customizable
+
+# Or run specific test categories
+./custom-functions-dev.sh test parser
+./custom-functions-dev.sh test sema
+./custom-functions-dev.sh test codegen
+```
+
+## Build Script Commands
+
+### Configure
+
+```bash
+# Debug build (default - best for development)
+./custom-functions-dev.sh configure debug
+
+# Release build (for performance testing)
+./custom-functions-dev.sh configure release
+
+# Minimal build (fastest iteration)
+./custom-functions-dev.sh configure minimal
+```
+
+### Build
+
+```bash
+# Build clang
+./custom-functions-dev.sh build
+
+# Build specific target
+./custom-functions-dev.sh build check-clang-sema
+
+# Build with custom job count
+BUILD_JOBS=8 ./custom-functions-dev.sh build
+```
+
+### Test
+
+```bash
+# Run all customizable functions tests
+./custom-functions-dev.sh test customizable
+
+# Run parser tests
+./custom-functions-dev.sh test parser
+
+# Run semantic analysis tests
+./custom-functions-dev.sh test sema
+
+# Run code generation tests
+./custom-functions-dev.sh test codegen
+
+# Run AST tests
+./custom-functions-dev.sh test ast
+
+# Run all Clang tests
+./custom-functions-dev.sh test all
+
+# Run tests matching specific pattern
+./custom-functions-dev.sh test "custom.*syntax"
+```
+
+### Utility Commands
+
+```bash
+# Show build information
+./custom-functions-dev.sh info
+
+# Clean build directory
+./custom-functions-dev.sh clean
+
+# Rebuild from scratch
+./custom-functions-dev.sh rebuild
+
+# Show help
+./custom-functions-dev.sh help
+```
+
+## Build Optimization
+
+The build script uses several optimizations for faster development:
+
+- **Ninja build system**: Parallel builds with dependency tracking
+- **Optimized TableGen**: Faster build times
+- **Split DWARF**: Faster linking with debug info
+- **Single target (X86)**: Reduces build time
+- **Minimal LLVM projects**: Only builds Clang, not other tools
+
+### Build Times (Approximate)
+
+On a typical development machine (8 cores):
+
+- **Initial build**: 20-40 minutes (full Clang build)
+- **Incremental parser change**: 30-60 seconds
+- **Incremental Sema change**: 1-2 minutes
+- **Incremental test-only**: 5-10 seconds
+
+## Development Tips
+
+### Fast Iteration Cycle
+
+1. **Use incremental builds**: The script automatically detects changes
+2. **Test specific categories**: Don't run all tests every time
+3. **Use minimal build mode**: For rapid prototyping
+
+```bash
+# Example fast iteration
+vim clang/lib/Parse/ParseDecl.cpp
+./custom-functions-dev.sh build
+./custom-functions-dev.sh test parser
+```
+
+### Debugging Build Issues
+
+```bash
+# Show detailed build information
+./custom-functions-dev.sh info
+
+# Clean and rebuild
+./custom-functions-dev.sh rebuild debug
+
+# Verbose build output
+cd build-custom-functions && ninja -v clang
+```
+
+### Running Individual Tests
+
+```bash
+# Build directory contains lit tool
+cd build-custom-functions
+
+# Run a specific test file
+./bin/llvm-lit -v ../clang/test/Parser/cxx-customizable-functions.cpp
+
+# Run with verbose output
+./bin/llvm-lit -v -a ../clang/test/SemaCXX/customizable-functions-*.cpp
+```
+
+### Using the Built Clang
+
+```bash
+# Direct path to built clang
+./build-custom-functions/bin/clang --version
+
+# Test the custom keyword (when implemented)
+./build-custom-functions/bin/clang -std=c++20 -fcustomizable-functions -fsyntax-only test.cpp
+
+# See generated AST
+./build-custom-functions/bin/clang -std=c++20 -fcustomizable-functions -Xclang -ast-dump test.cpp
+```
+
+## Project Structure
+
+```
+llvm-project/
+├── custom-functions-dev.sh # Main build script
+├── CUSTOMIZABLE_FUNCTIONS_DEV.md # This file
+├── build-custom-functions/ # Build output directory
+│ ├── bin/clang # Built clang binary
+│ └── compile_commands.json # For IDE integration
+├── clang/
+│ ├── docs/
+│ │ ├── CustomizableFunctionsDesign.md # Design document
+│ │ └── CustomizableFunctionsTestPlan.md # Test plan
+│ ├── include/clang/
+│ │ ├── Basic/
+│ │ │ ├── TokenKinds.def # Add 'custom' keyword here
+│ │ │ └── LangOptions.def # Add language option here
+│ │ ├── Parse/
+│ │ │ └── Parser.h # Parser interface
+│ │ ├── Sema/
+│ │ │ ├── Sema.h # Semantic analysis interface
+│ │ │ └── SemaCustomFunction.h # Custom function transformation (new)
+│ │ └── AST/
+│ │ └── Decl.h # AST node declarations
+│ ├── lib/
+│ │ ├── Parse/
+│ │ │ └── ParseDecl.cpp # Parse 'custom' keyword
+│ │ ├── Sema/
+│ │ │ ├── SemaDecl.cpp # Semantic analysis
+│ │ │ └── SemaCustomFunction.cpp # Transform logic (new)
+│ │ └── AST/
+│ │ └── Decl.cpp # AST implementation
+│ └── test/
+│ ├── Parser/
+│ │ └── cxx-customizable-functions-*.cpp
+│ ├── SemaCXX/
+│ │ └── customizable-functions-*.cpp
+│ ├── CodeGenCXX/
+│ │ └── customizable-functions-*.cpp
+│ └── AST/
+│ └── customizable-functions-*.cpp
+```
+
+## Common Development Tasks
+
+### Adding a New Keyword
+
+1. Add to `clang/include/clang/Basic/TokenKinds.def`
+2. Rebuild: `./custom-functions-dev.sh build`
+3. Test: `./custom-functions-dev.sh test parser`
+
+### Adding Parser Support
+
+1. Modify `clang/lib/Parse/ParseDecl.cpp`
+2. Add tests in `clang/test/Parser/`
+3. Build and test:
+ ```bash
+ ./custom-functions-dev.sh build
+ ./custom-functions-dev.sh test parser
+ ```
+
+### Adding Semantic Analysis
+
+1. Create `clang/lib/Sema/SemaCustomFunction.cpp`
+2. Add hook in `clang/lib/Sema/SemaDecl.cpp`
+3. Add tests in `clang/test/SemaCXX/`
+4. Build and test:
+ ```bash
+ ./custom-functions-dev.sh build
+ ./custom-functions-dev.sh test sema
+ ```
+
+### Adding Code Generation
+
+1. Modify `clang/lib/CodeGen/CGDecl.cpp`
+2. Add tests in `clang/test/CodeGenCXX/`
+3. Build and test:
+ ```bash
+ ./custom-functions-dev.sh build
+ ./custom-functions-dev.sh test codegen
+ ```
+
+## IDE Integration
+
+### Compile Commands (for clangd, CLion, etc.)
+
+The build script automatically creates a symlink to `compile_commands.json`:
+
+```bash
+llvm-project/compile_commands.json -> build-custom-functions/compile_commands.json
+```
+
+This enables IDE features like:
+- Code completion
+- Jump to definition
+- Error highlighting
+- Refactoring tools
+
+### VS Code
+
+Install the clangd extension and it will automatically find the compile commands.
+
+### CLion
+
+CLion will detect the CMake project automatically. Point it to `build-custom-functions/`.
+
+## Testing Strategy
+
+### Test Categories
+
+1. **Parser Tests**: Syntax validation
+ - `./custom-functions-dev.sh test parser`
+
+2. **Semantic Tests**: Type checking, constraints
+ - `./custom-functions-dev.sh test sema`
+
+3. **CodeGen Tests**: LLVM IR generation
+ - `./custom-functions-dev.sh test codegen`
+
+4. **AST Tests**: AST structure verification
+ - `./custom-functions-dev.sh test ast`
+
+5. **Integration Tests**: End-to-end workflows
+ - `./custom-functions-dev.sh test customizable`
+
+### Writing New Tests
+
+Create test files following this pattern:
+
+```cpp
+// RUN: %clang_cc1 -std=c++20 -fcustomizable-functions -fsyntax-only -verify %s
+
+// Test case description
+custom void my_test() { } // expected-no-diagnostics
+
+// Or with expected errors
+custom void bad_test(); // expected-error {{custom functions must have a body}}
+```
+
+Place in appropriate directory:
+- Parser tests: `clang/test/Parser/`
+- Semantic tests: `clang/test/SemaCXX/`
+- CodeGen tests: `clang/test/CodeGenCXX/`
+
+## Troubleshooting
+
+### Build Fails
+
+```bash
+# Clean and rebuild
+./custom-functions-dev.sh rebuild
+
+# Check for CMake issues
+cd build-custom-functions
+cmake ..
+
+# Verbose build to see errors
+ninja -v clang
+```
+
+### Tests Fail
+
+```bash
+# Run specific test with verbose output
+cd build-custom-functions
+./bin/llvm-lit -v ../clang/test/Parser/cxx-customizable-functions.cpp -a
+
+# Check test expectations
+cat ../clang/test/Parser/cxx-customizable-functions.cpp
+```
+
+### Performance Issues
+
+```bash
+# Use release build for performance testing
+./custom-functions-dev.sh configure release
+./custom-functions-dev.sh build
+
+# Increase parallel jobs (if you have RAM)
+BUILD_JOBS=16 ./custom-functions-dev.sh build
+```
+
+## Environment Variables
+
+- `BUILD_JOBS`: Number of parallel build jobs (default: nproc)
+- `BUILD_TYPE`: Override build type (Debug/Release)
+- `CC`: C compiler to use
+- `CXX`: C++ compiler to use
+
+Example:
+```bash
+export BUILD_JOBS=12
+export CC=clang
+export CXX=clang++
+./custom-functions-dev.sh configure debug
+```
+
+## References
+
+- [Design Document](clang/docs/CustomizableFunctionsDesign.md)
+- [Test Plan](clang/docs/CustomizableFunctionsTestPlan.md)
+- [LLVM Testing Infrastructure](https://llvm.org/docs/TestingGuide.html)
+- [Clang Internals Manual](https://clang.llvm.org/docs/InternalsManual.html)
+
+## Next Steps
+
+Once the build is configured:
+
+1. Review the design documents
+2. Start with Phase 1: Add keyword and basic parsing
+3. Add tests incrementally
+4. Iterate quickly with the build script
+
+Happy coding!
diff --git a/clang/docs/CustomizableFunctionsDesign.md b/clang/docs/CustomizableFunctionsDesign.md
new file mode 100644
index 0000000000000..e11d7c794c819
--- /dev/null
+++ b/clang/docs/CustomizableFunctionsDesign.md
@@ -0,0 +1,1037 @@
+# Customizable Functions Design Document
+
+**Author:** TBD
+**Date:** 2025-11-17
+**Status:** Draft
+
+## Table of Contents
+
+1. [Introduction](#introduction)
+2. [Motivation](#motivation)
+3. [Proposed Syntax](#proposed-syntax)
+4. [Semantics](#semantics)
+5. [Implementation Plan](#implementation-plan)
+6. [Testing Strategy](#testing-strategy)
+7. [Open Questions](#open-questions)
+8. [References](#references)
+
+---
+
+## Introduction
+
+This document proposes adding a new language feature to Clang: **Customizable Functions** using the `custom` keyword. This feature would allow library authors to easily create Customization Point Objects (CPOs) using the tag_invoke pattern without manually writing extensive boilerplate code.
+
+### Goals
+
+- Reduce boilerplate for implementing the tag_invoke CPO pattern
+- Improve library interface design and customization points
+- Maintain compatibility with existing C++20/23 code
+- Generate efficient, zero-overhead abstractions
+
+### Non-Goals
+
+- Replacing existing customization mechanisms (ADL, specialization)
+- Changing the behavior of tag_invoke itself
+- Adding runtime dispatch overhead
+
+---
+
+## Motivation
+
+### Current State: Manual CPO Implementation
+
+Library authors currently need to write significant boilerplate to create proper CPOs:
+
+```cpp
+namespace my_lib {
+ namespace CPO_DETAIL {
+ // Default implementation
+ void do_thing(auto& t) { t.method(); }
+
+ // Functor class
+ struct do_thing_fn {
+ // tag_invoke overload
+ template <typename T>
+ constexpr auto operator()(T& t) const
+ requires requires { tag_invoke(*this, t); }
+ {
+ return tag_invoke(*this, t);
+ }
+
+ // Fallback overload
+ template <typename T>
+ constexpr auto operator()(T& t) const
+ requires (!requires { tag_invoke(*this, t); })
+ {
+ return do_thing(t);
+ }
+ };
+ }
+
+ // Inline constexpr instance
+ inline constexpr CPO_DETAIL::do_thing_fn do_thing{};
+}
+```
+
+This pattern:
+- Is verbose and error-prone
+- Requires understanding of advanced C++ techniques
+- Must be repeated for each customization point
+- Is difficult to maintain and modify
+
+### Proposed State: Declarative Syntax
+
+With customizable functions, the same functionality becomes:
+
+```cpp
+namespace my_lib {
+ custom void do_thing(auto& t) {
+ t.method();
+ }
+}
+```
+
+The compiler automatically generates the CPO boilerplate, making the code:
+- More concise and readable
+- Less error-prone
+- Easier to maintain
+- Self-documenting
+
+### Real-World Use Cases
+
+1. **Generic Algorithm Libraries**: Customization points for ranges, algorithms
+2. **Serialization/Deserialization**: Custom serializers for user types
+3. **Logging/Debugging**: Customizable formatting and output
+4. **Resource Management**: Custom allocators, deleters
+5. **Async/Await Patterns**: Customizable awaitable operations
+
+---
+
+## Proposed Syntax
+
+### Basic Syntax
+
+```cpp
+custom <return-type> <function-name>(<parameters>) <body>
+```
+
+### Examples
+
+#### Simple Free Function
+```cpp
+namespace lib {
+ custom void print(auto const& value) {
+ std::cout << value;
+ }
+}
+```
+
+#### With Explicit Return Type
+```cpp
+custom int compute(int x, int y) {
+ return x + y;
+}
+```
+
+#### Multiple Parameters
+```cpp
+custom void transform(auto& container, auto&& func) {
+ for (auto& elem : container) {
+ func(elem);
+ }
+}
+```
+
+#### With Constraints
+```cpp
+custom void process(auto& t)
+ requires std::copyable<decltype(t)>
+{
+ t.process();
+}
+```
+
+#### Template Function
+```cpp
+template <typename T>
+custom void serialize(T const& value, std::ostream& out) {
+ out << value;
+}
+```
+
+### Syntax Restrictions
+
+- `custom` keyword must appear before the return type
+- Cannot be used with:
+ - Member functions (initially; may be relaxed later)
+ - Constructors/destructors
+ - Operators (except when implementing operator() for the generated functor)
+ - Virtual functions
+- Must have a function body (not just a declaration)
+- Can be used in any namespace (including global)
+
+---
+
+## Semantics
+
+### Transformation Overview
+
+When the compiler encounters a `custom` function, it performs the following transformation:
+
+1. **Create Detail Namespace** (optional, configurable)
+ - Named `CPO_DETAIL` or `<function_name>_detail`
+ - Contains the default implementation and functor class
+
+2. **Generate Default Implementation**
+ - Original function body becomes a hidden implementation function
+ - Used as the fallback when tag_invoke is not available
+
+3. **Generate Functor Class**
+ - Named `<function_name>_fn`
+ - Implements two operator() overloads:
+ - Primary: calls tag_invoke (when available)
+ - Fallback: calls default implementation
+
+4. **Create CPO Instance**
+ - Inline constexpr variable with original function name
+ - Type is the functor class
+ - This becomes the actual customization point
+
+### Generated Code Structure
+
+For a `custom` function:
+```cpp
+custom RetType func_name(Params...) { body }
+```
+
+The compiler generates:
+```cpp
+namespace CPO_DETAIL {
+ // Default implementation (hidden)
+ RetType func_name_impl(Params...) { body }
+
+ // Functor class
+ struct func_name_fn {
+ // Primary overload: use tag_invoke if available
+ template <typename... Args>
+ constexpr auto operator()(Args&&... args) const
+ noexcept(noexcept(tag_invoke(*this, std::forward<Args>(args)...)))
+ requires requires { tag_invoke(*this, std::forward<Args>(args)...); }
+ {
+ return tag_invoke(*this, std::forward<Args>(args)...);
+ }
+
+ // Fallback overload: use default implementation
+ template <typename... Args>
+ constexpr auto operator()(Args&&... args) const
+ noexcept(noexcept(func_name_impl(std::forward<Args>(args)...)))
+ requires (!requires { tag_invoke(*this, std::forward<Args>(args)...); }
+ && requires { func_name_impl(std::forward<Args>(args)...); })
+ {
+ return func_name_impl(std::forward<Args>(args)...);
+ }
+ };
+}
+
+// The actual CPO
+inline constexpr CPO_DETAIL::func_name_fn func_name{};
+```
+
+### Name Lookup and ADL
+
+The generated CPO follows standard C++ name lookup rules:
+
+1. **Unqualified calls** to the CPO trigger ADL
+2. **tag_invoke** is found via ADL in the namespace of the arguments
+3. Users customize by defining `tag_invoke` in their namespace:
+
+```cpp
+namespace user {
+ struct MyType { };
+
+ // Customization
+ void tag_invoke(lib::do_thing_fn, MyType& t) {
+ // Custom implementation
+ }
+}
+
+// Usage
+user::MyType obj;
+lib::do_thing(obj); // Finds user::tag_invoke via ADL
+```
+
+### Overload Resolution
+
+The two operator() overloads in the functor are constrained to be mutually exclusive:
+- Primary: `requires tag_invoke(*this, args...)`
+- Fallback: `requires !tag_invoke(*this, args...) && default_impl(args...)`
+
+This ensures:
+- No ambiguity during overload resolution
+- tag_invoke always preferred when available
+- Fallback only selected when tag_invoke is not viable
+
+### Template Instantiation
+
+For templated custom functions:
+```cpp
+template <typename T>
+custom void process(T& value) { ... }
+```
+
+The functor class itself is templated on the CPO's template parameters, and the operator() remains templated on the call-site arguments.
+
+---
+
+## Implementation Plan
+
+### Phase 1: Core Infrastructure (Minimal Viable Product)
+
+**Goal**: Get basic `custom` keyword parsing and simple code generation working.
+
+#### 1.1 Add Keyword and Language Option
+
+**Files**:
+- `clang/include/clang/Basic/TokenKinds.def`
+- `clang/include/clang/Basic/LangOptions.def`
+
+**Tasks**:
+- [ ] Add `custom` as a C++20 keyword with flag `KEYCUSTOMFN`
+- [ ] Add language option `CustomizableFunctions` (default: disabled)
+- [ ] Add `-fcustomizable-functions` / `-fno-customizable-functions` flags
+
+**Testing**:
+- [ ] Verify keyword is recognized when feature is enabled
+- [ ] Verify keyword is not recognized when feature is disabled
+
+#### 1.2 Extend Parser
+
+**Files**:
+- `clang/include/clang/Parse/Parser.h`
+- `clang/lib/Parse/ParseDecl.cpp`
+- `clang/include/clang/Sema/DeclSpec.h`
+
+**Tasks**:
+- [ ] Add `isCustomFunction` flag to `DeclSpec`
+- [ ] Modify `ParseDeclarationSpecifiers` to recognize `custom` keyword
+- [ ] Set the flag when `custom` is encountered
+- [ ] Ensure proper error handling for invalid uses
+
+**Testing**:
+- [ ] Parse simple `custom void foo() {}`
+- [ ] Reject `custom` on member functions
+- [ ] Reject `custom` on declarations without definitions
+- [ ] Proper error messages for invalid syntax
+
+#### 1.3 Extend AST
+
+**Files**:
+- `clang/include/clang/AST/Decl.h`
+- `clang/lib/AST/Decl.cpp`
+- `clang/lib/AST/DeclPrinter.cpp`
+- `clang/lib/AST/ASTDumper.cpp`
+
+**Tasks**:
+- [ ] Add `IsCustomFunction` bit to `FunctionDeclBitfields`
+- [ ] Add `isCustomFunction()` / `setCustomFunction()` methods
+- [ ] Update `DeclPrinter` to print `custom` keyword
+- [ ] Update `ASTDumper` to show custom function flag
+
+**Testing**:
+- [ ] AST dump shows custom function annotation
+- [ ] AST printer reproduces `custom` keyword
+
+#### 1.4 Basic Semantic Analysis
+
+**Files**:
+- `clang/lib/Sema/SemaDecl.cpp`
+- `clang/include/clang/Sema/Sema.h`
+
+**Tasks**:
+- [ ] Create `ActOnCustomFunctionDecl()` hook
+- [ ] Validate custom function constraints:
+ - Must have a body
+ - Cannot be member function
+ - Cannot be virtual
+ - Cannot be main()
+- [ ] Mark function as custom in AST
+
+**Testing**:
+- [ ] Semantic errors for invalid custom functions
+- [ ] Accept valid custom function declarations
+
+### Phase 2: Code Generation (Core Transformation)
+
+**Goal**: Generate the CPO boilerplate during Sema.
+
+###...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/168420
More information about the llvm-commits
mailing list