[cfe-dev] [RFC] Actions and initiation rules in Clang's refactoring engine

[Alex Lorenz via cfe-dev]

Hi all,

This is a follow-up to my initial refactoring proposal [1], in which I mentioned that I will send out a separate RFC that deals with the initiation of refactoring actions. This email outlines the proposed design ("action and rules") and describes how actions can control their own initiation. If you are interested in looking at code, I have a code sample [2] that takes Xcode's "Add missing switch cases" refactoring action, implements it in the new engine, and extends it to provide a global source transformation (add missing cases to all enums that operate on a particular switch) that is accessible via clang-refactor. The sample is not 100% complete, but it's pretty much almost there.

## Actions and Rules

A refactoring action is a class that defines a list of related refactoring rules (operations). These rules get grouped under a common umbrella - a single clang-refactor subcommand. An individual rule specifies how a function that performs a source transformation (i.e. refactoring) should run and when the call to that function can be initiated. Here's an outline of an action and a rule in code:

```
class TransformCode: public RefactoringAction {                // A refactoring action
  RefactoringActionRules createActionRules() const override {
    return {
      refactoring_action_rules::apply(transformCode, ...)      // A refactoring action rule
    };
  }
  static RefactoringResult transformCode(...) { ... }
}
```

Why are the rules grouped in actions? I'll try to answer this with an example. Let's consider the action that adds missing cases to a switch. In Xcode 9, you can add missing case statements to one switch at a time. However, it could be useful to have an operation that works on all switches that operate on a particular enum, as one could automatically update them after adding a new enum constant. This is when the distinction between rules and actions becomes useful, as it gives you the ability to create a single action that adds missing cases to a switch using two __very__ different rules - the first rule describes a local operation that's initiated when you select a single switch, while the other describes a global operation that works with different TUs and is initiated once you pass in the name of the enum to clang-refactor (it's also possible to have a similar rule that can be initiated when the user selects the enum declaration in the editor).

Each individual refactoring action maps to one subcommand in the clang-refactor tool. Clang-refactor will determine which rule should be invoked based on the command line arguments that are passed to it after the subcommand. These arguments are automatically processed by the tool, and the refactoring actions should never interact with any command line arguments directly. Individual rules that include a source selection requirement can be connected to IDEs by binding them to the editor command objects, e.g.:

`auto Rule = MyEditorCommand.bind(apply(performRefactoring, requiredSelection(...)));`.

### Rules

What exactly is a refactoring action rule? Simply put, a rule is a function and a list of requirements that map to the corresponding arguments for that function. Here's a rule with a single  `requiredOption` requirement that will print "Sample rule: abc" when a "-string-text=abc" argument is passed to clang-refactor:

```
apply([](std::string string) { llvm::outs() << "Sample rule: " << string << "\n"; },
      requiredOption(RefactoringOption<std::string>("string-text")));
```

Different combinations of requirements can create very different operations, like local operations that work with the raw token stream, or global operations that work with many parsed TUs. Each requirement produces a value that's passed to the rule's function. The following requirements will be supported by the refactoring engine:

-  `requiredSelection`. Rules with this requirement can be initiated using source selection only (clang-refactor will support selection). This requirement and the type that's passed to the rule's function is described in the next section.

-  `requiredOption`. Rules with this requirement can be initiated only when the specified option is given to clang-refactor (editors can support these rules if they provide the required options). This requirement can be constructed using a previously created `RefactoringOption<Type>` value. A `Type` value is passed to the rule's function, unless the option is mapped (more on that later).

-  `requiredIndexerCapabilites<...>`. Rules with this requirement can be initiated only when a particular client (clangd/libclang/clang-refactor) supports the set of specified indexer capabilities. An indexer object is passed to the rule's function. The indexer object has a type that's determined by the specified capabilities, and provides an API that exposes only these capabilities. This design guarantees that a refactoring action can __never__ produce an indexer operation that's not supported by a client that initiated the action.

-  `option`. Similar to `requiredOption`. However, a rule that includes an `option` requirement can be initiated even when that option is not specified. An `Optional<Type>` value is passed to the rule's function.

-  `join`. Can be used to group multiple `option`s into a tuple for convenience. An appropriate tuple value is passed to the rule's function. It might make sense to join other requirements in the future.

-  `requiredContinuation`. Doesn't affect initiation. Refactoring continuations have to be specified in the rule because the indexer API in the new engine will work with tagged continuations that provide a name for a particular continuation function and act as alternative "entry points" to the action. This convention will ensure that continuations that have serializable state can be invoked in a separate process. This in turn will enable the creation of a refactoring tool that can distribute work across processes and machines (on a per-TU/file basis) without any changes to the code of a particular refactoring action.

Some requirements can contain user-specified functions that might change their behaviour. They can also modify the type of value that's passed to the rule's function. The `requiredOption` and `option` requirements support the `map` operation, which transforms the option's value to a value of a different type. One could use `map` to convert options to action-specific types. The snippet below shows how `map` can be used to ensure that a string USR represents a valid enum declaration in the project (the validation and error handling is performed automatically by the engine):

```
requiredOption(RefactoringOption<std::string>("enum-usr")),
  .map([] (std::string USR) { return indexer::symbolForUSR<EnumDecl>(USR); })
```

### Selection Requirements

The `requiredSelection` requirement is quite different to the other ones. It's actually just a function that accepts a valid selection constraint and returns a value of any type that's passed to the rule's function or a diagnostic that ensures that the rule won't be initiated. For example, this requirement will pass `MySelection` value to the rule's function when the rule is initiated successfully:

```
struct MySelection {
  CallExpr &CE;
  DeclContext *EnclosingContext;
};

...

auto SelectionRule = requiredSelection([] (const ASTRefactoringOperation &Op,
                                           selection::Code<CallExpr> Selection) -> DiagOr<MySelection> {
  CallExpr &Call = Selection.get();
  if (!Call.getNumArgs())
    return Op.Diag(diag::err_refactoring_missing_arguments);
  return MySelection { Call, Selection.getEnclosingContext() };
});
```

If the user selects `function()` in the editor, it won't list this particular refactoring action as available.  However, if the user selects `function(1)`, then the rule will be initiated and the refactoring action will be performed successfully. The clang-refactor tool will report the `err_refactoring_missing_arguments` diagnostic when the user gives it a source range that corresponds to the selection of  `function()`.

A selection constraint is a special type that can be interpreted by the engine. The valid selection constraints are listed below:

- The `selection::Code<>` constraint is satisfied only when some portion of code that is contained in one declaration is selected.
- The `selection::Code<Expr>` constraint is satisfied only when one expression/statement of a given type is selected.
- The `selection::ASTDecl<Decl>` constraint is satisfied only when a single declaration of a given type is selected.
- The `selection::RawSelectionRange` constraint is always satisfied (provided there's a selection range!). It allows actions with custom selection rules.

What "selection" actually means depends on the particular type of the constraint. In general, selection is the prerogative of the refactoring engine (and not of the action), and it will try to optimize the selection rules for the best possible user experience. These choices will be documented appropriately.

## Conclusion

I believe the proposed design satisfies the goals that I've defined in my initial email. I've tested this design using code that implements all the actions that are supported by Xcode 9 and an additional refactoring action that’s currently implemented in a separate tool (clang-reorder-fields). I was quite satisfied with the results and have decided to settle on this particular design.

I'm looking forward to any feedback that you might have,
Cheers,
Alex

[1]: http://lists.llvm.org/pipermail/cfe-dev/2017-June/054286.html
[2]: https://gist.github.com/hyp/6e3abc6464eca7fb570674041b1df1ed

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170728/466f82b7/attachment.html>