[cfe-dev] RFC: YAML as an intermediate format for clang::tooling::Replacement data on disk

[Vane, Edwin]

Hi all,

This discussion began on cfe-commits as the result of a commit (Tareq's poor header replacement patch that keeps getting reverted due to Windows build-bot issues). The start of the thread is here if you want background info: http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20130729/084881.html.

The proposal: The C++11 Migrator has a need to write Replacement data: offset, length, and replacement text, to disk. The replacement data describes changes made to a header while transforming one or more TU's. All the replacement data would be gathered up after an entire code-base is transformed by a separate tool and merged together to produce actual changes to headers. So the point is to serialize Replacement data as a form of inter-process communication using the file system as the communication link. Real inter-process communication is a possibility but not portable.

YAML was suggested off-hand in some past patch discussion. Support for reading/writing YAML exists in LLVM. YAML is also meant for human-readable data serialization. It seems like a good choice.

Another option that has been suggested is to just output diffs but as Manuel outlines in http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20130729/084904.html, there are some major drawbacks to this technique.

It's worth nothing we don't care to make these change description files full of serialized replacements available to general-purpose diff/merge tools. The only intended consumer is the C++11 Migrator Post-Processor. A prototype of this tool exists at https://github.com/revane/migmerge_git but is soon to be replaced by a tool that has no third-party dependencies so it can live in clang-tools-extra with the migrator.

Are there any other suggestions to address the flow of data between migrator and post-processor? Comments?

--
Edwin Vane
  Software Developer
  Intel of Canada, Inc.