[cfe-dev] [RFC] Embedding compilation database info in object files.

[Sean Silva]

On Wed, Jul 17, 2013 at 9:44 PM, Manuel Klimek <klimek at google.com> wrote:
>
> We have done similar things before internally, but considered it more to
> be a hack ;)
>
> I think the direction that we want to go to is to have an option in clang
> to append to a compilation database while running - that way, no
> post-processing step is required, which again needs to be somehow put into
> the build flow. The only part missing is somebody with enough time on their
> hands for whom this is high enough priority.
>

Wouldn't this approach (appending to a compilation database) have issues
with filesystem contention and/or write atomicity in multicore/distributed
builds (without involving a "real database" for the database storage)?
Also, wouldn't a post-processing step be needed in order to remove outdated
entries appended from a previous incremental build (consider: `make;
<rename some file in the project>; make`)?

The approach I proposed has two extremely desirable properties that I think
would be hard to achieve with an approach that carries the information in
an external "side channel", as in the approach you suggested:
1. The compilation database info is always up to date as long as the build
products are up to date, since the information follows the "causal chain"
leading to the final programs/libraries.
2. It works "everywhere clang does" since it makes no assumptions about
build systems, filesystems, or anything else; the data is carried along a
datapath that already works (namely, that information emitted by the
compiler will end up in build products).

Also, the format of the embedded entry could be streamlined to make it
utterly trivial to extract, e.g. a simple string
"@ClangCompilationDatabaseEntryMD5JSON<hex md5sum of $JSON>$JSON", and then
you could reliably extract the compdb entries with a single linear scan of
arbitrary binary files; with that it seems like it would be feasible for
most use cases (possibly adding an optional caching step) to have clang
tools directly accept binaries containing such data as the compilation
database itself!

-- Sean Silva
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20130718/8aed25d2/attachment.html>