[cfe-dev] [RFC] Embedding compilation database info in object files.

[Manuel Klimek]

On Sat, Jul 20, 2013 at 6:44 AM, Sean Silva <silvas at purdue.edu> wrote:

> On Fri, Jul 19, 2013 at 6:36 PM, Joshua Cranmer 🐧 <Pidgeot18 at gmail.com>wrote:
>
>> On 7/17/2013 11:36 PM, Sean Silva wrote:
>>
>>> tl;dr: compiler embeds compilation db info in object files; you can then
>>> collect it afterwards with simple tools. Good idea?
>>>
>>> It seems like for a while now, we have been looking for a way that clang
>>> can assist users in creating JSON compilation databases, but solutions seem
>>> limited to specific build systems or platforms. I came up with a neat
>>> little hack that may be a viable way for clang to help create compilation
>>> databases "everywhere clang runs", with fairly good user experience.
>>>
>>> I believe the following user experience is achievable "everywhere clang
>>> runs":
>>> 1. Add some option to the compiler command line.
>>> 2. Rebuild.
>>> 3. Feed all of your built object files/executables to a small tool we
>>> ship and out comes a compilation database.
>>>
>>
>> Quite frankly, I don't see this as being a superior user-experience to
>> the following scenario:
>> 1. Add an option to the compiler command line to dump the database to a
>> file.
>> 2. Rebuild.
>> 3. You have your compilation database automatically!
>>
>> The primary difficulty of this approach from an implementation
>> perspective is locking the database for writing when compiling with make
>> -jN.
>
>
> That approach may simply not be achievable in the case where the
> compilations are happening on physically distinct machines that do not
> share an underlying filesystem. I consider 'achievable "everywhere clang
> runs"' to be an important property; it's easy to come up with many
> far-superior user experiences by restricting consideration to particular
> build environments.
>

Ah, that's where we disagree.
I think this is for "simple" set ups. If you have the resources to write
your own distributed build system, I assume you also have the resources to
implement your own compilation database that works against the build graph
a distributed build system has to somehow put together anyway.

Also, especially for distributed build systems, you still have not answered
the question:
If I edit file a/b/c/d.cc, what algorithm do I run through to get the
compile command line for that file?

Cheers,
/Manuel


>
> Also, I believe that should someone want to integrate the compilation
> database into an automated build process, the "update a random database
> file specified on command line" approach has serious issues since the
> nature of its dependency on other parts of the build process is difficult
> to describe within the semantics of existing build systems. For example,
> Ninja is fundamentally based on "run this command with these files as input
> to get these files as output", and it's not clear how to express "this file
> is updated by running these commands, with such and such
> invalidation/staleness semantics etc." within that semantic model.
> Embedding the compilation database information in build products integrates
> well with build systems since the compilation database information is
> simply another aspect of the compiler's output and follows the same
> dependency rules.
>
>
>> I've seen concerns about impact on build-times for this, but considering
>> natural variation in compilation times (since not all files are the same
>> size), and the fact that we're already rebuilding world anyways, there
>> should be very little contention for this file in the grand scheme of
>> things.
>>
>> --
>> Joshua Cranmer
>> Thunderbird and DXR developer
>> Source code archæologist
>>
>>
>> ______________________________**_________________
>> cfe-dev mailing list
>> cfe-dev at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/**mailman/listinfo/cfe-dev<http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev>
>>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20130720/5485187e/attachment.html>