[cfe-commits] Design proposal: Add custom compilation database formats as plugins

[Manuel Klimek]

On Thu, Jul 19, 2012 at 11:54 AM, Arnaud de Grandmaison
<arnaud.allarddegrandmaison at parrot.com> wrote:
> Or maybe:
>  - add a private field 'hook' in CompilationDatabase (a function pointer to
> the user callback),
>  - initialize it by default to NULL
>  - add a setter to CompilationDatabase for this hook field

This is the problematic part, as it requires changing the tool's code
for each environment, if I understand you correctly. One of the
reasons we want the plugins is that we can use upstream clang tools
without keeping a branch that changes their code to register our
internal environment stuff.

Cheers,
/Manuel

>  - and use the hook if non null in findCompilationDatabaseFromDirectory
>
> The burden of linking in the other required libraries is left to the user of
> the hook --- which seems good to me : we can not generally decide for the
> user which libraries should be linked in.
>
> This can probably be extended by changing the hook to a vector (of hooks).
> This would allow registering several user backends to a specific
> CompilationDatabase instance, and have it try to use them in turn, until one
> succeed.
>
> Cheers,
> --
> Arnaud de Grandmaison
>
>
> On 07/19/2012 11:32 AM, Daniel Jasper wrote:
>
> While I also like the plugin concept much better, I am still unsure about
> the fundamental problem of getting other libraries linked in. The problem is
> that the linker will simply throw away any object file that has no
> references to it. After some discussions on IRC, I see two options:
>
> 1. Conditionally compile in a custom registration method, i.e.:
> in CompilationDatabase.cpp:
>   void someInit() {
>     #ifdef USE_CUSTOM_COMPILATION_DATABASES
>     RegisterCompilationDatabases();
>     #endif
>   }
> And then implement RegisterCompilationDatabases() together with the custom
> compilation database.
> Pros: Very easy to understand and to use right.
> Cons: It would be hard to link together multiple such implementations.
>
> 2. Conditionally compile in custom anchors, i.e;.:
> in CompilationDatabase.cpp:
>   #ifdef COMPILATION_DATABASE_ANCHORS
>   COMPILATION_DATABASE_ANCHORS;
>   #endif
> And then e..g. "-DCOMPILATION_DATABASE_ANCHORS='extern volatile int
> MyAnchorSource; static int MyAnchorDest = MyAnchorSource;'"
> Pros: Multiple custom compilation databases can be added. Nice clean
> separation (conditional code can be somewhere on the bottom of
> "CompilationDatabase.cpp".
> Cons: Seems a bit tangled and easy to get the -D part wrong.
>
> I will try to get #2 to work (possibly simplifying it a bit with more macros
> :-/). Does anyone see a better solution?
>
>
> On Thu, Jul 19, 2012 at 10:42 AM, Manuel Klimek <klimek at google.com> wrote:
>>
>> On Thu, Jul 19, 2012 at 12:34 AM, Sean Silva <silvas at purdue.edu> wrote:
>> >> 3) It is *incredibly* easy to write a minimal JSON writer that only
>> >> supports the features we need.
>> >
>> > Ah, ok, I seem to have overestimated how difficult/annoying this would
>> > be to write.
>>
>> Writing JSON is simple: it's a well-defined format with very little
>> chance that it ever changes. No need to have a "writer", just print it
>> out:
>>
>> stream << "[\n";
>> for (tu : tus) {
>>   stream << "  {\n      " << "\"directory\": " <<
>> shellEscape(tu.directory())
>>   ...
>> }
>> stream << "]\n";
>>
>> The hard part is the shell escaping. CMake already had that built in.
>>
>> >> We can easily place a copy of the code into LLVM or re-license it
>> >> however it helps.
>> >
>> > This might be good, to help keep things consistent. Are you thinking
>> > something with a role like ninja's ninja_syntax.py?
>> >
>> > --Sean Silva
>> >
>> > On Wed, Jul 18, 2012 at 3:15 PM, Chandler Carruth <chandlerc at google.com>
>> > wrote:
>> >> On Wed, Jul 18, 2012 at 3:11 PM, Sean Silva <silvas at purdue.edu> wrote:
>> >>>
>> >>> > For ninja in particular, I have long thought that the correct
>> >>> > approach
>> >>> > is to write something which can convert 'ninja -t commands' into the
>> >>> > JSON
>> >>> > format, or to build a tool for writing the JSON database directly
>> >>> > into
>> >>> > ninja.
>> >>>
>> >>> This was my line of thought as well.
>> >>>
>> >>> Since the C++ stdlib unfortunately doesn't have JSON support, what do
>> >>> you think about a simpler, more plaintext-y compilation database
>> >>> (PlaintextCompilationDatabase?); it seems like that might be a win
>> >>> since it is simpler to output. For example, the format would just be
>> >>>
>> >>> /path/to/dir/
>> >>> clang++ foo.cpp
>> >>> foo.cpp
>> >>> <empty line>
>> >>>
>> >>> Just a mirror of the JSON one, but with a format easier to output (god
>> >>> forbid there's a newline in one of the command lines or filenames). On
>> >>> the other hand, throwing together a simple "good-enough" JSON writer
>> >>> isn't *too* difficult; nonetheless, for tools written in C/C++, this
>> >>> could lower the bar to entry. Do you know what CMake currently does?
>> >>> Does it have its own mini JSON writer?
>> >>
>> >>
>> >> We didn't go with the plaintext route for three important reasons:
>> >>
>> >> 1) Filenames do have whitespace in them. The format should be robust
>> >> against
>> >> that.
>> >> 2) Many other tools and languages should have a minimal barrier to read
>> >> them. An existing format eases this.
>> >> 3) It is *incredibly* easy to write a minimal JSON writer that only
>> >> supports
>> >> the features we need.
>> >>
>> >> Manuel contributed the CMake support for the JSON database, and it
>> >> includes
>> >> just such a trivial JSON writer. =] We can easily place a copy of the
>> >> code
>> >> into LLVM or re-license it however it helps.
>> >>
>> >> That said, the latest version of CMake already has support for JSON +
>> >> Ninja
>> >> -- we didn't contribute it, so I don't know what strategy they
>> >> followed, but
>> >> you should look at that and talk to the ninja and CMake developers
>> >> before
>> >> going too far here.
>> > _______________________________________________
>> > cfe-commits mailing list
>> > cfe-commits at cs.uiuc.edu
>> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>> _______________________________________________
>> cfe-commits mailing list
>> cfe-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>
>
>
>
> --
> Arnaud de Grandmaison
> Senior CPU engineer
> Business Unit Digital Tuner
>
> Parrot S.A.
> 174, quai de Jemmapes
> 75010 Paris - France
> Phone: +33 1 48 03 84 59