[cfe-dev] Getting source files into the CompilationDatabase interfaces

[Manuel Klimek]

On Wed, Nov 6, 2013 at 9:52 AM, Val Markovic <strahinja at google.com> wrote:

> On Wed, Nov 6, 2013 at 9:34 AM, Manuel Klimek <klimek at google.com> wrote:
>
>> Hi,
>>
>> as we've seen more use of the compilation database (for example through
>> the YCM vim plugin), we noticed that for networked build systems and server
>> applications the information we currently expose in the interfaces (path,
>> file-name, command line arguments) is not enough - we also need to be able
>> to get all source code, which we then can put into clang's VFS to get a
>> fully build-system independent clang run over a translation unit.
>>
>
> This is a good idea, but do keep in mind (you probably do already, but
> I'll mention it just in case) that for autocompletion and similar
> code-comprehension features in a source code editor the canonical state of
> a source file is not what the filesystem sees, it's what the user has in
> his editor buffer. The state in the buffer may be unsaved but the user
> still wants it used for code-completion.
>
> So it's perfectly fine if libclang can say "here's what I think is the
> state of the relevant source files" as long as the caller can then override
> this with their own data.
>

Yes, of course - the libclang change would be that you have some additional
functions (much like for how to iterate over command line arguments) that
would allow you to fetch all file names and contents for those files - in
the end, the client can of course exchange any file with its own content
before handing it off the parser.


>
>
>> We think that getting the source information (perhaps optionally) as part
>> of a getCompileCommands run is a good fit - a build system always must know
>> how to provide the required sources, and as such it seems to be a natural
>> fit.
>>
>> I'd propose to change the CompilationDatabase interface
>> (tools/clang/include/clang/Tooling/CompilationDatabase.h) to that end, and
>> see two possible solutions, which both have different pros and cons:
>> 1. Add a map from std::string (file-name) -> std::string (source content)
>> to the CompileCommand class (in
>> tools/clang/include/clang/Tooling/CompilationDatabase.h); let specific
>> CompilationDatabases optionally fill in that information
>> 2. Do not modify CompileCommand - instead, add a
>> getCompileCommandsAndSources(StringRef FilePath) method that returns a
>> vector<pair<CompileCommand, map<string, string>>> which also includes the
>> sources; I'm reluctant to split the call into two, as the compile command
>> and the sources are tightly coupled (if a user syncs in the background,
>> both tend to change at the same time)
>>
>
> I'm curious how either of the two APIs would be exposed through the
> libclang C API.
>

See above...


>
>
>>
>> I lean towards solution (1), mainly because it seems less invasive. On
>> the other hand, it might make it less obvious to users of libclang what's
>> going on (for both solutions I'd propose extending the libclang interface
>> basically in a very similar way to how the C++ code changes).
>>
>
One open question with #1 is: what do we do for the JSONCompilationDatabase
/ FixedCompilationDatabase? Do we always read all files from disk? On the
one hand, it seems like often unnecessary overhead, on the other hand, it
might take away problematic race conditions with files changing underneath
the client...


>
>> Opinions?
>>
>> Thanks,
>> /Manuel
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20131106/ae3140b0/attachment.html>