[cfe-dev] Relative Paths in Compilation Database

Mon Apr 6 01:41:43 PDT 2015

hi there,

i'm also writing a tool (Bear) which generate the compilation database. and
had a few bugs in my code about escaping the command. if it's not too late
i would also propose a format for the problem. ;)

{
  directory: "/my/path"
  cmd: ["path to binary", "-c", "filename"],
...
}

which would reformat the command field as a JSON array. would still give
the readability of the command and keep the order of the arguments. (i'm
not sure that the recommended multiple "argument" key would be correct
JSON.)

how does it sound for you?

regards,
Laszlo

On Mon, Apr 6, 2015 at 1:11 AM, Manuel Klimek <klimek at google.com> wrote:

> Also, I think we want to introduce a new keyword for the binary.
> So both of these should work:
> {
>   directory: "/my/path"
>   binary: "path to binary"
>   argument: "-c"
>   argument: "filename"
> ...
> }
> or the shell-escaped command inside the json escaped string:
> {
>   directory: "/my/path"
>   command: "path\\ to\\ binary -c filename"
> ...
> }
> The thing to note is that both shell escaping and shell unescaping are
> sufficiently hard on non-unix platforms, and even on unix platforms it's
> just not nice to go from one to the other.
>
> Cheers,
> /Manuel
>
> On Sun, Apr 5, 2015 at 4:01 PM Manuel Klimek <klimek at google.com> wrote:
>
>> Yes, I think it's easy enough to make compatible
>>
>> On Sun, Apr 5, 2015, 8:42 AM Daniel Dilts <diltsman at gmail.com> wrote:
>>
>>> I like that a lot better.  I will look at what that is going to take.
>>>
>>> If it goes that direction, will it be necessary to also support the
>>> older format?  I know there are tools, like CMake, that can produce this
>>> file format.
>>>
>>> On Sun, Apr 5, 2015 at 8:26 AM, Manuel Klimek <klimek at google.com> wrote:
>>>
>>>> On Sun, Apr 5, 2015 at 8:19 AM Daniel Dilts <diltsman at gmail.com> wrote:
>>>>
>>>>> So, escape the backslashes once for JSON and a second time for Clang?
>>>>>
>>>>
>>>> Well, a second time for "shell".
>>>> A different idea that has come up was to just add (optional) 'arg'
>>>> fields to the json, instead of the shell escaping, which seems much more
>>>> pleasant to work with across platforms, like this:
>>>> {
>>>>   directory: '/my/path'
>>>>   command: 'just the exec'
>>>>   argument: '-c'
>>>>   argument: 'filename'
>>>> ...
>>>> }
>>>> Patches for that would be welcome :)
>>>>
>>>>
>>>>>
>>>>> If this is going to be necessary on Windows, it probably ought to be
>>>>> added to the documentation for the compilation database file format.
>>>>>
>>>>> On Fri, Apr 3, 2015 at 7:49 PM, Manuel Klimek <klimek at google.com>
>>>>> wrote:
>>>>>
>>>>>> Why not escape the backslashes when writing the compilation db?
>>>>>>
>>>>>> On Fri, Apr 3, 2015, 1:54 PM Daniel Dilts <diltsman at gmail.com> wrote:
>>>>>>
>>>>>>> I finally had the opportunity to look into this.
>>>>>>>
>>>>>>> CommandLineArgumentParser in JSONCompilationDatabase.cpp deals with
>>>>>>> escapes by discarding all backslash characters.  I imagine that this works
>>>>>>> pretty well on Linux, not perfectly, but well enough.
>>>>>>>
>>>>>>> Windows uses backslash as a separator in file paths, so this breaks
>>>>>>> most file paths on Windows.
>>>>>>>
>>>>>>> How would one go about fixing this so it works on both Windows and
>>>>>>> Linux?  Obviously the different escape sequences need to be implemented.
>>>>>>> What I really mean is, how do you make it so that it uses the correct set
>>>>>>> of escapes depending on platform?
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Mar 17, 2015 at 12:57 PM, Manuel Klimek <klimek at google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I haven't had time to look.
>>>>>>>>
>>>>>>>> On Tue, Mar 17, 2015 at 6:06 PM Daniel Dilts <diltsman at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> What appears to be the exact problem?  Are relative paths just not
>>>>>>>>> properly implemented, or is the problem more specific than that?
>>>>>>>>>
>>>>>>>>> On Tue, Mar 17, 2015 at 4:07 AM, Manuel Klimek <klimek at google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I think this is a known problem. Patches very welcome :)
>>>>>>>>>>
>>>>>>>>>> On Tue, Mar 17, 2015 at 1:14 AM Daniel Dilts <diltsman at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I am calling runClangTidy using a JSONCompilationDatabase that I
>>>>>>>>>>> generate from a string.  Everything works perfectly until I have a relative
>>>>>>>>>>> path.
>>>>>>>>>>>
>>>>>>>>>>> With the following database I use "D:\\CMakeTest\\bld\\..\\src\\main.cpp"
>>>>>>>>>>> as the file to open:
>>>>>>>>>>> [
>>>>>>>>>>>    {
>>>>>>>>>>>       "directory": "D:\\CMakeTest\\bld\\",
>>>>>>>>>>>       "command"  : "D:/llvm/build/Debug/bin/clang.exe
>>>>>>>>>>> -I\"C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\VC\\include\"
>>>>>>>>>>> -I\"C:\\Program Files (x86)\\Microsoft Visual Studio
>>>>>>>>>>> 12.0\\VC\\atlmfc\\include\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>> Kits\\8.1\\Include\\um\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>> Kits\\8.1\\Include\\shared\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>> Kits\\8.1\\Include\\winrt\"   -I\"C:\\Program Files (x86)\\Microsoft Visual
>>>>>>>>>>> Studio 12.0\\VC\\include\" -I\"C:\\Program Files (x86)\\Microsoft Visual
>>>>>>>>>>> Studio 12.0\\VC\\atlmfc\\include\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>> Kits\\8.1\\Include\\um\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>> Kits\\8.1\\Include\\shared\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>> Kits\\8.1\\Include\\winrt\"   -DWIN32 -D_WINDOWS -D_DEBUG
>>>>>>>>>>> -DCMAKE_INTDIR=\"Debug\" ..\\src\\main.cpp",
>>>>>>>>>>>       "file"     : "..\\src\\main.cpp"
>>>>>>>>>>>    },
>>>>>>>>>>> ]
>>>>>>>>>>>
>>>>>>>>>>> This gives me an output of "Error while processing
>>>>>>>>>>> D:\\CMakeTest\\bld\\..\\src\\main.cpp."
>>>>>>>>>>>
>>>>>>>>>>> When I use the following database with
>>>>>>>>>>> "D:\\CMakeTest\\src\\main.cpp" as the file to open, everything works.
>>>>>>>>>>>
>>>>>>>>>>> [
>>>>>>>>>>>    {
>>>>>>>>>>>       "directory": "D:\\CMakeTest\\src\\",
>>>>>>>>>>>       "command"  : "D:/llvm/build/Debug/bin/clang.exe
>>>>>>>>>>> -I\"C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\VC\\include\"
>>>>>>>>>>> -I\"C:\\Program Files (x86)\\Microsoft Visual Studio
>>>>>>>>>>> 12.0\\VC\\atlmfc\\include\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>> Kits\\8.1\\Include\\um\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>> Kits\\8.1\\Include\\shared\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>> Kits\\8.1\\Include\\winrt\"   -I\"C:\\Program Files (x86)\\Microsoft Visual
>>>>>>>>>>> Studio 12.0\\VC\\include\" -I\"C:\\Program Files (x86)\\Microsoft Visual
>>>>>>>>>>> Studio 12.0\\VC\\atlmfc\\include\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>> Kits\\8.1\\Include\\um\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>> Kits\\8.1\\Include\\shared\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>> Kits\\8.1\\Include\\winrt\"   -DWIN32 -D_WINDOWS -D_DEBUG
>>>>>>>>>>> -DCMAKE_INTDIR=\"Debug\" main.cpp",
>>>>>>>>>>>       "file"     : "main.cpp"
>>>>>>>>>>>    },
>>>>>>>>>>> ]
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Is there something that I am doing wrong?  The only changes
>>>>>>>>>>> between the two compilation databases are the directory and file entries.
>>>>>>>>>>> Both should refer to the same absolute path.
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> cfe-dev mailing list
>>>>>>>>>>> cfe-dev at cs.uiuc.edu
>>>>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20150406/3915f1bc/attachment.html>