[cfe-dev] Relative Paths in Compilation Database
Kim Gräsman
kim.grasman at gmail.com
Mon Apr 6 03:27:51 PDT 2015
Hey all,
This sounds like a good development, and I'd like to throw in my 2 cents.
I looked at a related problem before, and I think it boils down to the
same challenge -- Manuel and I discussed it on the bug here:
http://llvm.org/bugs/show_bug.cgi?id=19687
I ran out of steam before I got around to any real implementation, but
I think the primary challenge is that CMake and Ninja (or other
compdb-producers) don't necessarily have the arguments in list form
either. They usually seem to just have a command-line.
So the complexity of splitting the command-line into arguments can
either go in all the world's generators, or in the single consumer in
LibTooling.
- Kim
On Mon, Apr 6, 2015 at 10:41 AM, Laszlo Nagy
<rizsotto.mailinglist at gmail.com> wrote:
> hi there,
>
> i'm also writing a tool (Bear) which generate the compilation database. and
> had a few bugs in my code about escaping the command. if it's not too late i
> would also propose a format for the problem. ;)
>
> {
> directory: "/my/path"
> cmd: ["path to binary", "-c", "filename"],
> ...
> }
>
> which would reformat the command field as a JSON array. would still give the
> readability of the command and keep the order of the arguments. (i'm not
> sure that the recommended multiple "argument" key would be correct JSON.)
>
> how does it sound for you?
>
> regards,
> Laszlo
>
>
> On Mon, Apr 6, 2015 at 1:11 AM, Manuel Klimek <klimek at google.com> wrote:
>>
>> Also, I think we want to introduce a new keyword for the binary.
>> So both of these should work:
>> {
>> directory: "/my/path"
>> binary: "path to binary"
>> argument: "-c"
>> argument: "filename"
>> ...
>> }
>> or the shell-escaped command inside the json escaped string:
>> {
>> directory: "/my/path"
>> command: "path\\ to\\ binary -c filename"
>> ...
>> }
>> The thing to note is that both shell escaping and shell unescaping are
>> sufficiently hard on non-unix platforms, and even on unix platforms it's
>> just not nice to go from one to the other.
>>
>> Cheers,
>> /Manuel
>>
>> On Sun, Apr 5, 2015 at 4:01 PM Manuel Klimek <klimek at google.com> wrote:
>>>
>>> Yes, I think it's easy enough to make compatible
>>>
>>>
>>> On Sun, Apr 5, 2015, 8:42 AM Daniel Dilts <diltsman at gmail.com> wrote:
>>>>
>>>> I like that a lot better. I will look at what that is going to take.
>>>>
>>>> If it goes that direction, will it be necessary to also support the
>>>> older format? I know there are tools, like CMake, that can produce this
>>>> file format.
>>>>
>>>> On Sun, Apr 5, 2015 at 8:26 AM, Manuel Klimek <klimek at google.com> wrote:
>>>>>
>>>>> On Sun, Apr 5, 2015 at 8:19 AM Daniel Dilts <diltsman at gmail.com> wrote:
>>>>>>
>>>>>> So, escape the backslashes once for JSON and a second time for Clang?
>>>>>
>>>>>
>>>>> Well, a second time for "shell".
>>>>> A different idea that has come up was to just add (optional) 'arg'
>>>>> fields to the json, instead of the shell escaping, which seems much more
>>>>> pleasant to work with across platforms, like this:
>>>>> {
>>>>> directory: '/my/path'
>>>>> command: 'just the exec'
>>>>> argument: '-c'
>>>>> argument: 'filename'
>>>>> ...
>>>>> }
>>>>> Patches for that would be welcome :)
>>>>>
>>>>>>
>>>>>>
>>>>>> If this is going to be necessary on Windows, it probably ought to be
>>>>>> added to the documentation for the compilation database file format.
>>>>>>
>>>>>> On Fri, Apr 3, 2015 at 7:49 PM, Manuel Klimek <klimek at google.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Why not escape the backslashes when writing the compilation db?
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Apr 3, 2015, 1:54 PM Daniel Dilts <diltsman at gmail.com> wrote:
>>>>>>>>
>>>>>>>> I finally had the opportunity to look into this.
>>>>>>>>
>>>>>>>> CommandLineArgumentParser in JSONCompilationDatabase.cpp deals with
>>>>>>>> escapes by discarding all backslash characters. I imagine that this works
>>>>>>>> pretty well on Linux, not perfectly, but well enough.
>>>>>>>>
>>>>>>>> Windows uses backslash as a separator in file paths, so this breaks
>>>>>>>> most file paths on Windows.
>>>>>>>>
>>>>>>>> How would one go about fixing this so it works on both Windows and
>>>>>>>> Linux? Obviously the different escape sequences need to be implemented.
>>>>>>>> What I really mean is, how do you make it so that it uses the correct set of
>>>>>>>> escapes depending on platform?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Mar 17, 2015 at 12:57 PM, Manuel Klimek <klimek at google.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> I haven't had time to look.
>>>>>>>>>
>>>>>>>>> On Tue, Mar 17, 2015 at 6:06 PM Daniel Dilts <diltsman at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> What appears to be the exact problem? Are relative paths just not
>>>>>>>>>> properly implemented, or is the problem more specific than that?
>>>>>>>>>>
>>>>>>>>>> On Tue, Mar 17, 2015 at 4:07 AM, Manuel Klimek <klimek at google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> I think this is a known problem. Patches very welcome :)
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Mar 17, 2015 at 1:14 AM Daniel Dilts <diltsman at gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I am calling runClangTidy using a JSONCompilationDatabase that I
>>>>>>>>>>>> generate from a string. Everything works perfectly until I have a relative
>>>>>>>>>>>> path.
>>>>>>>>>>>>
>>>>>>>>>>>> With the following database I use
>>>>>>>>>>>> "D:\\CMakeTest\\bld\\..\\src\\main.cpp" as the file to open:
>>>>>>>>>>>> [
>>>>>>>>>>>> {
>>>>>>>>>>>> "directory": "D:\\CMakeTest\\bld\\",
>>>>>>>>>>>> "command" : "D:/llvm/build/Debug/bin/clang.exe
>>>>>>>>>>>> -I\"C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\VC\\include\"
>>>>>>>>>>>> -I\"C:\\Program Files (x86)\\Microsoft Visual Studio
>>>>>>>>>>>> 12.0\\VC\\atlmfc\\include\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>>> Kits\\8.1\\Include\\um\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>>> Kits\\8.1\\Include\\shared\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>>> Kits\\8.1\\Include\\winrt\" -I\"C:\\Program Files (x86)\\Microsoft Visual
>>>>>>>>>>>> Studio 12.0\\VC\\include\" -I\"C:\\Program Files (x86)\\Microsoft Visual
>>>>>>>>>>>> Studio 12.0\\VC\\atlmfc\\include\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>>> Kits\\8.1\\Include\\um\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>>> Kits\\8.1\\Include\\shared\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>>> Kits\\8.1\\Include\\winrt\" -DWIN32 -D_WINDOWS -D_DEBUG
>>>>>>>>>>>> -DCMAKE_INTDIR=\"Debug\" ..\\src\\main.cpp",
>>>>>>>>>>>> "file" : "..\\src\\main.cpp"
>>>>>>>>>>>> },
>>>>>>>>>>>> ]
>>>>>>>>>>>>
>>>>>>>>>>>> This gives me an output of "Error while processing
>>>>>>>>>>>> D:\\CMakeTest\\bld\\..\\src\\main.cpp."
>>>>>>>>>>>>
>>>>>>>>>>>> When I use the following database with
>>>>>>>>>>>> "D:\\CMakeTest\\src\\main.cpp" as the file to open, everything works.
>>>>>>>>>>>>
>>>>>>>>>>>> [
>>>>>>>>>>>> {
>>>>>>>>>>>> "directory": "D:\\CMakeTest\\src\\",
>>>>>>>>>>>> "command" : "D:/llvm/build/Debug/bin/clang.exe
>>>>>>>>>>>> -I\"C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\VC\\include\"
>>>>>>>>>>>> -I\"C:\\Program Files (x86)\\Microsoft Visual Studio
>>>>>>>>>>>> 12.0\\VC\\atlmfc\\include\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>>> Kits\\8.1\\Include\\um\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>>> Kits\\8.1\\Include\\shared\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>>> Kits\\8.1\\Include\\winrt\" -I\"C:\\Program Files (x86)\\Microsoft Visual
>>>>>>>>>>>> Studio 12.0\\VC\\include\" -I\"C:\\Program Files (x86)\\Microsoft Visual
>>>>>>>>>>>> Studio 12.0\\VC\\atlmfc\\include\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>>> Kits\\8.1\\Include\\um\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>>> Kits\\8.1\\Include\\shared\" -I\"C:\\Program Files (x86)\\Windows
>>>>>>>>>>>> Kits\\8.1\\Include\\winrt\" -DWIN32 -D_WINDOWS -D_DEBUG
>>>>>>>>>>>> -DCMAKE_INTDIR=\"Debug\" main.cpp",
>>>>>>>>>>>> "file" : "main.cpp"
>>>>>>>>>>>> },
>>>>>>>>>>>> ]
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Is there something that I am doing wrong? The only changes
>>>>>>>>>>>> between the two compilation databases are the directory and file entries.
>>>>>>>>>>>> Both should refer to the same absolute path.
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> cfe-dev mailing list
>>>>>>>>>>>> cfe-dev at cs.uiuc.edu
>>>>>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
More information about the cfe-dev
mailing list