[cfe-dev] Some error in clangd

Sam McCall via cfe-dev cfe-dev at lists.llvm.org
Tue Apr 2 09:24:02 PDT 2019 

Indeed, the heuristics are valuing base filenames over directory structure
match in this case:

sed -i s%/root/workspace/eos4%$PWD%g compile_commands.json
mkdir -p libraries/chain/include/eosio/chain
touch libraries/chain/include/eosio/chain/{types,resource_limits,zzz}.hpp

clang-check -debug-only=interpolate
libraries/chain/include/eosio/chain/types.hpp
interpolate: chose .../libraries/wasm-jit/Source/IR/*Types*.cpp as proxy
for .../libraries/chain/include/eosio/chain/*types*.hpp preferring c++
score=2
(this is a bad result: prefer e.g. .../libraries/chain/abi_serializer.cpp)

clang-check -debug-only=interpolate
libraries/chain/include/eosio/chain/resource_limits.hpp
interpolate: chose .../libraries/*chain*/*resource_limits*.cpp as proxy for
.../libraries/chain/include/eosio/*chain*/*resource_limits*.hpp preferring
c++ score=3
(this is a good result)

clang-check -debug-only=interpolate
libraries/chain/include/eosio/chain/zzz.hpp
interpolate: chose .../libraries/*chain*/abi_serializer.cpp as proxy for
.../libraries/chain/include/eosio/*chain*/zzz.hpp preferring c++ score=1
(this is a good result)

--- details ---

Points are awarded to entries in compile_commands:
 - 2 points for filename match without extension (1 for prefix match)
 - up to 3 points for directory structure match (1 for each of last 2
segments, 1 for the rest)

For files in this directory, the relevant directory structure parts are
.../libraries/chain/include eosio chain.
So:
  - anything under ".../libraries/chain/include" would get 1 point (there
are no cpp files there)
  - anything with "eosio" in the last 4 directory segments gets 1 point (no
matches)
  - anything with "chain" in the last 4 directory segments gets 1 point (*1
point for sources in "chain" subproject*)
With this directory structure, we do manage to pair headers/sources in the
same project, but we only give a 1 point bonus for doing so, and so the 2
point bonus for matching basename can take precedence.

--- how to fix in clang ---

We only need to tweak the scores here by one point: a tie will be broken in
favor of the longest prefix match, which would be when sources are in the
same subproject with this directory structure.

One option is to make the name match worth less: 0.5/1 rather than 1/2
points, though I do worry this will break some cases in the other direction.

Another option is to recognize this directory structure match as
particularly good, and boost it more - 1 seems relatively low for e.g.
.../chain/abi_serializer.cpp matching .../chain/types.hpp where it's the
immediate parent directories that match.
This would be somewhat involved: the index data structure doesn't currently
capture where the word "chain" appeared within the cpp filename.

I'll look into this...

--- how to work around ---

The only thing I can suggest is to change your compilation database, either:
 - adding an explicit compile command for types.hpp, and avoid overwriting
it, or
 - only generate the CDB for a smaller subset of the project that avoids
Types.cpp

On Mon, Apr 1, 2019 at 6:37 PM Sam McCall <sammccall at google.com> wrote:

> Apologies, missed the attachment in the original email. Will look in the
> morning!
>
> On Mon, Apr 1, 2019 at 2:17 PM Sam McCall <sammccall at google.com> wrote:
>
>> Hi! Sorry the heuristics aren't working here.
>> The most likely thing is the bare name "types" is matching some *.cpp
>> file from a base library, and we're weighing that too heavily and the
>> directory not heavily enough. This is probably due to heuristics tuned for
>> the LLVM project itself, which has few dependencies and parallel
>> source/include trees.
>> Can you attach compile_commands.json please?
>>
>> On Mon, Apr 1, 2019 at 11:24 AM Ilya Biryukov <ibiryukov at google.com>
>> wrote:
>>
>>> I don't think there's any simple solution to this. You could try
>>> patching your 'compile_commands.json' to add the header files in there, but
>>> CMake won't do this automatically for you.
>>> I believe the following project on GitHub tries to do something similar,
>>> but I haven't tried it myself: https://github.com/Sarcasm/compdb
>>>
>>>
>>> +Sam McCall <sammccall at google.com>, now that you're back from leave,
>>> any ideas for improving the heuristics to find proper matching .cpp files
>>> in more cases?
>>>
>>> On Mon, Apr 1, 2019 at 11:11 AM yus <suyu.nu at gmail.com> wrote:
>>>
>>>> Excuse me, is there any solution for my problem?
>>>> Thanks
>>>>
>>>> yus <suyu.nu at gmail.com> 于2019年3月23日周六 上午9:40写道：
>>>>
>>>>> HI,
>>>>>
>>>>> Got it.
>>>>> So my problem has no relationship with front-end vim-YoucompleteMe
>>>>> I just need to wait for your solution or workaround.
>>>>>
>>>>> Thanks for your help.
>>>>>
>>>>> Brs,
>>>>> Su Yu
>>>>>
>>>>> Ilya Biryukov <ibiryukov at google.com> 于2019年3月22日周五 下午8:43写道：
>>>>>
>>>>>> On Fri, Mar 22, 2019 at 12:41 PM yus <suyu.nu at gmail.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> By the way,
>>>>>>> If a header has no source file (*.cpp), how it been handled.
>>>>>>>
>>>>>> In that case we would fallback to something like `clang -xc++
>>>>>> test.hpp`.
>>>>>> But that almost never happens, our heuristics will always try to pick
>>>>>> some cpp file first.
>>>>>> E.g. in the extreme case when compile_commands.json has just a single
>>>>>> file, we will take the compile command from that file for *all* headers
>>>>>> we encounter.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Brs,
>>>>>>> Su Yu
>>>>>>>
>>>>>>> yus <suyu.nu at gmail.com> 于2019年3月22日周五 下午7:37写道：
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Many thanks for your prompt answer.
>>>>>>>> It's an open source project.
>>>>>>>> But it's a little difficult to build. It has many dependency
>>>>>>>> Hope it can help.
>>>>>>>> https://github.com/EOSIO/eos
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>
>>>>>>>> Brs,
>>>>>>>> Su Yu
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Ilya Biryukov <ibiryukov at google.com> 于2019年3月22日周五 下午5:18写道：
>>>>>>>>
>>>>>>>>> Hi Su,
>>>>>>>>>
>>>>>>>>> Since compile_commands.json only contains compile commands for
>>>>>>>>> .cpp files, we use a heuristics-based approach to infer compile commands
>>>>>>>>> for headers.
>>>>>>>>> More precisely, based on the path to the header file we try to
>>>>>>>>> guess the best .cpp file for a header (e.g. an implementation file) and
>>>>>>>>> pick the compile command from it.
>>>>>>>>>
>>>>>>>>> In your case, we fail to guess a good compile command. I don't
>>>>>>>>> know of a good workaround, unfortunately.
>>>>>>>>> +Sam McCall <sammccall at google.com> is the author of this
>>>>>>>>> approach, he might have ideas on how to fix this.
>>>>>>>>>
>>>>>>>>> Is the project you're working on open-source? If so, could you
>>>>>>>>> provide a link to the source code? If not, compilation database should also
>>>>>>>>> be enough to reproduce.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Mar 21, 2019 at 10:41 PM yus via cfe-dev <
>>>>>>>>> cfe-dev at lists.llvm.org> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> I encounter an error in clangd
>>>>>>>>>>
>>>>>>>>>> I[12:49:12.197] Updating file
>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp with
>>>>>>>>>> command*
>>>>>>>>>> [/root/workspace/eos4/build/libraries/wasm-jit/Source/IR] */usr/bin/clang++-4.0
>>>>>>>>>> -D ENABLE_SIMD_PROTOTYPE=0 -D ENABLE_THREADING_PROTOTYPE=0 -D
>>>>>>>>>> IR_API=DLL_EXPORT -D PRETEND_32BIT_ADDRESS_SPACE=0 -D WAVM_METRICS_OUTPUT=0
>>>>>>>>>> -I /root/workspace/eos4/libraries/wasm-jit/Include -I
>>>>>>>>>> /root/workspace/eos4/libraries/wasm-jit/Include/IR -Wall
>>>>>>>>>> -Wno-invalid-partial-specialization -O3 -D NDEBUG -c -std=gnu++14
>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp
>>>>>>>>>> -resource-dir=/root/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/clang/lib/clang/7.0.0
>>>>>>>>>> V[12:49:12.199] Preamble for file
>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp cannot
>>>>>>>>>> be reused. Attempting to rebuild it.
>>>>>>>>>> V[12:49:12.238] Built preamble of size 213512 for file
>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp
>>>>>>>>>> I[12:49:12.242] Dropped diagnostic outside main file: : too many
>>>>>>>>>> errors emitted, stopping now
>>>>>>>>>>
>>>>>>>>>> some other file in same folder update correctly
>>>>>>>>>>
>>>>>>>>>> I[12:49:03.898] Updating file
>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/resource_limits.hpp
>>>>>>>>>> with command *[/root/workspace/eos4/build/libraries/chain] */usr/bin/clang++-4.0
>>>>>>>>>> -I /root/workspace/eos4/libraries/chain/include -I
>>>>>>>>>> /root/workspace/eos4/build/libraries/chain/include -I
>>>>>>>>>> /root/workspace/eos4/libraries/chain/../wasm-jit/Include -I
>>>>>>>>>> /root/workspace/eos4/libraries/chain/../../externals/binaryen/src -I
>>>>>>>>>> /root/workspace/eos4/libraries/utilities/include -I
>>>>>>>>>> /root/workspace/eos4/libraries/utilities/../wasm-jit/Include -I
>>>>>>>>>> /root/workspace/eos4/libraries/fc/include -I /root/opt/boost/include -I
>>>>>>>>>> /usr/local/include -I /root/workspace/eos4/libraries/fc/vendor/websocketpp
>>>>>>>>>> -I /root/workspace/eos4/libraries/chainbase/include -I
>>>>>>>>>> /root/workspace/eos4/libraries/wasm-jit/Source/Runtime/../../../chain/include
>>>>>>>>>> -I /root/workspace/eos4/libraries/softfloat/source/include -I
>>>>>>>>>> /root/workspace/eos4/libraries/softfloat/source/8086-SSE -I
>>>>>>>>>> /root/workspace/eos4/libraries/softfloat/build/Linux-x86_64-GCC -I
>>>>>>>>>> /root/workspace/eos4/libraries/builtins -I
>>>>>>>>>> /root/workspace/eos4/libraries/builtins../softfloat/source/include -Wall
>>>>>>>>>> -Wno-invalid-partial-specialization -O3 -D NDEBUG -c -std=gnu++14
>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/resource_limits.hpp
>>>>>>>>>> -resource-dir=/root/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/clang/lib/clang/7.0.0
>>>>>>>>>> V[12:49:03.901] Preamble for file
>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/resource_limits.hpp
>>>>>>>>>> cannot be reused. Attempting to rebuild it.
>>>>>>>>>> V[12:49:10.676] <<<
>>>>>>>>>> {"id":"3","jsonrpc":"2.0","method":"textDocument/definition","params":{"position":{"character":51,"line":53},"textDocument":{"uri":"file:///root/workspace/eos4/libraries/chain/include/eosio/chain/resource_limits.hpp"}}}
>>>>>>>>>>
>>>>>>>>>> I don't know why, and what's the difference. why the flags are
>>>>>>>>>> different.
>>>>>>>>>> How to fix the problem
>>>>>>>>>>
>>>>>>>>>> root at 39470746ee31:~/workspace/eos4# /usr/bin/clang++-4.0 -D
>>>>>>>>>> ENABLE_SIMD_PROTOTYPE=0 -D ENABLE_THREADING_PROTOTYPE=0 -D
>>>>>>>>>> IR_API=DLL_EXPORT -D PRETEND_32BIT_ADDRESS_SPACE=0 -D WAVM_METRICS_OUTPUT=0
>>>>>>>>>> -I /root/workspace/eos4/libraries/wasm-jit/Include -I
>>>>>>>>>> /root/workspace/eos4/libraries/wasm-jit/Include/IR -Wall
>>>>>>>>>> -Wno-invalid-partial-specialization -O3 -D NDEBUG -c -std=gnu++14
>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp
>>>>>>>>>> -resource-dir=/root/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/clang/lib/clang/7.0.0
>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp:6:10:
>>>>>>>>>> fatal error: 'eosio/chain/name.hpp' file not found
>>>>>>>>>> #include <eosio/chain/name.hpp>
>>>>>>>>>>          ^~~~~~~~~~~~~~~~~~~~~~
>>>>>>>>>> 1 error generated.
>>>>>>>>>>
>>>>>>>>>> attach the error and compilation database
>>>>>>>>>>
>>>>>>>>>> Thanks.
>>>>>>>>>>
>>>>>>>>>> Brs
>>>>>>>>>> Su Yu
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> cfe-dev mailing list
>>>>>>>>>> cfe-dev at lists.llvm.org
>>>>>>>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Regards,
>>>>>>>>> Ilya Biryukov
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Ilya Biryukov
>>>>>>
>>>>>
>>>
>>> --
>>> Regards,
>>> Ilya Biryukov
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190402/670a1eb3/attachment.html>

More information about the cfe-dev mailing list