[cfe-dev] Some error in clangd

[Sam McCall via cfe-dev]

Add an entry to compile_commands.json:
{
  "file": ".../libraries/chain/include/eosio/chain/types.hpp",
  "directory": ".../build/libraries/chain",
  "command": "/usr/bin/clang++-4.0 ... -c
.../libraries/chain/include/eosio/chain/types.hpp"
}

where the ... path is adjusted for your system, and the command is taken
from the entry from some other relevant file (e.g. abi_serializer.cpp)

You'll also need to prevent your build system from overwriting this change.

---

Another thought on a proper fix: we know that types.hpp is a good match for
resource_limits.hpp in the same directory, which in turn is a good match
for resource_limits.cpp. This transitive connection might be a useful
signal. Unfortunately we don't know that resource_limits.hpp exists when
loading the interpolating CDB. Maybe listing the parent directory of
types.hpp is viable though.

On Wed, Apr 3, 2019 at 4:30 AM yus <suyu.nu at gmail.com> wrote:

> Hi,
>
> Many thanks for your analysis and suggestion.
> For you suggestion, I know what to do but don't know how to do.
>
> Can you give some details about the option 1:
> - adding an explicit compile command for types.hpp, and avoid overwriting
> it,
> How to do it.
>
> Thanks.
>
> Brs,
> Su Yu
>
> Sam McCall <sammccall at google.com> 于2019年4月3日周三 上午12:24写道：
>
>> Indeed, the heuristics are valuing base filenames over directory
>> structure match in this case:
>>
>> sed -i s%/root/workspace/eos4%$PWD%g compile_commands.json
>> mkdir -p libraries/chain/include/eosio/chain
>> touch libraries/chain/include/eosio/chain/{types,resource_limits,zzz}.hpp
>>
>> clang-check -debug-only=interpolate
>> libraries/chain/include/eosio/chain/types.hpp
>> interpolate: chose .../libraries/wasm-jit/Source/IR/*Types*.cpp as proxy
>> for .../libraries/chain/include/eosio/chain/*types*.hpp preferring c++
>> score=2
>> (this is a bad result: prefer e.g. .../libraries/chain/abi_serializer.cpp)
>>
>> clang-check -debug-only=interpolate
>> libraries/chain/include/eosio/chain/resource_limits.hpp
>> interpolate: chose .../libraries/*chain*/*resource_limits*.cpp as proxy
>> for .../libraries/chain/include/eosio/*chain*/*resource_limits*.hpp
>> preferring c++ score=3
>> (this is a good result)
>>
>> clang-check -debug-only=interpolate
>> libraries/chain/include/eosio/chain/zzz.hpp
>> interpolate: chose .../libraries/*chain*/abi_serializer.cpp as proxy for
>> .../libraries/chain/include/eosio/*chain*/zzz.hpp preferring c++ score=1
>> (this is a good result)
>>
>> --- details ---
>>
>> Points are awarded to entries in compile_commands:
>>  - 2 points for filename match without extension (1 for prefix match)
>>  - up to 3 points for directory structure match (1 for each of last 2
>> segments, 1 for the rest)
>>
>> For files in this directory, the relevant directory structure parts are
>> .../libraries/chain/include eosio chain.
>> So:
>>   - anything under ".../libraries/chain/include" would get 1 point (there
>> are no cpp files there)
>>   - anything with "eosio" in the last 4 directory segments gets 1 point
>> (no matches)
>>   - anything with "chain" in the last 4 directory segments gets 1 point (*1
>> point for sources in "chain" subproject*)
>> With this directory structure, we do manage to pair headers/sources in
>> the same project, but we only give a 1 point bonus for doing so, and so the
>> 2 point bonus for matching basename can take precedence.
>>
>> --- how to fix in clang ---
>>
>> We only need to tweak the scores here by one point: a tie will be broken
>> in favor of the longest prefix match, which would be when sources are in
>> the same subproject with this directory structure.
>>
>> One option is to make the name match worth less: 0.5/1 rather than 1/2
>> points, though I do worry this will break some cases in the other direction.
>>
>> Another option is to recognize this directory structure match as
>> particularly good, and boost it more - 1 seems relatively low for e.g.
>> .../chain/abi_serializer.cpp matching .../chain/types.hpp where it's the
>> immediate parent directories that match.
>> This would be somewhat involved: the index data structure doesn't
>> currently capture where the word "chain" appeared within the cpp filename.
>>
>> I'll look into this...
>>
>> --- how to work around ---
>>
>> The only thing I can suggest is to change your compilation database,
>> either:
>>  - adding an explicit compile command for types.hpp, and avoid
>> overwriting it, or
>>  - only generate the CDB for a smaller subset of the project that avoids
>> Types.cpp
>>
>> On Mon, Apr 1, 2019 at 6:37 PM Sam McCall <sammccall at google.com> wrote:
>>
>>> Apologies, missed the attachment in the original email. Will look in the
>>> morning!
>>>
>>> On Mon, Apr 1, 2019 at 2:17 PM Sam McCall <sammccall at google.com> wrote:
>>>
>>>> Hi! Sorry the heuristics aren't working here.
>>>> The most likely thing is the bare name "types" is matching some *.cpp
>>>> file from a base library, and we're weighing that too heavily and the
>>>> directory not heavily enough. This is probably due to heuristics tuned for
>>>> the LLVM project itself, which has few dependencies and parallel
>>>> source/include trees.
>>>> Can you attach compile_commands.json please?
>>>>
>>>> On Mon, Apr 1, 2019 at 11:24 AM Ilya Biryukov <ibiryukov at google.com>
>>>> wrote:
>>>>
>>>>> I don't think there's any simple solution to this. You could try
>>>>> patching your 'compile_commands.json' to add the header files in there, but
>>>>> CMake won't do this automatically for you.
>>>>> I believe the following project on GitHub tries to do something
>>>>> similar, but I haven't tried it myself:
>>>>> https://github.com/Sarcasm/compdb
>>>>>
>>>>>
>>>>> +Sam McCall <sammccall at google.com>, now that you're back from leave,
>>>>> any ideas for improving the heuristics to find proper matching .cpp files
>>>>> in more cases?
>>>>>
>>>>> On Mon, Apr 1, 2019 at 11:11 AM yus <suyu.nu at gmail.com> wrote:
>>>>>
>>>>>> Excuse me, is there any solution for my problem?
>>>>>> Thanks
>>>>>>
>>>>>> yus <suyu.nu at gmail.com> 于2019年3月23日周六 上午9:40写道：
>>>>>>
>>>>>>> HI,
>>>>>>>
>>>>>>> Got it.
>>>>>>> So my problem has no relationship with front-end vim-YoucompleteMe
>>>>>>> I just need to wait for your solution or workaround.
>>>>>>>
>>>>>>> Thanks for your help.
>>>>>>>
>>>>>>> Brs,
>>>>>>> Su Yu
>>>>>>>
>>>>>>> Ilya Biryukov <ibiryukov at google.com> 于2019年3月22日周五 下午8:43写道：
>>>>>>>
>>>>>>>> On Fri, Mar 22, 2019 at 12:41 PM yus <suyu.nu at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> By the way,
>>>>>>>>> If a header has no source file (*.cpp), how it been handled.
>>>>>>>>>
>>>>>>>> In that case we would fallback to something like `clang -xc++
>>>>>>>> test.hpp`.
>>>>>>>> But that almost never happens, our heuristics will always try to
>>>>>>>> pick some cpp file first.
>>>>>>>> E.g. in the extreme case when compile_commands.json has just a
>>>>>>>> single file, we will take the compile command from that file for
>>>>>>>> *all* headers we encounter.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Brs,
>>>>>>>>> Su Yu
>>>>>>>>>
>>>>>>>>> yus <suyu.nu at gmail.com> 于2019年3月22日周五 下午7:37写道：
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Many thanks for your prompt answer.
>>>>>>>>>> It's an open source project.
>>>>>>>>>> But it's a little difficult to build. It has many dependency
>>>>>>>>>> Hope it can help.
>>>>>>>>>> https://github.com/EOSIO/eos
>>>>>>>>>>
>>>>>>>>>> Thanks.
>>>>>>>>>>
>>>>>>>>>> Brs,
>>>>>>>>>> Su Yu
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ilya Biryukov <ibiryukov at google.com> 于2019年3月22日周五 下午5:18写道：
>>>>>>>>>>
>>>>>>>>>>> Hi Su,
>>>>>>>>>>>
>>>>>>>>>>> Since compile_commands.json only contains compile commands for
>>>>>>>>>>> .cpp files, we use a heuristics-based approach to infer compile commands
>>>>>>>>>>> for headers.
>>>>>>>>>>> More precisely, based on the path to the header file we try to
>>>>>>>>>>> guess the best .cpp file for a header (e.g. an implementation file) and
>>>>>>>>>>> pick the compile command from it.
>>>>>>>>>>>
>>>>>>>>>>> In your case, we fail to guess a good compile command. I don't
>>>>>>>>>>> know of a good workaround, unfortunately.
>>>>>>>>>>> +Sam McCall <sammccall at google.com> is the author of this
>>>>>>>>>>> approach, he might have ideas on how to fix this.
>>>>>>>>>>>
>>>>>>>>>>> Is the project you're working on open-source? If so, could you
>>>>>>>>>>> provide a link to the source code? If not, compilation database should also
>>>>>>>>>>> be enough to reproduce.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Mar 21, 2019 at 10:41 PM yus via cfe-dev <
>>>>>>>>>>> cfe-dev at lists.llvm.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> I encounter an error in clangd
>>>>>>>>>>>>
>>>>>>>>>>>> I[12:49:12.197] Updating file
>>>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp with
>>>>>>>>>>>> command*
>>>>>>>>>>>> [/root/workspace/eos4/build/libraries/wasm-jit/Source/IR] */usr/bin/clang++-4.0
>>>>>>>>>>>> -D ENABLE_SIMD_PROTOTYPE=0 -D ENABLE_THREADING_PROTOTYPE=0 -D
>>>>>>>>>>>> IR_API=DLL_EXPORT -D PRETEND_32BIT_ADDRESS_SPACE=0 -D WAVM_METRICS_OUTPUT=0
>>>>>>>>>>>> -I /root/workspace/eos4/libraries/wasm-jit/Include -I
>>>>>>>>>>>> /root/workspace/eos4/libraries/wasm-jit/Include/IR -Wall
>>>>>>>>>>>> -Wno-invalid-partial-specialization -O3 -D NDEBUG -c -std=gnu++14
>>>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp
>>>>>>>>>>>> -resource-dir=/root/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/clang/lib/clang/7.0.0
>>>>>>>>>>>> V[12:49:12.199] Preamble for file
>>>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp cannot
>>>>>>>>>>>> be reused. Attempting to rebuild it.
>>>>>>>>>>>> V[12:49:12.238] Built preamble of size 213512 for file
>>>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp
>>>>>>>>>>>> I[12:49:12.242] Dropped diagnostic outside main file: : too
>>>>>>>>>>>> many errors emitted, stopping now
>>>>>>>>>>>>
>>>>>>>>>>>> some other file in same folder update correctly
>>>>>>>>>>>>
>>>>>>>>>>>> I[12:49:03.898] Updating file
>>>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/resource_limits.hpp
>>>>>>>>>>>> with command *[/root/workspace/eos4/build/libraries/chain] */usr/bin/clang++-4.0
>>>>>>>>>>>> -I /root/workspace/eos4/libraries/chain/include -I
>>>>>>>>>>>> /root/workspace/eos4/build/libraries/chain/include -I
>>>>>>>>>>>> /root/workspace/eos4/libraries/chain/../wasm-jit/Include -I
>>>>>>>>>>>> /root/workspace/eos4/libraries/chain/../../externals/binaryen/src -I
>>>>>>>>>>>> /root/workspace/eos4/libraries/utilities/include -I
>>>>>>>>>>>> /root/workspace/eos4/libraries/utilities/../wasm-jit/Include -I
>>>>>>>>>>>> /root/workspace/eos4/libraries/fc/include -I /root/opt/boost/include -I
>>>>>>>>>>>> /usr/local/include -I /root/workspace/eos4/libraries/fc/vendor/websocketpp
>>>>>>>>>>>> -I /root/workspace/eos4/libraries/chainbase/include -I
>>>>>>>>>>>> /root/workspace/eos4/libraries/wasm-jit/Source/Runtime/../../../chain/include
>>>>>>>>>>>> -I /root/workspace/eos4/libraries/softfloat/source/include -I
>>>>>>>>>>>> /root/workspace/eos4/libraries/softfloat/source/8086-SSE -I
>>>>>>>>>>>> /root/workspace/eos4/libraries/softfloat/build/Linux-x86_64-GCC -I
>>>>>>>>>>>> /root/workspace/eos4/libraries/builtins -I
>>>>>>>>>>>> /root/workspace/eos4/libraries/builtins../softfloat/source/include -Wall
>>>>>>>>>>>> -Wno-invalid-partial-specialization -O3 -D NDEBUG -c -std=gnu++14
>>>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/resource_limits.hpp
>>>>>>>>>>>> -resource-dir=/root/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/clang/lib/clang/7.0.0
>>>>>>>>>>>> V[12:49:03.901] Preamble for file
>>>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/resource_limits.hpp
>>>>>>>>>>>> cannot be reused. Attempting to rebuild it.
>>>>>>>>>>>> V[12:49:10.676] <<<
>>>>>>>>>>>> {"id":"3","jsonrpc":"2.0","method":"textDocument/definition","params":{"position":{"character":51,"line":53},"textDocument":{"uri":"file:///root/workspace/eos4/libraries/chain/include/eosio/chain/resource_limits.hpp"}}}
>>>>>>>>>>>>
>>>>>>>>>>>> I don't know why, and what's the difference. why the flags are
>>>>>>>>>>>> different.
>>>>>>>>>>>> How to fix the problem
>>>>>>>>>>>>
>>>>>>>>>>>> root at 39470746ee31:~/workspace/eos4# /usr/bin/clang++-4.0 -D
>>>>>>>>>>>> ENABLE_SIMD_PROTOTYPE=0 -D ENABLE_THREADING_PROTOTYPE=0 -D
>>>>>>>>>>>> IR_API=DLL_EXPORT -D PRETEND_32BIT_ADDRESS_SPACE=0 -D WAVM_METRICS_OUTPUT=0
>>>>>>>>>>>> -I /root/workspace/eos4/libraries/wasm-jit/Include -I
>>>>>>>>>>>> /root/workspace/eos4/libraries/wasm-jit/Include/IR -Wall
>>>>>>>>>>>> -Wno-invalid-partial-specialization -O3 -D NDEBUG -c -std=gnu++14
>>>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp
>>>>>>>>>>>> -resource-dir=/root/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/clang/lib/clang/7.0.0
>>>>>>>>>>>> /root/workspace/eos4/libraries/chain/include/eosio/chain/types.hpp:6:10:
>>>>>>>>>>>> fatal error: 'eosio/chain/name.hpp' file not found
>>>>>>>>>>>> #include <eosio/chain/name.hpp>
>>>>>>>>>>>>          ^~~~~~~~~~~~~~~~~~~~~~
>>>>>>>>>>>> 1 error generated.
>>>>>>>>>>>>
>>>>>>>>>>>> attach the error and compilation database
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>
>>>>>>>>>>>> Brs
>>>>>>>>>>>> Su Yu
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> cfe-dev mailing list
>>>>>>>>>>>> cfe-dev at lists.llvm.org
>>>>>>>>>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Regards,
>>>>>>>>>>> Ilya Biryukov
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Regards,
>>>>>>>> Ilya Biryukov
>>>>>>>>
>>>>>>>
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Ilya Biryukov
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190403/2a43ed9d/attachment.html>