[cfe-dev] clangd/libclang: how to emulate other compilers?

[Tom Honermann via cfe-dev]

Coverity also does something similar for compatibility with different 
Clang releases and distributions.

The most significant problems we run into are differences in builtin 
functions (either signature changes or added/removed builtins) and 
differences in recognized compiler options (rarely option syntax 
changes, generally unrecognized options).

For builtins, we audit differences in the Builtins*.def files across 
releases and distributions (assuming we can get access to the source for 
the distribution) and define macros that, based on compiler 
identification macros (__clang_major__, __clang_minor__, 
__apple_build_version__, etc...) provide support for missing builtins or 
incompatible builtin signatures.

For compiler options, we do similar audits of the (generated) 
Options.inc header and have support for mapping unrecognized options to 
recognized ones or just dropping them in hopes that the option doesn't 
affect the recognized language dialect.

These audits are fairly straight forward to script for Clang. 
Unfortunately, I'm not at liberty to share our scripts.

We've often wished for the ability to do similar audits for gcc, but gcc 
doesn't define builtins in a consistent manner.

Tom.

On 4/17/2018 4:35 PM, Doug Schaefer via cfe-dev wrote:
> Eclipse CDT does something similar. For each unique collection of compiler command line arguments (i.e. for files that build with the same options) we add -E -P -v -dD and call the real compiler to get the list of all include paths and symbols used for a given parse, including both user and built-in ones. Works with both clang and gcc.
> 
> For compiler specifics, we cheat and add macros that work us through the parse errors. You likely lose info but at least it rounds up to a pretty good parse.
> 
> I'm at the very beginning of trying to figure out how we'd do something similar with clangd. This feature will be a must for users with embedded systems toolchains which are mainly based on gcc.
> 
> Doug.
> 
> -----Original Message-----
> From: cfe-dev [mailto:cfe-dev-bounces at lists.llvm.org] On Behalf Of Milian Wolff via cfe-dev
> Sent: Tuesday, April 17, 2018 4:03 PM
> To: Clang Dev <cfe-dev at lists.llvm.org>
> Subject: [cfe-dev] clangd/libclang: how to emulate other compilers?
> 
> Hey all,
> 
> how does clangd or other users of the libclang handle situations where you want to parse code that is dependent on a certain other compiler or compiler environment? The most common scenario being embedded projects that rely on the compiler-builtin defines and include paths to find the sysroot include paths and such.
> 
> For KDevelop, which is using libclang, we have tried to build a sort of emulation layer that originally yielded good results. The approach is as
> followed:
> 
> 1) We use the actual compiler that is used to compile a given project, e.g.
> gcc, arm-none-eabi-gcc, ...
> 
> 2) We take this compiler and query it for its builtin defines:
> /usr/bin/gcc -xc++ -std=c++11 -dM -E - < /dev/null
> 
> 3) And also query the include paths:
> /usr/bin/gcc -xc++ -std=c++11 -v -E - < /dev/null
> 
> 4) Then for the libclang calls to clang_parseTranslationUnit2 we pass `- nostdinc -nostdinc++` followed by the defines and includes we got from 2) and 3).
> 
> Now, for simply things this actually worked quite well. But once you include a file that heavily relies on the compiler, such as all the SIMD intrinsic headers, you are easily drowning in parse errors. And once you have too many parse errors, clang will just give up. We have tried to workaround this via compatibility headers such as [1], but it keeps breaking.
> 
> More recently, we now also got bug reports where the user system has clang3 and they use that to to compile the code, but then download a KDevelop AppImage built against libclang v5 (e.g. via AppImage). Once again this easily yields tons of parse errors when encountering system headers that are using intrinsics specific to clang v3.
> 
> I am now thinking about removing the emulation layer described above. But then it will be essentially impossible to work on a lot of embedded projects which rely on the cross compiler defines and include paths...
> 
> So, once again - how do other users of libclang handle this scenario? What is the plan for clangd in this regard?
> 
> Thanks
> --
> Milian Wolff
> mail at milianw.de
> https://urldefense.proofpoint.com/v2/url?u=http-3A__milianw.de&d=DwIGaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=oOJsFtgkFCCcDlfLWKmfrVDtpXLihXJTBGJfSapXCV0&m=98IoWzNzKkOfNQYNIW0JrMNB1lfOKhvm_zc0s6gLvYw&s=OfD_gbOYZgrZeC_9kNIxaBuPzmUcMLEityEAtEeqGrE&e=
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_cfe-2Ddev&d=DwIGaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=oOJsFtgkFCCcDlfLWKmfrVDtpXLihXJTBGJfSapXCV0&m=98IoWzNzKkOfNQYNIW0JrMNB1lfOKhvm_zc0s6gLvYw&s=cOy5d4H7BOeruOJup0Ly3HK_uD6q-wKbguD005HPKkM&e=
>