[cfe-dev] LLVM, Clang Development IDEs

Manuel Klimek via cfe-dev cfe-dev at lists.llvm.org
Mon Sep 14 11:08:49 PDT 2015 

On Fri, Sep 11, 2015 at 7:15 AM Vladimir Voskresensky - Oracle via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

>
> On 09/11/15 01:55 PM, Manuel Klimek via cfe-dev wrote:
>
> On Fri, Sep 11, 2015 at 12:46 PM Vladimir Voskresensky - Oracle via
> cfe-dev <cfe-dev at lists.llvm.org> wrote:
>
>>
>> On 09/11/15 12:44 PM, Manuel Klimek via cfe-dev wrote:
>>
>> On Fri, Sep 11, 2015 at 11:40 AM Vladimir Voskresensky - Oracle <
>> vladimir.voskresensky at oracle.com> wrote:
>>
>>> Hello Manuel,
>>>
>>>
>>> On 09/11/15 11:59 AM, Manuel Klimek wrote:
>>>
>>> On Thu, Sep 10, 2015 at 7:59 PM Vladimir Voskresensky - Oracle via
>>> cfe-dev <cfe-dev at lists.llvm.org> wrote:
>>>
>>>> Hello Keith,
>>>>
>>>> I'm from Oracle (previously from Sun Microsystems) and use NetBeans C++
>>>> IDE for
>>>> developing Clang based tools.
>>>>
>>>
>>> Oh, this is awesome :)
>>>
>>> I've demoed this to Argyrios Kyrtzidis ~year ago and he was impressed by
>>> it's parsing speed :-)
>>> NB needed just 1 minute to parse whole LLVM+Clang 3.4 codebase on my
>>> laptop.
>>> Also I was complaining that migrating to i.e. Clang's preprocessor makes
>>> us 2x slower (which is still the case for upcoming NB 8.1, but we trying to
>>> restore our speed)
>>>
>>>
>>>
>>>> Till 8.0 version Netbeans had own parser (as Eclipse). Starting from
>>>> upcoming
>>>> 8.1 NB is trying to use some clang components in experimental mode.
>>>>
>>>
>>> Will this by any chance use the compilation database integration?
>>>
>>> NetBeans for a long time has own "build interceptor". It helps to put
>>> code bases with even really complex build systems into IDE.
>>> When developer uses Project with Existing Sources wizard and specify
>>> commands which he proceed in cmd shell, then IDE executes them and
>>> interpose compiler invocations to extract cwd and all flags passed to
>>> compiler.
>>> Then all is persisted in project properties, so user gain "Compile File"
>>> for free, because IDE for each file knows how it was compiled.
>>> For CMake based codebases json database is produced and used to extract
>>> flags.
>>> Am I answering your question? Or do you mean smth different?
>>>
>>
>> Let me rephrase: for example, YouCompleteMe supports using libclang & its
>> compilation database interface to get the necessary compile flags for C++
>> files. Due to that support, I can take an arbitrary internal build system
>> and add support for YCM by providing a libclang with a special
>> implementation of the CompilationDatabase.
>>
>> What you describe is the YCM approach how to help libclang to find
>> compile flags to create correct TU.
>> YCM should create implementation of CompilationDatabase and register it
>> for libclang to see it (or generate compile_commands.json file) , right?
>>
>
> No, the trick is that YCM doesn't know about how to generate a compilation
> database - the build system integration knows (for example, CMake and ninja
> can generate the .json files, for our internal distributed build system
> (similar to bazel.io) we use a specific internal integration).
>
> Then the most complex part (generate compilation db) is just provided for
> you. Great.
>
>
>
> But, from our experience the most difficult part here is: how to fill this
>> CompilationDatabase content for arbitrary build system?
>> I.e. for projects with alone my_favorite_buld_all.sh script?
>> So, above I just shortly described how NB historically gets information
>> about compiled flags.
>> http://netbeans.org/kb/docs/cnd/quickstart.html#existingsourcesprojects
>> It is similar to scan-build.
>>
>>
>> Is that possible with NetBeans?
>>
>> Possible what? :-)
>> Possible to wrap flags gathered by scan-build-like interposer into
>> CompilationDatabase for libclang usage? Yes it's possible.
>> But we don't use libclang or Tooling APIs.
>> I.e. we init Preprocessor manually, because have to disable all target
>> build-ins and provide all our own settings for system paths, system macros
>> and so one.
>> Also we provide own FileSystem impl (great add-on in 3.6!!!), because we
>> support Remote Development and real parsed files have to be treated in the
>> environment emulating Remote Host.
>>
>
> Ah, is all of this open source? Are you planning to contribute it back to
> clang to make the tooling better?
>
> yes, it's in open source. There is no sense to contribute FS impl, because
> it wraps NB's FileObjects into CLang's Files.
>
> http://hg.netbeans.org/main/file/tip/cnd.apt/src/org/netbeans/modules/cnd/apt/impl/support/clank/ClankFileObjectBasedFileSystem.java
>
>
>
> I guess the answer to my questions is "no",
>
> The answer to your original question
> "Will this by any chance use the compilation database integration?"
> is "no"
> Because we have all this information already. We don't need clang for that
> task.
> We generate compilation database for any arbitrary complex systems by own
> mechanism.
>

Well, the problem is once you hit a single highly integrated codebase of >
100MLOC on networked file systems (meaning latency at least 1 order of
magnitude higher than local disk) most of the IDEs I've tried fail
miserably.


> But we can read compile_commands.json format as well and use it to
> configure NetBeans (and in 8.1 we use it to init Clang classes).
>

If I put O(500k) files into it, many of which take 20-60 seconds to parse,
what will happen?


> then - what I really want to do is be able to use the already existing
> integration we have for our internal build system with clang tools via the
> compilation-database to use NetBeans, but from what you say it seems like
> that's not possible.
>
> If "existing integration you have for your internal build system" can dump
> cmake-complaint compile_commands.json, then you can use NetBeans.
>

See questions above.

If you are really interested in trying NB[1] then let's exclude cfe-dev?
>

I'm interested in using NB, but I'd also first like to understand some of
the engineering trade-offs you made - as the maintainer for our tooling
infrastructure in clang, I'd like to get to a point where we get a lot of
integration for free, instead of everybody needing to re-invent the wheel
because we don't meet the requirements.


> Write me personally and I will try to advise how to configure/tune IDE for
> your project.
> I'm eager to do that, because we are very interested in user's experience
> on big code bases.
> In fact we develop NetBeans as an IDE platform for Oracle Solaris Studio
> IDE where customers are dealing with really huge enterprise apps (recently
> I talked to customer successfully using Studio IDE for 50K Files/15MLoc C++
> boost-intensive codebase)
>

Well, the problem I have is that I have:
- a libclang / CompilationDatabase implementation that works for our
internal codebase, but has rather non-standard latency (can have 1-2
minutes latency on first load of a file, if none of the files around have
been touched yet)
- it is basically impossible to get all compile commands for the whole
codebase at once

The way people work around this when they use IDEs is that they generate a
"view" of part of the codebase, but this takes considerable amount of time
of effort to implement.

Cheers,
/Manuel


>
> Thanks,
> Vladimir.
>
>
> Cheers,
> /Manuel
>
>
>>
>>
>> Vladimir.
>>
>>
>>
>>>
>>>
>>> Vladimir.
>>>
>>>
>>>
>>>>
>>>> Vladimir.
>>>>
>>>> On 09/10/15 03:17 AM, Keith Smith via cfe-dev wrote:
>>>> > Mats, Renalto - Thanks for the information
>>>> >
>>>> > I beg to differ that Eclipse CDT hasn't caught on. The originator of
>>>> > Eclipse CDT, QNX, and the maintainers, use Eclipse CDT as their IDE
>>>> > for their OS. QNX is in many high end car nav systems today.
>>>> >
>>>> > Eclipse CDT is the basis for many embedded tool chains used by
>>>> > firmware engineers, both in Linux and Windows.
>>>> >
>>>> > Eclipse CDT may not have caught on as a host OS, host app development
>>>> > IDE, but it is used extensively.
>>>> >
>>>> > I have used it for over ten years now. It has had its limitations,
>>>> > like no 'headless' builds, but that has been corrected.
>>>> >
>>>> > Anyway thanks for the info. I was afraid that emacs and vi(m) would be
>>>> > part of the response. :( Don't use either at present.
>>>> >
>>>> > Keith Smith
>>>> >
>>>> > On Wed, Sep 9, 2015 at 10:35 AM, mats petersson <
>>>> mats at planetcatfish.com> wrote:
>>>> >>
>>>> >> On 9 September 2015 at 15:03, Renato Golin <renato.golin at linaro.org>
>>>> wrote:
>>>> >>> On 9 September 2015 at 14:29, mats petersson <
>>>> mats at planetcatfish.com>
>>>> >>> wrote:
>>>> >>>> Technically, I'm not an LLVM or Clang developer [by which I mean,
>>>> I'm
>>>> >>>> not
>>>> >>>> contributing code to LLVM or Clang, although I do have a patch for
>>>> clang
>>>> >>>> that may make it in at some point], but I do use Emacs with cscope.
>>>> >>> Honest question: how does cscope copes with C++11 constructs? I
>>>> >>> finally gave up emacs when cscope was the only thing I could use and
>>>> >>> it wasn't enough. Maybe I missed something?
>>>> >>
>>>> >> I have not tried on big projects, but I use cscope  on C++ in my
>>>> hobby
>>>> >> project compiler, which uses limited C++11 features, and it's not
>>>> failing in
>>>> >> any obvious way for this use-case. But llvm is "out of tree", and I
>>>> >> typically use google and the online doxygen pages for LLVM searches.
>>>> >>
>>>> >> My main use is in my day-job, which is nearly all C, so C++11 is not
>>>> a big
>>>> >> issue - but the build we use has all of clang and llvm in the
>>>> sources, and
>>>> >> cscope is not failing in any obvious way, and I can search for
>>>> "getType" and
>>>> >> it finds a load of them. But I'm sure there may be more subtle
>>>> things that I
>>>> >> don't notice because when I use cscope in this project, I'm typically
>>>> >> searching for C symbols, not C++ things.
>>>> >>>
>>>> >>>
>>>> >>>> I'm not trying to start a war with Renato about "vi(m) vs
>>>> (x)emacs" -
>>>> >>>> it's
>>>> >>>> pointless,
>>>> >>> That was a joke. :)
>>>> >>
>>>> >> Sorry, my "sarcasticly pointing out the pointlessness of a editor
>>>> flame-war"
>>>> >> obviously didn't have the (right) sarcasm font... ;)
>>>> >>>
>>>> >>>
>>>> >>>> it's just one of those choices one makes at some point in life -
>>>> >>>> once you know enough to do things with ease in one, you end up not
>>>> >>>> liking
>>>> >>>> the other.
>>>> >>> Yup. Especially as you get older... :)
>>>> >>
>>>> >> I've been old quite some time now... ;)
>>>> >>
>>>> >> --
>>>> >> Mats
>>>> >>>
>>>> >>>
>>>> >>>> I'm sufficiently damaged that I type ESC+w to copy text in
>>>> >>>> the browser - which of course doesn't work... :(
>>>> >>> I type :wq and "i" everywhere, too. :)
>>>> >>>
>>>> >>> cheers,
>>>> >>> --renato
>>>> >>
>>>> >
>>>> >
>>>>
>>>> _______________________________________________
>>>> cfe-dev mailing list
>>>> cfe-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>
>>>
>>>
>>
>> _______________________________________________
>> cfe-dev mailing listcfe-dev at lists.llvm.orghttp://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
>>
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
>
>
> _______________________________________________
> cfe-dev mailing listcfe-dev at lists.llvm.orghttp://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20150914/1fbc307a/attachment.html>

More information about the cfe-dev mailing list