[cfe-dev] RFC: Place libs in Clang-dedicated directories (affects openmp, libcxx, libunwind, compiler-rt)

Tue Mar 12 07:43:10 PDT 2019

Hi Joel,

Sorry for the late response, was on vacation. Find a few more
clarifications inline.
Hopefully still useful, even though the actual changes have already landed.

On Wed, Feb 27, 2019 at 7:28 PM Joel E. Denny <jdenny.ornl at gmail.com> wrote:

> Hi Ilya,
>
> On Wed, Feb 27, 2019 at 6:08 AM Ilya Biryukov <ibiryukov at google.com>
> wrote:
> >
> > Hi Joel,
> >
> > clangd, clang-tidy and other tools do not require to be built from the
> same revision as the host compiler that the project uses to build code. In
> fact, the compiler is not necessarily clang, it can be gcc or MSVC.
> > However, the internal clang headers (the ones -resource-dir points at)
> must correspond to the same version of the code that the clang frontend is
> built from.
> > So the aforementioned tools ship their own version of clang's internal
> headers and pass -resource-dir to the clang frontend to make sure the
> frontend picks them up. I.e. if the host compiler is also clang, the tools
> must not pick the host clang's internal headers.
> > The tools take other compilation arguments from a compilation database
> (compile_commands.json).
>
> Thanks.  That clears up a lot for me.
>
> Naively, I would've thought any project (clangd, clang-tidy, or any
> project external to LLVM) would specify -I to tell the compiler
> (clang, gcc, or MSVC) where to find the project's required headers
> when building the project.  Using -resource-dir sounds like a special
> shortcut for the case where the project is clang-based (clangd or
> clang-tidy) and the compiler building the project is clang.  Is that
> right?  What do clangd and clang-tidy specify to the compiler if the
> compiler is instead gcc or MSVC?  Do those have an option equivalent
> to -resource-dir?
>
The underlying "compiler" for clangd and clang-tidy is always clang (as
they are calling into the clang frontend), so the tools
would just override -resource-dir as normal. This only works because clang
is good enough at understanding gcc flags
and clang-cl is good enough at understanding MVSC flags (although I haven't
really checked MSVC flags work, but in
principle they could).

> I don't mean to be arguing against the usage of -resource-dir for this
> purpose.  I'm just trying to understand it.
>
> >
> > Note that the internal headers is the only thing that the tools need to
> override, e.g. this should not affect the C++ standard library found by the
> tools.
> > For that to work, we have to make sure the internal headers is the only
> thing affected by "-resource-dir", we don't want the tools to see a
> different standard library (or not find a standard library at all).
>
> Makes sense.
>
> >
> > So far in cases where -resource-dir was used for finding libc++,
> replacing it with "compiler install dir" ("Driver.InstalledDir")  seemed to
> do the job.
>
> Does "resource-dir used for finding libc++" imply libc++ is in the
> resource directory and so LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=True?
>
The same recipe seemed to work in that case.
LLVM_ENABLE_PER_TARGET_RUNTIME_DIR relied on -resource-dir before D59013
landed, and seems to use install dir now.

> Thanks.
>
> Joel
>
> >
> > On Wed, Feb 27, 2019 at 12:09 AM Joel E. Denny <jdenny.ornl at gmail.com>
> wrote:
> >>
> >> Hi Ilya,
> >>
> >> On Mon, Feb 25, 2019 at 5:32 AM Ilya Biryukov <ibiryukov at google.com>
> wrote:
> >> >
> >> > > From this point of view, what LLVM_ENABLE_PER_TARGET_RUNTIME_DIR
> breaks the tools and the proposed alternative fixes this problem
> >> > what  LLVM_ENABLE_PER_TARGET_RUNTIME_DIR does now ...
> >> >
> >> > On Mon, Feb 25, 2019 at 11:31 AM Ilya Biryukov <ibiryukov at google.com>
> wrote:
> >> >>
> >> >> Using the resource-dir in the header search paths would break tools
> that use compilation database, e.g. clang-tidy and clangd.
> >> >> They override the resource-dir as it's very common for them to be
> built from a different revision than the used compiler. They rely on the
> fact that the same standard library can be found with the overridden
> resouce-dir.
> >> >>
> >> >> From this point of view, what LLVM_ENABLE_PER_TARGET_RUNTIME_DIR
> breaks the tools and the proposed alternative fixes this problem.
> >>
> >> Thanks for pointing this out.  I'd like to better understand, but I'm
> >> only familiar with clang-tidy and clangd from a high level.  Can you
> >> explain a bit more about how they interact with the used compiler and
> >> how they use the overridden resource directory?
> >>
> >> Thanks.
> >>
> >> Joel
> >>
> >> >>
> >> >> On Mon, Feb 25, 2019 at 11:17 AM Petr Hosek via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
> >> >>>
> >> >>> On Wed, Feb 20, 2019 at 6:14 PM Joel E. Denny <
> jdenny.ornl at gmail.com> wrote:
> >> >>>>
> >> >>>> My alternative to LLVM_ENABLE_PER_TARGET_RUNTIME_DIR is the
> preceding
> >> >>>> bullets.  In other words, you wouldn't need to specify
> >> >>>> LLVM_ENABLE_PER_TARGET_RUNTIME_DIR because it would effectively be
> >> >>>> always on (except the directories might be different than now if
> the
> >> >>>> version locking issue is important, as noted above).  Is that what
> >> >>>> you're asking?
> >> >>>
> >> >>>
> >> >>> That would be my preference. I always hoped that
> LLVM_ENABLE_PER_TARGET_RUNTIME_DIR would eventually become the default. It
> would be nice to finish the Darwin support so we can completely deprecate
> the old layout, but I don't know how far along beanz is in his effort. We
> should also update openmp to stop using the custom Android-specific runtime
> layout.
> >> >>>
> >> >>> There's also the unresolved question of where should libc++ headers
> and libraries go. Currently, in LLVM_ENABLE_PER_TARGET_RUNTIME_DIR we use
> the resource dir, but some people expressed the opinion that we shouldn't
> be using these for libc++ et al. since they're not version-locked to Clang.
> This is different from what GCC does (e.g. GCC would use
> $prefix/lib/gcc/x86_64-linux-gnu/8/libstdc++.a) and it's one of the reasons
> why I used the resource dir for libc++ et al. when implementing
> LLVM_ENABLE_PER_TARGET_RUNTIME_DIR.
> >> >>>
> >> >>> So concretely, today LLVM_ENABLE_PER_TARGET_RUNTIME_DIR uses the
> following layout:
> >> >>>
> >> >>> headers: $prefix/lib/clang/$version/include(/$triple)(/c++/v1)
> >> >>> libraries: $prefix/lib/clang/$version/$triple/lib/$name.$ext
> >> >>>
> >> >>> The alternative that doesn't use resource dir for libc++ would be
> the following:
> >> >>>
> >> >>> compiler-rt:
> >> >>>   headers: $prefix/lib/clang/$version/include
> >> >>>   libraries: $prefix/lib/clang/$version/$triple/lib/$name.$ext
> >> >>>
> >> >>> libc++, libc++abi, libunwind:
> >> >>>   headers: $prefix/include/c++/v1
> >> >>>   libraries: $prefix/lib/$triple/$name.$ext
> >> >>>
> >> >>> Making this change should be trivial, it's the matter of changing
> three CMakeLists.txt files (libunwind, libc++abi and libc++) and Clang
> driver in one place. However, if we're going to make that change, I'd like
> to get a broader consensus. It'd be also useful to get feedback from libc++
> maintainers on this.
> >> >>> _______________________________________________
> >> >>> cfe-dev mailing list
> >> >>> cfe-dev at lists.llvm.org
> >> >>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Regards,
> >> >> Ilya Biryukov
> >> >
> >> >
> >> >
> >> > --
> >> > Regards,
> >> > Ilya Biryukov
> >
> >
> >
> > --
> > Regards,
> > Ilya Biryukov
>

-- 
Regards,
Ilya Biryukov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190312/01992eb4/attachment.html>