[cfe-dev] RFC: Place libs in Clang-dedicated directories (affects openmp, libcxx, libunwind, compiler-rt)

Tue Mar 12 10:15:49 PDT 2019

On Tue, Mar 12, 2019 at 7:43 AM Ilya Biryukov <ibiryukov at google.com> wrote:

> Hi Joel,
>
> Sorry for the late response, was on vacation. Find a few more
> clarifications inline.
> Hopefully still useful, even though the actual changes have already landed.
>
> On Wed, Feb 27, 2019 at 7:28 PM Joel E. Denny <jdenny.ornl at gmail.com>
> wrote:
>
>> Hi Ilya,
>>
>> On Wed, Feb 27, 2019 at 6:08 AM Ilya Biryukov <ibiryukov at google.com>
>> wrote:
>> >
>> > Hi Joel,
>> >
>> > clangd, clang-tidy and other tools do not require to be built from the
>> same revision as the host compiler that the project uses to build code. In
>> fact, the compiler is not necessarily clang, it can be gcc or MSVC.
>> > However, the internal clang headers (the ones -resource-dir points at)
>> must correspond to the same version of the code that the clang frontend is
>> built from.
>> > So the aforementioned tools ship their own version of clang's internal
>> headers and pass -resource-dir to the clang frontend to make sure the
>> frontend picks them up. I.e. if the host compiler is also clang, the tools
>> must not pick the host clang's internal headers.
>> > The tools take other compilation arguments from a compilation database
>> (compile_commands.json).
>>
>> Thanks.  That clears up a lot for me.
>>
>> Naively, I would've thought any project (clangd, clang-tidy, or any
>> project external to LLVM) would specify -I to tell the compiler
>> (clang, gcc, or MSVC) where to find the project's required headers
>> when building the project.  Using -resource-dir sounds like a special
>> shortcut for the case where the project is clang-based (clangd or
>> clang-tidy) and the compiler building the project is clang.  Is that
>> right?  What do clangd and clang-tidy specify to the compiler if the
>> compiler is instead gcc or MSVC?  Do those have an option equivalent
>> to -resource-dir?
>>
> The underlying "compiler" for clangd and clang-tidy is always clang (as
> they are calling into the clang frontend), so the tools
> would just override -resource-dir as normal. This only works because clang
> is good enough at understanding gcc flags
> and clang-cl is good enough at understanding MVSC flags (although I
> haven't really checked MSVC flags work, but in
> principle they could).
>
>
>> I don't mean to be arguing against the usage of -resource-dir for this
>> purpose.  I'm just trying to understand it.
>>
>> >
>> > Note that the internal headers is the only thing that the tools need to
>> override, e.g. this should not affect the C++ standard library found by the
>> tools.
>> > For that to work, we have to make sure the internal headers is the only
>> thing affected by "-resource-dir", we don't want the tools to see a
>> different standard library (or not find a standard library at all).
>>
>> Makes sense.
>>
>> >
>> > So far in cases where -resource-dir was used for finding libc++,
>> replacing it with "compiler install dir" ("Driver.InstalledDir")  seemed to
>> do the job.
>>
>> Does "resource-dir used for finding libc++" imply libc++ is in the
>> resource directory and so LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=True?
>>
> The same recipe seemed to work in that case.
> LLVM_ENABLE_PER_TARGET_RUNTIME_DIR relied on -resource-dir before D59013
> landed, and seems to use install dir now.
>

FYI D59013 was reverted because it broke Windows bots, but should re-land
soon as D59168.

> Thanks.
>>
>> Joel
>>
>> >
>> > On Wed, Feb 27, 2019 at 12:09 AM Joel E. Denny <jdenny.ornl at gmail.com>
>> wrote:
>> >>
>> >> Hi Ilya,
>> >>
>> >> On Mon, Feb 25, 2019 at 5:32 AM Ilya Biryukov <ibiryukov at google.com>
>> wrote:
>> >> >
>> >> > > From this point of view, what LLVM_ENABLE_PER_TARGET_RUNTIME_DIR
>> breaks the tools and the proposed alternative fixes this problem
>> >> > what  LLVM_ENABLE_PER_TARGET_RUNTIME_DIR does now ...
>> >> >
>> >> > On Mon, Feb 25, 2019 at 11:31 AM Ilya Biryukov <ibiryukov at google.com>
>> wrote:
>> >> >>
>> >> >> Using the resource-dir in the header search paths would break tools
>> that use compilation database, e.g. clang-tidy and clangd.
>> >> >> They override the resource-dir as it's very common for them to be
>> built from a different revision than the used compiler. They rely on the
>> fact that the same standard library can be found with the overridden
>> resouce-dir.
>> >> >>
>> >> >> From this point of view, what LLVM_ENABLE_PER_TARGET_RUNTIME_DIR
>> breaks the tools and the proposed alternative fixes this problem.
>> >>
>> >> Thanks for pointing this out.  I'd like to better understand, but I'm
>> >> only familiar with clang-tidy and clangd from a high level.  Can you
>> >> explain a bit more about how they interact with the used compiler and
>> >> how they use the overridden resource directory?
>> >>
>> >> Thanks.
>> >>
>> >> Joel
>> >>
>> >> >>
>> >> >> On Mon, Feb 25, 2019 at 11:17 AM Petr Hosek via cfe-dev <
>> cfe-dev at lists.llvm.org> wrote:
>> >> >>>
>> >> >>> On Wed, Feb 20, 2019 at 6:14 PM Joel E. Denny <
>> jdenny.ornl at gmail.com> wrote:
>> >> >>>>
>> >> >>>> My alternative to LLVM_ENABLE_PER_TARGET_RUNTIME_DIR is the
>> preceding
>> >> >>>> bullets.  In other words, you wouldn't need to specify
>> >> >>>> LLVM_ENABLE_PER_TARGET_RUNTIME_DIR because it would effectively be
>> >> >>>> always on (except the directories might be different than now if
>> the
>> >> >>>> version locking issue is important, as noted above).  Is that what
>> >> >>>> you're asking?
>> >> >>>
>> >> >>>
>> >> >>> That would be my preference. I always hoped that
>> LLVM_ENABLE_PER_TARGET_RUNTIME_DIR would eventually become the default. It
>> would be nice to finish the Darwin support so we can completely deprecate
>> the old layout, but I don't know how far along beanz is in his effort. We
>> should also update openmp to stop using the custom Android-specific runtime
>> layout.
>> >> >>>
>> >> >>> There's also the unresolved question of where should libc++
>> headers and libraries go. Currently, in LLVM_ENABLE_PER_TARGET_RUNTIME_DIR
>> we use the resource dir, but some people expressed the opinion that we
>> shouldn't be using these for libc++ et al. since they're not version-locked
>> to Clang. This is different from what GCC does (e.g. GCC would use
>> $prefix/lib/gcc/x86_64-linux-gnu/8/libstdc++.a) and it's one of the reasons
>> why I used the resource dir for libc++ et al. when implementing
>> LLVM_ENABLE_PER_TARGET_RUNTIME_DIR.
>> >> >>>
>> >> >>> So concretely, today LLVM_ENABLE_PER_TARGET_RUNTIME_DIR uses the
>> following layout:
>> >> >>>
>> >> >>> headers: $prefix/lib/clang/$version/include(/$triple)(/c++/v1)
>> >> >>> libraries: $prefix/lib/clang/$version/$triple/lib/$name.$ext
>> >> >>>
>> >> >>> The alternative that doesn't use resource dir for libc++ would be
>> the following:
>> >> >>>
>> >> >>> compiler-rt:
>> >> >>>   headers: $prefix/lib/clang/$version/include
>> >> >>>   libraries: $prefix/lib/clang/$version/$triple/lib/$name.$ext
>> >> >>>
>> >> >>> libc++, libc++abi, libunwind:
>> >> >>>   headers: $prefix/include/c++/v1
>> >> >>>   libraries: $prefix/lib/$triple/$name.$ext
>> >> >>>
>> >> >>> Making this change should be trivial, it's the matter of changing
>> three CMakeLists.txt files (libunwind, libc++abi and libc++) and Clang
>> driver in one place. However, if we're going to make that change, I'd like
>> to get a broader consensus. It'd be also useful to get feedback from libc++
>> maintainers on this.
>> >> >>> _______________________________________________
>> >> >>> cfe-dev mailing list
>> >> >>> cfe-dev at lists.llvm.org
>> >> >>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Regards,
>> >> >> Ilya Biryukov
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Regards,
>> >> > Ilya Biryukov
>> >
>> >
>> >
>> > --
>> > Regards,
>> > Ilya Biryukov
>>
>
>
> --
> Regards,
> Ilya Biryukov
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190312/c4074f49/attachment.html>