[cfe-dev] Tools, resource dir and toolchains

[Chris Bieneman via cfe-dev]

New versions of clang (I think 6.x and later) have a flag `-print-resource-dir` that can give you the resource directory path without needing to do crazy text parsing.

-Chris

> On Aug 22, 2018, at 1:48 PM, Kim Gräsman via cfe-dev <cfe-dev at lists.llvm.org> wrote:
> 
> I've had a few days to mull on this, and I think it should be pretty
> easy to locate the prefix of a given Clang version with some basic
> probing similar to what's done in Clang's Toolchain classes for GCC
> sysroots, etc.
> 
> - Find all 'clang*' on the PATH
> - Look for a version suffix matching what we depend on (e.g. clang-7)
> - Invoke best-match clang with --version and parse
> - Step up twice from that path, e.g. /usr/bin/clang/../../ -> /usr/
> - Append lib/clang/<version>/
> 
> That's the resource dir a tool should use.
> 
> So as long as `clang` is available on the PATH, this should just work.
> Otherwise, users can always provide the resource dir with
> `-resource-dir`.
> 
> And we'd need to make the relevant clang packages prerequisites of the tool.
> 
> Does that make sense?
> 
> - Kim
> On Fri, Aug 17, 2018 at 9:17 PM Kim Gräsman <kim.grasman at gmail.com> wrote:
>> 
>> Hi all,
>> 
>> This is a somewhat thorny and detail-ridden question, so please bear with me.
>> 
>> For IWYU, we've recently started work on addressing the
>> "builtin-headers" problem, where a Clang tool can't find e.g.
>> <stddef.h> as described here:
>> https://clang.llvm.org/docs/LibTooling.html#builtin-includes
>> 
>> One of our goals has been to allow users to install IWYU to a
>> non-Clang prefix, but I think we've run into a complete road-block.
>> 
>> - The Driver constructor initializes the resource dir to $(dirname
>> /path/to/clang)/../lib/clang/$version
>> - This can be overridden with the `-resource-dir` switch or, if we're
>> a tool building on Clang, in any bootstrap code where the Driver is
>> created
>> 
>> So far, so good. If we're careful to overwrite
>> `compiler->getHeaderSearchOpts().ResourceDir` with a path before using
>> the Driver to set up a compilation, Clang will use our custom resource
>> dir.
>> 
>> For the builtin headers, this works well, the path calculation there
>> is just `$resourcedir/include`.
>> 
>> But the resource dir is used in all sorts of places in the Toolchain
>> hierarchy and in InitHeaderSearch.cpp to locate various SDKs,
>> sysroots, auxiliary include paths, etc. Often in a way that they step
>> up from the resource dir to some (seemingly) unrelated directory. For
>> example:
>> https://github.com/llvm-mirror/clang/blob/master/lib/Frontend/InitHeaderSearch.cpp#L266
>> https://github.com/llvm-mirror/clang/blob/master/lib/Frontend/InitHeaderSearch.cpp#L332
>> https://github.com/llvm-mirror/clang/blob/master/lib/Frontend/InitHeaderSearch.cpp#L473
>> 
>> Here lies the challenge --
>> 
>> - For the `clang` binary, all path resolution will work out, because
>> its resource dir and itself are installed together to a common prefix
>> (I assume?)
>> - All of this path arithmetic is done in the Clang libraries and
>> always relative to the running executable
>> - When the Clang libraries are linked into a Clang tool, that tool
>> must also be installed to the same prefix or the Clang libraries'
>> assumptions about where to find toolchain paths are broken
>> - And if a tool overrides the resource dir, it basically pulls the rug
>> out from under itself -- there's no realistic way it can duplicate all
>> the subtrees necessary (this is probably mostly problematic for tools
>> that need to run in a cross-compilation scenario, but there are
>> examples in InitHeaderSearch.cpp above where target-specific include
>> paths break down)
>> 
>> It seems none of this is discoverable at build-time, either,
>> everything is figured out at runtime based on the running executable.
>> That makes sense, as the library code doesn't know where it will
>> ultimately be installed.
>> 
>> It would be nice if a tool was able to say "I depend on Clang being
>> installed", and it had some magic way to find out *where* a matching
>> Clang version was installed. That way it could just set its
>> resource-dir to that Clang path and everything would work. I wonder if
>> tools could use something like `llvm-config` at runtime to figure out
>> the install prefix for a given version?
>> 
>> If you've read this far, thanks!
>> 
>> - Kim
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev