[PATCH] D65430: Add `--dependency-files` option, which is equivalent to compiler option -MD.

[Petr Hosek via Phabricator via llvm-commits]

phosek added a comment.

In D65430#1614759 <https://reviews.llvm.org/D65430#1614759>, @MaskRay wrote:

> In D65430#1614683 <https://reviews.llvm.org/D65430#1614683>, @ruiu wrote:
>
> > In D65430#1614557 <https://reviews.llvm.org/D65430#1614557>, @MaskRay wrote:
> >
> > > In D65430#1606209 <https://reviews.llvm.org/D65430#1606209>, @emaste wrote:
> > >
> > > > In D65430#1605980 <https://reviews.llvm.org/D65430#1605980>, @peter.smith wrote:
> > > >
> > > > > My understanding is that many developers use makefile/ninja generation systems such as cmake rather than hand-write the file themselves. As such would this get much use unless it was integrated into these generators? May be worth approaching them to see if they have any requirements/observations about the option?
> > > >
> > > >
> > > > Having it integrated into e.g. cmake would be a great project for someone to take on, but for us (FreeBSD) it would be useful by itself. Our build is about 175K lines of hand-written Makefiles and we could plug it in with a small change in a couple of places. In any case we shouldn't hold up lld support waiting on prospective cmake changes IMO.
> > >
> > >
> > > Ed, does `ld.lld a.o -o /dev/null | sed 's/(.*//' | sort -u | sed '$!s/$/ \\/';}` generate a usable Makefile fragment? I think the scenarios where people write Makefile are not very common now (think cmake/bazel/meson/scons/buck/...). Makefile and build.ninja are mostly used as an "assembly language". I think a dependency graph feature will be useful, but a Makefile fragment is probably not the best format. (I actually like Makefile more than the alternatives but I know the trend is that people are moving away from hand-written Makefile) At least it looks like the output can be easily generated from `ld.lld -t` output.
> >
> >
> > I mentioned Makefile because `-MD` was created with that usage in mind (at least originally), but I guess that file format can be understood by other build tools. Am I missing something? Even though some build tools can understand C/C++ file dependencies without using a compiler, there's still some languages that the build systems cannot read, but still they can read auto-generated Makefile-compatible dependency file, no?
>
>
> Different build systems use different syntax :/
>
> Some are represented as a function call, e.g. `executable('main', 'main.c')`. Some are `cc_binary(name="main", srcs=["main.c"])`. llvm has `LLVMBuild.txt`. A Makefile fragment cannot be consumed by any of the build systems ;-)
>
> In a hand-written Makefile, `a.d` is usually used in the following pattern:
>
>   a.o a.d: a.c
>     gcc -MD -c $<
>  
>   -include a.d   # suppress the error if a.d does not exist
>
>
> `a.c` has many direct and indirect includes, e.g. `stdio.h`, `stddef.h`, `a.h`.
>  When any of the dependency is updated, `a.o` will be considered stale and will get rebuilt next time `make` is executed.
>
> `-MD` is useful in that it gives the information that isn't available before: we can't tell what files `a.c` includes.
>
> In the `ld.lld --write-dependencies=` case, the dependencies (bitcode files, object files, shared objects, and archive members that get pulled out) are already known. If it isn't, we'll get a "undefined reference" error. So I don't know who will be the consumer of the dependency output.


One use case for `ld.lld --dependency-file=` is auto-linking which is supported by lld for both ELF and COFF. In case of auto-linking, you don't know which additional dependencies are going to be included until you process all input files. Another case that's specific to ELF are linker scripts which allows specifying additional inputs using the `INPUT` statement e.g. libc++ installs the following linker script `INPUT(libc++.so.1 -lunwind -lc++abi)` as `libc++.so`.

Regarding the depfile format, the simplified Makefile format is the de-facto standard for depfiles and is emitted and supported by build systems like Make or Ninja (which is generated by number of other high-level build systems like CMake, Meson, GN, etc.). Using this format would make this functionality immediately usable.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D65430/new/

https://reviews.llvm.org/D65430