[PATCH] D118252: Adding cmake to the exported list of tarballs

[Konrad Wilhelm Kleine via Phabricator via llvm-commits]

kwk added a comment.

In D118252#3272970 <https://reviews.llvm.org/D118252#3272970>, @serge-sans-paille wrote:

> @kkleine I have not tested that approach, but would that be possible to bundle the content of the `cmake` directory within each other tarball, so that the existence of `cmake/` is transparent to downstream user?
>
> I'm not 100% sure it's a good thing to do, as it makes building from tarball even more different from building from the mono repo, but as a packager I do see value in the approach.

First answering your question about the technical possibility:

`git archive` has an option (`--add-file`) to a file to the generated acrhive. You can use that option multiple times to add more than one file. The added file will be placed at the root of the tar ball. I think it is best to show you how this could be done. For simplicity I'll just use `tar` and not `tar.xz`:

  # Let's create the shared cmake tarball once for all projects to consume it:
  cd llvm-project/
  cd cmake/
  git archive --prefix=cmake-shared/ HEAD . > cmake-shared.tar
  
  # Let's pick a small project (`libcxx`) so its quick and easy to get results.
  # We'll create the tarball for that project (analogue to how we do it in `export.sh`).
  cd ../libcxx
  git archive --add-file=../cmake/cmake-shared.tar --prefix=libcxx/ HEAD . > libcxx-HEAD.tar
  
  # Now let's print out the top level elements in the resulting tarball
  tar -tf libcxx-HEAD.tar | cut -d "/" -f 1 | sort | uniq
  
  # Output:
  # cmake-shared.tar
  # libcxx

**NOTICE**: The resulting tarball contains the `cmake-shared.tar` as a standalone file. This is the cleanest way I could find to get the shared cmake directory into the outer-tarball.

Then, there's another way that looked promising but produced a rather odd looking tree structure in the end.

Instead of adding a single compressed file, we could add all the files from the top-level `cmake` directory individually:

  git archive $(for f in $(find ../cmake/ -type f); do echo "--add-file=$f"; done) --prefix=libcxx/ HEAD . > libcxx-HEAD.tar
  
  tar -tf libcxx-HEAD.tar | cut -d "/" -f 1 | sort | uniq
  
  # Output:
  # CheckLinkerFlag.cmake
  # EnableLanguageNolink.cmake
  # ExtendPath.cmake
  # FindPrefixFromConfig.cmake
  # HandleCompilerRT.cmake
  # HandleOutOfTreeLLVM.cmake
  # libcxx
  # README.rst
  # SetPlatformToolchainTools.cmake

**NOTICE:** The absensce of any nesting of the cmake files. Their placement in the subtree is gone.

Now that I've presented what's possible I wanted to talk a bit about the scenario and if it is really such a big problem, to include the cmake tarball into packages that need it.

1. Yes it's bad that packages need to know about this. But eventually this time had to come that projects outsource shared pieces.
2. The `llvm` subproject` is needed by many other projects. Do we include in them? No. Sure, it is much bigger in size but just looking at what the current practice is now...
3. Do we really want to include the shared cmake tree into projects that (yet) don't use it? What if they suddenly do, refactor this script again to include them? Why not make this a pull-in approach and leave it to the packagers to deal with? For clang in Fedora we already consume clang and clang-tools-extra and I think it doesn't necessarily hurt to include a third tarball there.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D118252/new/

https://reviews.llvm.org/D118252