[PATCH] D32721: Accept archive files with no symbol table instad of warning on them.

Thu May 4 21:56:48 PDT 2017

On Thu, May 4, 2017 at 9:40 PM, Davide Italiano <davide at freebsd.org> wrote:

> On Thu, May 4, 2017 at 4:30 PM, Rui Ueyama <ruiu at google.com> wrote:
> > Then maybe -DCMAKE_RANLIB=/usr/bin/true does magic? (This conversation
> is an
> > instance why I think this feature is useful :)
> >
>
> Replying to both of you together.
>
> Sean, my cmake version is 3.6. IIRC I tried this also with 3.4 in the
> golden days, but I'm not 100% sure.
>
> Rui, no magic here, I think. Setting RANLIB=/usr/bin/true is just an
> optimization as the work by ranlib is actually done by llvm-ar. People
> may want to set this to a real ranlib with older versions of ar(1)
> from GNU binutils, but even with llvm, setting RANLIB to be
> llvm-ranlib shouldn't change anything.

That explains it. I think RANLIB=/bin/true is actually a key.

If you forget to specify RANLIB, generated make/ninja files run
/usr/bin/ranlib on archive files. That strips symbol tables from archive
files containing bitcode because ranlib reconstructs symbol tables and it
doesn't know anything about LLVM bitcode files.

> On Thu, May 4, 2017 at 4:28 PM, Sean Silva via llvm-commits
> > <llvm-commits at lists.llvm.org> wrote:
> >>
> >>
> >>
> >> On Thu, May 4, 2017 at 4:26 PM, Davide Italiano <davide at freebsd.org>
> >> wrote:
> >>>
> >>> In particular, this is the conf I have in my script:
> >>>
> >>> cmake -GNinja
> >>>
> >>> -DCMAKE_C_COMPILER=/home/davide/work/llvm-monorepo/
> build-release/bin/clang
> >>>
> >>> -DCMAKE_CXX_COMPILER=/home/davide/work/llvm-monorepo/
> build-release/bin/clang++
> >>> -DCMAKE_C_FLAGS="-fuse-ld=gold -flto=thin"
> >>> -DCMAKE_CXX_FLAGS="-fuse-ld=gold -flto=thin"
> >>> -DCMAKE_AR=/home/davide/work/llvm-monorepo/build-release/bin/llvm-ar
> >>> -DCMAKE_RANLIB=/usr/bin/true ../llvm
> >>>
> >>> (I just build llvm/clang/lld FWIW).
> >>
> >>
> >> What CMake version? Maybe this issue has been fixed in newer CMake's.
> >>
> >> -- Sean Silva
> >>
> >>>
> >>>
> >>> On Thu, May 4, 2017 at 4:24 PM, Davide Italiano <davide at freebsd.org>
> >>> wrote:
> >>> > On Thu, May 4, 2017 at 4:12 PM, Sean Silva via llvm-commits
> >>> > <llvm-commits at lists.llvm.org> wrote:
> >>> >>
> >>> >>
> >>> >> On Thu, May 4, 2017 at 3:58 PM, Mehdi AMINI <joker.eph at gmail.com>
> >>> >> wrote:
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> 2017-05-04 15:50 GMT-07:00 Sean Silva <chisophugis at gmail.com>:
> >>> >>>>
> >>> >>>>
> >>> >>>>
> >>> >>>> On Wed, May 3, 2017 at 6:28 PM, Rui Ueyama via llvm-commits
> >>> >>>> <llvm-commits at lists.llvm.org> wrote:
> >>> >>>>>
> >>> >>>>> On Wed, May 3, 2017 at 6:12 PM, Mehdi AMINI <joker.eph at gmail.com
> >
> >>> >>>>> wrote:
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>> 2017-05-03 18:07 GMT-07:00 Rui Ueyama <ruiu at google.com>:
> >>> >>>>>>>
> >>> >>>>>>> On Wed, May 3, 2017 at 5:58 PM, Mehdi AMINI <
> joker.eph at gmail.com>
> >>> >>>>>>> wrote:
> >>> >>>>>>>>
> >>> >>>>>>>>
> >>> >>>>>>>> On Wed, May 3, 2017 at 5:51 PM Rui Ueyama <ruiu at google.com>
> >>> >>>>>>>> wrote:
> >>> >>>>>>>>>
> >>> >>>>>>>>> On Wed, May 3, 2017 at 5:48 PM, Mehdi AMINI
> >>> >>>>>>>>> <joker.eph at gmail.com>
> >>> >>>>>>>>> wrote:
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>
> >>> >>>>>>>>>> On Wed, May 3, 2017 at 5:44 PM Rui Ueyama <ruiu at google.com>
> >>> >>>>>>>>>> wrote:
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> On Wed, May 3, 2017 at 5:40 PM, Mehdi AMINI
> >>> >>>>>>>>>>> <joker.eph at gmail.com>
> >>> >>>>>>>>>>> wrote:
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> On Wed, May 3, 2017 at 5:26 PM Rui Ueyama <
> ruiu at google.com>
> >>> >>>>>>>>>>>> wrote:
> >>> >>>>>>>>>>>>>
> >>> >>>>>>>>>>>>> On Wed, May 3, 2017 at 5:13 PM, Mehdi AMINI
> >>> >>>>>>>>>>>>> <joker.eph at gmail.com> wrote:
> >>> >>>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>> Clang is incrementally linking in a matter of a few
> >>> >>>>>>>>>>>>>> seconds, so
> >>> >>>>>>>>>>>>>> 0.5s to read the symbols is a double digit percentage of
> >>> >>>>>>>>>>>>>> that.
> >>> >>>>>>>>>>>>>> And there are over 50 binaries in LLVM, not just one.
> >>> >>>>>>>>>>>>>
> >>> >>>>>>>>>>>>>
> >>> >>>>>>>>>>>>> We do not support incremental linking,
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> I'm talking about ThinLTO incremental linking, which we
> >>> >>>>>>>>>>>> support.
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> How can that be faster than the regular build?
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>
> >>> >>>>>>>>>> Not sure what you mean: on my mac ld64 links clang in less
> >>> >>>>>>>>>> than 2s.
> >>> >>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>> But that is ld64. We are talking about LLD, no?
> >>> >>>>>>>>
> >>> >>>>>>>>
> >>> >>>>>>>> I don't see where you're going: lld is supposed to be fast,
> >>> >>>>>>>> isn't it?
> >>> >>>>>>>> I assume it has to be able to outspeed ld64.
> >>> >>>>>>>> So I'm giving you a reference of what is "a regular" build
> time
> >>> >>>>>>>> and
> >>> >>>>>>>> that should explain why you .5s overhead is not trivial.
> >>> >>>>>>>
> >>> >>>>>>>
> >>> >>>>>>> That is hypothetical. If LLD is already able to link Clang with
> >>> >>>>>>> ThinLTO in 2 seconds, it may make sense to warn on 0.5 second
> >>> >>>>>>> loss, but
> >>> >>>>>>> that's in reality not the case, so I'm not convinced that we
> >>> >>>>>>> should warn on
> >>> >>>>>>> it right now.
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>> I disagree: we should define what is a *correct* build setting,
> >>> >>>>>> and
> >>> >>>>>> warn if it is not honored.
> >>> >>>>>> Your changing this definition here, and I doubt it'll be easy to
> >>> >>>>>> revert
> >>> >>>>>> in the future, which is why I don't like this.
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> That's a valid concern, and this change certainly relaxes the
> >>> >>>>> definition
> >>> >>>>> what is correct (or at least what is not incorrect). But I still
> >>> >>>>> think that
> >>> >>>>> that's beneficial for users overall. You are extremely familiar
> >>> >>>>> with LLVM
> >>> >>>>> LTO, but ordinary developers don't know much about LLVM or build
> >>> >>>>> systems
> >>> >>>>> compared to you. Attempting to use llvm-ar took a fair amount of
> >>> >>>>> time even
> >>> >>>>> to me (which I hope better than the average Unix user). If we
> print
> >>> >>>>> out a
> >>> >>>>> warning on every linker invocation, we'd probably be teaching
> users
> >>> >>>>> to
> >>> >>>>> ignore warnings rather than let them change the build
> >>> >>>>> configuration, as it
> >>> >>>>> just works with a marginal performance difference.
> >>> >>>>
> >>> >>>>
> >>> >>>>
> >>> >>>> In my experience (and mentioned in this thread too), changing the
> >>> >>>> archiver is very difficult. One necessary requirement for
> emitting a
> >>> >>>> warning
> >>> >>>> is that it has to be actionable.
> >>> >>>>
> >>> >>>> For example, in CMake, I only know how to do it after looking at
> >>> >>>> Rafael's
> >>> >>>> https://github.com/espindola/llvm-scripts (and I have no idea how
> >>> >>>> Rafael
> >>> >>>> figured it out).
> >>> >>>
> >>> >>>
> >>> >>> As a side note: this is suboptimal I believe, cmake has -DCMAKE_AR
> /
> >>> >>> -DCMAKE_RANLIB options (just like -DCMAKE_C_COMPILER).
> >>> >>
> >>> >>
> >>> >> IIRC, I've tried to use those and they didn't work. My memory is
> >>> >> vague, but
> >>> >> I think the issue was that CMAKE_AR and CMAKE_RANLIB were not
> >>> >> propagated to
> >>> >> all configuration checks, so that some of them spuriously failed
> >>> >> because the
> >>> >> CFLAGS had -flto but the CMAKE_AR/CMAKE_RANLIB was not being
> >>> >> respected.
> >>> >>
> >>> >
> >>> > hmm, my memory is very fuzzy too but I remember self-hosting LLVM
> with
> >>> > -DCMAKE_AR=myar -DCMAKE_RANLIB=/bin/true (and maybe setting
> -DCMAKE_NM
> >>> > as well, but I'm not 100% sure). Everything went smoothly, but it's
> >>> > been a while so things may have changed.
> >>> >
> >>> > --
> >>> > Davide
> >>> >
> >>> > "There are no solved problems; there are only problems that are more
> >>> > or less solved" -- Henri Poincare
> >>>
> >>>
> >>>
> >>> --
> >>> Davide
> >>>
> >>> "There are no solved problems; there are only problems that are more
> >>> or less solved" -- Henri Poincare
> >>
> >>
> >>
> >> _______________________________________________
> >> llvm-commits mailing list
> >> llvm-commits at lists.llvm.org
> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
> >>
> >
>
>
>
> --
> Davide
>
> "There are no solved problems; there are only problems that are more
> or less solved" -- Henri Poincare
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170504/b6a7232e/attachment.html>