[llvm] r188188 - Misc enhancements to LTO:

[Shuxin Yang]

On 8/12/13 1:03 PM, Nick Lewycky wrote:
> On 12 August 2013 12:22, Shuxin Yang <shuxin.llvm at gmail.com 
> <mailto:shuxin.llvm at gmail.com>> wrote:
>
>     On 8/12/13 12:16 PM, Nick Lewycky wrote:
>
>         Shuxin Yang wrote:
>
>             Author: shuxin_yang
>             Date: Mon Aug 12 13:29:43 2013
>             New Revision: 188188
>
>             URL: http://llvm.org/viewvc/llvm-project?rev=188188&view=rev
>             Log:
>             Misc enhancements to LTO:
>
>                1. Add some helper classes for partitions. They are
>             designed in a
>                   way such that the top-level LTO driver will not see
>             much difference
>                   with or without partitioning.
>
>                2. Introduce work-dir. Now all intermediate files
>             generated during
>                   LTO phases will be saved under work-dir. User can
>             specify the workdir
>                   via -lto-workdir=/path/to/dir. By default the
>             work-dir will be
>                   erased before linker exit. To keep the workdir, do
>             -lto-keep, or -lto-keep=1.
>
>                  TODO: Erase the workdir, if the linker exit prematurely.
>                    We are currently not able to remove directory on
>             signal. The support
>                    routines simply ignore directory.
>
>                3. Add one new API lto_codegen_get_files_need_remove().
>                   Linker and LTO plugin will communicate via this API
>             about which files
>                  (including directories) need to removed before linker
>             exit.
>
>
>         Please revert. Adding new flags to libLTO is the wrong
>         direction (in spite of the ones that exist -- consider those
>         grandfathered in).
>
>     It dose not make sense. Without flags, how do you manage to triage
>     the correctness and performance problem?
>
>
> Something else has flags,

What are "something else"?  As far as I know, there are only two fall 
into this category:

    - Apple linker, and
    - GNU gold.

   The former communicate with the libLTO directly with these APIs, 
while GNU gold communicate
with the libLTO via, which I called adapter.

    Directly calling these APIs is really bad idea. I manage to convince 
the black-belt guru Nick @ Apple to
not directly calling these APIs in the new ld design. The linker's 
should be LTO oblivious, the linker
should expose symbol-related interface instead of LTO-control interfaces.

 >which in turn drives libLTO through the API.

Depending on the what kind of info "something" else need to drive the 
libLTO.
In general it is very bad idea, if "something else" need micro-management.

Take Apple ld as example,  if I want to change LTO in a way such that I 
don't want to load all module,
I just want to load summary info. Current APIs are not sufficient. I 
have to modify the API, or add new APIs
to that matter, in the mean time, I need release the new ld to the user 
in order to accomodate the change.
that is nightmare.

Sure, "something else" can control the libLTO, if it want. In my case, 
if "something else" want specify
  a workdir, then go ahead. Otherwise, the libLTO use default one. Is 
there any wrong here?


>     Adding flags to linker instead, I think that is wrong direction.
>     Linker dose not have data structure which libLTO dose.
>
>
> This is the discussion to have. What things do you need here which you 
> don't think should be exposed through the API, and yet you want to be 
> exposed for you?
I actually discuss with Nick @ Apple before.  The conclusion is linker 
must be LTO oblivious,
it should think in symbol-way, and talk in symbol way (as with GNU 
gold). It would otherwise
  very very troublesome both for linker and libLTO.

On the other hand, we now have two linkers support LTO. There are 
different way to control
the libLTO (even for simple task, like save intermediate files), how messy?

I'd like to move all these stuff to libLTO to have a unified control.


>
>     libLTO is intended to be used as a library, it may not get a
>     chance to parse flags.
>     It has to. Prior to my change, linkers (GNU linker and Apple ld)
>     pass arch to linker, via a function
>     confusingly called, something like "add.*debug.*options".
>
>
> Can't. If we allow this, every flag in every part of LLVM that libLTO 
> links against is baked into the C ABI forever.
>
> Of course addDebugOptions does allow this, but it's named (and I 
> thought documented in the comments) such that anybody using it knows 
> they're using a non-stable non-production debugging API. Anybody using 
> addDebugOptions for something other than debugging libLTO is living 
> outside the ABI guarantees.
addDebugOptions is misnomer. It is also passes essential flags like 
-arch=x86.  Without such flags,
the LTO dose not even compile.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20130812/9850fbcc/attachment.html>