[patch] Misc changes to LTOCodeGenerator

[Shuxin Yang]

Ping...

It actually took me lots of effort to convince people inside Apple as to 
It is *terribly* bad idea to expose
control of libLTO.{dylib,so} to linkers, and I hope in the long run, we 
will ditch those damn lto_xxx APIs,
or just expose to those who just want to plugin a home-brew linker for 
hobby project...

It seems I need to dual effort to convince people as well...

Why it is terrible to use those lto_xxx APIs in the projects he/she want 
to make money out of it, and is
release in separate SW?

Considering design a PC with 3 components involved : CPU, graphic 
adapter, display. Suppose the
adapater is installed on old PCI bus, and display can hook to VGA port.

    Design 1: put CPU in computer case, and graphic-adapter stay with 
display, they two form monitor.
                   Dig a hole on the computer case to install a socket 
for PCI bus.
                   And the monitor hook to the external-PCI-socket?

    Design 2: CPU and adapter stay in computer case,  which communicate 
with display with VCG.

   Design 1 sound like a disaster right? Linker directly calling lto_xxx 
APIs are similar to the design-1
in many ways.

   In our case, things are even messy. We support at least two linkers, 
each has its esoteric way to
"control" the libLTO. (Do we have unified way to save the 
merged-linker?).   Adding control
to linkers is just like cancer grow....


On 8/8/13 5:22 PM, Shuxin Yang wrote:
> Hi,
>
>  This patch is to :
>  1) restructure the LTOCodegenerator little bit such that partitioning 
> can comfortably kick in.
>    (The code about partitioning is not include in this patch)
>  2) introduce workdir to better manage intermediate files generated 
> during the LTO compilation.
>  3) tries to unify some debugging functionalities of Apple-linker and 
> GNU-gold.
>
>  For the first goal, I introduce several classes:
>   o. IPOPartition: To depict a partition.
>   o. PostIPODriver: The driver for the Post-IPO compilation.
>       We are going to support several variants of the drivers, all of 
> them will derive
>       from this base class.
>
>   o. IPOFile : To manage misc info about an intermediate file 
> generated during the IPO or
>                PostIPO phases, like file-name, absolute-path, 
> last-err-code, last-error-string.
>                It is not a be wrapper of file IOs.
>   o. and other misc classes.
>
>   They are designed in a way such the the top-level driver 
> LTOCodeGenerator will not see much
> difference with or without partition.  This is achieved by viewing a 
> merged module as
> single degenerated partition if the partition is not enabled.
>
> The workdir
> ===========
>   To better manage the intermediate files (*.bc file, obj file etc), 
> we put all intermediate
> files into a temp workdir.  If the path of workdir is not specified, 
> the workdir is created
> under $PWD, with the name leading with "llvm-ipo".
>
>   This workdir will be erased if the user dose not instruct the 
> compiler to keep it.
>
>   In order to properly erase the workdir, I have to introduce one API:
>     lto_codegen_get_files_need_remove().
>
>   Apple-ld will not see this change, as Apple-linker communicates with 
> libLTO.dylib with
> memory buffer instead of object files. After the libLTO.dylib hands 
> the buffer the
> linker, it can safely erase the workdir, i.e. it is up to libLTO.dylib 
> it erase the workdir.
>
>   The GNU-gold linker's adaptor (tools/gold/gold-plugin.cpp) is 
> modified slightly in order
> to accommodate this change.
>
>   TODO: I don't know to install a sig-handler to erase the workdir if 
> the linker quit
>   prematurely. Currently, the delete-on-signal support-routine ignores 
> directory. I don't
>   know if this is intentional.
>
> Misc
> =====
>    Now that we are able to keep intermediate file, the "obj-path" flag in
> GNU-gold-linker adaptor is not useful any more, and is removed by this 
> patch.
>
>    Apple-ld's -Wl,-save-temps is not useful either. On the other hand, 
> it will
> become meaningless if partition is enabled. The reason is that 
> -Wl,-save-temps
> is to instruct linker to save "merged-module". However, the partitioning,
> the merged-module is "merged" module at beginning; it become 
> "remaining module"
> right after partition.
>
>    Maintaining two linker is annoying, I'd like to shift such 
> functionality to
> the LTO driver.
>
> Thanks
> Shuxin