[LLVMdev] RFC: ThinLTO Impementation Plan

[Teresa Johnson]

On Fri, May 15, 2015 at 2:54 PM, Duncan P. N. Exon Smith
<dexonsmith at apple.com> wrote:
>
>> On 2015-May-15, at 07:30, Teresa Johnson <tejohnson at google.com> wrote:
>>
>>>> a. Clang/LLVM/gold linker options:
>>>>
>>>> An early set of clang/llvm patches is needed to provide options to
>>>> enable ThinLTO (off by default), so that the rest of the
>>>> implementation can be disabled by default as it is added.
>>>> Specifically, clang options -fthinlto (used instead of -flto) will
>>>> cause clang to invoke the phase-1 emission of LLVM bitcode and
>>>> function summary/index on a compile step, and pass the appropriate
>>>> option to the gold plugin on a link step. The -thinlto option will be
>>>> added to the gold plugin and llvm-lto tool to launch the phase-2 thin
>>>> archive step. The -thinlto option will also be added to the ‘opt’ tool
>>>> to invoke it as a phase-3 parallel backend instance.
>>>
>>> I'm not sure I follow the `opt` part of this.  That's a developer
>>> tool, not something we ship.  It also doesn't have a backend (doesn't
>>> do CodeGen).  What am I missing?
>>
>> For the prototype I was using llvm-lto as my backend driver. I
>> realized that this was probably not the best option as we don't need
>> all of the LTO handling built into that driver, and it isn't listed as
>> a tool on http://llvm.org/docs/CommandGuide/, so my feeling was that
>> 'opt' was better supported and a better alternative. Unfortunately
>> when I was writing this up I forgot that 'opt' generates bitcode not
>> an object file.
>
> Both `opt` and `llvm-lto` are designed as tools for LLVM developers;
> the latter in particular is just for testing LTO-related things without
> needing to talk to a linker (so, it's platform agnostic).
>
> Neither is supported for end users.
>
>>
>> Another option would be to use clang and allow it to accept bitcode
>> and bypass parsing under an appropriate ThinLTO option. AFAICT there
>> isn't currently an option for clang to accept bitcode. Do you think
>> this is the right approach?
>
> Clang handles bitcode just fine.  The `-save-temps` option even
> round-trips through bitcode:
>
> $ lrun -w staging clang t.c -save-temps -###
> /Users/dexonsmith/data/llvm.symbols+asserts/staging/bin/clang t.c -save-temps -###
> clang version 3.7.0 (trunk 236655) (llvm/trunk 236658)
> Target: x86_64-apple-darwin14.4.0
> Thread model: posix
>  "/Volumes/Data/llvm.symbols+asserts/staging/bin/clang-3.7" "-cc1" "-triple" "x86_64-apple-macosx10.10.0" "-E" "-disable-free" "-main-file-name" "t.c" "-mrelocation-model" "pic" "-pic-level" "2" "-mthread-model" "posix" "-mdisable-fp-elim" "-masm-verbose" "-munwind-tables" "-target-cpu" "core2" "-target-linker-version" "250" "-dwarf-column-info" "-fno-unique-section-names" "-resource-dir" "/Volumes/Data/llvm.symbols+asserts/staging/bin/../lib/clang/3.7.0" "-fdebug-compilation-dir" "/tmp" "-ferror-limit" "19" "-fmessage-length" "160" "-stack-protector" "1" "-mstackrealign" "-fblocks" "-fobjc-runtime=macosx-10.10.0" "-fencode-extended-block-signature" "-fmax-type-align=16" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-o" "t.i" "-x" "c" "t.c"
>  "/Volumes/Data/llvm.symbols+asserts/staging/bin/clang-3.7" "-cc1" "-triple" "x86_64-apple-macosx10.10.0" "-emit-llvm-bc" "-emit-llvm-uselists" "-disable-free" "-main-file-name" "t.c" "-mrelocation-model" "pic" "-pic-level" "2" "-mthread-model" "posix" "-mdisable-fp-elim" "-masm-verbose" "-munwind-tables" "-target-cpu" "core2" "-target-linker-version" "250" "-dwarf-column-info" "-fno-unique-section-names" "-resource-dir" "/Volumes/Data/llvm.symbols+asserts/staging/bin/../lib/clang/3.7.0" "-fdebug-compilation-dir" "/tmp" "-ferror-limit" "19" "-fmessage-length" "160" "-stack-protector" "1" "-mstackrealign" "-fblocks" "-fobjc-runtime=macosx-10.10.0" "-fencode-extended-block-signature" "-fmax-type-align=16" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-disable-llvm-optzns" "-o" "t.bc" "-x" "cpp-output" "t.i"
>  "/Volumes/Data/llvm.symbols+asserts/staging/bin/clang-3.7" "-cc1" "-triple" "x86_64-apple-macosx10.10.0" "-S" "-disable-free" "-main-file-name" "t.c" "-mrelocation-model" "pic" "-pic-level" "2" "-mthread-model" "posix" "-mdisable-fp-elim" "-masm-verbose" "-munwind-tables" "-target-cpu" "core2" "-target-linker-version" "250" "-dwarf-column-info" "-fno-unique-section-names" "-resource-dir" "/Volumes/Data/llvm.symbols+asserts/staging/bin/../lib/clang/3.7.0" "-fdebug-compilation-dir" "/tmp" "-ferror-limit" "19" "-fmessage-length" "160" "-stack-protector" "1" "-mstackrealign" "-fblocks" "-fobjc-runtime=macosx-10.10.0" "-fencode-extended-block-signature" "-fmax-type-align=16" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-o" "t.s" "-x" "ir" "t.bc"
>  "/Volumes/Data/llvm.symbols+asserts/staging/bin/clang-3.7" "-cc1as" "-triple" "x86_64-apple-macosx10.10.0" "-filetype" "obj" "-main-file-name" "t.c" "-target-cpu" "core2" "-mrelax-all" "-o" "t.o" "t.s"
>  "/usr/bin/ld" "-demangle" "-dynamic" "-arch" "x86_64" "-macosx_version_min" "10.10.0" "-o" "a.out" "t.o" "-lSystem"
>

Ok good. I tried again and it worked fine, my bitcode file must have
been out of date when I got an error this morning. I will change the
proposal to just use clang as the backend driver, using an option to
specify that we are doing ThinLTO and the path to the combined
function index/summary.

Thanks,
Teresa


-- 
Teresa Johnson | Software Engineer | tejohnson at google.com | 408-460-2413