[cfe-dev] Clang 3.5 Release Pre-Pre-Pre-Announcement

[Alp Toker]

On 21/05/2014 02:31, Reid Kleckner wrote:
> On Tue, May 20, 2014 at 3:54 PM, Alp Toker <alp at nuanti.com 
> <mailto:alp at nuanti.com>> wrote:
>
>
>     On 19/05/2014 04:41, Rafael Espíndola wrote:
>
>             TLDR; We _really_ want to calculate these dependencies at
>             build time with
>             LLVMBuild instead of annotating the source.
>
>         Is there a convenient way to feed this to link.exe?
>
>
>     We can pass in exports through a /def file, similar to exports
>     with other linkers.
>
>     To calculate the exports, someone will have to write a script/tool
>     that collects undefined symbols, then goes through visible symbols
>     in other modules to derive the export lists.
>
>     That'll generate minimal export lists, tuned for each platform and
>     configuration, permitting optimal dead stripping and optimisation.
>     And it'll save us from the burden of annotating loads and loads of
>     definitions in the source that only work for a few setups.
>
>
> It's an interesting idea, but I'm pretty opposed to this approach.
>
> First, this approach doesn't work for exported data.  You need 
> annotations to import data, and I can say from experience that we do 
> this in LLVM.

There's not much exported data compared to all the symbol exports, so 
there wouldn't be much cost in annotating these if needed.

But it's generally not a good idea to export data like that in defined 
interface anyway. I'd be surprised if there's much of that going on 
between LLVM / clang that can't be detected from symbols or tidied up.

>
> Second, I tried to rig up something that exports all symbols with 
> dumpbin once and it was ridiculously slow.  I think anything that 
> tried to scrape imports of one set of object files from another would 
> also be really slow.  Even if we used llvm-nm and made it fast, it'd 
> be part of our build system.  Ew.

We have a tool in our internal toolchain that handles this reading stdin 
and deriving exports with std::set and we link *huge* code bases using 
it. It's crazy fast so I don't buy the "I tried this and it was slow" 
argument.

ISTR that exporting minimal dependencies actually speeds up the final 
link by reducing redundantly exported symbols.

>
> Third, suppose we want to build LLVMSupport.dll.  Now we need to build 
> LLVMSupport.lib, then everything that uses LLVMSupport.dll, and then 
> scrape imports from all those static archives or object files.  This 
> is a *huge* web of dependencies.  Now building llvm-as.exe which 
> dynamically links LLVMSupport.dll depends on building clangSema.lib. 
>  That's a non-starter.

LTO also requires a full build so by that same argument, LTO would be a 
non-starter?

This is a technique for producing optimal release shared library / DLL 
builds, not for day-to-day development in your debug build.

To take the LTO analogy further, using annotations for linker hints is 
kind of like trying to annotate your entire source tree with __const__ 
and __pure__ attributes for readonly/readnone.

(Except it's more painful because the set of exports and imports changes 
all the time based on build configuration, platform and which modules 
happen to be using which functions changing day to day.)

> If we want to support DSOs exporting a C++ interface, IMO we should 
> define the DSO boundary up front and use annotations.  I think the 
> logical DSO boundaries would be LLVMSupport, LLVM, and Clang.

Logical to whom? That's only one of several supported build configurations.

For our use case, for example, we'd also need libclang.so so we'd 
introduce a *different* set of export macros than yours into the source 
code. Ditto if someone wants another configuration. And who's going to 
maintain all these annotations, individual committers?

Alp.



-- 
http://www.nuanti.com
the browser experts