[PATCH] Change representation of dllexport/dllimport

[Robinson, Paul]

> From: Nick Lewycky [mailto:nlewycky at google.com] 
> Sent: Friday, September 13, 2013 4:13 PM
> To: Robinson, Paul
> Cc: Chandler Carruth; reviews+D1110+public+872068f434b7231c at llvm-reviews.chandlerc.com; llvm-commits
> Subject: Re: [PATCH] Change representation of dllexport/dllimport
>
> On 12 September 2013 14:08, Robinson, Paul <Paul_Robinson at playstation.sony.com> wrote:
> > > From: Nick Lewycky [mailto:nlewycky at google.com]
> > > Sent: Thursday, September 12, 2013 11:43 AM
> > > To: Robinson, Paul
> > > Cc: Chandler Carruth; reviews+D1110+public+872068f434b7231c at llvm-reviews.chandlerc.com; llvm-commits
> > > Subject: Re: [PATCH] Change representation of dllexport/dllimport
> > >
> > > > On 11 September 2013 15:26, Robinson, Paul <Paul_Robinson at playstation.sony.com> wrote:
> > > > With the caveat that I didn't actually do the work, it looks like:
> > > > - We emit lists of dllimport/dllexport symbols to a linker-command
> > > >  section, conceptually similar to what Windows does.
> > > > - We invented a DLLExportLinkOnceLinkage for the special case of inline
> > > >  functions marked dllexport.  For this we added a new IR attribute.
> > > > - We fiddled a few places in lib/Target to get the right kinds of references
> > > >  to happen (relocs & which instruction is used). These look like some
> > > >  combination of ABI stuff and "do what the linker wants to see" stuff.
> > > >
> > > > Is that the kind of info you were looking for?
> > > >
> > > Yes, thanks. This is roughly what I was guessing -- a way of encoding the
> > > exposed symbols list (see tools/gold/gold.exports for an llvm example) in
> > > the source code.
> > >
> > > I do not understand how visibility is orthogonal to dllimport/dllexport.
> > > I can't think of any way that hidden + dllimport/dllexport makes sense.
> > > Neither does protected + dllimport, since there is no local copy of the
> > > symbol to bind to (we're importing it). Protected + dllexport could make
> > > sense, except that I don't see how it differs from protected (or on a
> > > regular windows system, dllexport). The only combination that makes sense
> > > is default + dllimport/dllexport.
> > >
> > > Could you give me an example where you'd actually want to use these
> > > visibilities in combination?
> > > Nick
> > I understand where you're coming from, the first time somebody proposed
> > to dllexport a hidden symbol was a big double-take for me too.
> >
> > Okay, for dllimport the visibility options are probably the same as any
> > other kind of external reference.
> >
> > For dllexport, if you don't have an explicit export list, your linker/loader
> > is trying to infer exports based on visibility, and exporting a hidden
> > symbol is absolutely the wrong thing to do.  I get that.
> >
> > For dllexport when you DO have an explicit export list, your linker/loader
> > isn't making any inferences; it knows a-priori what's visible outside the
> > boundaries of the DSO.  That means the "visibility" applies only to uses
> > _within_ the DSO.  Now let's postulate one TU that contributes to your DSO,
> > which defines utility functions for your exported API but aren't used
> > elsewhere within your DSO.  If those are visibility=default then they will
> > require GOT references (because you're compiling -fPIC, or at least I am)
> > even though you know there's no possibility that's actually needed within
> > the DSO.  But, if I make them visibility=hidden, the compiler can generate
> > cheaper shorter references without compromising the availability of the
> > symbol outside the DSO.
>
> I think I get it. You know the symbol is in the same DSO/DLL but not in the 
> same TU. Then you can avoid having to reload the GOT. We don't have any way 
> to express this.
>
> > And the ELF visibility makes sense within the
> > boundaries of the DSO.  Everybody's happy.
> >
> > So: because my target has an explicit export list, dllexport doesn't have
> > to imply anything special about visibility.
>
> I think visibility was added as an easy way to get data from the frontend 
> (the visibility attribute) down to the ELF .o files, and then we added some
> relevant optimizations in the code generator level. What you're suggesting 
> is that visibility really means something more like:
>
> * callers/accesses within our TU only
> * callers/accesses within our DSO/DLL
> * callers/accesses may be external
> * (callers/accesses may be done by a users of dlopen on this binary even 
> though it defines a main?)
>
> There's a couple other semantics attached to the existing visibilities. 
> Protected implies that symbol interposition can't happen, but that's really 
> already controlled by llvm linkage. We already know whether you can inline, 
> for example. If it's strong, we'll inline, if it's weak we won't.
>
> Does this make sense? The part I'm weakest on is the meanings of 
> dllimport/dllexport, I know how to look up what ELF does.

Heh. I'm kind of the opposite: cut my teeth on a system that had 
import/export lists but not visibility. Or preemption. Import/export 
makes sense to me, and all this visibility stuff is a weird hack.
Which I am still playing catch-up on; I jumped into this conversation
perhaps a little sooner than I should have, but it was happening and
I didn't want to completely miss the party.

So, try-try-again... see if my restated understanding lines up with
yours, and your patience is very much appreciated.

> * callers/accesses within our TU only
Sounds like binding=local, not actually a visibility.  My mistake,
sorry for causing confusion.

> * callers/accesses within our DSO/DLL
This would be visibility=hidden.  No need to go through GOT, I think,
because a hidden symbol can't be preempted.  This is the performance
advantage I keep reading about on web pages about visibility.

> * callers/accesses may be external
Normally this means visibility=default.  I suppose the idea is
that -fvisibility=hidden + declspec(dllexport) automatically
promotes to visibility=default.  I'm in a little bit of a bind
because we haven't actually done the experiment yet, but I think
my target will want to promote to visibility=protected instead.
(Reid Kleckner reported that this is what ARM does, earlier in the
thread.) Making this target-specific behavior would be nice.

I will observe that my toolchain can already dllexport a hidden
symbol, by compiling -fvisibility=hidden and then using linker
commands instead of the dllexport attribute.  Because the net
effect is like exporting a protected symbol, my linker guys are
okay with it, even though semantically it's a little weird.

> * (callers/accesses may be done by a users of dlopen on this binary even 
> though it defines a main?)

Um, I think you need to explain that particular scenario a little more.
I'm guessing the existence of main means that the binary assumes
its symbols can't be preempted?  

>
> Nick 

--paulr