[cfe-dev] add new option -fnovisibility for clang

[Jason Liu via cfe-dev]

>
>     SYM_V_UNSPECIFIED has no corresponding visibility bit on ELF
>     SYM_V_INTERNAL ~= STV_INTERNAL
>     SYM_V_HIDDEN   ~= STV_INTERNAL
>     SYM_V_PROTECTED ~= STV_PROTECTED
>     SYM_V_EXPORTED  ~= STV_DEFAULT


These mappings are mostly right.

If a symbol is marked __attribute__((visibility("default"))) in the
> source code, it will get the SYM_V_EXPORTED bit and cannot be restricted
> by an exportlist (like a GNU version script).

Exportlist could still override them in some scenarios. But in some cases,
it could result in errors reported by the linker.
The mixed use of export list and visibility attribute could bring confusion
in code development, and therefore is generally not recommended.

The main purpose of -fnovisibility is to ignore
> __attribute__((visibility("default"))),
> so that such a symbol can still be restricted by an exportlist.
>
 The main purpose of the option is to get the default behavior to be
consistent with other compilers on AIX (xlclang and gcc),
which is no visibility is set on the symbol even when attribute is
presented.
Users could use the export list to control the symbol visibility. If no
export list is present, no symbol is exported.

On Tue, Sep 15, 2020 at 6:42 PM Fāng-ruì Sòng <maskray at google.com> wrote:

> On 2020-09-15, Jason Liu wrote:
> >> What is ultimately the behavior for "unspecified" symbols, when no
> >export-list is provided by the user?
> >
> >If symbols have `unspecified` visibility, and no export list is provided,
> >symbols will not get exported.
> >
> >> Are you saying that on the existing AIX compilers/linkers, the
> >export-list cannot be used to make a symbol "hidden" (non-exported), if it
> >had been explicitly annotated visibility("default") in the source
> (assuming
> >visibility declarations weren't being ignored)? That only "unspecified"
> >symbols can be modified by the export-list?
> >
> >The export list is able to override the visibility settings in the object
> >file in some situations, but not in all situations. The scenario you
> >described above could be overridden by export list.
> >There is a comprehensive list for all scenarios here:
> >
> https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility-part2/index.html
> >in
> >Table 1.
> >
> >Again, this is more for getting the default right on AIX. GCC and xlclang
> >on AIX have the consistent default behavior as what we described for
> >-mignore-xcoff-visibility.
>
> Hi Jason, am I right for the following points?
>
> The AIX/XCOFF visibility properties are roughly:
>
>    SYM_V_UNSPECIFIED has no corresponding visibility bit on ELF
>    SYM_V_INTERNAL ~= STV_INTERNAL
>    SYM_V_HIDDEN   ~= STV_INTERNAL
>    SYM_V_PROTECTED ~= STV_PROTECTED
>    SYM_V_EXPORTED  ~= STV_DEFAULT
>
> If a symbol is marked __attribute__((visibility("default"))) in the
> source code, it will get the SYM_V_EXPORTED bit and cannot be restricted
> by an exportlist (like a GNU version script).
>
> The main purpose of -fnovisibility is to ignore
> __attribute__((visibility("default"))),
> so that such a symbol can still be restricted by an exportlist.
>
> >On Tue, Sep 15, 2020 at 5:20 PM James Y Knight <jyknight at google.com>
> wrote:
> >
> >> On Tue, Sep 15, 2020 at 3:29 PM Jason Liu via cfe-dev <
> >> cfe-dev at lists.llvm.org> wrote:
> >>
> >>> > I remain unclear as to how that is a useful or important feature. Are
> >>> you worried about compatibility with code which incorrectly specified
> >>> visibility("hidden"), when it actually did need to export the symbol?
> >>>
> >>> I guess I understand the point of confusion, so I will try to clarify
> it
> >>> a little bit.
> >>> It goes back to Clang does not have "unspecified" visibility mapping
> yet.
> >>> On AIX, the baseline visibility is not "default" or export. The
> baseline
> >>> visibility is "unspecified" when no attribute is marked on the symbol.
> >>>
> >>
> >> What is ultimately the behavior for "unspecified" symbols, when no
> >> export-list is provided by the user?
> >>
> >> And -mignore-xcoff-visibility means let all the symbols have
> "unspecified"
> >>> visibility, whether or not any visibility attribute is specified on the
> >>> symbol. So that user could use export list in the link step to control
> the
> >>> visibility on AIX, which gives a more fine-grained control.
> >>>
> >>
> >> On GNU/ELF, the export-list functionality (called version-script there)
> >> allows you to remove symbols from the list to be exported, but not add
> >> them. So, visibility("default") marks it as potentially-exported, but
> the
> >> version-script can override and make the symbol hidden if it wants to.
> >>
> >> Are you saying that on the existing AIX compilers/linkers, the
> export-list
> >> cannot be used to make a symbol "hidden" (non-exported), if it had been
> >> explicitly annotated visibility("default") in the source (assuming
> >> visibility declarations weren't being ignored)? That only "unspecified"
> >> symbols can be modified by the export-list?
> >>
> >>
> >>
> >>> As it is right now, the llc on AIX actually treats
> >>> `GlobalValue::DefaultVisibility` as "unspecified" as it is the baseline
> >>> visibility mode on AIX. So it is a bit messy there, and we will need to
> >>> clean that up when we add "unspecified" visibility attribute to clang
> for
> >>> AIX. But I don't think it should prevent us from making the default
> >>> behavior(unspecified visibility for all symbols) right on AIX for now.
> >>> Hope that would clean up the confusion.
> >>>
> >>>
> >>> On Tue, Sep 15, 2020 at 2:35 PM Fāng-ruì Sòng via cfe-dev <
> >>> cfe-dev at lists.llvm.org> wrote:
> >>>
> >>>> On Tue, Sep 15, 2020 at 11:21 AM James Y Knight via cfe-dev
> >>>> <cfe-dev at lists.llvm.org> wrote:
> >>>> >
> >>>> > On Fri, Sep 11, 2020 at 3:55 PM Hubert Tong <
> >>>> hubert.reinterpretcast at gmail.com> wrote:
> >>>> >>
> >>>> >> On Fri, Sep 11, 2020 at 2:58 PM James Y Knight <
> jyknight at google.com>
> >>>> wrote:
> >>>> >>>
> >>>> >>> On Thu, Sep 10, 2020 at 10:30 PM Hubert Tong <
> >>>> hubert.reinterpretcast at gmail.com> wrote:
> >>>> >>>>
> >>>> >>>> On Thu, Sep 10, 2020 at 9:07 PM James Y Knight <
> jyknight at google.com>
> >>>> wrote:
> >>>> >>>>>
> >>>> >>>>>
> >>>> >>>>>
> >>>> >>>>> On Thu, Sep 10, 2020, 6:57 PM Hubert Tong <
> >>>> hubert.reinterpretcast at gmail.com> wrote:
> >>>> >>>>>>
> >>>> >>>>>> On Thu, Sep 10, 2020 at 6:30 PM James Y Knight via cfe-dev <
> >>>> cfe-dev at lists.llvm.org> wrote:
> >>>> >>>>>>>
> >>>> >>>>>>> But why does AIX want to ignore visibility restrictions
> encoded
> >>>> via attribute? Especially by default?
> >>>> >>>>>>
> >>>> >>>>>> One reason is that AIX performs more early/less lazy symbol
> >>>> resolution (even with runtime linking) than, say, Linux. So, if a
> project
> >>>> goes with an export-by-default model (via attribute in some scope),
> they
> >>>> can cause link-time errors from symbol references to undefined
> symbols from
> >>>> code that would otherwise have been discarded as unreferenced by the
> >>>> linker. It seems such code is not uncommon (and has the questionable
> effect
> >>>> of exporting template instantiations based on "client provided" types
> that
> >>>> are not part of the "library"/"utility" code in question).
> >>>> >>>>>
> >>>> >>>>>
> >>>> >>>>> But, the default visibility is "default" -- which is the most
> >>>> export-y visibility setting there is. So, without specifying
> visibility
> >>>> options on the command line, the only thing you can do with visibility
> >>>> attributes in code is to remove exports, not add additional ones.
> >>>> >>>>>
> >>>> >>>>> What am I missing?
> >>>> >>>>
> >>>> >>>> That AIX is not an ELF platform. AIX has a default visibility for
> >>>> XCOFF called "unspecified". This is because the ELF-based visibility
> >>>> options do not quite match the base case on AIX. That the most
> export-y
> >>>> visibility is named "default" at the source level is an ELF-ism.
> >>>> >>>
> >>>> >>>
> >>>> >>> But Clang doesn't have an "unspecified" visibility, only default,
> >>>> protected, and hidden. And this proposal doesn't propose to add one,
> >>>> either. So I don't see how this command-line option makes sense in
> Clang,
> >>>> right now.
> >>>> >>
> >>>> >> It makes sense for Clang on AIX if it improves the consistency of
> >>>> behaviour with other compilers on that platform. GCC on AIX does not
> >>>> support encoding visibility into XCOFF and the default behaviour of
> the IBM
> >>>> xlclang/xlclang++ compiler on AIX is to not encode visibility into
> XCOFF.
> >>>> That there will be future complementary changes does not seem to be a
> >>>> practical reason for leaving Clang gratuitously different on AIX from
> other
> >>>> AIX compilers.
> >>>> >
> >>>> >
> >>>> > If clang did not implement this feature, symbols marked
> >>>> visibility("hidden") or visibility("protected") will actually be
> marked
> >>>> hidden/protected in the output on AIX. If clang does implement this
> >>>> feature, such symbols will instead be marked as exported
> >>>> default-visibility. There is no change for symbols marked
> >>>> visibility("default").
> >>>> >
> >>>> > So, the request here is that Clang on AIX should ignore the attempt
> to
> >>>> hide symbols, resulting in symbols being exported, even though the
> code
> >>>> thought it was hiding them? That's the desired behavior change?
> >>>> >
> >>>> > I remain unclear as to how that is a useful or important feature.
> Are
> >>>> you worried about compatibility with code which incorrectly specified
> >>>> visibility("hidden"), when it actually did need to export the symbol?
> >>>>
> >>>> The same feeling here... Aren't most
> >>>> __attribute__((visibility("hidden"))) guarded by macros? AIX can turn
> >>>> off the macros...
> >>>> _______________________________________________
> >>>> cfe-dev mailing list
> >>>> cfe-dev at lists.llvm.org
> >>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> >>>>
> >>> _______________________________________________
> >>> cfe-dev mailing list
> >>> cfe-dev at lists.llvm.org
> >>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> >>>
> >>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20200915/c017ee53/attachment-0001.html>