[cfe-dev] add new option -fnovisibility for clang

[Fāng-ruì Sòng via cfe-dev]

On 2020-09-15, Jason Liu wrote:
>> What is ultimately the behavior for "unspecified" symbols, when no
>export-list is provided by the user?
>
>If symbols have `unspecified` visibility, and no export list is provided,
>symbols will not get exported.
>
>> Are you saying that on the existing AIX compilers/linkers, the
>export-list cannot be used to make a symbol "hidden" (non-exported), if it
>had been explicitly annotated visibility("default") in the source (assuming
>visibility declarations weren't being ignored)? That only "unspecified"
>symbols can be modified by the export-list?
>
>The export list is able to override the visibility settings in the object
>file in some situations, but not in all situations. The scenario you
>described above could be overridden by export list.
>There is a comprehensive list for all scenarios here:
>https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility-part2/index.html
>in
>Table 1.
>
>Again, this is more for getting the default right on AIX. GCC and xlclang
>on AIX have the consistent default behavior as what we described for
>-mignore-xcoff-visibility.

Hi Jason, am I right for the following points?

The AIX/XCOFF visibility properties are roughly:

   SYM_V_UNSPECIFIED has no corresponding visibility bit on ELF
   SYM_V_INTERNAL ~= STV_INTERNAL
   SYM_V_HIDDEN   ~= STV_INTERNAL
   SYM_V_PROTECTED ~= STV_PROTECTED
   SYM_V_EXPORTED  ~= STV_DEFAULT

If a symbol is marked __attribute__((visibility("default"))) in the
source code, it will get the SYM_V_EXPORTED bit and cannot be restricted
by an exportlist (like a GNU version script).

The main purpose of -fnovisibility is to ignore __attribute__((visibility("default"))),
so that such a symbol can still be restricted by an exportlist.

>On Tue, Sep 15, 2020 at 5:20 PM James Y Knight <jyknight at google.com> wrote:
>
>> On Tue, Sep 15, 2020 at 3:29 PM Jason Liu via cfe-dev <
>> cfe-dev at lists.llvm.org> wrote:
>>
>>> > I remain unclear as to how that is a useful or important feature. Are
>>> you worried about compatibility with code which incorrectly specified
>>> visibility("hidden"), when it actually did need to export the symbol?
>>>
>>> I guess I understand the point of confusion, so I will try to clarify it
>>> a little bit.
>>> It goes back to Clang does not have "unspecified" visibility mapping yet.
>>> On AIX, the baseline visibility is not "default" or export. The baseline
>>> visibility is "unspecified" when no attribute is marked on the symbol.
>>>
>>
>> What is ultimately the behavior for "unspecified" symbols, when no
>> export-list is provided by the user?
>>
>> And -mignore-xcoff-visibility means let all the symbols have "unspecified"
>>> visibility, whether or not any visibility attribute is specified on the
>>> symbol. So that user could use export list in the link step to control the
>>> visibility on AIX, which gives a more fine-grained control.
>>>
>>
>> On GNU/ELF, the export-list functionality (called version-script there)
>> allows you to remove symbols from the list to be exported, but not add
>> them. So, visibility("default") marks it as potentially-exported, but the
>> version-script can override and make the symbol hidden if it wants to.
>>
>> Are you saying that on the existing AIX compilers/linkers, the export-list
>> cannot be used to make a symbol "hidden" (non-exported), if it had been
>> explicitly annotated visibility("default") in the source (assuming
>> visibility declarations weren't being ignored)? That only "unspecified"
>> symbols can be modified by the export-list?
>>
>>
>>
>>> As it is right now, the llc on AIX actually treats
>>> `GlobalValue::DefaultVisibility` as "unspecified" as it is the baseline
>>> visibility mode on AIX. So it is a bit messy there, and we will need to
>>> clean that up when we add "unspecified" visibility attribute to clang for
>>> AIX. But I don't think it should prevent us from making the default
>>> behavior(unspecified visibility for all symbols) right on AIX for now.
>>> Hope that would clean up the confusion.
>>>
>>>
>>> On Tue, Sep 15, 2020 at 2:35 PM Fāng-ruì Sòng via cfe-dev <
>>> cfe-dev at lists.llvm.org> wrote:
>>>
>>>> On Tue, Sep 15, 2020 at 11:21 AM James Y Knight via cfe-dev
>>>> <cfe-dev at lists.llvm.org> wrote:
>>>> >
>>>> > On Fri, Sep 11, 2020 at 3:55 PM Hubert Tong <
>>>> hubert.reinterpretcast at gmail.com> wrote:
>>>> >>
>>>> >> On Fri, Sep 11, 2020 at 2:58 PM James Y Knight <jyknight at google.com>
>>>> wrote:
>>>> >>>
>>>> >>> On Thu, Sep 10, 2020 at 10:30 PM Hubert Tong <
>>>> hubert.reinterpretcast at gmail.com> wrote:
>>>> >>>>
>>>> >>>> On Thu, Sep 10, 2020 at 9:07 PM James Y Knight <jyknight at google.com>
>>>> wrote:
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> On Thu, Sep 10, 2020, 6:57 PM Hubert Tong <
>>>> hubert.reinterpretcast at gmail.com> wrote:
>>>> >>>>>>
>>>> >>>>>> On Thu, Sep 10, 2020 at 6:30 PM James Y Knight via cfe-dev <
>>>> cfe-dev at lists.llvm.org> wrote:
>>>> >>>>>>>
>>>> >>>>>>> But why does AIX want to ignore visibility restrictions encoded
>>>> via attribute? Especially by default?
>>>> >>>>>>
>>>> >>>>>> One reason is that AIX performs more early/less lazy symbol
>>>> resolution (even with runtime linking) than, say, Linux. So, if a project
>>>> goes with an export-by-default model (via attribute in some scope), they
>>>> can cause link-time errors from symbol references to undefined symbols from
>>>> code that would otherwise have been discarded as unreferenced by the
>>>> linker. It seems such code is not uncommon (and has the questionable effect
>>>> of exporting template instantiations based on "client provided" types that
>>>> are not part of the "library"/"utility" code in question).
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> But, the default visibility is "default" -- which is the most
>>>> export-y visibility setting there is. So, without specifying visibility
>>>> options on the command line, the only thing you can do with visibility
>>>> attributes in code is to remove exports, not add additional ones.
>>>> >>>>>
>>>> >>>>> What am I missing?
>>>> >>>>
>>>> >>>> That AIX is not an ELF platform. AIX has a default visibility for
>>>> XCOFF called "unspecified". This is because the ELF-based visibility
>>>> options do not quite match the base case on AIX. That the most export-y
>>>> visibility is named "default" at the source level is an ELF-ism.
>>>> >>>
>>>> >>>
>>>> >>> But Clang doesn't have an "unspecified" visibility, only default,
>>>> protected, and hidden. And this proposal doesn't propose to add one,
>>>> either. So I don't see how this command-line option makes sense in Clang,
>>>> right now.
>>>> >>
>>>> >> It makes sense for Clang on AIX if it improves the consistency of
>>>> behaviour with other compilers on that platform. GCC on AIX does not
>>>> support encoding visibility into XCOFF and the default behaviour of the IBM
>>>> xlclang/xlclang++ compiler on AIX is to not encode visibility into XCOFF.
>>>> That there will be future complementary changes does not seem to be a
>>>> practical reason for leaving Clang gratuitously different on AIX from other
>>>> AIX compilers.
>>>> >
>>>> >
>>>> > If clang did not implement this feature, symbols marked
>>>> visibility("hidden") or visibility("protected") will actually be marked
>>>> hidden/protected in the output on AIX. If clang does implement this
>>>> feature, such symbols will instead be marked as exported
>>>> default-visibility. There is no change for symbols marked
>>>> visibility("default").
>>>> >
>>>> > So, the request here is that Clang on AIX should ignore the attempt to
>>>> hide symbols, resulting in symbols being exported, even though the code
>>>> thought it was hiding them? That's the desired behavior change?
>>>> >
>>>> > I remain unclear as to how that is a useful or important feature. Are
>>>> you worried about compatibility with code which incorrectly specified
>>>> visibility("hidden"), when it actually did need to export the symbol?
>>>>
>>>> The same feeling here... Aren't most
>>>> __attribute__((visibility("hidden"))) guarded by macros? AIX can turn
>>>> off the macros...
>>>> _______________________________________________
>>>> cfe-dev mailing list
>>>> cfe-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>
>>