[llvm-dev] New x86-64 micro-architecture levels

[Carlos O'Donell via llvm-dev]

On 7/22/20 6:34 AM, Florian Weimer wrote:
> * Jan Beulich:
> 
>> On 21.07.2020 20:04, Florian Weimer wrote:
>>> * Premachandra Mallappa:
>>> 
>>>> [AMD Public Use]
>>>> 
>>>> Hi Floarian,
>>>> 
>>>>> I'm including a proposal for the levels below.  I use single
>>>>> letters for them, but I expect that the concrete
>>>>> implementation of this proposal will use names like
>>>>> “x86-100”, “x86-101”, like in the glibc patch referenced
>>>>> above.  (But we can discuss other approaches.)
>>>> 
>>>> Personally I am not a big fan of this, for 2 reasons 1. uses
>>>> just x86 in name on x86_64 as well
>>> 
>>> That's deliberate, so that we can use the same x86-* names for
>>> 32-bit library selection (once we define matching
>>> micro-architecture levels there).
>> 
>> While indeed I did understand it to be deliberate, in the light of 
>> 64-bit only ISA extensions (like AMX, and I suspect we're going to 
>> see more) I nevertheless think Premachandra has a point here.
> 
> Let me explain how I ended up there.  Maybe I'm wrong.

I did a review of your analysis, and it is my opinion that your
conclusion is correct.

> Previously, I observed that it is difficult to set LD_PRELOAD and 
> LD_LIBRARY_PATH on combined x86-64/i386 systems, so that the right 
> libraries are loaded for both variants, and users aren't confused by 
> dynamic linker warning messages.  On some systems, it is possible to
> use dynamic string tokens ($LIB), but not all.

The case of LD_PRELOAD is the most difficult because it is a direct request
to the dynamic loader to load a particular library. If the library to be
loaded is an absolute path then you'll always get warning messages if you
need to execute child processes that inherited LD_PRELOAD for an architecture
that doesn't match the architecture to be executed.

The case of LD_LIBRARY_PATH is generally less troublesome because you are
adding search paths, and the library loading can be suppressed by other
mechanisms that include search path pruning.

It is also possible that $LIB does not match what is actually required
for the system to operate correctly and it depends on /etc/ld.so.conf
(and included files) for correctness (despite it being a cache, see
glibc bug 22359). This is an ISV problem that the ISV can solve.

> Eventually, it will be possible to add and restrict glibc-hwcaps 
> subdirectories by setting an environment variable.  The original
> patch series only contains ld.so command line options because I
> wanted to avoid a discussion about the precise mechanism for setting
> the environment variable (current glibc has two approaches).  But the
> desire to provide this functionality is there: for adding additional 
> glibc-hwcaps subdirectories to be searched first, and for
> restricting selection to a subset of the built-in
> (automatically-selected) subdirectories.

If you allow the addition of subdirectories, those subdirectories
can then be processed as directories are normally processed and we
can indeed avoid emitting an error message. The addition of directories
is not a direct request to the loader to load a specific shared object.

> I was worried that we would run into the same problem as with 
> LD_PRELOAD, where x86-64 and i386 binaries may have different 
> requirements.  I wanted to minimize the conflict by sharing the
> names (eventually, once we have 32-bit variants).

Right, this would make it easier to deploy from the ISV side.

> But thinking about this again, I'm not sure if my worry is
> warranted. The main selection criteria is still the library load
> path, and that is already provided by some different means (e.g.
> $LIB).  Within the library path, there is the glibc-hwcaps
> subdirectory, but since it is nested under a specific library path
> subdirectory (determined by the architecture), adding subdirectories
> to be searched which do not exist on the file system, or surpressing
> directories which would not be searched in the first place, is not a
> problem.  The situation is completely benign and would not warrant
> any error message from the dynamic loader.

I agree completely.
 
> If this analysis is correct, there is no reason to share the 
> subdirectory names between x86-64 and i386 binaries, and we can put
> “64” somewhere in the x86-64 strings.

We can choose not to share the paths. In fact it may make it easier to
explain to users that they are distinct.

In summary:

The conclusion is that x86-64 and i386 shared objects can use different
directories because they are just search paths, and such search paths
have different semantics from explicit load requests like LD_PRELOAD,
therefore they can be suppressed at runtime without the need to issue
an error or warning diagnostic.

Notes:
- We may wish to have an LD_DEBUG settings that helps catch issues
  with various paths, but that's a diagnostic settings whose semantics
  we can iron out as we discover developers making bad choices.

-- 
Cheers,
Carlos.