[lldb-dev] [PATCH] Host.cpp and RegisterContextLinux_x86_64.cpp fixes

[Greg Clayton]

> This will allows for architectures to be specified in an alternate byte order (big endian ARM) and also allow us to water down the current specific arch down to a target triple that has more generic enums for the arch, vendor and OS.
> 
> Many classes that have static FindPlugin() functions current take an ArchSpec which, in its current form, isn't enough to make an informed decision on which plug-in to chose. With the lldb::Triple added, it will be much easier and more correct. This patch touches a bunch of areas of the code, but I should commit it soon.
> 
>> 
>>> - Linux doesn't support RTLD_FIRST so apply that mode conditionally.
>> 
>> Why conditionally?  Perhaps we should just not pass this flag at all so
>> that we get similar behavior from this method on all posixy systems --
>> just a thought.
> 
> I will check to make sure RTLD_FIRST isn't something we need on MacOSX with our dynamic linker guru, and if not, then I will leave the code as is. Else, I will move a copy of Host::DynamicLibraryOpen() over into Host.mm (the apple specific Host functions) and conditionally compile it out in Host.cpp for non-apple variants.

So looking at our darwin man pages for dlopen, the RTLD_FIRST seems like a nice flag to specify:

RTLD_FIRST   The retuned handle is tagged so that any dlsym() calls on the handle will only search the image specified, and not subsequent images. If path is NULL and the option RTLD_FIRST is used, the handle returned will only search the main executable.

We don't use NULL for the file handle for calls to dlopen, though we could. Is there an equivalent option on dlsym() on linux? I don't want to do code like:

FileSpec plugin_spec ("/plugins/disassembler.dylib");
void *lib_handle = Host::DynamicLibraryOpen (plugin_spec, error);
if (lib_handle)
{
    void *init_callback = Host::DynamicLibraryGetSymbol (lib_handle, "InitPlugin", error);
    if (init_callback)
    {
        // Now on linux we might have a function that comes from another shared library...
    }
}

Due to the global symbol namespace we will run into issue unless we open things locally (RTLD_LOCAL), this is how unix dl calls only looks into a single shared library. The main problem is the RTLD_GLOBAL is sticky, so if anyone has opened a dynamic library already it will already be global and it will be subject to dlsym() calls that kick up any symbol in any shared library...

A few solutions to this is to define our own DynamicLibrary struct that contains both a "void *" for the handle, and a FileSpec for the path to the shared library. Then any calls to Host::DynamicLibraryGetSymbol will need to verify their symbols came from the shared library mentioned in the "void *".

I will submit a patch that should fix these issues as well as add some extra args to the Host::DynamicLibraryOpen and Host::DynamicLibraryGetSymbol calls that will allow for more control over this process.

Greg Clayton