[PATCH] D31528: [ELF][MIPS] Multi-GOT implementation

Fri May 5 13:38:46 PDT 2017

Hi Rui,

> but the amount of engineering work to understand the MIPS ABI and modify existing code
Agreed, this is costly but is also one of the reasons why change is so rare. A reasonable number of the active MIPS developers in the community are relative newcomers to the architecture and are still learning some aspects.

> won't be responsible for MIPS if something breaks due to a difference between MIPS and non-MIPS
Obviously Simon, myself and other MIPS users will hope this is be relatively rare but some fallout from overall improvements to LLD seems reasonable; please try to support anyone who works on fixing the MIPS issues though so it doesn’t remain broken for long.

> And I'll keep mildly pushing MIPS ABIs to be more in line with others until it's
Personally, I would actually appreciate that as it helps to keep these things alive and increase motivation for change. Perhaps Simon could extract the various recommendations you have made and we can put them into the ABI section of our compiler group wiki at dmz-portal.mips.com.

Thanks,
Matthew
From: Rui Ueyama [mailto:ruiu at google.com]
Sent: 05 May 2017 02:10
To: Matthew Fortune
Cc: Sean Silva; llvm-commits (llvm-commits at lists.llvm.org)
Subject: Re: [PATCH] D31528: [ELF][MIPS] Multi-GOT implementation

Matthew,

Thank you for writing it up! I can imagine that being asked a lot of questions that are not directly related to this patch and getting complaints about an ABI that you guys didn't invent are annoying, so I really appreciate your patience. I think I'm obliged as a maintainer to ask code/design that I don't understand and express concerns if something seems more complicated than it should be, and I appreciate you noted my concerns.

It seems to me that there's no strong desire in the MIPS community to address my concerns though. Looks like you guys are OK with the existing divergences and just want to make improvements as-needed basis. That is understandable given that you guys have a large number of running systems with the current ABI, and toolchains already support it, although it is not desirable for those who wants to implement it from scratch like LLD.

In reality, MIPS will be different from other ELF ABIs in various places in the foreseeable future.

I'm not a person who can change the MIPS ABI. So the only choices we have in the end are to support it or not support it. Do we want to support it? I think the answer is yes, as other LLVM tools support it.

My main concern is not the sheer amount of code to support MIPS but the amount of engineering work to understand the MIPS ABI and modify existing code. I'm now familiar with the MIPS ABI even though I didn't work for MIPS because I had to understand it to write code that works for all architectures. Reading documentations about MIPS, investigating undocumented part of the ABI, and understanding it actually took a long time. Not many people have that much time.

Here is I'd like to propose. I'm currently very careful not to break the MIPS support, and that's one reason why I don't welcome changes that I don't understand easily. I'll continue making best effort to keep LLD/MIPS working, but ultimately I (and other contributors who are not working on MIPS) won't be responsible for MIPS if something breaks due to a difference between MIPS and non-MIPS. And I'll keep mildly pushing MIPS ABIs to be more in line with others until it's done (I'm sorry but I cannot give it up for those who would work on LLD in 2030!). Does this sound good?

On Thu, May 4, 2017 at 1:33 AM, Matthew Fortune <Matthew.Fortune at imgtec.com<mailto:Matthew.Fortune at imgtec.com>> wrote:
Hi Rui,

I’ll try to answer your specific question on ownership of the MIPS ABIs and also recap some of the important ABI changes or proposals that have happened, some of which others have already pointed at.

If we go with the last person who touched it owns it stance then I think I would own the MIPS ABIs under that policy! The real position is that nobody to my knowledge claims to own the MIPS ABI and I believe that applies to most architectures as well. As architecture owners, Imagination’s engineers are still only one of the many stakeholders in the ABI and generally we look to the largest user base to hold the golden reference for the ABI. For MIPS this is upstream binutils BFD linker and glibc and I would generally expect these to be the places to debate and discuss ABI changes as the level of compatibility and correctness these projects aim for is extremely high.

A quick summary of recent and proposed ABI changes for MIPS (I can get links to mail archives to all of these if necessary but have not included them right now):

•         Introduction of 64-bit FPRs in o32. This took over 12 months to design, agree and commit to the initial repos of binutils and glibc with many more months before seeding the remaining c libraries, linkers and other affected tools. It is pretty much true that this was the first major change to the MIPS ABIs in several years but having done the work it is easy to see why! I knew an ABI change was hard but this was more work than I could have ever expected. The effort was invested as it was an essential ABI change to support new architectural features, if it had been a nice-to-have it would not have been worth the effort. Incidentally this feature had been a nice-to-have, but not essential, for at about 10 years prior to me implementing it.

•         PIE shared library debug – DT_MIPS_RLD_MAP_REL. This is an extension required to support finding the shared libraries used by a PIE because MIPS unique DT_MIPS_RLD_MAP would not work with PIE. During this design the community concluded that while the original solution obviously needed fixing the underlying achievement of keeping the dynamic section read-only at runtime was a worthy goal and the solution retained that (potentially unique) MIPS benefit.

•         ifunc – This is still in review and has slowed down a bit in pace. The general issues of how it will work with the GOT have been resolved and it should progress.

•         PT_GNU_STACK – This is also still in development and is complex for MIPS because of the Linux kernel using the user-stack for execution of code during some exception handling. This is a multi-component problem.

•         .gnu.hash – a solution to this has been proposed by an engineer from Cisco and is designed to work with the MIPS ABI nuances in a compatible way. This is held up by copyright assignment issues as the code is implemented for binutils which requires FSF copyright assignment for inclusion.

•         Recent discussions on the reducing the size of dynamic relocations for PIE are interesting because MIPS has a partial solution to this already due to the implicitly relocated local section of the GOT which costs no executable space.

•         Older issues that have been addressed include the addition of a static object model for n64 which was not originally designed to be non-PIC, addition of PLTs for o32 and n32 and likely more that I can’t remember.

Also on a positive note, I‘m pleased to say that we have active maintenance on all of binutils LD, GOLD, and LLD which covers the major static linkers currently so the potential for change is good. Your input here is really valuable and is a fantastic source of information to guide potential improvements; it is appreciated. The only challenge is finding either community contributors for the changes or business reasons to invest in the change. We do a bit of both at Imagination but obviously on a case by case basis.

I hope that is enough reassurance that MIPS ABIs are not forgotten, they are just incredibly complex and expensive to maintain and change.

Thanks,
Matthew

From: llvm-commits [mailto:llvm-commits-bounces at lists.llvm.org<mailto:llvm-commits-bounces at lists.llvm.org>] On Behalf Of Sean Silva via llvm-commits
Sent: 03 May 2017 18:48
To: Rui Ueyama
Cc: llvm-commits
Subject: Re: [PATCH] D31528: [ELF][MIPS] Multi-GOT implementation

On May 2, 2017 4:26 PM, "Rui Ueyama" <ruiu at google.com<mailto:ruiu at google.com>> wrote:
On Sun, Apr 30, 2017 at 10:02 PM, Sean Silva <chisophugis at gmail.com<mailto:chisophugis at gmail.com>> wrote:

On Wed, Apr 26, 2017 at 6:57 AM, Rui Ueyama via llvm-commits <llvm-commits at lists.llvm.org<mailto:llvm-commits at lists.llvm.org>> wrote:
I think I disagree. As I wrote there are still a lot of unnecessary divergences from the other ELF standards in the MIPS ABI, and in a long run, I believe these issues should be addressed, even if it means we need more code in a (possibly long) transition period.

I think we should first and foremost focus on making LLD a production MIPS linker for today's environment. For example, making LLD be /usr/bin/ld for FreeBSD on MIPS.

The ABI is a very cross-functional thing, involving kernel (x N OS's), dynamic linker/loader (x N libc's), static linker (x N static linkers), compiler (x N compilers), MC/assembler (x N assemblers), package management (x N distros), and other components (and there is likely not a perfect mailing list where active maintainers of all affected components are reading, making the organizational complexity extremely large). The costs of such a transition could easily enter into the millions of dollars of engineer time (including user time, as some things are likely to be incompatible). Speaking for myself as an LLD developer, I don't think it's appropriate for LLD developers to be making requests to change an ABI that we don't even implement at production quality.

It sounds like we're pretty close to supporting MIPS at production quality, so I think we should move forward with that. Once that is done, we can concretely evaluate what things we want to be changed. I don't think it's worth even starting a discussion right now.

My 2 cents: I suspect that once we do support MIPS at production quality, we'll look at all the MIPS-specific code in LLD and say "this is annoying, but not bad enough to be worth churning the world with a new ABI". (i.e., it's a "frog in boiling water" situation, but the water is only at 35 degrees C and not rising at a significant rate)

I agree that we eventually have to check this in, but what I'm trying to do is to understand the situation and and ask why why MIPS ABI has not been updated so long. It seems MIPS ABI is very conservative while ABIs for other architectures are not that much.

What is an ABI that you would consider not as conservative? You make it sound like most ABI's change in backward-incompatible ways "frequently".

-- Sean Silva

I could imagine a few examples (lack of clear ownership of the ABI?) but no one really seem to answer my question.

I believe, for example, we can agree that the very odd middle-endian header should have been fixed like 10 years ago.

I think that 10 years ago it was probably clear that the middle-endian header was undesirable and annoying, but I don't think it's at all obvious that 10 years ago a transition should have been started in order to change it; the costs of doing that are enormous and probably not worth it. We would probably be stuck in a situation like .ctors/.dtors vs .init_array/.fini_array -- 10+ years later, we still have to support both, with no indication that we will ever be able to delete support for one of them (like the xkcd comic that Joerg posted).

I think you would agree that making a new MIPS ABI just to end up in a situation like .{c,d}tors/.{init,fini}_array would just make things worse for us. What makes you so sure that that isn't going to happen? Is it worth the risk?

That is a valid concern and should be answered by people from the MIPS community. To me, it seems they should make a change to their ABI at least for the .gnu.hash section.

-- Sean Silva

I understand that it is of course not your fault. Who is maintaining the MIPS ABI?

On Wed, Apr 26, 2017 at 3:17 AM, Simon Atanasyan <simon at atanasyan.com<mailto:simon at atanasyan.com>> wrote:
I do not think that hypothetical MIPS ABI will be _much_ _more_
similar to other ABIs. For example, as to me I would keep MIPS GOT
almost the same because it's efficient and works well. Even if a
dynamic linker starts to recognize multi-GOT, that removes only a few
lines of code from LLD. And during a long transition period we will
have to support both variants. Special rules for .dynsym ordering and
unsupported --hash-style=gnu are tightly related to MIPS GOT. There is
an oddity in calculation addends for "paired" relocations (R_MIPS_HI16
/ R_MIPS_LO16) but we can consider this issue as fixed in N32 and N64
ABI. Special sections like .reginfo, .MIPS.options, and .MIPS.abiflags
are required to describe varieties of MIPS hardware. It's possible to
rename default entry symbol s/__start/_start/, but it will not make
LLD code much cleaner.

In general, requirement to remove incompatibilities from MIPS ABI
looks like a requirement to make MIPS ISA more similar to say X86 ISA
for simplification of LLVM MC library code base. MIPS requires new ABI
and finally I think it gets the new ABI, but I do not expect that this
makes LLD code base smaller and cleaner.

On Tue, Apr 25, 2017 at 7:37 PM, Rui Ueyama via llvm-commits
<llvm-commits at lists.llvm.org<mailto:llvm-commits at lists.llvm.org>> wrote:
> Simon, can you start a discussion in the MIPS ABI mailing list (if exists)
> that you are getting a push-back due to the incompatibilities from other ELF
> standards?
--
Simon Atanasyan

_______________________________________________
llvm-commits mailing list
llvm-commits at lists.llvm.org<mailto:llvm-commits at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170505/f9457b7f/attachment.html>