[llvm-dev] [RFC] Lanai backend

Wed Feb 10 00:24:33 PST 2016

Don't forget the cost in time and effort. In some cases, those may outstrip
the physical hardware costs.

On Tue, Feb 9, 2016 at 11:50 PM Sean Silva <chisophugis at gmail.com> wrote:

> On Tue, Feb 9, 2016 at 11:01 PM, Pete Cooper via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>>
>> On Feb 9, 2016, at 10:24 PM, Chandler Carruth <chandlerc at google.com>
>> wrote:
>>
>> You've raised an important point here Pete, and while I disagree pretty
>> strongly with it (regardless of whether Lanai makes sense or not), I'm glad
>> that you've surfaced it where we can clearly look at the issue.
>>
>> The idea of "it really should have users outside of just the people who
>> have access to the HW" I think is deeply problematic for the project as a
>> whole. Where does it stop?
>>
>> While I may have the theoretical ability to get access to an AVR,
>> Hexagon, MSP430, SystemZ, or XCore processor... It is a practical
>> impossibility. There is no way that I, or I suspect 95% of LLVM
>> contributors, will be able to run code for all these platforms. And for
>> some of them, I suspect it is already the case that their only users have
>> access to specialized, quite hard to acquire hardware (both Hexagon[1] and
>> SystemZ come to mind).
>>
>> You’re right.  I don’t even have access to a 486 which we probably
>> support.
>>
>> My point was just that I thought this would be our first backend for
>> which it is impossible for anyone to use.  It may be hard for me to get an
>> MSP430, but I *can*.
>>
>
> This actually got me curious really how difficult it is for each of LLVM's
> backends, so I did an `ls` in lib/Target and here is what I know of as far
> as getting a machine that you can run your own code on:
>
> AArch64/ - obvious
> AMDGPU/ - obvious
> ARM/ - obvious
> AVR/ - just get an arduino (e.g.
> http://store-usa.arduino.cc/products/a000066)
> BPF/ - run linux on most any hardware I think?
> CppBackend/ - can we delete this :)
> Hexagon/ - e.g. https://developer.qualcomm.com/hardware/dragonboard-810
> ($500)
> MSP430/ - e.g. https://store.ti.com/msp-exp430fr5969.aspx (< $20)
> Mips/ - e.g. https://www.arduino.cc/en/Main/ArduinoBoardYun or even your
> friendly WRT54GL router.
> NVPTX/ - obvious
> PowerPC/ - lots of stuff in the embedded space for relatively cheap (e.g.
> 100's of $). If you have deep enough pockets you can buy servers.
> Sparc/ - If you have deep enough pockets Oracle will sell you servers. You
> can probably find something workstation-y on ebay (talk to netbsd folks
> apparently http://wiki.netbsd.org/ports/sparc/
> http://wiki.netbsd.org/ports/sparc64/). You can also probably synthesize
> OpenSPARC for an FPGA setup in the <$10,000 range.
> SystemZ/ - If you have deep enough pockets IBM will sell you one.
> WebAssembly/ - You can probably pull some sort of bleeding edge V8 or
> Mozilla repo. (being developed in the open, so "obvious" I guess)
> X86/ - obvious
> XCore/ - There are dev boards for <$200
> http://www.xmos.com/support/boards?product=18230
>
> This ignores all the issues of ISA subsets that we support. E.g. right now
> I don't think it's possible to buy an x86 with AVX512 (but Skylake server
> is coming soon), and some of the more esoteric ISA subsets for e.g. the
> supercomputer PPC stuff or the high-end MIPS stuff (e.g. the chips used in
> cellular base stations) are probably in SystemZ territory to get ahold of.
>
> -- Sean Silva
>
>
>
>
>>  I can’t get a Lanai, possibly ever.
>>
>> Now maybe you’re right, and Hexagon, or one of the other backends you
>> mention, is unable to be useful in practice to the community.  If that is
>> the case then the precedent has already been set that we as a community are
>> happy to work with (great) contributors who also provide some code we may
>> not get a direct benefit from and occasionally have to help maintain with
>> changes to common code.
>>
>> I just didn’t want Lanai (or any other backend) to set that precedent
>> without a discussion.  And it sure looks like we have got that discussion.
>> I definitely never wanted to imply anything negative about Lanai or the
>> folks who work on it.
>>
>>
>> But I really don't think we want to kick any of these backends out of
>> LLVM, or that they were a mistake to include in the first place.
>>
>> Nor do I.
>>
>> So the question really is whether the lack of *theoretical* access
>> matters substantially more than lack of *practical* access.
>>
>> Again, depends if this point is what matters, but maybe its already been
>> answered by other backends.
>>
>>
>> This policy would also set an extremely alarming precedent to me at
>> least, maybe one that you're not even intending: what's the difference
>> between hardware and an operating system? I don't think we want to limit
>> the operating system support of Clang to those which are readily
>> accessible. Lots of folks don't have a practical ability to use Apple's
>> operating system because they don't own any of Apple's hardware. And the
>> reverse is of course true -- I think if you work at Apple and only own
>> Apple hardware, you probably shoudn't be expected to boot Windows or Linux
>> and debug something.
>>
>> I’m not really sure how this applies to the OS.  Do you mean not
>> accepting patches which allow clang to run on an OS outside of the common
>> ones?  If so, yeah, that would be a problem, and not something I want to
>> see.  But there’s also a difference.  Much of the OS code is #defined,
>> whereas any backend which isn’t experimental is something of a burden.  I’m
>> not saying the burden isn’t worth it, but it is one nonetheless as your DAG
>> combine example below is a great example of.
>>
>>
>> We have a growing history of support diverse platforms, both hardware and
>> software. And when we do this, we rely on the community contributors who
>> care deeply about those platforms to bridge the gap and support those
>> developers who don't have ready access to the platform. I don't want to see
>> this change, as I think it has been absolutely critical to the community
>> growing and thriving over the past few years. My best memory of this is
>> with AMDGPU (which is why I brought it up, I agree with others that MSP430
>> or XCore are perhaps better examples here). I was making a change to a DAG
>> combine of some form and it ended up regressing the AMDGPU (R600 at the
>> time) regression tests. I was completely lost. While in theory, I could go
>> pick up such a GPU and install it in a machine to test things out, in
>> practice I'm never going to do that. If that were the requirement, I would
>> just have to abandon such changes. But instead, Tom and Matt bent over
>> backwards to help me out, looked at all the test case updates I sent them
>> and helped verify that indeed, the changes were harmless or even helpful. I
>> didn't need hardware, an emulator, or even an ISA spec. And I think this is
>> *awesome*.
>>
>> I’ve had a similar experience just a couple of weeks ago with someone
>> (who i’d never had any interaction with before) who took my reverted
>> patches and debugged and fixed the reason they broke on Windows with MSVC.
>> Its awesome to be part of this.  I’m not expecting anyone to have access to
>> even a tiny fraction of the platforms we target, but personally I would
>> prefer that they can have access to those platforms.
>>
>>
>> So regardless of what we do with Lanai, I feel really strongly that we
>> don't want to use "have other users" or "can run the code" as the bar for
>> supporting platforms.
>>
>> Personally I do want to set the bar there, but it seems I might be the
>> only one, so i’m not going to keep forcing people to follow along with an
>> ever increasing email chain.
>>
>> Cheers,
>> Pete
>>
>>
>>
>> But all that really doesn't mean we should necessarily adopt Lanai. I
>> think this is probably the nearest tradeoff the community has looked at
>> (that's actually one reason I worked internally to see if we could
>> contribute the backend). And it is definitely is a tradeoff.
>>
>> IMO, it is worth the tradeoff, just like it was for BPF, AMDGPU, and
>> others. Our goal should be to get more upstream contributors and LLVM
>> developers, and to the extent that we can do that by encouraging people
>> working on new (to the community) platforms, whether software or hardware,
>> to work on them upstream and in-tree, I think we should.
>>
>> Personally, I'm not concerned about how encumbered in any sense SystemZ
>> is or isn't (I have no idea about such things). I don't care that I'm
>> completely unlikely to be able to get a SystemZ system, or even an
>> emulator. I'm probably not going to spend a whole lot of time trying to
>> read the entire ISA spec. To me, while those things would *help* motivate
>> the issue, they're not the operative factor in my decision. Instead, the
>> fact that IBM contributed SystemZ, maintained it, and was able to do more
>> work in-tree was justification enough.
>>
>> My 2 cents, sorry I took so many words to express them. =]
>> -Chandler
>>
>>
>> [1] Yes, I know Hexagon chips are present all over Android devices, but I
>> have no idea how one would go about actually programming most of them.
>> Maybe it's actually much easier than I imagine, if so, apologize. My
>> impression has just been that Qualcomm (the producer of the Hexagon chips)
>> is by far the largest user of the backend if not the exclusive user. None
>> of this is a criticism of course! =]
>>
>> On Tue, Feb 9, 2016 at 9:35 PM Pete Cooper via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> On Feb 9, 2016, at 9:15 PM, Hal Finkel <hfinkel at anl.gov> wrote:
>>>
>>> ----- Original Message -----
>>>
>>> From: "Pete Cooper via llvm-dev" <llvm-dev at lists.llvm.org>
>>> To: "Sean Silva" <chisophugis at gmail.com>
>>> Cc: "llvm-dev" <llvm-dev at lists.llvm.org>
>>> Sent: Tuesday, February 9, 2016 10:59:58 PM
>>> Subject: Re: [llvm-dev] [RFC] Lanai backend
>>>
>>>
>>> Hi Sean
>>>
>>>
>>> I think you’ve summed it up really well here.
>>>
>>>
>>> Personally I don’t think we should accept backends for which there is
>>> no way to run the code. The burden (however small) on the community
>>> to having an in-tree backend they can’t use is too high IMO.
>>>
>>>
>>> As you point out ‘no way to run the code’ may mean not having access
>>> to HW, or having HW but no API.
>>>
>>>
>>> Out of curiosity, would the existence of some kind of open-source
>>> emulator affect your opinion on this? Or does it need to be actual hardware?
>>>
>>> I guess hardware isn’t even the correct term given we have BPF which I
>>> think is a virtual ISA of some kind.
>>>
>>> I’d be ok with targeting an emulator, but with the caveat that people
>>> are actually using the emulator.  An emulator which exists purely to get
>>> this backend in to LLVM isn’t useful, but one which people do use for
>>> education or otherwise is useful.  But it really should have users outside
>>> of just the people who have access to the real HW.
>>>
>>> Perhaps this can be an experimental backend to give time for an emulator
>>> and community to grow around it, and then we can decide later whether to
>>> promote to a first class backend?  Then we still have options, including
>>> just continuing as an experimental backend for longer if thats best.
>>>
>>> As a small digression, I would say that this has raised a very valid
>>> discussion about the code that ends up in LLVM and is built as standard.  I
>>> don’t know, nor am I going to look, but I think the same could (should?) be
>>> applied to other code like passes.  If a pass isn’t used by anyone in tree
>>> then should it be in tree?
>>>
>>> Pete
>>>
>>>
>>> -Hal
>>>
>>>
>>> NVPTX is a good example. Now you can take the output from LLVM and
>>> run it on HW. It may or may not be how Nvidia do it in their code,
>>> but that doesn’t matter, you can do it. Same for AMDGPU.
>>>
>>>
>>> So -1 from me to having backends we can’t make use of.
>>>
>>>
>>> Finally, one option is to have perpetually experimental backends.
>>> Then all the code is in tree but no-one in tree should ever be
>>> expected to update it. That does have the big advantage that all of
>>> the code is there to discuss and the maintainers can make
>>> contributions to common code and gain/provide help in the community.
>>> They can also be involved in discussions which impact them such as
>>> changes to common code.
>>>
>>>
>>> Cheers,
>>> Pete
>>>
>>>
>>>
>>>
>>> On Feb 9, 2016, at 4:18 PM, Sean Silva via llvm-dev <
>>> llvm-dev at lists.llvm.org > wrote:
>>>
>>>
>>> One data point (IIRC) is that the NVPTX backend sat in tree for a
>>> long time without a way to actually use them. But lately this has
>>> been opening up (e.g. http://llvm.org/docs/CompileCudaWithLLVM.html
>>> ). However, the obstacle for NVPTX was mostly a software
>>> proprietary-ness (no way to plug it into the driver stack really,
>>> except via nvidia's own proprietary software), whereas the actual
>>> hardware was available. For the Lanai stuff, it seems like the
>>> hardware is fundamentally not available for purchase.
>>>
>>>
>>> The reverse situation is with e.g. Apple's GPU backends, where the
>>> devices are readily available, but (AFAIK) even if the backend were
>>> open-source you couldn't run the code produced by the open-source
>>> compiler.
>>>
>>>
>>> Or to put it in matrix form (this is all heavily prefixed by "AFAIK";
>>> corrections welcome):
>>>
>>>
>>> AMDGPU: InTree:Yes DevicesAvailable:Yes CanIRunTheCode:Yes
>>> NVPTX: InTree:Yes DevicesAvailable:Yes CanIRunTheCode :Yes
>>> Lanai: InTree:? DevicesAvailable:No CanIRunTheCode :No
>>> Apple GPU's: InTree:No DevicesAvailable:Yes CanIRunTheCode :No
>>>
>>> I couldn't come up with a good name for "Can I Run The Code" column.
>>> Basically it means: "assuming the backend were in open source, could
>>> I actually run the code produced by the open source backend
>>> somehow?".
>>>
>>>
>>> I had a quick look at lib/Target and it seems like every backend we
>>> have has "CanIRunTheCode:Yes" in theory.
>>> IIRC, the NVPTX stuff used to actually be "No" though?
>>>
>>>
>>> Anyway, just a random thought. Not sure what the conclusion is.
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>
>>> --
>>> Hal Finkel
>>> Assistant Computational Scientist
>>> Leadership Computing Facility
>>> Argonne National Laboratory
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160210/5aa02288/attachment-0001.html>