[llvm-dev] SPIRV-LLVM as an external tool

Wed Feb 21 15:07:02 PST 2018

On 02/21/2018 02:43 AM, Tomeu Vizoso via llvm-dev wrote:
> On 02/21/2018 10:58 AM, Nicholas Wilson wrote:
>> I agree that having the conversion live as part of LLVM is the best option going forward.
>>
>> libclc is an library implementation of the OpenCL builtins and I think its fine for that aspect of the converter,
>> i.e. consumption of SPIR-V into LLVM IR in preparation for code generation by LLVM by its various backends as a library
>> to link the generated code to, to live in an external repo.
>>
>> This is fine because SPIR<->LLVM IR is trivial, SPIR-V<->LLVM is not.
>>
>> Having the SPIR-V <-> LLVM-IR conversion part of the tool as an external repo as opposed to the llvm would
>> make it very difficult to use as a fontend:
>>
>> First the changes to the build pipeline are a hinderance. In my fork of SPIRV-LLVM the backend is a target
>> because I couldn’t get CMake to link the libraries to LDC as a support library. This would be made even more
>> complicated with at external project. I would probably resort to invoking an external program despite integrated
>> conversion being the best solution.
>>
>> Secondly (unless you've changed things) all of the builtins are not intrinsics, but C++ Itanium with extensions mangled.
>> This makes it so that anyone not implementing a C++ frontend has to deal with it, presumably by hardcoding the
>> names since C++ is very non-trivial. Even for projects like LDC (the LLVM based D compiler) which have a C++ mangler,
>> we still have to hardcode the names because Windows does not use Itanium and the compiler is not set up to do two
>> different forms of C++ mangling, and even if that were possible the non-standard extensions make it infeasible.
>> Whereas any LLVM frontend worth is salt has a way of exposing intrinsics.
>>
>> The backend as it currently stands is not based on tablegen and that is required AFAIU to support intrinsics.
>> I have tablegen descriptions of the SPIR-V format mostly complete (all core instruction relevant to OpenCL +
>> most of the ones for Vulkan, + all the OpenCL extension instructions) though not the intrinsics.
>> However I didn’t get around to using them because I have more important things to work on in the front end.
>>
>> Then there is the question of what to do about Vulkan? I would love to have support for Vulkan in LLVM.
>> That would be relatively simple to set up for with SPIRV as a target, even more so with a tablegen based
>> infrastructure. I have no idea how that would work as an external project.
>>
>> I also think that having it in an external project will greatly reduce the visibility in comparison to a target.
>> How many of you have cloned libclc for example? Not many I suspect. Probably because you are not
>> working on implementing OpenCL for a backend. And while libclc is useful to those implementing OpenCL
>> they are vastly outnumbered by those using OpenCL i.e. producing SPIR/SPIR-V. My usage of SPIR-V is
>> I generate it from my source files though LDC/LLVM then I hand it off to the OpenCL implementation.
>> Nowhere in that pipeline do I need something like libclc. Now libclc may be used in the implementation
>> but that’s a different copy of LLVM to the one that produced it.
>>
>> TL;DR.
>>
>> This really needs to be a target so that we can use intrinsics, have it as part of regular LLVM releases
>> so that it can be used by many frontends. Having an external project for assisting backends consuming
>> SPIR-V (a la libclc) is fine but that shouldn’t mean that the processes used for generating SPIR-V need to go
>> into a separate project.
> 
> Yes, for compilers it would be much more convenient to have a backend, but what OpenCL implementations need (for example) is only a bidirectional converter.
> 
> I don't think anybody will oppose to the converter becoming a backend at some point (as long as we keep exposing the conversion passes as today), but at the current pace it will take years and I don't think it's fair for everybody that we keep this codebase in the limbo for much longer.
> 
> Currently there's people shipping this code in products, and have to carry heavily modified forks because there's no community upstream to contribute to.
> 
> So, I think we really need at this point to find some place where the community can gather and work together on stability, intrinsics, have a proper backend, etc.
> 
> This proposal is only about where the collaboration would happen in the near future. If at some point we have also a backend, and people want to have it along the other backends in the main LLVM repo, that will be great. But in the meantime we need a place to work together.
> 

I agree with this sentiment.  This code has been floating around for a
long time and an external tool is the path of least resistance to get
something working that we can all start collaborating on.  I think it's
more important to provide something working now than to wait until we solve
the backend vs external tool debate.  Just because we start with an external
tool doesn't mean that has to be the long-term solution.

-Tom

> Regards,
> 
> Tomeu
> 
> 
>> Nic
>>
>>> On 21 Feb 2018, at 4:15 pm, Tomeu Vizoso via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>
>>> Hi,
>>>
>>> for a few months already I have been asking around for opinions on how
>>> people could best work together on Khronos' SPIR-V <-> LLVM-IR converter
>>> and some consensus seems to have formed.
>>>
>>> Most of the people I talked to favored having the converter become an
>>> external tool within the LLVM project, similar to libclc. I think that
>>> the LLVM project's processes, infrastructure and community form the best
>>> place for this collaboration to happen.
>>>
>>> I hope that having the converter as part of LLVM can help expand LLVM's
>>> value proposition in heterogeneous computing, in part by complementing
>>> the OpenCL C backend in clang.
>>>
>>> Thus I would like to ask what the LLVM community needs to see from us
>>> before SPIRV-LLVM can be accepted as an external tool, if at all.
>>>
>>> Thanks,
>>>
>>> Tomeu
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev