[cfe-dev] [RFC][OpenCL] Add builtin types and functions from the standard headers implicitly in the driver

[Reid Kleckner via cfe-dev]

My short, not carefully considered, input is that the suggested approach of
tablegen + a header seems reasonable.

The modules / PCH approach has been considered for the ever-growing
immintrin.h on x86, but I don't believe it got much traction:
https://lists.llvm.org/pipermail/cfe-dev/2016-September/050980.html

On Fri, Feb 5, 2021 at 2:13 PM James Y Knight via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> The whole tablegen thing seems like a sad path to have gone down, although
> I can certainly see the practical benefits. Substantially the same problem
> of compilation-speed exists for <x86intrin.h> (and friends), and I really
> don't think we want to start defining all of *those* with a tablegen rule.
>
> It would be really nice to instead somehow take advantage of the modules
> infrastructure to fix this problem -- I'd really love it if we could
> somehow start shipping a pre-built module artifact for our giant intrinsics
> headers, included with the compiler distribution. And then use that by
> default, regardless of whether users are otherwise enabling modules. If we
> got that to work, we could use the same solution for both X86 and opencl.
>
>
> However, independent of that discussion -- and more to the immediate issue
> you're trying to raise -- your proposal seems like it's resulting in a very
> complex set of options, and I'm not sure what the purpose is.
>
> IIUC, the overall desire is to have, by default, these tens-of-thousands
> of prototypes available to all OpenCL compilations. But, I don't see any
> reason why users should care exactly HOW these are provided. I'd expect
> that a given prototype should be provided either by the tablegen-based
> builtin code, OR an auto-included header file -- not both, and not neither.
>
> What difference does it make if the builtin-tablegen code doesn't provide
> 100% of the declarations, so long as the remainder are provided by an
> automatically-included header? Why do you want to make users choose between
> getting a half-baked set of function prototypes (tablegen version) and slow
> compilation (auto-including giant header), when you can have the fully
> correct set of functions AND nearly-as-fast compilation, by simply
> supplementing tablegen with an auto-included header providing the remainder?
>
> And then you need just a single user-visible option: to disable the
> automatic declarations (via both tablegen + autoinclude).
>
> On Wed, Feb 3, 2021 at 11:58 AM Anastasia Stulova via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
>> Hello,
>>
>> I would like to check if there is any feedback to the following proposal
>> for
>> improving the interface of standard type and function includes.
>>
>> Background
>>
>> Currently the default clang driver provides incomplete functionality for
>> OpenCL
>> because the headers with builtin function declarations are not included
>> by
>> default. The header can only be added using frontend flags requiring
>> ‘-cc1’ or
>> other frontend forwarding options
>> (https://clang.llvm.org/docs/UsersManual.html#opencl-header). This means
>> it
>> is inaccessible to users in a conventional way.
>>
>> We propose to add the implicit header by default when a source is
>> compiled in
>> OpenCL mode. A review for this has been uploaded by Matt a few months ago:
>> https://reviews.llvm.org/D78979. Note that the standard library
>> functionality is
>> added by default in OpenCL C without using include directives in the
>> compiling
>> sources.  This means all builtin function declarations (there are 17000
>> of
>> them) are to be parsed every time the source is compiled because which
>> builtins
>> are used by the kernels is not known beforehand. This impacts the
>> compilation
>> speed. For example, parsing a simple kernel with the builtin function
>> declarations is 138 times slower in a Debug build and 13 times slower in
>> a
>> Release build than parsing the same kernel without those declarations.
>>
>> To mitigate the overhead of parsing the full header, a fast Tablegen
>> based
>> solution has been developed
>> (https://llvm.org/devmtg/2019-10/talk-abstracts.html#lit5). The parsing
>> speed with
>> this mechanism for builtin function declarations is only 1.3 times slower
>> in a Debug
>> build and 1.05 times slower in a Release build compared to clang without
>> the
>> builtins. While this mechanism covers most of OpenCL standard
>> functions it lacks two main classes of builtins: builtins defined by
>> vendor extensions and builtins with enum arguments.
>>
>> Proposal
>>
>> We propose the following changes in the clang driver interface for
>> OpenCL:
>> - Enable the fast Tablegen based builtin function declaration mechanism
>> by
>> default in the clang driver. This makes the majority of OpenCL builtin
>> functions available.
>> - In addition, include the small header opencl-c-base.h by default in the
>> clang
>> driver. This header provides basic types and constants.
>> No frontend or driver flags will be needed to allow using the majority of
>> OpenCL
>> types and functions from the standard, at a very low parsing speed
>> increase.
>>
>> Since the Tablegen mechanism has some small overhead and it is not fully
>> complete, we propose to add the following additional clang driver flags:
>> 1. Add a new clang driver flag -cl-no-stdinc (*) that disables such extra
>> includes to
>> minimize further compilation speed or for the use cases that don’t
>> require
>> standard libraries. The majority of OpenCL clang tests will use this
>> option.
>> 2. [Optionally, if there is enough interest] Add a new clang driver flag
>> -cl-all-stdinc (*) that will include the full header instead of using the
>> Tablegen
>> mechanism, at the cost of a significant increase in parsing time.
>>
>> At present we propose no change to the ‘cc1’ interface, but in the future
>> it is
>> expected that the functionality will be aligned between driver and
>> frontend
>> command line interfaces for the OpenCL headers.
>>
>> (*) The exact spelling of command line options is to be discussed.
>>
>> Summary
>>
>> This proposal enhances the clang driver with full functionality of OpenCL
>> C by
>> adding builtin function declarations implicitly without the need for any
>> frontend flags to be given in the command line.
>>
>> The default clang behavior proposed is not expected to negatively impact
>> users
>> of clang as the parsing speed difference remains within the same order of
>> magnitude. While the fast header mechanism matures, a fallback mechanism
>> will
>> be provided if needed that would allow switching to the slow header with
>> the
>> full functionality using a new driver flag. For backward compatibility,
>> another
>> flag is provided to disable all OpenCL declarations that are not builtin
>> into
>> the compiler.
>>
>> The solution proposed improves the driver interface and reduces risks of
>> forcing the OpenCL community to update their use of clang due to
>> significant
>> regression of the compilation speed.
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20210208/dc0ed9ed/attachment.html>