[PATCH] D140088: Add LLVM type support for fp8

Pooya Jannaty via Phabricator via llvm-commits llvm-commits at lists.llvm.org
Fri Jan 6 11:26:11 PST 2023


pjannaty added a comment.

In D140088#4015926 <https://reviews.llvm.org/D140088#4015926>, @nikic wrote:

> Is there an RFC for this change? In https://discourse.llvm.org/t/rfc-add-apfloat-and-mlir-type-support-for-fp8-e5m2/65279 the addition of an LLVM IR type was explicitly out of scope. See also in particular the comment by @efriedma here: https://discourse.llvm.org/t/rfc-add-apfloat-and-mlir-type-support-for-fp8-e5m2/65279/3
>
> I'm personally rather skeptical about adding new ad-hoc floating-point types to IR without at least some restructuring of the existing system. At the IR level, my expectation would be that all the different float types get consolidated into a single type that is parameterized over the floating-point semantics.

Hi Nikita et al, Thanks for the feedback. The idea of parametrizing APFloat sounds like a neat one, though also complex given both the amount of overhauling that is needed to get it right, as well as the need for proper design and dealing with a myriad of exceptions (e.g. NaNs, Infs, etc), for which my team doesn't have LLVM competence (or bandwidth) yet to implement on our own. From a pragmatic standpoint, I am afraid this effort would require many months of concerted planning, deliberation, implementation and testing before it would be ready to serve as a basis for FP8 extensions.

As you may know, the introduction of FP8 is one of the main features of NVIDIA's new Hopper GPUs, so we are trying very hard to adopt it across all DL frameworks. We hope LLVM will be a benefit in this process, since its features are inherited by many different frameworks. This patch (which defines the types) and another currently in development (to introduce ptx codegen) is needed as part of supporting native Hopper instructions for FP8 casts. My team has already implemented FP8 support in XLA and we now have end-to-end support in TF (and JAX and others) through XLA. If the changes cannot be provided in LLVM in a reasonable timeframe, we will need to pursue framework specific hacks to enable efficient type conversions, which is something we'd very much like to avoid.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D140088/new/

https://reviews.llvm.org/D140088



More information about the llvm-commits mailing list