<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Yeah I agree Eli, there are two different things going on here:<div class=""><br class=""></div><div class="">1) Refactoring Clang ABI lowering code so other clients can use it.</div><div class="">2) The LLVM IR representation of calling conventions, which clang happens to use.</div><div class=""><br class=""></div><div class="">The historical mistake that was made was that I/we conflated having a “reasonable default lowering” with trying to make it align with the C calling conventions. There were several hopes behind this (all of which are failed experiments) including the idea of having LLVM IR be more or less portable across targets.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Doing it all over again, a better design would be:</div><div class=""><br class=""></div><div class="">1) LLVM IR functions with no attributes would be some target dependent ABI, something generally good for general targets (e.g. JIT compilers) that don’t care about ABIs, and good for writing .ll file testcases.</div><div class=""><br class=""></div><div class="">2) We introduce attributes to fully control the handling of parameters and results, things like inreg/inmem etc. We would introduce a gaggle of *target specific* attributes to say “pass this float in an integer reg; pass this according to this wierd target specific parameterization; etc”.</div><div class=""><br class=""></div><div class="">3) We would have clang (an everything else that cared) always generate the fully specified attributes when it has to match a specific fixed calling convention. This is (inherently) target specific. Generating target specific attributes is fine.</div><div class=""><br class=""></div><div class="">4) IPO optimizations could generally drop these attributes to remove constraints on code generation when the optimizer can tell the ABI doesn’t matter.</div><div class=""><br class=""></div><div class="">We can make incremental progress on this today, by introducing those attributes and changing clang.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">OTOH none of the above is directed at the original point upthread which was “how do we reuse this from other targets”. The problem is that you have several problems to solve:</div><div class=""><br class=""></div><div class="">1) You want to define the basic psABI logic once in a common place. You want extensions on top (e.g. FORTRAN ABI, C++ ABI, …) to be able to be layered on top.</div><div class=""><br class=""></div><div class="">2) The common psABI logic depends inherently on the C type system - a structure with two floats can be passed differently than a complex float, etc. However, it doesn’t make sense to embed the C type system into LLVM IR and definitely does not make sense to propagate this “high level” information through LLVM IR - lowering it out makes sense.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">This is why MLIR is important. It has an extensible type system that can encode (e.g.) C and do so by pointing to external existing type systems (so you don’t have to rebuild IR representations of this), and has a mature extension framework for callouts to extended APIs ("Operation interfaces”). MLIR can also obviously represent the LLVM IR type system and general attributes.</div><div class=""><br class=""></div><div class="">If Clang and other frontends were generating MLIR, this “ABI lowering” would be any other pass using the <a href="https://mlir.llvm.org/docs/DialectConversion/#type-conversion" class="">dialect conversion / type converter</a> infra. Lowering from that to LLVM is then an isomorphic transformation.</div><div class=""><br class=""></div><div class="">This change has lots of other advantages as well of course.</div><div class=""><br class=""></div><div class="">-Chris</div><div class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Jun 4, 2020, at 8:47 AM, Eli Friedman <<a href="mailto:efriedma@quicinc.com" class="">efriedma@quicinc.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">In LLVM, ABI information currently comes from three sources:<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><ol start="1" type="1" style="margin-bottom: 0in; margin-top: 0in;" class=""><li class="MsoListParagraph" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">The function type<o:p class=""></o:p></li><li class="MsoListParagraph" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">The calling convention<o:p class=""></o:p></li><li class="MsoListParagraph" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">Attributes<o:p class=""></o:p></li></ol><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">I’m getting the following from your description of what you think needs to change:<o:p class=""></o:p></div><ol start="1" type="1" style="margin-bottom: 0in; margin-top: 0in;" class=""><li class="MsoListParagraph" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">ABI attributes shouldn’t be mixed with other attributes; we should have some data structure dedicated to ABI information.<o:p class=""></o:p></li><li class="MsoListParagraph" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">ABI information should be explicitly target-specific: instead of using attributes like “inreg” that have target-specific meanings, each target-specific ABI attribute should have its own target-specific name.<o:p class=""></o:p></li><li class="MsoListParagraph" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">We should depend more on explicit ABI information, as opposed to depending on each target’s default rules.<o:p class=""></o:p></li><li class="MsoListParagraph" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">We should document each ABI supported by clang.<o:p class=""></o:p></li><li class="MsoListParagraph" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">LLVM function types should be fixed to correspond more closely to C function types.<o:p class=""></o:p></li></ol><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">I think messing with LLVM function types is a giant sinkhole that would destroy any comprehensive proposal, though. The LLVM type system is not the C type system; LLVM structs are not C structs, and LLVM functions are not C functions. And any changes are very high impact: messing with struct or function types would impact basically every file in LLVM.<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">From the standpoint of LLVM IR optimizations and lowering, the place we’re currently at with function types is actually pretty convenient, mostly, even if generating LLVM IR is inconvenient. Making the LLVM IR representation closer to the machine, as opposed to the frontend, is good for optimization: it’s hard to model the cost of code implicitly generated during isel. And first-class structs/arrays are pretty awful to work with in LLVM IR; optimizations strongly prefer working with simple values. Really, I think we want to break up arguments more, not less.<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">I agree the way ABI markings are represented in IR is lacking, though, and we need ABI-specific documentation for the way the lowering works. Wrapping up the current clang code in a friendlier interface only goes so far.<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">-Eli<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="border-style: none none none solid; border-left-width: 1.5pt; border-left-color: blue; padding: 0in 0in 0in 4pt;" class=""><div class=""><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0in 0in;" class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><b class="">From:</b><span class="Apple-converted-space"> </span>llvm-dev <<a href="mailto:llvm-dev-bounces@lists.llvm.org" style="color: blue; text-decoration: underline;" class="">llvm-dev-bounces@lists.llvm.org</a>><span class="Apple-converted-space"> </span><b class="">On Behalf Of<span class="Apple-converted-space"> </span></b>James Y Knight via llvm-dev<br class=""><b class="">Sent:</b><span class="Apple-converted-space"> </span>Wednesday, June 3, 2020 9:54 PM<br class=""><b class="">To:</b><span class="Apple-converted-space"> </span>Chris Lattner <<a href="mailto:clattner@nondot.org" style="color: blue; text-decoration: underline;" class="">clattner@nondot.org</a>><br class=""><b class="">Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:llvm-dev@lists.llvm.org" style="color: blue; text-decoration: underline;" class="">llvm-dev@lists.llvm.org</a>; cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" style="color: blue; text-decoration: underline;" class="">cfe-dev@lists.llvm.org</a>>;<span class="Apple-converted-space"> </span><a href="mailto:flang-dev@lists.llvm.org" style="color: blue; text-decoration: underline;" class="">flang-dev@lists.llvm.org</a><br class=""><b class="">Subject:</b><span class="Apple-converted-space"> </span>[EXT] Re: [llvm-dev] [cfe-dev] Clang/LLVM function ABI lowering (was: Re: [RFC] Refactor Clang: move frontend/driver/diagnostics code to LLVM)<o:p class=""></o:p></div></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div class=""><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">While MLIR may be one part of the solution, I think it's also the case that the function-ABI interface between Clang and LLVM is just wrong and should be fixed -- independently of whether Clang might use MLIR in the future.<o:p class=""></o:p></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">I've mentioned this idea before, I think, but never got around to writing up a real proposal. And I still haven't. Maybe this email could inspire someone else to work on that.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Essentially, I'd like to see the code in Clang responsible for function parameter-type mangling as part of its ABI lowering deleted. Currently, there is a secret "LLVM IR" ABI used between Clang and LLVM, which involves expanding some arguments into multiple arguments, adding a smattering of "inreg" or "byval" attributes, and converting some types into other types. All in a completely target-dependent, complex, and undocumented manner.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">So, while the IR function syntax appears at first glance to be generic and target-independent, that's not at all true. Sadly, in some cases, clang must even know how many registers different calling conventions use, and count numbers of available registers left, in order to choose the right set of those "generic" attributes to put on a parameter.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">So: not only does a frontend need to understand the C ABI rules, they also need to understand that complex dance for how to convert that into LLVM IR -- and that's both completely undocumented, and a huge mess.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Instead, I believe clang should always pass function parameters in a "naive" fashion. E.g. if a parameter type is "struct X", the llvm function should be lowered to LLVM IR with a function parameter of type %struct.X. The decision on whether to then pass that in a register (or multiple registers), on the stack, padded and then passed on the stack, etc, should be the responsibility of LLVM. Only in the case of C++ types which<i class="">must</i> be passed indirectly for correctness, independent of calling convention ABI, should clang be explicitly making the decision to pass indirectly.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Of course, the tricky part is that LLVM doesn't -- and shouldn't -- have the full C type system available to it, and the full C type system typically is required to evaluate the ABI rules (e.g., distinguishing a "_Complex float" from a struct containing two floats).<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Therefore, in order to communicate the correct ABI information to LLVM, I'd like clang to also emit <i class="">explicitly-ABI-specific</i> data (metadata?), reflecting the extra information that the ABI rules require the backend to know about the type. E.g., for X86_64, clang needs to inform LLVM of the classification for each parameter's type into MEMORY, INTEGER, SSE, SSEUP, X87, X87UP, COMPLEX_X87. Or, for PPC64 elfv2, Clang needs to inform LLVM when a structure should be treated as a "homogenous aggregate" of floating-point or vector type. (In both cases, that information cannot correctly be extracted from the LLVM IR struct type, only from the C type system.)<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">We should document what data is needed, for each architecture/abi. This required data should be as straightforward an application of the ABI document's rules as possible -- and be only the minimum data necessary.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">If this is done, frontends (either a new one, or Clang itself) who want to use the C ABI have a significantly simpler task. It remains non-trivial -- you do still need to understand ABI-specific rules, and write ABI-specific code to generate ABI-specific metadata. But, at least the interface boundary has become something which is readily-understandable and implementable based on the ABI documents.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">All that said, an MLIR encoding of the C type system can still be useful -- it could contain the code which distills the C types into the ABI-specific metadata. But, I see that as less important than getting the fundamentals in LLVM-IR into a better shape. Even frontends without a C type system representation should still be able to generate LLVM IR which conforms in their own manner to the documented ABIs -- without it being super painful. Also, the code in Clang now is really confusing, and nearly unmaintainable; it would be a clear improvement to be able to eliminate the majority of it, not just move it into an MLIR dialect.<o:p class=""></o:p></div></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div class=""><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">On Wed, Jun 3, 2020 at 7:26 PM Chris Lattner via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank" style="color: blue; text-decoration: underline;" class="">cfe-dev@lists.llvm.org</a>> wrote:<o:p class=""></o:p></div></div><blockquote style="border-style: none none none solid; border-left-width: 1pt; border-left-color: rgb(204, 204, 204); padding: 0in 0in 0in 6pt; margin-left: 4.8pt; margin-right: 0in;" class=""><div class=""><div class=""><blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class=""><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">On Jun 2, 2020, at 4:21 PM, comex via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank" style="color: blue; text-decoration: underline;" class="">cfe-dev@lists.llvm.org</a>> wrote:<o:p class=""></o:p></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div class=""><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">While this is a different area of the codebase, another thing that<br class="">would benefit greatly from being moved out of Clang is function call<br class="">ABI handling. Currently, that handling is split awkwardly between<br class="">Clang and LLVM proper, forcing frontends that implement C FFI to<br class="">either recreate the Clang parts themselves (like Rust does), depend on<br class="">Clang (like Swift does), or live with FFI just not working with some<br class="">function signatures. I'm not sure what Flang currently does, but my<br class="">understanding is that Flang does support C FFI, so it would probably<br class="">benefit from this as well. Just something to consider. :)<o:p class=""></o:p></div></div></div></blockquote><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">For what its worth, I think there is a pretty clear path on this, but it hinges on Clang moving to MLIR as its code generation backend (an intermediary to generating LLVM IR).<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">The approach is to factor the ABI lower part of clang out of Clang itself into a specific dialect lowering pass, that works on a generic C type system (plus callout to extended type systems). MLIR has all the infra to support this, it is just a massive job to refactor all the things to change clang’s architecture.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">I also don’t think there is broad consensus on the direction for Clang here, but given that Flang is already using MLIR for this, maybe it would make sense to start work there.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">If you’re curious, I co-delivered a talk about this recently, the slides <a href="https://docs.google.com/presentation/d/11-VjSNNNJoRhPlLxFgvtb909it1WNdxTnQFipryfAPU/edit#slide=id.g7d334b12e5_0_4" target="_blank" style="color: blue; text-decoration: underline;" class="">are available here</a>.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">-Chris<o:p class=""></o:p></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">_______________________________________________<br class="">cfe-dev mailing list<br class=""><a href="mailto:cfe-dev@lists.llvm.org" target="_blank" style="color: blue; text-decoration: underline;" class="">cfe-dev@lists.llvm.org</a><br class=""><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" target="_blank" style="color: blue; text-decoration: underline;" class="">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a></div></blockquote></div></div></div></div></div></blockquote></div><br class=""></div></body></html>