[llvm-dev] RFC: Adding support for the z/OS platform to LLVM and clang

Fri Jun 12 07:10:05 PDT 2020

> On Jun 10, 2020, at 15:11, Kai Peter Nacke via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> 
> As part of IBM’s ongoing efforts to improve the z/OS ecosystem, our 
> current plans involve adding support for the z/OS platform to LLVM and 
> Clang. Our goal is to have a viable C and C++ LLVM compiler and runtime 
> library that generates code for, and runs on z/OS.
> 
> Long term, we expect to have a compiler and library that supports the 
> platform more fully. We intend to support the native character encoding 
> (EBCDIC), file system - z/OS UNIX System Services (z/OS UNIX) files and 
> datasets, addressing modes (31-bit and 64-bit, and possibly 24-bit), 
> different floating and fixed point formats, systems programming 
> capabilities, language co-processors, and generating code output in the 
> z/OS object file formats (GOFF and XOBJ).
> 
> The tentative plan is to initially support the z/OS Unix System Services 
> (z/OS UNIX) interface with EBCDIC and ASCII 64-bit code generation. In 
> particular, our intent would be to focus on:
> 1) C++ standard library support. We would be making changes to the library 
> so it can work on z/OS.  We would need some design discussions with the 
> community for issues such as character encodings.
> 2) EBCDIC source file input. We would need reviews at the Clang level when 
> dealing with reading source files and dealing with multiple code pages.
> 3) GOFF object file output. We would need reviews in LLVM to add a new 
> object file output format.
> Our plans include setting up z/OS build bots and we will update the list 
> when we have them ready to go.
> 
> To begin, we plan to add patches that would:
> - Set the new triple for z/OS
> - Make changes to the build recipes and tools (cmake, etc.) as needed to 
> allow building for z/OS
> Following that, we intend to start on the focus areas listed above:
> 
> 1) Add patches to enable building and using the C++ standard library on 
> z/OS. In particular, issues dealing with EBCDIC would need to be 
> addressed. We would need to have functions in the headers (e.g. iostream) 
> that work on ASCII encoded strings, and functions that work on EBCDIC 
> encoded strings. These would need to work with the underlying system C 
> library (e.g. printf) that provides the actual functionality. For example, 
> currently, the z/OS C library has (at least) two sets of functions (ASCII 
> and EBCDIC versions). The one used by the application is selected at 
> compile time during the system header file processing which selects the 
> correct function via mapping the programmer function name (e.g. printf) 
> into one that the application will link to (e.g. __printf for EBCDIC and 
> \174\174A00118 for ASCII).

I must admit I am wary of adding even more complexity to the libc++ headers. We have a big problem of configuration explosion in libc++, and that will unfortunately not help. If there are ways to support EDBCDIC non-intrusively, that would be better. In all cases, let's talk. Ideally, we could have an exploratory chat where someone from your side can show me what you're trying to achieve before we go down the route of having a full patch posted to Phab.

> We would also add patches to disable 
> functionality when on z/OS where there is no support for the 
> functionality. For example, thread specific locales would be disabled when 
> in a non-POSIX mode.
>  Our intent is that follow on patches would incrementally add support in 
> tandem with the compiler for features that require it and for other z/OS 
> specifics such as various floating/fixed point formats.

At the risk of sounding grumpy, this scares me too. Conditionally removing parts of libc++ is tricky. It basically creates a different "compile-time code path" through #ifdefs every time we do that. Those are a source of bugs and a maintenance burden, and I've been trying to remove as many as I can. For example, you'll notice I'm fairly aggressive about removing workarounds for old untested Apple platforms. It's much easier when the underlying system just provides what the library needs.

But in all cases, let's have a chat to see the extent of the changes you need and we can go from there. Also, I would suggest setting up build bots for libc++ on z/OS from the very start -- one can only claim to support a system if it's tested continuously, so patches will be received with a lot more enthusiasm if they are tested somehow.

If you're on the CppLang Slack, feel free to drop me a line and we can have an informal chat.

Cheers,
Louis

> 
> 2) Add patches to Clang to allow EBCDIC and ASCII (ISO-8859-1) encoded 
> input source files. This would be done at the file open time to allow the 
> rest of Clang to operate as if the source was UTF-8 and so require no 
> changes downstream. Feedback on this plan is welcome from the Clang 
> community.
>  Our intent is that later patches would handle execution character set 
> differences. Collaboration with the community here would be useful in 
> areas such as adding in exec-charset and library selection options and 
> strategies.
>  Our intent is also to make changes to support any platform issues, 
> processing native C header files, and idiosyncrasies on z/OS such as 
> having no native strnlen function. We would update test tooling to handle 
> character encoding issues as needed. Further design discussion will take 
> place on the Clang mailing list.
> 
> 3) Add patches to LLVM that will stub out GOFF object binary generation. 
> We would not be generating assembly (HLASM in z/OS), and instead only 
> generate the binary object directly for the initial round of changes. 
> Assembly generation would follow in later stages once we have a working 
> compiler on z/OS. Feedback on this plan of direction is appreciated.
>  Our intent is that patches that incrementally add support for GOFF 
> object generation such as code sections and records would follow. The next 
> steps after support for the object file format would be handling the z/OS 
> XPLINK calling convention. This would involve changes to both Clang and 
> LLVM and we intend to follow the same style of functional component 
> responsibility as is done for other platforms calling conventions. If we 
> believe deviations from this is necessary, we plan on notifying the 
> community and ensuring the reasons behind the deviations are valid and 
> accepted.
> 
> Any feedback or comments are welcome.
> 
> Notice: IBM’s statements regarding its plans, directions, and intent are 
> subject to change or withdrawal without notice at IBM’s sole discretion. 
> Information regarding potential future products is intended to outline our 
> general product direction and it should not be relied on in making a 
> purchasing decision. The information mentioned regarding potential future 
> products is not a commitment, promise, or legal obligation to deliver any 
> material, code or functionality. Information about potential future 
> products may not be incorporated into any contract. The development, 
> release, and timing of any future features or functionality described for 
> our products remains at our sole discretion.
> 
> Best regards,
> Kai Nacke
> IT Architect
> 
> IBM Deutschland GmbH
> Vorsitzender des Aufsichtsrats: Sebastian Krause
> Geschäftsführung: Gregor Pillen (Vorsitzender), Agnes Heftberger, Norbert 
> Janzen, Markus Koerner, Christian Noll, Nicole Reimer
> Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart, 
> HRB 14562 / WEEE-Reg.-Nr. DE 99369940
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev