[llvm-dev] High Performance containers

[Francois Fayard via llvm-dev]

Hi,

Let me present myself : I work on High Performance Computing, mainly on number crunching where the languages used are mainly Fortran and C++. This field is moving more and more from pure number crunching where Fortran shines to a mix of numbers, texts and other data that you cannot easily deal with Fortran. Unfortunately, the C++ standard library suffers from its age and the fact that it has never been thought for performance.

As a consequence, I develop an Open Source library of containers and utility that could be useful for HPC people. But the more I look at LLVM, the more I find that our problems are very close. Moreover, I find it disappointing that when Chandler Carruth gives a talk about LLVM containers, people cannot ask for a standalone library that they can use in their code. It turns out that the world I live in is very close to the LLVM world:
- No exceptions (painful with highly multithreaded applications, and when mixing languages)
- Need an efficient array container with small size optimization
- Need an efficient hash map, hash set (with open addressing)
- Need an efficient string that can plays smoothly with UTF8 and its folklore (filenames being byte arrays on Linux, UCS2/UTF16 on Windows, etc)
- Need an easily used formatting library (the way Python does with format)
- Need an easy way to instrument the containers (such as checking statistics on malloc size, on the number of copies vs moves, etc)
- Need an efficient way to “return errors” that must be checked with rich type information
- Using the same Open Source licence

In the end, I believe that LLVM problems for performance are common to many people. My background is mainly on x86-64, ILP, vectorization, multithreading and memory layout optimizations. I am sure that mixing different background can make a great library that could be useful to many C++ developers. What I am looking for is for developers with LLVM experience in core containers design too share ideas to build such a great library. The problem being that an API can sometimes kill performance, it is very important to share experience when designing it.

The project is still young and available here : https://github.com/insideloop/InsideLoop <https://github.com/insideloop/InsideLoop>

Let me know if you think that such a library could be useful for you and if you would like to contribute. And, if it is a success, in a few years, why not using it in some LLVM parts...

François Fayard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170816/2e51e358/attachment.html>