[llvm-dev] [RFC 0/2] Propose a new pointer trait.

[Tomasz Kapela via llvm-dev]

Hi,
I'm part of an engineering team doing research on persistent memory support and
we have stumbled upon an interesting problem. The issue is, we would like to be
able to use the standard library containers in a persistent memory context
(think NVDIMM-N). What I mean is that you allocate a container from said
memory, use it like you normally would. After the application terminates,
expectedly or otherwise, you can place the container back into the newly
created process's address space - it will be in the last consistent state in
which it was before termination (interruption/crash/exit). The obvious way to
achieve this is to substitute the allocator. The programming model is based on
memory mapped files and because of that the pointer type used by the allocator
is a fancy pointer, so that it can do all the offset calculations.

User data consistency is provided via a transaction mechanism provided by the
library. We snapshot data before modifying them to be able to roll the
transaction back in case of an error. The usage model we are proposing is this
(pseudocode, not the actual API):

int main() {
    auto handle = pool::open("path/to/file", ...);
    using pvector = std::vector<foo, persistent_allocator<foo>>;
    {
        auto tx = transaction::start(handle); // transactions are per pool file
        auto vec_ptr = allocate_from_persistent_memory<pvector>();
        vec_ptr->emplace_back(foo, parameters);
    }
}

The problem occurs when standard library containers have metadata. For example
rb tree nodes carry their color, which will not be included in the
transaction's undo log. To address this we propose to define a new type in the
std::pointer_traits, which could be used to wrap the containers' metadata in
a user-defined type. This should be fully backward compatible and as such not
mandatory.

This is a standard level change and we would like to gather feedback for this
idea before we start the whole process. This feature is not persistent memory
specific, even we use it as part of providing transactional data consistency.

This is in fact a working proof of concept available at
(https://github.com/pmem/libcxx) combined with the allocator from
(https://github.com/pmem/nvml).

I will be happy to answer any questions you might have and await your
feedback.

Thanks,
Tom


Tomasz Kapela (2):
  pm: change deque typedef
  pm: add persistency_type typedef to pointer_traits

 include/__hash_table                               | 20 ++++++++------
 include/__tree                                     |  8 +++---
 include/deque                                      |  8 +++---
 include/list                                       |  7 +++--
 include/map                                        |  7 +++--
 include/memory                                     | 31 ++++++++++++++++++++++
 include/set                                        |  9 +++++--
 .../pointer.traits.types/persistency_type.pass.cpp | 25 +++++++++++++++++
 8 files changed, 94 insertions(+), 21 deletions(-)
 create mode 100644 test/std/utilities/memory/pointer.traits/pointer.traits.types/persistency_type.pass.cpp

-- 
2.9.3