Viper: An Efficient Hybrid PMem-DRAM Key-Value Store


Key-value stores (KVSs) have found wide application in modern software systems. For persistence, their data resides in slow secondary storage, which requires KVSs to employ various techniques to increase their read and write performance from and to the underlying medium. Emerging persistent memory (PMem) technologies offer data persistence at close-to-DRAM speed, making them a promising alternative to classical disk-based storage. However, simply drop-in replacing existing storage with PMem does not yield good results, as block-based access behaves differently in PMem than on disk and ignores PMem’s byte addressability, layout, and unique performance characteristics. In this paper, we propose three PMem-specific access patterns and implement them in a hybrid PMem-DRAM KVS called Viper. We employ a DRAM-based hash index and a PMem-aware storage layout to utilize the random-write speed of DRAM and efficient sequential-write performance PMem. Our evaluation shows that Viper significantly outperforms existing KVSs for core KVS operations while providing full data persistence. Moreover, Viper outperforms existing PMem-only, hybrid, and disk-based KVSs by 4–18x for write workloads, while matching or surpassing their get performance.

In Proceedings of the VLDB Endowment 2021
Hendrik Makait
Hendrik Makait
OSS Engineer

I’m a data and software engineer building systems at the intersection of large-scale data management and machine learning. I work as an OSS Engineer on the Dask Foundations team at Coiled evolving Dask into a performant, production-grade distributed computing platform. Previously, I worked on data management platforms for machine learning at SiaSearch and Scale AI. I studied CS at TU Berlin while redesigning message brokers for modern hardware at Hasso Plattner Institute.