Skip to content
This repository has been archived by the owner on May 3, 2022. It is now read-only.

Latest commit

 

History

History
61 lines (49 loc) · 3 KB

README.md

File metadata and controls

61 lines (49 loc) · 3 KB

DXRAM: A distributed in-memory key-value storage

Build Status Build Status

Developed by the operating systems group of the department of computer science of the Heinrich-Heine-University Düsseldorf, DXRAM is a distributed in-memory key-value store for low-latency cloud applications, e.g. social networks, search engines and I/O-bound long running scientific computations running in a data center. It is designed to efficiently manage billions of small data objects which are typical for graph based applications.

For example, Facebook runs approx. 1,000 memcached servers keeping around 75% of its data always in RAM because back-end database accesses are too slow. We argue that it is possible to go one step further by keeping all data always in RAM relieving developers from cache management and synchronization of caches with secondary storage.

Important

DXRAM is a research project that's under development. We do not recommend using the system in a production environment or with production data without having an external backup. Expect to encounter bugs. However, we are looking forward to bug reports and code contributions.

Features

  • Distributed object lookup and scalable meta-data management with super-peer overlay
  • Optimized for up to 10 GBit/s Ethernet network and up to 56 GBit/s InfiniBand
  • All objects always in RAM
  • Custom, highly efficient and performant memory management for billions of small objects (< 64 bytes) per node
  • Smart asynchronous logging to disk (optimized for SSD) for persistency
  • Replication of logs to remote nodes for fault tolerance
  • Crash-recovery failure-model
  • Object migrations for storage nodes to handle hot spots
  • Run computations on storage nodes using lightweight jobs or tasks submitted to coordinated compute groups

Contents

License

Copyright (C) 2018 Heinrich-Heine-Universitaet Duesseldorf, Institute of Computer Science, Department Operating Systems. Licensed under the GNU General Public License.