Loading…
Attending this event?
Welcome to Percona Live Online 2021
Online Open Source Database Conference
REGISTER HERE!
Back To Schedule
Thursday, May 13 • 11:00 - 12:00
Scaling Out Distributed Storage Fabric with RocksDB

Log in to save this to your schedule, view media, leave feedback and see who's attending!

Video will become available 10 minutes before session start


Engineers at Nutanix have been working on the challenge of building a next-generation architecture for its distributed storage fabric. Scaling this architecture to the needs of the future required three primary objectives: significant improvements in sustained random write performance, support for large-capacity deep storage nodes for multi-petabyte scale and reducing storage latency by a significant magnitude.

These goals required re-imagining the core approach to how metadata is stored in the fabric management system and move the metadata controllers closer to where is the data is stored.

After extensive research and testing, RocksDB was chosen as the core component for this project, based on its open-source pedigree and proven reliability and industry adoption. Within a few months, the engineering team was able to ramp up expertise, build confidence with the open-source technology and eventually grow its adoption into several core products at Nutanix.

In this technical talk, we will share some of the early lessons learned in adopting RocksDB, contrast with the legacy approach and other competing key-value storage projects and discuss some innovative enhancements we were able to make to fit our performance goals and objectives.

One of the significant improvements has been the addition of async read/write support to RocksDB. Currently, the open source RocksDB exposes blocking I/O APIs which can limit overall system throughput under resource constraints. We developed a Fibers/Co-routine based non-blocking I/O solution for RocksDB. Our results show significant throughput improvement with efficient resource usage.

In addition to this presentation, we plan to publish a series of blogs to discuss various topics and projects that have built on this enhanced RocksDB implementation.

Speakers
RT

Raghav Tulshibagwale

Staff Engineer, Core Data Path, Nutanix Inc.
Raghav is a Staff Engineer and technical lead in the metadata subsystem for Nutanix distributed filesystem. Prior to Nutanix, Raghav worked as Principal Engineer in the Database Kernel Team at Pivotal/Greenplum.
PK

Pulkit Kapoor

MTS, Core Data Path, Nutanix, Inc.


Thursday May 13, 2021 11:00 - 12:00 EDT
Room #4
Feedback form isn't open yet.