Python Binding

Python bindings for RocksDB enable Python applications to leverage RocksDB’s high-performance embedded database capabilities with idiomatic Python interfaces.

Available Python Bindings

There are multiple Python binding options for RocksDB, each with different features and maintenance status:

RocksDict

Recommended: Modern, actively maintained Python binding with dict-like interface and comprehensive features.

python-rocksdb

Older binding, no longer actively maintained. Consider migrating to RocksDict.

pyrocksdb

Legacy binding, unmaintained. Not recommended for new projects.

This guide focuses on RocksDict, the recommended and actively maintained Python binding for RocksDB.

RocksDict - Recommended Binding

Installation

pip
Poetry
conda

pip install rocksdict

poetry add rocksdict

conda install -c conda-forge rocksdict

RocksDict includes pre-compiled wheels for Linux, macOS, and Windows, so no compilation is required.

Quick Start

RocksDict provides a dict-like interface that feels natural to Python developers:

from rocksdict import Rdict

# Open or create a database
db = Rdict("/path/to/db")

# Dict-like operations
db["key1"] = "value1"
db["key2"] = "value2"

# Read values
print(db["key1"])  # Output: value1

# Check existence
if "key1" in db:
    print("Key exists")

# Delete keys
del db["key1"]

# Iterate over keys
for key in db.keys():
    print(f"{key}: {db[key]}")

# Close the database
db.close()

Basic Operations

from rocksdict import Rdict

db = Rdict("/path/to/db")

# Write operations
db["user:1000"] = {"name": "Alice", "age": 30}
db["user:1001"] = {"name": "Bob", "age": 25}

# Read operations
user = db["user:1000"]
print(user["name"])  # Alice

# Get with default
user = db.get("user:9999", {"name": "Unknown"})

# Update
db["user:1000"] = {"name": "Alice", "age": 31}

# Delete
del db["user:1001"]

db.close()

from rocksdict import Rdict

db = Rdict("/path/to/db")

# Batch write
with db.write_batch() as batch:
    batch["key1"] = "value1"
    batch["key2"] = "value2"
    batch["key3"] = "value3"
    # Commit happens automatically at end of context

# Delete multiple keys
with db.write_batch() as batch:
    del batch["key1"]
    del batch["key2"]

db.close()

from rocksdict import Rdict

db = Rdict("/path/to/db")

# Iterate over all keys
for key in db.keys():
    print(key)

# Iterate over all values
for value in db.values():
    print(value)

# Iterate over items
for key, value in db.items():
    print(f"{key}: {value}")

# Prefix scan
for key in db.keys():
    if key.startswith("user:"):
        print(f"{key}: {db[key]}")

db.close()

Configuration Options

Configure RocksDB when opening a database:

from rocksdict import Rdict, Options

# Create options
opts = Options()
opts.create_if_missing(True)
opts.set_max_open_files(10000)
opts.set_use_fsync(False)
opts.set_bytes_per_sync(8388608)
opts.optimize_for_point_lookup(1024)
opts.set_table_cache_num_shard_bits(6)
opts.set_max_write_buffer_number(32)
opts.set_write_buffer_size(536870912)
opts.set_target_file_size_base(1073741824)
opts.set_min_write_buffer_number_to_merge(4)
opts.set_level_zero_stop_writes_trigger(2000)
opts.set_level_zero_slowdown_writes_trigger(0)
opts.set_compaction_style("universal")

# Open database with options
db = Rdict("/path/to/db", options=opts)

db.close()

Column Families

RocksDict supports column families for logical data partitioning:

from rocksdict import Rdict

# Open with column families
db = Rdict("/path/to/db", column_families={"users", "posts", "comments"})

# Access specific column family
users_cf = db.get_column_family("users")
posts_cf = db.get_column_family("posts")

# Write to different column families
users_cf["user:1000"] = {"name": "Alice"}
posts_cf["post:5000"] = {"title": "Hello World"}

# Read from column families
user = users_cf["user:1000"]
post = posts_cf["post:5000"]

db.close()

Snapshots

Create point-in-time snapshots for consistent reads:

from rocksdict import Rdict

db = Rdict("/path/to/db")

# Write some data
db["counter"] = 100

# Create a snapshot
snapshot = db.snapshot()

# Modify data
db["counter"] = 200

# Read from snapshot (still sees old value)
print(snapshot["counter"])  # 100

# Read from current db (sees new value)
print(db["counter"])  # 200

# Release snapshot
snapshot.close()
db.close()

Data Serialization

Automatic Serialization
Binary Data
Custom Serialization

RocksDict automatically serializes Python objects:

from rocksdict import Rdict

db = Rdict("/path/to/db")

# Store Python objects directly
db["user"] = {"name": "Alice", "age": 30, "active": True}
db["numbers"] = [1, 2, 3, 4, 5]
db["settings"] = {"theme": "dark", "notifications": True}

# Retrieve as Python objects
user = db["user"]
print(user["name"])  # Alice

Store raw binary data:

from rocksdict import Rdict

db = Rdict("/path/to/db", raw_mode=True)

# Store raw bytes
db[b"key1"] = b"value1"
db[b"key2"] = b"\x00\x01\x02\x03"

# Retrieve as bytes
value = db[b"key1"]
print(value)  # b'value1'

Use custom serialization for specific needs:

import json
from rocksdict import Rdict

db = Rdict("/path/to/db", raw_mode=True)

# Custom JSON serialization
def set_json(db, key, value):
    db[key.encode()] = json.dumps(value).encode()

def get_json(db, key):
    data = db[key.encode()]
    return json.loads(data.decode())

# Usage
set_json(db, "user", {"name": "Alice", "age": 30})
user = get_json(db, "user")
print(user["name"])  # Alice

Performance Tips

Use Batch Writes

Group multiple writes into batches to improve throughput and reduce write amplification.

Tune Buffer Sizes

Adjust write buffer and cache sizes based on your memory constraints and workload patterns.

Enable Compression

Use compression to reduce storage space. RocksDB supports multiple compression algorithms.

Optimize for Workload

Use optimize_for_point_lookup() for read-heavy workloads or adjust compaction settings for write-heavy workloads.

Batch Operations for Performance

from rocksdict import Rdict
import time

db = Rdict("/path/to/db")

# Slow: Individual writes
start = time.time()
for i in range(10000):
    db[f"key_{i}"] = f"value_{i}"
print(f"Individual writes: {time.time() - start:.2f}s")

# Fast: Batch writes
start = time.time()
with db.write_batch() as batch:
    for i in range(10000):
        batch[f"key_{i}"] = f"value_{i}"
print(f"Batch writes: {time.time() - start:.2f}s")

db.close()

Context Manager Support

RocksDict supports Python context managers for automatic resource cleanup:

from rocksdict import Rdict

# Automatic cleanup with context manager
with Rdict("/path/to/db") as db:
    db["key"] = "value"
    print(db["key"])
    # Database is automatically closed when exiting the context

Error Handling

from rocksdict import Rdict, RocksDBError

try:
    db = Rdict("/path/to/db")
    
    # Attempt operations
    value = db["nonexistent_key"]
    
except KeyError as e:
    print(f"Key not found: {e}")
except RocksDBError as e:
    print(f"RocksDB error: {e}")
except Exception as e:
    print(f"Unexpected error: {e}")
finally:
    if 'db' in locals():
        db.close()

Comparison: python-rocksdb (Legacy)

If you’re maintaining code using the older python-rocksdb binding:

The python-rocksdb binding is unmaintained. Consider migrating to RocksDict for continued support and new features.

import rocksdb

# Open database
opts = rocksdb.Options()
opts.create_if_missing = True
db = rocksdb.DB("/path/to/db", opts)

# Basic operations (bytes only)
db.put(b"key", b"value")
value = db.get(b"key")

# Iteration
it = db.iterkeys()
it.seek_to_first()
for key in it:
    print(key)

del db

from rocksdict import Rdict

# Open database
db = Rdict("/path/to/db")

# Dict-like operations with auto-serialization
db["key"] = "value"
value = db["key"]

# Pythonic iteration
for key in db.keys():
    print(key)

db.close()

Type Hints and Modern Python

RocksDict works well with modern Python features:

from typing import Dict, Optional
from rocksdict import Rdict
from dataclasses import dataclass

@dataclass
class User:
    name: str
    age: int
    email: str

class UserStore:
    def __init__(self, db_path: str):
        self.db = Rdict(db_path)
    
    def save_user(self, user_id: str, user: User) -> None:
        self.db[f"user:{user_id}"] = {
            "name": user.name,
            "age": user.age,
            "email": user.email
        }
    
    def get_user(self, user_id: str) -> Optional[User]:
        data = self.db.get(f"user:{user_id}")
        if data:
            return User(**data)
        return None
    
    def close(self) -> None:
        self.db.close()

# Usage
store = UserStore("/path/to/db")
user = User(name="Alice", age=30, email="alice@example.com")
store.save_user("1000", user)
retrieved = store.get_user("1000")
print(retrieved.name)  # Alice
store.close()

Resources

RocksDict GitHub

Official RocksDict repository with documentation and examples

python-rocksdb Docs

Legacy python-rocksdb documentation (unmaintained)

PyPI Package

Install RocksDict from Python Package Index

RocksDB Wiki

General RocksDB documentation and guides

Next Steps

Configuration

Learn about RocksDB configuration options

Performance Tuning

Optimize RocksDB for your Python application

Other Bindings

Explore RocksDB bindings for other languages

Basic Operations

Master RocksDB fundamentals

Get Started

Core Concepts

Developer Guide

Advanced Topics

Language Bindings

Tools & Utilities

Available Python Bindings

RocksDict

python-rocksdb

pyrocksdb

RocksDict - Recommended Binding

Installation

Quick Start

Basic Operations

Configuration Options

Column Families

Snapshots

Data Serialization

Performance Tips

Use Batch Writes

Tune Buffer Sizes

Enable Compression

Optimize for Workload

Batch Operations for Performance

Context Manager Support

Error Handling

Comparison: python-rocksdb (Legacy)

Type Hints and Modern Python

Resources

RocksDict GitHub

python-rocksdb Docs

PyPI Package

RocksDB Wiki

Next Steps

Configuration

Performance Tuning

Other Bindings

Basic Operations

​Available Python Bindings

RocksDict

python-rocksdb

pyrocksdb

​RocksDict - Recommended Binding

​Installation

​Quick Start

​Basic Operations

​Configuration Options

​Column Families

​Snapshots

​Data Serialization

​Performance Tips

Use Batch Writes

Tune Buffer Sizes

Enable Compression

Optimize for Workload

​Batch Operations for Performance

​Context Manager Support

​Error Handling

​Comparison: python-rocksdb (Legacy)

​Type Hints and Modern Python

​Resources

RocksDict GitHub

python-rocksdb Docs

PyPI Package

RocksDB Wiki

​Next Steps

Configuration

Performance Tuning

Other Bindings

Basic Operations

Available Python Bindings

RocksDict - Recommended Binding

Installation

Quick Start

Basic Operations

Configuration Options

Column Families

Snapshots

Data Serialization

Performance Tips

Batch Operations for Performance

Context Manager Support

Error Handling

Comparison: python-rocksdb (Legacy)

Type Hints and Modern Python

Resources

Next Steps