Iterators

Iterators provide an efficient way to scan through RocksDB data in sorted order. This guide covers basic iteration patterns, seeking strategies, and performance considerations.

Creating an Iterator

Create an iterator using NewIterator():

#include "rocksdb/db.h"
#include "rocksdb/iterator.h"
#include "rocksdb/options.h"

using ROCKSDB_NAMESPACE::Iterator;
using ROCKSDB_NAMESPACE::ReadOptions;

ReadOptions read_options;
Iterator* it = db->NewIterator(read_options);

// Use the iterator...

// Clean up
delete it;

You must delete the iterator when you’re done with it. Iterators hold resources that need to be released.

Basic Iteration Patterns

Forward Iteration

Iterate from beginning to end:

Iterator* it = db->NewIterator(ReadOptions());

// Start from the beginning
it->SeekToFirst();

while (it->Valid()) {
    Slice key = it->key();
    Slice value = it->value();
    
    // Process key-value pair
    std::cout << key.ToString() << ": " << value.ToString() << std::endl;
    
    it->Next();
}

// Check for errors
assert(it->status().ok());
delete it;

Reverse Iteration

Iterate from end to beginning:

Iterator* it = db->NewIterator(ReadOptions());

// Start from the end
it->SeekToLast();

while (it->Valid()) {
    Slice key = it->key();
    Slice value = it->value();
    
    std::cout << key.ToString() << ": " << value.ToString() << std::endl;
    
    it->Prev();  // Move backwards
}

assert(it->status().ok());
delete it;

Seeking Operations

Seek to Specific Key

Iterator* it = db->NewIterator(ReadOptions());

// Seek to key >= "user:1000"
it->Seek("user:1000");

if (it->Valid()) {
    // Iterator is positioned at "user:1000" or the next key
    std::cout << "Found: " << it->key().ToString() << std::endl;
} else {
    std::cout << "No keys >= user:1000" << std::endl;
}

delete it;

SeekForPrev - Seek in Reverse

Iterator* it = db->NewIterator(ReadOptions());

// Seek to key <= "user:1000"
it->SeekForPrev("user:1000");

if (it->Valid()) {
    // Iterator is positioned at "user:1000" or the previous key
    std::cout << "Found: " << it->key().ToString() << std::endl;
}

delete it;

Seek()
SeekForPrev()
SeekToFirst()
SeekToLast()

Positions iterator at the first key greater than or equal to target

Iterator API Reference

Core Methods

class Iterator {
public:
    // Position iterator
    virtual void SeekToFirst() = 0;
    virtual void SeekToLast() = 0;
    virtual void Seek(const Slice& target) = 0;
    virtual void SeekForPrev(const Slice& target) = 0;
    
    // Navigate
    virtual void Next() = 0;
    virtual void Prev() = 0;
    
    // Query state
    virtual bool Valid() const = 0;
    virtual Slice key() const = 0;
    virtual Slice value() const = 0;
    virtual Status status() const = 0;
};

Key Iterator State Methods

Valid()

Returns true if the iterator is positioned at a valid key-value pair.

key()

Returns the current key. Only valid when Valid() is true.

value()

Returns the current value. Only valid when Valid() is true.

status()

Returns the iterator’s error status. Always check after iteration completes.

Range Queries

Bounded Range Iteration

Iterate over a specific key range:

ReadOptions read_options;

// Set upper bound (exclusive)
Slice upper_bound("user:2000");
read_options.iterate_upper_bound = &upper_bound;

Iterator* it = db->NewIterator(read_options);

// Iterate from user:1000 to user:2000 (exclusive)
it->Seek("user:1000");

while (it->Valid()) {
    std::cout << it->key().ToString() << std::endl;
    it->Next();
}

assert(it->status().ok());
delete it;

Setting iterate_upper_bound can significantly improve performance by allowing RocksDB to skip unnecessary data.

Prefix Iteration

Efficiently iterate over keys with a common prefix:

#include "rocksdb/slice_transform.h"

using ROCKSDB_NAMESPACE::NewFixedPrefixTransform;

// Configure prefix extractor (do this when opening the DB)
Options options;
options.prefix_extractor.reset(NewFixedPrefixTransform(4));

// Later, iterate with prefix
ReadOptions read_options;
read_options.prefix_same_as_start = true;

Iterator* it = db->NewIterator(read_options);

// Iterate over all keys with prefix "user"
it->Seek("user");

while (it->Valid()) {
    // Will only iterate over keys starting with "user"
    std::cout << it->key().ToString() << std::endl;
    it->Next();
}

delete it;

Snapshot Iteration

Iterate over a consistent snapshot:

using ROCKSDB_NAMESPACE::Snapshot;

// Create a snapshot
const Snapshot* snapshot = db->GetSnapshot();

ReadOptions read_options;
read_options.snapshot = snapshot;

// Iterator reads from the snapshot
Iterator* it = db->NewIterator(read_options);

for (it->SeekToFirst(); it->Valid(); it->Next()) {
    // Data is consistent with the snapshot
    std::cout << it->key().ToString() << std::endl;
}

assert(it->status().ok());
delete it;

// Release the snapshot
db->ReleaseSnapshot(snapshot);

Iterators created with a snapshot will read data as it existed when the snapshot was created, even if the data is subsequently modified or deleted.

Performance Optimization

Read-Ahead

Enable read-ahead for sequential scans:

ReadOptions read_options;

// Enable read-ahead (in bytes)
read_options.readahead_size = 2 << 20;  // 2MB

Iterator* it = db->NewIterator(read_options);

for (it->SeekToFirst(); it->Valid(); it->Next()) {
    // Process data...
}

delete it;

Skip Block Cache

For large scans that shouldn’t pollute the cache:

ReadOptions read_options;
read_options.fill_cache = false;

Iterator* it = db->NewIterator(read_options);
// Large scan won't evict useful data from cache
delete it;

Tailing Iterator

For monitoring new data in real-time:

ReadOptions read_options;
read_options.tailing = true;

Iterator* it = db->NewIterator(read_options);

while (keep_monitoring) {
    it->SeekToLast();
    
    if (it->Valid()) {
        // Process latest data
        ProcessKey(it->key(), it->value());
    }
    
    // Wait before checking again
    std::this_thread::sleep_for(std::chrono::seconds(1));
}

delete it;

Multi-Column Family Iteration

NewIterators for Multiple Column Families

using ROCKSDB_NAMESPACE::ColumnFamilyHandle;

std::vector<ColumnFamilyHandle*> column_families;
std::vector<Iterator*> iterators;

Status s = db->NewIterators(ReadOptions(), column_families, &iterators);
assert(s.ok());

for (size_t i = 0; i < iterators.size(); i++) {
    Iterator* it = iterators[i];
    
    for (it->SeekToFirst(); it->Valid(); it->Next()) {
        // Process data from column family i
    }
    
    assert(it->status().ok());
    delete it;
}

Error Handling

Always check iterator status after iteration:

Iterator* it = db->NewIterator(ReadOptions());

for (it->SeekToFirst(); it->Valid(); it->Next()) {
    // Process entries
}

// IMPORTANT: Check status after iteration
if (!it->status().ok()) {
    std::cerr << "Iterator error: " << it->status().ToString() << std::endl;
    // Handle error...
}

delete it;

An iterator can become invalid due to errors (I/O errors, corruption). Always check status() after iteration completes.

Common Patterns

Count Keys in Range

size_t CountKeysInRange(DB* db, const Slice& start, const Slice& end) {
    ReadOptions read_options;
    read_options.iterate_upper_bound = &end;
    
    Iterator* it = db->NewIterator(read_options);
    
    size_t count = 0;
    for (it->Seek(start); it->Valid(); it->Next()) {
        count++;
    }
    
    assert(it->status().ok());
    delete it;
    
    return count;
}

Batch Processing

void ProcessInBatches(DB* db, size_t batch_size) {
    Iterator* it = db->NewIterator(ReadOptions());
    
    it->SeekToFirst();
    
    while (it->Valid()) {
        std::vector<std::pair<std::string, std::string>> batch;
        
        // Collect batch
        for (size_t i = 0; i < batch_size && it->Valid(); i++, it->Next()) {
            batch.emplace_back(it->key().ToString(), it->value().ToString());
        }
        
        // Process batch
        ProcessBatch(batch);
    }
    
    assert(it->status().ok());
    delete it;
}

Best Practices

Always delete iterators

Iterators hold resources. Delete them when done.

Check status after iteration

Errors can occur during iteration. Always check status() after the loop.

Use bounded iteration

Set iterate_upper_bound to improve performance for range queries.

Consider fill_cache

Set fill_cache = false for large scans to avoid cache pollution.

Use snapshots for consistency

Create a snapshot if you need a consistent view across multiple operations.

Next Steps

Basic Operations

Learn fundamental CRUD operations

Transactions

Implement ACID transactions

Performance Tuning

Optimize iterator performance

Get Started

Core Concepts

Developer Guide

Advanced Topics

Language Bindings

Tools & Utilities

Creating an Iterator

Basic Iteration Patterns

Forward Iteration

Reverse Iteration

Seeking Operations

Seek to Specific Key

SeekForPrev - Seek in Reverse

Iterator API Reference

Core Methods

Key Iterator State Methods

Range Queries

Bounded Range Iteration

Prefix Iteration

Snapshot Iteration

Performance Optimization

Read-Ahead

Skip Block Cache

Tailing Iterator

Multi-Column Family Iteration

NewIterators for Multiple Column Families

Error Handling

Common Patterns

Count Keys in Range

Batch Processing

Best Practices

Next Steps

Basic Operations

Transactions

Performance Tuning

​Creating an Iterator

​Basic Iteration Patterns

​Forward Iteration

​Reverse Iteration

​Seeking Operations

​Seek to Specific Key

​SeekForPrev - Seek in Reverse

​Iterator API Reference

​Core Methods

​Key Iterator State Methods

​Range Queries

​Bounded Range Iteration

​Prefix Iteration

​Snapshot Iteration

​Performance Optimization

​Read-Ahead

​Skip Block Cache

​Tailing Iterator

​Multi-Column Family Iteration

​NewIterators for Multiple Column Families

​Error Handling

​Common Patterns

​Count Keys in Range

​Batch Processing

​Best Practices

​Next Steps

Basic Operations

Transactions

Performance Tuning

Creating an Iterator

Basic Iteration Patterns

Forward Iteration

Reverse Iteration

Seeking Operations

Seek to Specific Key

SeekForPrev - Seek in Reverse

Iterator API Reference

Core Methods

Key Iterator State Methods

Range Queries

Bounded Range Iteration

Prefix Iteration

Snapshot Iteration

Performance Optimization

Read-Ahead

Skip Block Cache

Tailing Iterator

Multi-Column Family Iteration

NewIterators for Multiple Column Families

Error Handling

Common Patterns

Count Keys in Range

Batch Processing

Best Practices

Next Steps