MergeOperator

MergeOperator allows defining custom merge semantics for associative and non-associative operations in RocksDB. Instead of reading, modifying, and writing values, merge operands can be efficiently stacked and applied lazily.

Overview

RocksDB provides two interfaces for merge operations:

AssociativeMergeOperator: For simple, associative operations (e.g., numeric addition, string concatenation)
MergeOperator: For complex, non-associative operations with full control over merge semantics

AssociativeMergeOperator

Interface

class AssociativeMergeOperator : public MergeOperator {
 public:
  virtual bool Merge(
    const Slice& key,
    const Slice* existing_value,
    const Slice& value,
    std::string* new_value,
    Logger* logger
  ) const = 0;
};

Merge Method

const Slice&

Key associated with the merge operation.

const Slice*

Current value, or nullptr if the key doesn’t exist.

const Slice&

Merge operand to apply.

std::string*

Output parameter for the merge result.

Logger*

Logger for error messages.

bool

Returns true on success, false on error.

Example: Counter Merge Operator

#include "rocksdb/merge_operator.h"
#include <cstring>

using namespace ROCKSDB_NAMESPACE;

class CounterMergeOperator : public AssociativeMergeOperator {
 public:
  virtual bool Merge(
    const Slice& key,
    const Slice* existing_value,
    const Slice& value,
    std::string* new_value,
    Logger* logger
  ) const override {
    // Parse operand
    int64_t operand = 0;
    if (value.size() != sizeof(int64_t)) {
      return false;
    }
    memcpy(&operand, value.data(), sizeof(int64_t));

    // Get current value
    int64_t current = 0;
    if (existing_value != nullptr) {
      if (existing_value->size() != sizeof(int64_t)) {
        return false;
      }
      memcpy(&current, existing_value->data(), sizeof(int64_t));
    }

    // Add operand to current value
    int64_t result = current + operand;
    new_value->assign(reinterpret_cast<char*>(&result), sizeof(int64_t));
    return true;
  }

  const char* Name() const override {
    return "CounterMergeOperator";
  }
};

// Usage
Options options;
options.merge_operator.reset(new CounterMergeOperator());

DB* db;
DB::Open(options, "/tmp/testdb", &db);

// Increment counter
int64_t delta = 5;
Slice delta_slice(reinterpret_cast<char*>(&delta), sizeof(int64_t));
db->Merge(WriteOptions(), "counter", delta_slice);

// Another increment
delta = 3;
delta_slice = Slice(reinterpret_cast<char*>(&delta), sizeof(int64_t));
db->Merge(WriteOptions(), "counter", delta_slice);

// Read result (should be 8)
std::string value;
db->Get(ReadOptions(), "counter", &value);
int64_t result;
memcpy(&result, value.data(), sizeof(int64_t));
printf("Counter value: %ld\n", result);  // 8

FullMergeV3

virtual bool FullMergeV3(
  const MergeOperationInputV3& merge_in,
  MergeOperationOutputV3* merge_out
) const;

Applies a stack of merge operands on top of an existing value.

MergeOperationInputV3

struct MergeOperationInputV3 {
  using ExistingValue = std::variant<std::monostate, Slice, WideColumns>;
  using OperandList = std::vector<Slice>;

  const Slice& key;
  ExistingValue existing_value;
  const OperandList& operand_list;
  Logger* logger;
};

const Slice&

User key, including user-defined timestamp if applicable.

ExistingValue

Base value: no value, plain value, or wide-column value.

const OperandList&

List of merge operands to apply.

MergeOperationOutputV3

struct MergeOperationOutputV3 {
  using NewColumns = std::vector<std::pair<std::string, std::string>>;
  using NewValue = std::variant<std::string, NewColumns, Slice>;

  NewValue new_value;
  OpFailureScope op_failure_scope = OpFailureScope::kDefault;
};

NewValue

Result: new plain value, new wide-column value, or existing operand.

OpFailureScope

Scope of failure: kDefault, kTryMerge, or kMustMerge.

PartialMerge

virtual bool PartialMerge(
  const Slice& key,
  const Slice& left_operand,
  const Slice& right_operand,
  std::string* new_value,
  Logger* logger
) const;

Combines two merge operands into a single operand.

const Slice&

First operand.

const Slice&

Second operand.

std::string*

Output parameter for combined operand.

bool

Returns true if operands were combined, false if combination is not possible.

PartialMergeMulti

virtual bool PartialMergeMulti(
  const Slice& key,
  const std::deque<Slice>& operand_list,
  std::string* new_value,
  Logger* logger
) const;

Combines multiple merge operands into a single operand.

Example: String Append Merge Operator

#include "rocksdb/merge_operator.h"

using namespace ROCKSDB_NAMESPACE;

class StringAppendOperator : public AssociativeMergeOperator {
 public:
  virtual bool Merge(
    const Slice& key,
    const Slice* existing_value,
    const Slice& value,
    std::string* new_value,
    Logger* logger
  ) const override {
    // Append with delimiter
    if (existing_value != nullptr) {
      new_value->assign(existing_value->data(), existing_value->size());
      new_value->append(",");
    } else {
      new_value->clear();
    }
    new_value->append(value.data(), value.size());
    return true;
  }

  const char* Name() const override {
    return "StringAppendOperator";
  }
};

// Usage
Options options;
options.merge_operator.reset(new StringAppendOperator());

DB* db;
DB::Open(options, "/tmp/testdb", &db);

// Append strings
db->Merge(WriteOptions(), "list", "apple");
db->Merge(WriteOptions(), "list", "banana");
db->Merge(WriteOptions(), "list", "cherry");

// Read result
std::string value;
db->Get(ReadOptions(), "list", &value);
printf("List: %s\n", value.c_str());  // "apple,banana,cherry"

Example: Complex JSON Merge

#include "rocksdb/merge_operator.h"
#include <json/json.h>  // Using jsoncpp

using namespace ROCKSDB_NAMESPACE;

class JsonMergeOperator : public MergeOperator {
 public:
  virtual bool FullMergeV3(
    const MergeOperationInputV3& merge_in,
    MergeOperationOutputV3* merge_out
  ) const override {
    Json::Value base;
    Json::Reader reader;

    // Parse existing value
    if (std::holds_alternative<Slice>(merge_in.existing_value)) {
      const Slice& existing = std::get<Slice>(merge_in.existing_value);
      if (!reader.parse(existing.ToString(), base)) {
        return false;
      }
    }

    // Apply each operand
    for (const auto& operand : merge_in.operand_list) {
      Json::Value update;
      if (!reader.parse(operand.ToString(), update)) {
        return false;
      }

      // Merge JSON objects
      for (const auto& key : update.getMemberNames()) {
        base[key] = update[key];
      }
    }

    // Serialize result
    Json::FastWriter writer;
    merge_out->new_value = writer.write(base);
    return true;
  }

  virtual bool PartialMerge(
    const Slice& key,
    const Slice& left_operand,
    const Slice& right_operand,
    std::string* new_value,
    Logger* logger
  ) const override {
    // JSON merge is not associative, so we can't do partial merge
    return false;
  }

  const char* Name() const override {
    return "JsonMergeOperator";
  }
};

Advanced Features

ShouldMerge

virtual bool ShouldMerge(
  const std::vector<Slice>& operands
) const;

Controls when to invoke full merge during Get operations.

const std::vector<Slice>&

Merge operands in reversed order (most recent first).

bool

Returns true to trigger merge immediately.

AllowSingleOperand

virtual bool AllowSingleOperand() const;

Returns true if PartialMerge can be called with a single operand.

bool

Returns true to enable single-operand partial merge.

Failure Scopes

OpFailureScope

enum class OpFailureScope {
  kDefault,     // Fallback to kTryMerge
  kTryMerge,    // Fail operations that try to merge (flush, compaction)
  kMustMerge    // Fail operations that must merge (Get, iteration)
};

OpFailureScope

Flush and compaction fail, putting the DB in read-only mode.

OpFailureScope

Get and iteration fail, but flush/compaction can proceed by copying operands.

Performance Considerations

When to Use Merge

Good for:

Counters and statistics
Append-only data structures
Associative operations (sum, max, min)
High-frequency updates to the same key

Not ideal for:

Non-associative operations requiring all history
Operations needing full value for every update
Simple Put operations

Partial Merge Benefits

Implementing PartialMerge can significantly improve performance by:

Reducing full merge invocations
Merging operands during compaction
Reducing memory usage during Get

Memory Usage

Merge operands are stored until compaction or Get:

Many small operands can accumulate
Use ShouldMerge to limit operand stack size
Consider partial merge to reduce stack growth

Error Handling

Always validate input:

virtual bool FullMergeV3(
  const MergeOperationInputV3& merge_in,
  MergeOperationOutputV3* merge_out
) const override {
  // Validate operands
  for (const auto& operand : merge_in.operand_list) {
    if (operand.size() != sizeof(int64_t)) {
      // Set failure scope
      merge_out->op_failure_scope = OpFailureScope::kTryMerge;
      return false;
    }
  }
  // ... perform merge
  return true;
}

Database

Read & Write

Transactions

Storage

Utilities

MergeOperator

Overview

AssociativeMergeOperator

Interface

Merge Method

Example: Counter Merge Operator

MergeOperator

FullMergeV3

MergeOperationInputV3

MergeOperationOutputV3

PartialMerge

PartialMergeMulti

Example: String Append Merge Operator

Example: Complex JSON Merge

Advanced Features

ShouldMerge

AllowSingleOperand

Failure Scopes

OpFailureScope

Performance Considerations

When to Use Merge

Partial Merge Benefits

Memory Usage

Error Handling

See Also

​Overview

​AssociativeMergeOperator

​Interface

​Merge Method

​Example: Counter Merge Operator

​MergeOperator

​FullMergeV3

​MergeOperationInputV3

​MergeOperationOutputV3

​PartialMerge

​PartialMergeMulti

​Example: String Append Merge Operator

​Example: Complex JSON Merge

​Advanced Features

​ShouldMerge

​AllowSingleOperand

​Failure Scopes

​OpFailureScope

​Performance Considerations

​When to Use Merge

​Partial Merge Benefits

​Memory Usage

​Error Handling

​See Also

Overview

AssociativeMergeOperator

Interface

Merge Method

Example: Counter Merge Operator

MergeOperator

FullMergeV3

MergeOperationInputV3

MergeOperationOutputV3

PartialMerge

PartialMergeMulti

Example: String Append Merge Operator

Example: Complex JSON Merge

Advanced Features

ShouldMerge

AllowSingleOperand

Failure Scopes

OpFailureScope

Performance Considerations

When to Use Merge

Partial Merge Benefits

Memory Usage

Error Handling

See Also