Disaster recovery: restore, clone, and PITR for Postgres

PGO supports several patterns for recovering data and restoring service after failures. You can restore a cluster in place using the existing backup infrastructure, bootstrap an entirely new cluster from a pgBackRest backup or an existing cluster, perform point-in-time recovery to any moment covered by WAL archives, or run a live standby cluster that can be promoted with a single spec change. This page covers each pattern with working YAML examples derived directly from the operator’s spec.

In-place restore

An in-place restore re-initializes a running PostgresCluster from one of its own pgBackRest repositories. The cluster is shut down, the data directory is cleared, and pgBackRest restores the selected backup.

Enable the restore in the spec

Set spec.backups.pgbackrest.restore.enabled: true and specify the source repo.

spec:
  backups:
    pgbackrest:
      restore:
        enabled: true
        repoName: repo1

Apply the manifest

kubectl apply -f postgres.yaml

The operator creates a restore Job. Monitor its progress:

kubectl get jobs \
  -l postgres-operator.crunchydata.com/pgbackrest=restore \
  -l postgres-operator.crunchydata.com/cluster=hippo

kubectl logs -l postgres-operator.crunchydata.com/pgbackrest=restore \
  -l postgres-operator.crunchydata.com/cluster=hippo --tail=100

Disable the restore after completion

Once the restore is complete, set enabled: false to prevent accidental re-execution on the next reconcile.

spec:
  backups:
    pgbackrest:
      restore:
        enabled: false
        repoName: repo1

An in-place restore is destructive. All data in the cluster is overwritten. Confirm you are targeting the correct cluster and backup before applying.

Point-in-time recovery (PITR)

PITR lets you restore to any moment covered by continuous WAL archives in your pgBackRest repository. Pass pgBackRest restore options through spec.backups.pgbackrest.restore.options.

spec:
  backups:
    pgbackrest:
      restore:
        enabled: true
        repoName: repo1
        options:
          - --type=time
          - --target="2024-06-15 14:30:00+00"
          - --target-action=promote

The --target-action=promote option tells PostgreSQL to promote the instance to read-write once it reaches the target recovery time. Without it, the instance remains in recovery and accepts no writes.

Available --type options match pgBackRest’s restore command:

Type	Description
`default`	Restore to the end of the archive (most recent consistent state)
`time`	Restore to a specific timestamp (`--target`)
`xid`	Restore to a specific transaction ID
`lsn`	Restore to a specific log sequence number
`name`	Restore to a named restore point created with `pg_create_restore_point()`
`immediate`	Stop recovery as soon as a consistent state is reached

Bootstrap a new cluster from backup

spec.dataSource lets you pre-populate a brand-new PostgresCluster with data from a pgBackRest repository before the cluster accepts connections. This is the recommended way to stand up a clone or a replacement cluster from an off-site backup.

From a cloud repository
From an existing PostgresCluster

Use spec.dataSource.pgbackrest to restore from S3, GCS, or Azure into a new cluster. This is the only data source option that works with cloud (non-volume) repositories.

apiVersion: postgres-operator.crunchydata.com/v1beta1
kind: PostgresCluster
metadata:
  name: hippo-restore
spec:
  postgresVersion: 16
  dataSource:
    pgbackrest:
      stanza: db
      configuration:
        - secret:
            name: hippo-s3-creds
      global:
        repo1-s3-key: aws-s3-key
        repo1-s3-key-secret: aws-s3-key-secret
      repo:
        name: repo1
        s3:
          bucket: my-postgres-backups
          endpoint: s3.us-east-1.amazonaws.com
          region: us-east-1
  instances:
    - name: instance1
      dataVolumeClaimSpec:
        accessModes:
          - "ReadWriteOnce"
        resources:
          requests:
            storage: 50Gi
  backups:
    pgbackrest:
      repos:
        - name: repo1
          volume:
            volumeClaimSpec:
              accessModes:
                - "ReadWriteOnce"
              resources:
                requests:
                  storage: 50Gi

Use spec.dataSource.postgresCluster to bootstrap from a backup stored in an existing cluster’s pgBackRest repository. The source cluster must be in the same namespace, or you must specify clusterNamespace.

apiVersion: postgres-operator.crunchydata.com/v1beta1
kind: PostgresCluster
metadata:
  name: hippo-clone
spec:
  postgresVersion: 16
  dataSource:
    postgresCluster:
      clusterName: hippo
      repoName: repo1
  instances:
    - name: instance1
      dataVolumeClaimSpec:
        accessModes:
          - "ReadWriteOnce"
        resources:
          requests:
            storage: 50Gi
  backups:
    pgbackrest:
      repos:
        - name: repo1
          volume:
            volumeClaimSpec:
              accessModes:
                - "ReadWriteOnce"
              resources:
                requests:
                  storage: 50Gi

PITR when bootstrapping a new cluster

You can combine spec.dataSource with a point-in-time target by passing pgBackRest options through the options field.

spec:
  dataSource:
    postgresCluster:
      clusterName: hippo
      repoName: repo1
      options:
        - --type=time
        - --target="2024-06-15 08:00:00+00"
        - --target-action=promote

Clone a cluster using existing volumes

If you have PVCs from a stopped or deleted cluster, you can reuse them directly with spec.dataSource.volumes. This avoids a full backup-based restore and is faster for large datasets.

spec:
  dataSource:
    volumes:
      pgDataVolume:
        pvcName: hippo-instance1-0-pgdata
        directory: pgdata
      pgWALVolume:
        pvcName: hippo-instance1-0-pgwal
        directory: pg_wal
      pgBackRestVolume:
        pvcName: hippo-repo1
        directory: pgbackrest/repo1

The source PVCs must not be mounted by any running pods. Stop or delete the source cluster before reusing its volumes.

Standby cluster for live DR

A standby cluster continuously streams WAL from a pgBackRest repository and can be promoted to an independent primary at any time. This provides near-zero RPO for cross-cluster or cross-region disaster recovery.

apiVersion: postgres-operator.crunchydata.com/v1beta1
kind: PostgresCluster
metadata:
  name: hippo-dr
spec:
  postgresVersion: 16
  standby:
    enabled: true
    repoName: repo1   # Must match a repo in the primary cluster's pgBackRest config
  instances:
    - name: instance1
      replicas: 2
      dataVolumeClaimSpec:
        accessModes:
          - "ReadWriteOnce"
        resources:
          requests:
            storage: 50Gi
  backups:
    pgbackrest:
      configuration:
        - secret:
            name: hippo-s3-creds
      global:
        repo1-s3-key: aws-s3-key
        repo1-s3-key-secret: aws-s3-key-secret
      repos:
        - name: repo1
          s3:
            bucket: my-postgres-backups
            endpoint: s3.us-east-1.amazonaws.com
            region: us-east-1

Promote the standby

To take over traffic after a primary-side failure or for a planned migration:

Set standby.enabled to false

spec:
  standby:
    enabled: false

Apply the manifest

kubectl apply -f hippo-dr.yaml

PGO stops WAL replay and Patroni promotes one instance to primary. The cluster will begin accepting writes.

Update your application's connection string

Point your application to the new primary’s Service (hippo-dr-primary) or pgBouncer endpoint.

If the original primary cluster is still running when you promote the standby, you will have two writable clusters pointing at the same data. Ensure the original primary is offline or network-isolated before promoting the standby to avoid a split-brain scenario.

Verify recovery

After any restore or clone operation, confirm data integrity before returning to production traffic:

# Connect to the restored cluster
psql "$(kubectl get secret hippo-restore-pguser-hippo-restore \
  -o jsonpath='{.data.uri}' | base64 -d)"

# Check the recovery target timestamp
SELECT now(), pg_is_in_recovery();

# Verify key tables and row counts
SELECT schemaname, tablename, n_live_tup
FROM pg_stat_user_tables
ORDER BY n_live_tup DESC LIMIT 20;

Backups

Configure pgBackRest repos, schedules, and retention before a restore is needed.

High availability

Set up Patroni-managed automatic failover within a single cluster.

Create a cluster

Bootstrap a fresh PostgresCluster with the right storage configuration.

Custom Resource Reference

Full field reference for spec.dataSource, spec.standby, and spec.backups.pgbackrest.restore.

Get Started

Installation

Core Operations

Day-Two Operations

Security

Disaster recovery: restore, clone, and PITR for Postgres

In-place restore

Point-in-time recovery (PITR)

Bootstrap a new cluster from backup

PITR when bootstrapping a new cluster

Clone a cluster using existing volumes

Standby cluster for live DR

Promote the standby

Verify recovery

Backups

High availability

Create a cluster

Custom Resource Reference

​In-place restore

​Point-in-time recovery (PITR)

​Bootstrap a new cluster from backup

​PITR when bootstrapping a new cluster

​Clone a cluster using existing volumes

​Standby cluster for live DR

​Promote the standby

​Verify recovery

Backups

High availability

Create a cluster

Custom Resource Reference

In-place restore

Point-in-time recovery (PITR)

Bootstrap a new cluster from backup

PITR when bootstrapping a new cluster

Clone a cluster using existing volumes

Standby cluster for live DR

Promote the standby

Verify recovery