Troubleshoot common PGO and PostgreSQL cluster issues

Most PGO problems fall into a small set of categories: image pull failures, misconfigured storage, credential or TLS issues, and resource pressure. The diagnostic commands below apply to nearly every investigation. Start there, then work through the scenario that matches your symptom.

Essential diagnostic commands

Run these commands first whenever something is wrong with a cluster. Replace <namespace> and <cluster-name> with your values.

# Describe the PostgresCluster resource — check Status.Conditions for error messages
kubectl describe postgrescluster <cluster-name> -n <namespace>

# List all pods belonging to the cluster
kubectl get pods -n <namespace> \
  -l postgres-operator.crunchydata.com/cluster=<cluster-name>

# Stream logs from the database container on a specific instance pod
kubectl logs <pod-name> -n <namespace> -c database

# Stream logs from the pgBackRest container
kubectl logs <pod-name> -n <namespace> -c pgbackrest

# Show recent events in the namespace — includes scheduler, image pull, and OOM events
kubectl get events -n <namespace> --sort-by='.lastTimestamp'

# Show the operator's own logs — includes reconciliation errors and feature gate state
kubectl logs -n postgres-operator deployment/pgo --tail=100

Cluster stuck in “Creating” state

A cluster that never reaches a running state usually has one of three root causes: an image that cannot be pulled, a PVC that cannot be provisioned, or a pod that fails its readiness probe. Step 1 — Check pod status.

kubectl get pods -n <namespace> \
  -l postgres-operator.crunchydata.com/cluster=<cluster-name>

Look for pods in Pending, Init:Error, or CrashLoopBackOff state. Step 2 — Describe the stuck pod.

kubectl describe pod <pod-name> -n <namespace>

Examine the Events section at the bottom. Common findings:

Event message	Cause
`Failed to pull image … : unauthorized`	The image pull secret is missing or misconfigured.
`Failed to pull image … : not found`	The image tag or repository path is wrong.
`0/3 nodes are available: … Insufficient memory`	The cluster does not have enough schedulable resources.
`persistentvolumeclaim "…" not found`	The PVC was not created — see the PVC pending section below.

Step 3 — Check the image pull secret.

kubectl get secret <pull-secret-name> -n <namespace> -o jsonpath='{.type}'
# should output: kubernetes.io/dockerconfigjson

If the secret is missing or has the wrong type, recreate it with the correct credentials from the Crunchy Data Developer Portal.

The PostgresCluster status conditions surface reconciliation errors directly. Run kubectl get postgrescluster <name> -n <namespace> -o jsonpath='{.status.conditions}' to see a machine-readable summary.

Backup job failing

pgBackRest backup jobs fail for three main reasons: incorrect repository configuration (wrong bucket name, path, or region), missing or incorrect cloud credentials, and network connectivity problems between the backup pod and the object storage endpoint. Step 1 — Find the failing backup pod.

kubectl get pods -n <namespace> \
  -l postgres-operator.crunchydata.com/cluster=<cluster-name>,postgres-operator.crunchydata.com/pgbackrest-backup=true

Step 2 — Read the pgBackRest logs.

kubectl logs <backup-pod-name> -n <namespace>

pgBackRest error messages are verbose. Look for lines beginning with ERROR or FATAL. Step 3 — Check the Secret containing cloud credentials.

kubectl get secret <cluster-name>-pgbackrest-secret -n <namespace> -o yaml

For S3-compatible storage, verify that s3-key, s3-key-secret, and (if applicable) s3-endpoint are present and correctly base64-encoded. Step 4 — Verify network access from inside the backup pod.

kubectl exec -n <namespace> <backup-pod-name> -- \
  curl -I https://<s3-endpoint>/<bucket-name>/

A 403 Forbidden response means the bucket exists but the credentials are wrong. A connection timeout or DNS failure points to a network policy or endpoint misconfiguration.

If you are using a self-hosted S3-compatible store (MinIO, Ceph RGW), ensure the endpoint URL in the pgBackRest configuration uses https:// and that the CA certificate is trusted by the backup pod. You may need to mount a custom CA into the pod.

High availability failover not happening

Patroni manages failover. If a primary pod fails and Patroni does not elect a new leader within 30–60 seconds, investigate the Patroni state directly. Step 1 — Check Patroni’s view of the cluster.

kubectl exec -n <namespace> <any-instance-pod> -c database -- \
  patronictl -c /etc/patroni/postgres.yaml list

A healthy output shows one member with Role: Leader and the rest as Role: Replica with State: running. If all members show State: stopped or Role: standby leader, Patroni has lost quorum or cannot reach the DCS. Step 2 — Check for network policy blocking DCS traffic. Patroni uses Kubernetes API endpoints (not a separate DCS in PGO) for distributed consensus. If the instance pods cannot reach the Kubernetes API server, failover will stall. Test from inside a pod:

kubectl exec -n <namespace> <pod-name> -c database -- \
  curl -k https://kubernetes.default.svc/healthz

Step 3 — Review pod anti-affinity settings. If all instance pods are scheduled on the same node and that node goes down, no replicas are available to take over. Verify anti-affinity is configured:

kubectl get pod <pod-name> -n <namespace> \
  -o jsonpath='{.spec.affinity.podAntiAffinity}'

PGO sets preferred anti-affinity by default. For stricter placement guarantees, configure required anti-affinity rules in spec.instances[].affinity. See the high availability guide for examples.

Cannot connect to PostgreSQL

Connection failures usually stem from one of four causes: the service endpoint is not resolving, TLS certificate validation is failing, credentials are wrong, or a network policy is blocking the connection. Step 1 — Verify the service exists and has an endpoint.

kubectl get service -n <namespace> \
  -l postgres-operator.crunchydata.com/cluster=<cluster-name>

kubectl get endpoints -n <namespace> \
  -l postgres-operator.crunchydata.com/cluster=<cluster-name>

If the Endpoints object shows <none>, the primary pod is not ready or its labels do not match the service selector. Step 2 — Retrieve the connection Secret. PGO stores connection credentials in a Secret named <cluster-name>-pguser-<username>:

kubectl get secret <cluster-name>-pguser-<username> -n <namespace> \
  -o jsonpath='{.data}' | jq 'to_entries[] | .key + "=" + (.value | @base64d)'

The Secret contains host, port, dbname, user, password, and the TLS certificate fields. Step 3 — Test connectivity from inside the cluster.

kubectl run pg-test --rm -it --image=postgres:16 --restart=Never -- \
  psql "$(kubectl get secret <cluster-name>-pguser-<username> \
    -n <namespace> -o jsonpath='{.data.uri}' | base64 -d)"

Step 4 — Check TLS certificate validity. PGO enforces TLS on all connections. If you are connecting with sslmode=require but not supplying the cluster CA, the connection will fail certificate verification. Retrieve the CA from the Secret or disable host verification temporarily with sslmode=require (not sslmode=verify-full) to isolate the issue. See the TLS guide for how to manage and rotate certificates.

OOMKilled pods

A pod with OOMKilled in its last state has exceeded its memory limit. For PostgreSQL pods, this usually means shared_buffers, work_mem, or other memory parameters are set too high for the pod’s limit. Step 1 — Identify which container was killed.

kubectl describe pod <pod-name> -n <namespace> | grep -A 5 "Last State"

Step 2 — Check current resource requests and limits.

kubectl get pod <pod-name> -n <namespace> \
  -o jsonpath='{.spec.containers[*].resources}'

Step 3 — Adjust resource limits. Increase the memory limit in spec.instances[].resources or reduce PostgreSQL memory parameters in spec.patroni.dynamicConfiguration:

spec:
  instances:
    - name: instance1
      resources:
        requests:
          memory: "2Gi"
        limits:
          memory: "4Gi"
  patroni:
    dynamicConfiguration:
      postgresql:
        parameters:
          shared_buffers: "1GB"
          work_mem: "16MB"

As a rule of thumb, set shared_buffers to 25% of the container memory limit and work_mem conservatively — it is allocated per sort operation per connection, so high concurrency multiplies the footprint.

The crunchy-postgres-exporter sidecar (pgMonitor) also consumes memory. Account for its limit when sizing the pod. See the monitoring guide for exporter resource configuration.

PersistentVolumeClaim stuck in Pending state

A PVC that stays in Pending means the Kubernetes volume provisioner cannot satisfy the storage request. Step 1 — Describe the PVC.

kubectl describe pvc <pvc-name> -n <namespace>

Look for ProvisioningFailed events. Common causes:

Message	Cause
`no persistent volumes available`	No pre-provisioned PV matches the request (static provisioning only).
`storageclass.storage.k8s.io "…" not found`	The StorageClass name in your spec does not exist.
`exceeded quota`	The namespace has a `ResourceQuota` that limits total storage.
`Topology … does not match`	The PVC is in a zone where the StorageClass cannot provision.

Step 2 — Verify the StorageClass exists and is the default.

kubectl get storageclass

A StorageClass marked (default) is used when spec.instances[].dataVolumeClaimSpec.storageClassName is omitted. If no default exists, you must specify one explicitly. Step 3 — Check available capacity. For local-path or hostPath provisioners, verify that the target node has sufficient free disk space. For cloud provisioners, confirm you have not hit a regional volume quota.

FAQ

Answers to common questions about PGO setup and operations.

High availability guide

Configure Patroni, anti-affinity, and synchronous replication.

Backups guide

Set up and validate pgBackRest backup repositories.

TLS guide

Manage certificates and troubleshoot TLS connection failures.

Custom Resources

Configuration

Troubleshooting

Troubleshoot common PGO and PostgreSQL cluster issues

Essential diagnostic commands

Cluster stuck in “Creating” state

Backup job failing

High availability failover not happening

Cannot connect to PostgreSQL

OOMKilled pods

PersistentVolumeClaim stuck in Pending state

FAQ

High availability guide

Backups guide

TLS guide

​Essential diagnostic commands

​Cluster stuck in “Creating” state

​Backup job failing

​High availability failover not happening

​Cannot connect to PostgreSQL

​OOMKilled pods

​PersistentVolumeClaim stuck in Pending state

​Related pages

FAQ

High availability guide

Backups guide

TLS guide

Essential diagnostic commands

Cluster stuck in “Creating” state

Backup job failing

High availability failover not happening

Cannot connect to PostgreSQL

OOMKilled pods

PersistentVolumeClaim stuck in Pending state

Related pages