Monitor PostgreSQL clusters using PGO and pgMonitor

PGO integrates with pgMonitor to give you production-grade observability for your PostgreSQL clusters. When enabled, the operator injects a crunchy-postgres-exporter sidecar into each instance pod. That sidecar exposes PostgreSQL metrics on port 9187 in Prometheus format, which Prometheus can scrape and Grafana can visualize using pre-built dashboards.

Enable the exporter

Add a monitoring.pgmonitor.exporter section to your PostgresCluster spec to activate the metrics sidecar.

apiVersion: postgres-operator.crunchydata.com/v1beta1
kind: PostgresCluster
metadata:
  name: hippo
spec:
  postgresVersion: 16
  instances:
    - name: instance1
      dataVolumeClaimSpec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
  monitoring:
    pgmonitor:
      exporter: {}

The exporter field accepts the following options.

ExporterSpec fields

Field	Type	Description
`image`	string	Container image for the `crunchy-postgres-exporter` sidecar. Defaults to the `RELATED_IMAGE_PGEXPORTER` operator environment variable.
`resources`	ResourceRequirements	CPU and memory requests/limits for the exporter container. Changing this value causes PostgreSQL and the exporter to restart.
`configuration`	[]VolumeProjection	Projected volumes containing custom query files. A `queries.yml` file detected in any projected volume is loaded via the `--extend.query-path` flag. Changing this causes PostgreSQL and the exporter to restart.
`customTLSSecret`	SecretProjection	Projected secret with custom TLS certificates to encrypt output from the exporter web server.

Set resource limits

monitoring:
  pgmonitor:
    exporter:
      resources:
        requests:
          cpu: 10m
          memory: 32Mi
        limits:
          memory: 64Mi

Add custom queries

To extend the default query set with your own SQL, create a ConfigMap containing a queries.yml file and reference it via configuration.

# configmap with custom queries
apiVersion: v1
kind: ConfigMap
metadata:
  name: my-custom-queries
data:
  queries.yml: |
    pg_replication_lag:
      query: "SELECT EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp()))::INT AS seconds"
      metrics:
        - seconds:
            usage: "GAUGE"
            description: "Replication lag in seconds"

monitoring:
  pgmonitor:
    exporter:
      configuration:
        - configMap:
            name: my-custom-queries

The AppendCustomQueries feature gate (alpha, disabled by default) allows you to append custom queries to the default pgMonitor query set rather than replacing them. See Feature Gates for how to enable it.

Deploy Prometheus, Grafana, and Alertmanager

PGO ships a reference monitoring stack under config/monitoring/ in the operator repository. It deploys Prometheus, Grafana, and Alertmanager into a namespace using Kustomize.

Clone or download the monitoring config

The monitoring configuration lives at config/monitoring/ in the postgres-operator repository.

Set the target namespace

Open config/monitoring/kustomization.yaml and confirm the namespace field matches where you want to deploy the monitoring stack.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: postgres-operator
resources:
  - grafana
  - prometheus
  - alertmanager

Apply the configuration

kubectl apply -k config/monitoring/

If you are deploying on OpenShift, comment out the fsGroup line under securityContext in alertmanager/deployment.yaml, grafana/deployment.yaml, and prometheus/deployment.yaml before applying.

Configure Prometheus scraping

The reference prometheus.yml included in the monitoring config already contains the correct scrape job for the pgMonitor exporter. It selects pods using the label postgres-operator.crunchydata.com/crunchy-postgres-exporter=true, which PGO automatically applies to every exporter pod.

- job_name: 'crunchy-postgres-exporter'
  kubernetes_sd_configs:
    - role: pod
      selectors:
        - role: pod
          label: postgres-operator.crunchydata.com/crunchy-postgres-exporter=true
  relabel_configs:
    - source_labels: [__meta_kubernetes_pod_container_port_number]
      action: keep
      regex: 9187
    - source_labels: [__meta_kubernetes_namespace]
      target_label: kubernetes_namespace
    - source_labels: [__meta_kubernetes_pod_name]
      target_label: pod
    - source_labels: [__meta_kubernetes_namespace,__meta_kubernetes_pod_label_postgres_operator_crunchydata_com_cluster]
      target_label: pg_cluster
      separator: ":"
      replacement: '$1$2'

Prometheus requires a service account with permission to list and watch pods in the namespaces where your clusters run.

OpenTelemetry metrics (alpha)

PGO supports exporting metrics via an OpenTelemetry collector sidecar as an alternative to the pgMonitor exporter. This path is controlled by the OpenTelemetryMetrics feature gate, which is disabled by default (alpha stage). When enabled, configure the collector using spec.instrumentation:

spec:
  instrumentation:
    metrics:
      exporters:
        - prometheus

The reference Prometheus config includes a separate crunchy-otel-collector scrape job that selects pods with the label postgres-operator.crunchydata.com/crunchy-otel-collector=true.

See Feature Gates for full details on enabling OpenTelemetryMetrics and the related OpenTelemetryLogs gate.

Get Started

Installation

Core Operations

Day-Two Operations

Security

Monitor PostgreSQL clusters using PGO and pgMonitor

Enable the exporter

ExporterSpec fields

Set resource limits

Add custom queries

Deploy Prometheus, Grafana, and Alertmanager

Configure Prometheus scraping

OpenTelemetry metrics (alpha)

Next steps

Resize cluster resources

Feature gates reference

​Enable the exporter

​ExporterSpec fields

​Set resource limits

​Add custom queries

​Deploy Prometheus, Grafana, and Alertmanager

​Configure Prometheus scraping

​OpenTelemetry metrics (alpha)

​Next steps

Resize cluster resources

Feature gates reference

Enable the exporter

ExporterSpec fields

Set resource limits

Add custom queries

Deploy Prometheus, Grafana, and Alertmanager

Configure Prometheus scraping

OpenTelemetry metrics (alpha)

Next steps