Server Groups

Overview

Server Groups let you organize multiple servers into logical collections and operate on them as a unit. Use groups to target events (run jobs against a group rather than specific hosts), aggregate monitoring views, provide default alert actions, and create group snapshots.

A single server can belong to multiple groups, providing flexible organization across environments, roles, and regions.

Key Features

Event Targeting

Run jobs against groups instead of individual servers using selection algorithms

Automatic Assignment

Servers join groups automatically via hostname regex matching

Aggregated Views

Monitor all group members with unified metrics and status dashboards

Default Alerts

Configure alert actions that apply to all servers in the group

Creating Groups

Navigate to Groups

Go to Sidebar → Monitoring → Groups

Define Group Properties

Click “New Group…” and configure:

{
  "id": "prod_web",
  "title": "Production Web Servers",
  "icon": "server",
  "hostname_match": "^prod-web-\\d+$"
}

Set Alert Actions (Optional)

Configure default actions for alerts on group members:

"alert_actions": [
  {
    "enabled": true,
    "condition": "alert_new",
    "type": "email",
    "users": ["ops"]
  },
  {
    "enabled": true,
    "condition": "alert_cleared",
    "type": "web_hook",
    "web_hook": "slack_ops"
  }
]

Save and Apply

The group is created and hostname matching runs immediately

Group Membership

Servers can join groups through automatic or manual assignment:

Automatic Assignment

Groups use regular expressions to automatically match servers by hostname:

Match All Production Servers

{
  "id": "production",
  "title": "Production",
  "hostname_match": "^prod-"
}

Match Specific Pattern

{
  "id": "db_cluster",
  "title": "Database Cluster",
  "hostname_match": "^db-(primary|replica)-\\d+$"
}

Match Everything

{
  "id": "all_servers",
  "title": "All Servers",
  "hostname_match": ".+"
}

Automatic assignment re-evaluates whenever:

A server connects or changes hostname
Group definitions are created, updated, or deleted

Manual Assignment

Override automatic assignment for specific servers:

Open any Server page
Click “Edit Server…”
Set Groups explicitly
Auto Group behavior is disabled for that server

To restore automatic assignment:

Clear the server’s manual group list
Re-enable Auto Group in the server editor

Manual assignment disables automatic hostname matching for that server.

Event Targeting

Target events at groups instead of individual servers for dynamic server selection:

Selection Algorithms

Pick a random server from eligible group members.

{
  "targets": ["prod_web"],
  "algo": "random"
}

Best for: Load distribution across identical servers

Cycle through eligible servers, maintaining state across runs.

{
  "targets": ["prod_web"],
  "algo": "round_robin"
}

Best for: Fair distribution over time

Sort servers by hostname and pick first or last.

{
  "targets": ["prod_web"],
  "algo": "prefer_first"
}

Best for: Predictable server selection

Pick the server with lowest CPU load.

{
  "targets": ["prod_web"],
  "algo": "least_cpu"
}

Best for: Load-aware job placement

Pick the server with lowest memory usage.

{
  "targets": ["prod_web"],
  "algo": "least_mem"
}

Best for: Memory-intensive jobs

Pick the server with lowest value for a specific monitor.

{
  "targets": ["prod_web"],
  "algo": "monitor:queue_depth"
}

Best for: Custom metrics-based selection

Eligibility Rules

Servers must meet these criteria to be selected:

Online - Currently connected to xyOps
Enabled - Not administratively disabled
Not excluded by alerts - No active alerts with “limit jobs” enabled

If no eligible servers exist, the job can be queued (if configured) or fails immediately.

Default Alert Actions

Groups can define actions that run when alerts fire or clear on any member server:

Example Alert Actions

{
  "id": "critical_prod",
  "title": "Critical Production Servers",
  "hostname_match": "^prod-critical-",
  "alert_actions": [
    {
      "enabled": true,
      "condition": "alert_new",
      "type": "channel",
      "channel_id": "sev1"
    },
    {
      "enabled": true,
      "condition": "alert_cleared",
      "type": "email",
      "users": ["ops"],
      "email": "ops-team@example.com"
    }
  ]
}

Action Merging

For any alert, the final action list merges:

Alert definition’s own actions
All group default actions (for each group the server belongs to)
Universal alert actions from configuration

Actions are deduplicated by type + target (same email recipients, same webhook ID, etc.).

Group Views

Each group has a dedicated dashboard aggregating all member servers:

Summary

Group details, server count, and fleet breakdown (architectures, OSes, CPU types)

Server Table

All group members with live status, resource usage, and running jobs

Quick Look Charts

Real-time per-second metrics for CPU, memory, disk, and network (last minute)

Monitors

Last hour of configured monitors across all servers with filtering

Processes & Connections

Aggregated process list and network connections with filters

Jobs & Upcoming Jobs

Active jobs running on group members and predicted future jobs

Alerts

Active alerts affecting any member server

Snapshots and Watches

Capture group state for historical analysis or troubleshooting:

Manual Snapshots

Take an on-demand snapshot of the group:

POST /api/app/create_group_snapshot/v1

{
  "group": "prod_web"
}

Includes:

Group definition and member servers
Last minute of quick metrics per server
Active jobs on those servers
Alerts affecting the group

Group Watches

Automate snapshots every minute for a duration:

POST /api/app/watch_group/v1

Start 30-minute watch

{
  "id": "prod_web",
  "duration": 1800
}

Cancel watch

{
  "id": "prod_web",
  "duration": 0
}

Use watches during deployment windows or troubleshooting to capture continuous state.

API Reference

List All Groups

GET /api/app/get_groups/v1

Get Single Group

GET /api/app/get_group/v1?id=prod_web

Create Group

POST /api/app/create_group/v1

Request Body

{
  "title": "Production Web Servers",
  "hostname_match": "^prod-web-",
  "icon": "server",
  "alert_actions": []
}

ID is auto-generated using pattern g<random>. Optionally provide your own ID.

Update Group

POST /api/app/update_group/v1

Request Body

{
  "id": "prod_web",
  "title": "Production Web Tier",
  "hostname_match": "^prod-web-\\d+$"
}

Server group membership is automatically recalculated after updates.

Delete Group

POST /api/app/delete_group/v1

Request Body

{
  "id": "prod_web"
}

Reorder Groups

POST /api/app/multi_update_group/v1

Update sort order for multiple groups in one call.

Permissions

These privileges control group management:

create_groups - Create new groups
edit_groups - Modify existing groups
delete_groups - Remove groups
create_snapshots - Take group snapshots and set watches

Best Practices

Use Clear Hostname Patterns

Establish consistent hostname conventions for reliable automatic grouping:

prod-web-01, prod-web-02 (environment-role-number)
us-east-db-primary (region-role-function)

This makes regex matching straightforward and maintainable.

Multiple Groups Per Server

Leverage overlapping groups for orthogonal concepts:

Environment: production, staging
Role: web, database, cache
Region: us-east, eu-west

A server like prod-web-us-east-01 can belong to all three.

Centralize Alert Actions

Use group default alert actions to avoid repeating configuration on every server or alert definition.

Choose Appropriate Algorithms

Match your selection algorithm to job characteristics:

random or round_robin for stateless tasks
least_cpu or least_mem for resource-intensive jobs
monitor:<id> for custom capacity metrics

Monitor Group Health

Use group views to quickly assess fleet health and identify outliers in CPU, memory, or alerts.

Get Started

Core Concepts

Monitoring & Alerts

Organization

Operations

Plugins & Extensions

Advanced

Server Groups

Overview

Key Features

Event Targeting

Automatic Assignment

Aggregated Views

Default Alerts

Creating Groups

Group Membership

Automatic Assignment

Manual Assignment

Event Targeting

Selection Algorithms

Eligibility Rules

Default Alert Actions

Action Merging

Group Views

Snapshots and Watches

Manual Snapshots

Group Watches

API Reference

List All Groups

Get Single Group

Create Group

Update Group

Delete Group

Reorder Groups

Permissions

Best Practices

Build docs developers (and LLMs) love

Get Started

Core Concepts

Monitoring & Alerts

Organization

Operations

Plugins & Extensions

Advanced

Documentation Index

​Overview

​Key Features

Event Targeting

Automatic Assignment

Aggregated Views

Default Alerts

​Creating Groups

​Group Membership

​Automatic Assignment

​Manual Assignment

​Event Targeting

​Selection Algorithms

​Eligibility Rules

​Default Alert Actions

​Action Merging

​Group Views

​Snapshots and Watches

​Manual Snapshots

​Group Watches

​API Reference

​List All Groups

​Get Single Group

​Create Group

​Update Group

​Delete Group

​Reorder Groups

​Permissions

​Best Practices

Build docs developers (and LLMs) love

Overview

Key Features

Creating Groups

Group Membership

Automatic Assignment

Manual Assignment

Event Targeting

Selection Algorithms

Eligibility Rules

Default Alert Actions

Action Merging

Group Views

Snapshots and Watches

Manual Snapshots

Group Watches

API Reference

List All Groups

Get Single Group

Create Group

Update Group

Delete Group

Reorder Groups

Permissions

Best Practices