MediaWiki State: Deployment Infrastructure & Jobrunner

The mediawiki Salt state tree manages the full lifecycle infrastructure for deploying MediaWiki across the WikiOasis fleet. It is split into four sub-states: the main mediawiki state that sets up the staging server with the mwdeploy system user, deployment script, and config; mediawiki.proxy which provisions the deploy user on proxy* servers with HAProxy socket access; mediawiki.target which prepares mw* application servers to receive rsync pushes; and mediawiki.jobrunner which installs and runs the Wikimedia jobrunner and jobchron services for async job processing.

Pillar keys

All keys live under the top-level mediawiki: map in pillar/mediawiki/init.sls.

Key	Default	Description
`deploy_user`	`mwdeploy`	System user that runs all deploy operations
`staging_path`	`/srv/mediawiki-staging`	Path for the git-managed staging checkout
`prod_path`	`/srv/mediawiki`	Production MediaWiki path on each server
`canary_vhost`	`test.wikioasis.org`	Vhost used for canary validation before fleet rollout
`log_file`	`/var/log/mwdeploy.log`	Path to the mwdeploy operation log
`haproxy_backend`	`mediawiki`	HAProxy backend name to drain/restore during deploy
`haproxy_socket`	`/run/haproxy/admin.sock`	Path to the HAProxy admin socket
`backup_path`	`/srv/mediawiki-backup`	Path where the pre-deploy backup is stored
`mw_servers`	`[]`	List of application server FQDNs to push to
`proxy_servers`	`[]`	List of HAProxy server FQDNs to update routing on
`ssh_identity`	`/home/mwdeploy/.ssh/id_ed25519`	SSH private key used for all rsync/SSH operations
`deploy_ssh_public_key`	(none)	Public key to authorize on the deploy user’s account
`webhooks`	`{}`	Map of webhook targets; supports `discord` and `slack` URL keys

Production pillar values

# pillar/mediawiki/init.sls
mediawiki:
  deploy_user: mwdeploy
  staging_path: /srv/mediawiki-staging
  prod_path: /srv/mediawiki
  canary_vhost: test.wikioasis.org
  log_file: /var/log/mwdeploy.log
  haproxy_backend: mediawiki
  haproxy_socket: /run/haproxy/admin.sock
  backup_path: /srv/mediawiki-backup
  mw_servers:
    - task-us-east-011.ovvin.wonet
    - mw-us-east-011.ovvin.wonet
    - mw-us-east-012.ovvin.wonet
    - mw-us-east-021.ovvin.wonet
    - mw-us-east-022.ovvin.wonet
  proxy_servers:
    - proxy-us-east-011.ovvin.wonet
    - proxy-us-east-021.ovvin.wonet
  deploy_ssh_public_key: "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBw+m6ZQT6Q7MPgfd5STamRLSUADflff/9uKVtbBZluM mwdeploy@staging-us-east-021"
  ssh_identity: /home/mwdeploy/.ssh/id_ed25519
  webhooks:
    discord: ""
    slack: ""

mediawiki (staging server)

Applied to the staging server, this state sets up everything the mwdeploy script needs to operate.

Create the deploy user

Creates the mwdeploy system user with a home directory and shell. If deploy_ssh_public_key is set in pillar, that key is added to the user’s authorized_keys so the deploy user can be reached via SSH from the master.

Install the mwdeploy script

Manages /usr/local/bin/mwdeploy (mode 0755) from the Salt fileserver.

Write /etc/mwdeploy/config.yaml

Renders the Jinja config template with all pillar values. The deploy user is given group read access (0640, group mwdeploy).

Pre-create the log file

Creates /var/log/mwdeploy.log owned by the deploy user (replace: false — existing log is never truncated).

Create staging and production directories

Creates staging_path and prod_path owned by www-data so git operations (which run as www-data) can write to them.

Write sudoers for staging

Creates /etc/sudoers.d/mwdeploy_staging allowing the deploy user to run git and patch as www-data, rsync as root, and the mwscript.php maintenance runner as www-data.

Staging sudoers

mwdeploy ALL=(www-data) NOPASSWD: /usr/bin/git
mwdeploy ALL=(root) NOPASSWD: /usr/bin/rsync
mwdeploy ALL=(www-data) NOPASSWD: /usr/bin/patch
mwdeploy ALL=(www-data) NOPASSWD: /srv/mediawiki/scripts/mwscript.php

config.yaml template

The rendered /etc/mwdeploy/config.yaml is generated from salt://mediawiki/files/config.yaml.jinja:

# mwdeploy configuration — managed by Salt, do not edit manually
{%- set mw = salt['pillar.get']('mediawiki', {}) %}
{%- set webhooks = mw.get('webhooks', {}) %}

deploy_user: {{ mw.get('deploy_user', 'mwdeploy') }}
ssh_identity: {{ mw.get('ssh_identity', '/home/' ~ mw.get('deploy_user', 'mwdeploy') ~ '/.ssh/id_ed25519') }}
staging_path: {{ mw.get('staging_path', '/srv/mediawiki-staging') }}
prod_path: {{ mw.get('prod_path', '/srv/mediawiki') }}
canary_vhost: {{ mw.get('canary_vhost', 'test.wikioasis.org') }}
log_file: {{ mw.get('log_file', '/var/log/mwdeploy.log') }}
haproxy_backend: {{ mw.get('haproxy_backend', 'mediawiki') }}
haproxy_socket: {{ mw.get('haproxy_socket', '/run/haproxy/admin.sock') }}
backup_path: {{ mw.get('backup_path', '/srv/mediawiki-backup') }}

mw_servers:
{%- for server in mw.get('mw_servers', []) %}
  - {{ server }}
{%- endfor %}

proxy_servers:
{%- for server in mw.get('proxy_servers', []) %}
  - {{ server }}
{%- endfor %}

webhooks:
  discord: {{ webhooks.get('discord', '') | tojson }}
  slack: {{ webhooks.get('slack', '') | tojson }}

mediawiki.proxy

Applied to proxy* servers. Creates the mwdeploy system user with membership in the haproxy group so it can write to the HAProxy admin socket at /run/haproxy/admin.sock. This allows the deploy script to drain and restore backends during rolling deploys without requiring root.

salt 'proxy*' state.apply mediawiki.proxy

The mwdeploy user must be in the haproxy group before a deploy is attempted. Run mediawiki.proxy after any new proxy server is provisioned.

mediawiki.target

Applied to mw* application servers. Prepares each backend to receive rsync pushes from the staging server.

Creates the mwdeploy system user and .ssh directory
Authorizes the deploy_ssh_public_key from pillar (the staging server’s public key)
Creates the production directory at prod_path owned by www-data
Writes /etc/sudoers.d/mwdeploy_target:

mwdeploy ALL=(root) NOPASSWD: /usr/bin/rsync
mwdeploy ALL=(www-data) NOPASSWD: /srv/mediawiki/scripts/mwscript.php

salt 'mw*' state.apply mediawiki.target

mediawiki.jobrunner

Applied to mw* servers that run async MediaWiki jobs. Installs and runs the Wikimedia mediawiki-services-jobrunner and a companion mediawiki-jobchron cron dispatcher.

What it does

Clone the jobrunner repository

Uses git.latest to keep /srv/mediawiki-services-jobrunner up to date from the upstream Wikimedia GitHub repository.

Write the config

Renders /etc/mediawiki-jobrunner/config.json from Jinja, populating Redis host, statsd endpoint, job group settings, and the PHP dispatcher command.

Install systemd units

Manages /etc/systemd/system/mediawiki-jobrunner.service and mediawiki-jobchron.service from the Salt fileserver.

Enable and start services

Starts both services, configured to reload automatically when the repository, config, or unit file changes.

Jobrunner pillar keys

Key	Default	Description
`mediawiki_jobrunner:redis_host`	`redis-us-east-012.ovvin.wonet`	Redis host for job queues and aggregators
`mediawiki_jobrunner:redis_port`	`6379`	Redis port
`mediawiki_jobrunner:mw_path`	`/srv/mediawiki`	Path to the MediaWiki installation
`mediawiki_jobrunner:php_binary`	`/usr/bin/php`	PHP binary used by the dispatcher

config.json template

{
    "groups": {
        "basic": {
            "runners": 6,
            "include": ["*"],
            "low-priority": ["htmlCacheUpdate", "refreshLinks"]
        }
    },
    "limits": {
        "attempts": { "*": 3 },
        "claimTTL": { "*": 3600 },
        "real": { "*": 300 },
        "memory": { "*": "300M" }
    },
    "redis": {
        "aggregators": ["{{ jr.get('redis_host', 'redis-us-east-012.ovvin.wonet') }}"],
        "queues":      ["{{ jr.get('redis_host', 'redis-us-east-012.ovvin.wonet') }}"]
    },
    "statsd": "{{ monitoring_ip }}:9125",
    "dispatcher": "php /srv/mediawiki/scripts/mwscript.php maintenance/run.php runJobs --wiki=%(db)x --type=%(type)x --maxtime=%(maxtime)x --memory-limit=%(maxmem)x --result=json"
}

salt 'mw*' state.apply mediawiki.jobrunner

Core

Web Tier

Data & Search

Infrastructure

Monitoring

MediaWiki State: Deployment Infrastructure & Jobrunner

Pillar keys

Production pillar values

mediawiki (staging server)

Staging sudoers

config.yaml template

mediawiki.proxy

mediawiki.target

mediawiki.jobrunner

What it does

Jobrunner pillar keys

config.json template

Build docs developers (and LLMs) love

Core

Web Tier

Data & Search

Infrastructure

Monitoring

Documentation Index

​Pillar keys

​Production pillar values

​mediawiki (staging server)

​Staging sudoers

​config.yaml template

​mediawiki.proxy

​mediawiki.target

​mediawiki.jobrunner

​What it does

​Jobrunner pillar keys

​config.json template

Build docs developers (and LLMs) love

Pillar keys

Production pillar values

mediawiki (staging server)

Staging sudoers

config.yaml template

mediawiki.proxy

mediawiki.target

mediawiki.jobrunner

What it does

Jobrunner pillar keys

config.json template