Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/wikioasis/salt/llms.txt

Use this file to discover all available pages before exploring further.

Salt pillar data is served exclusively to the minion that requested it, so it is already more secure than putting secrets in state files. However, pillar files committed to a git repository are still visible to anyone with repository access. WikiOasis solves this by keeping all secrets in a dedicated pillar/private/ directory that is listed in .gitignore and never committed. The Salt master reads it at runtime; the git history never sees it.
pillar/private/*.sls is gitignored. Never commit the populated file to the repository — it contains plaintext database passwords, monitoring API credentials, and a private SSH key for database backups.

​
How the private pillar is wired in

pillar/top.sls assigns the private pillar to every minion via the '*' catch-all rule, and additionally to db*, apps*, and monitoring* explicitly: 
# pillar/top.sls (excerpt)
'*':
  - base
  - users
  - private          # ← every minion gets private data

'apps*':
  - php
  - private          # ← apps server needs private data

'db*':
  - mariadb
  - private          # ← backup credentials

'monitoring*':
  - monitoring
  - metal
  - private          # ← Icinga/Grafana passwords
Salt looks for pillar/private/init.sls relative to the pillar root configured in /etc/salt/master. If the file is absent, any state that calls salt['pillar.get']('monitoring:icinga_api_password') (for example) will return an empty string or raise an error, depending on how the state handles missing keys.

​
The example template

The repository ships pillar/private/init.sls.example as a documented template. Copy it to pillar/private/init.sls on the Salt master and fill in every CHANGE_ME placeholder before running any highstate. 
# pillar/private/init.sls.example
# Copy this file to pillar/private/init.sls on the Salt master with values filled in.
# It is gitignored (pillar/private/*.sls) and must never be committed.

monitoring:
  icinga_api_host: monitoring-us-east-021.ovvin.wonet
  icinga_api_password: CHANGE_ME
  ido_db_password: CHANGE_ME
  director_db_password: CHANGE_ME
  monitoring_db_password: CHANGE_ME
  grafana_admin_password: CHANGE_ME
  mysqld_exporter_password: CHANGE_ME

notifications:
  discord_webhook_url: https://discord.com/api/webhooks/YOUR_ID/YOUR_TOKEN
  slack_webhook_url: https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK

mediawiki:
  webhooks:
    discord: ""
    slack: ""

sentry_relay:
  dsn: CHANGE_ME

mariadb:
  backup:
    # MariaDB user Salt will create on each db* server for backup operations
    user: mariadb_backup
    password: CHANGE_ME
    destination:
      # IP or hostname of the external backup server
      host: CHANGE_ME
      user: mariadb-backup
      # Absolute path on the backup server; subdirs per host are created automatically
      path: /mnt/backups/mariadb
    # Private key whose public half is in authorized_keys on the backup server
    ssh_private_key: |
      -----BEGIN OPENSSH PRIVATE KEY-----
      CHANGE_ME
      -----END OPENSSH PRIVATE KEY-----

​
Secret categories

The private pillar holds four categories of secrets:

​
Monitoring credentials

KeyPurpose
monitoring.icinga_api_hostFQDN of the Icinga 2 API endpoint
monitoring.icinga_api_passwordPassword for the root Icinga API user
monitoring.ido_db_passwordMariaDB password for the icingadb IDO user
monitoring.director_db_passwordMariaDB password for the Icinga Director database user
monitoring.monitoring_db_passwordMariaDB password for the Icinga Web database user
monitoring.grafana_admin_passwordPassword for the Grafana admin account
monitoring.mysqld_exporter_passwordPassword for the Prometheus mysqld_exporter MariaDB user

​
Notification webhooks

KeyPurpose
notifications.discord_webhook_urlFull Discord incoming webhook URL for deployment alerts
notifications.slack_webhook_urlFull Slack incoming webhook URL for deployment alerts
mediawiki.webhooks.discordMediaWiki-specific Discord webhook (can differ from global)
mediawiki.webhooks.slackMediaWiki-specific Slack webhook

​
Sentry Relay

KeyPurpose
sentry_relay.dsnSentry DSN for the sentry_relay state to configure error reporting

​
MariaDB backup

KeyPurpose
mariadb.backup.userName of the MariaDB backup user created on each db* host
mariadb.backup.passwordPassword for the backup user
mariadb.backup.destination.hostIP or FQDN of the external backup server
mariadb.backup.destination.userSSH user on the backup server
mariadb.backup.destination.pathAbsolute path on the backup server; per-host subdirectories are created automatically
mariadb.backup.ssh_private_keyPEM-encoded private key (multi-line YAML block scalar) for rsync authentication to the backup server

​
Populating the private pillar

1

Create the private pillar directory

The pillar/private/ directory is tracked in git (so that the gitignore rules apply) but no .sls files inside it are committed.
# The directory should already exist; create it if not
mkdir -p /srv/wikioasis-salt/pillar/private
2

Copy the example template

cp /srv/wikioasis-salt/pillar/private/init.sls.example \
   /srv/wikioasis-salt/pillar/private/init.sls
3

Fill in all values

Open the file in your editor and replace every CHANGE_ME and YOUR_* placeholder with real credentials. Pay particular attention to the mariadb.backup.ssh_private_key block — it must be a valid PEM-encoded private key and must use the YAML literal block scalar syntax (|) to preserve newlines.
$EDITOR /srv/wikioasis-salt/pillar/private/init.sls
4

Verify pillar compilation

After saving, ask Salt to compile and display the private pillar for one minion to confirm it parses correctly. This command runs on the master and never sends the output to the minion.
# Inspect monitoring secrets as seen by the monitoring server
salt 'monitoring-us-east-021' pillar.get monitoring

# Quick check that the backup password is visible to db hosts
salt 'db-c1-us-east-021' pillar.get mariadb:backup:password
A correctly populated file will return the actual values (not CHANGE_ME).

​
File permissions

Because init.sls contains plaintext secrets, lock down its permissions on the Salt master: 
chmod 600 /srv/wikioasis-salt/pillar/private/init.sls
chown root:root /srv/wikioasis-salt/pillar/private/init.sls
For additional hardening, consider storing the private pillar outside the git working tree entirely and configuring a second pillar_roots entry in /etc/salt/master that points to an isolated directory (e.g. /etc/salt/private-pillar/). This prevents accidental git add from ever staging the file.

Build docs developers (and LLMs) love

Get started for freeTalk to us