Metal Salt State: Bare-Metal KVM Host Configuration

The metal Salt state configures the two bare-metal hypervisor hosts (metal-us-east-01 and metal-us-east-02) that run all WikiOasis virtual machines via KVM/libvirt under Proxmox. The top-level state is a simple include list that composes six sub-states, each responsible for a distinct infrastructure concern. Metal hosts have a public IP, an internal private bridge for VM traffic, and a vRack interface for inter-host connectivity.

Sub-state composition

# salt/metal/init.sls
include:
  - metal.ip_forwarding
  - metal.ssh
  - metal.dns_dhcp
  - metal.ipv6_routing
  - metal.apt_proxy

ip_forwarding

Enables net.ipv4.ip_forward via sysctl and manages iptables DNAT/MASQUERADE rules for port-forwarding traffic to VMs.

ssh

Deploys a hardened sshd_config from a Jinja template and reloads sshd on changes.

dns_dhcp

Registers VM hostname/MAC/IP records via the assign-host script, which drives the internal DNS and DHCP server.

ipv6_routing

Configures sysctl parameters for IPv6 forwarding and proxy NDP, and installs an if-up.d script to set up per-VM IPv6 routes on interface bring-up.

apt_proxy

Runs a Squid HTTP proxy on port 3129 that caches APT package downloads for all VMs on the internal network.

vm_ipv6

Assigns a /128 IPv6 address and default route to a specific VM based on its Proxmox VM ID and the host’s IPv6 prefix — applied on VM minions, not the metal host itself.

`metal.apt_proxy` — Squid APT cache

Installs the squid package, deploys the config file, and ensures the service is running and restarted on config changes. The proxy is used by VMs during the initial provisioning phase before Salt takes over; once Salt manages a host it enforces that /etc/apt/apt.conf.d/99proxy is absent so the proxy is no longer in the APT path.

# salt/metal/apt_proxy/init.sls
squid:
  pkg.installed: []
  service.running:
    - enable: True
    - watch:
      - file: /etc/squid/squid.conf
    - require:
      - pkg: squid

/etc/squid/squid.conf:
  file.managed:
    - source: salt://metal/apt_proxy/files/squid.conf
    - user: root
    - group: root
    - mode: '0644'
    - require:
      - pkg: squid

# The apt proxy config is only needed before Salt manages a host.
# Enforce it is absent once Salt takes over.
/etc/apt/apt.conf.d/99proxy:
  file.absent

The Squid configuration is intentionally minimal — it listens on 0.0.0.0:3129 and allows access only from localhost and RFC 1918 10.0.0.0/8 addresses:

# /etc/squid/squid.conf
http_port 0.0.0.0:3129

acl localnet src 127.0.0.1/32
acl localnet src 10.0.0.0/8
http_access allow localnet
http_access deny all

`metal.dns_dhcp` — Internal DNS and DHCP

Deploys the /usr/local/sbin/assign-host management script and idempotently registers every hostname defined in the dns_hosts pillar key. The unless guard checks the existing host table before running, making repeated applies safe.

# salt/metal/dns_dhcp/init.sls (condensed)
{%- set dns_hosts = salt['pillar.get']('dns_hosts', {}) %}

/usr/local/sbin/assign-host:
  file.managed:
    - source: salt://metal/dns_dhcp/files/assign-host
    - user: root
    - group: root
    - mode: "0750"

{%- for hostname, h in dns_hosts.items() %}
assign_host_{{ hostname }}:
  cmd.run:
    - name: assign-host add {{ hostname }} {{ h.mac if h.mac else '~' }} {{ h.ip }}
    - unless: assign-host list | awk 'NR>2{print $1}' | grep -qxF '{{ hostname }}'
    - require:
      - file: /usr/local/sbin/assign-host
{%- endfor %}

The dns_hosts pillar key is a flat map of hostname → {ip, mac}. A ~ MAC means DHCP is not required for that host (static IP assigned elsewhere).

`metal.ip_forwarding` — IPv4 Port Forwarding

Enables IPv4 forwarding via sysctl and installs a templated script that applies iptables DNAT and MASQUERADE rules for each forwarding rule defined in the pillar. The script is re-run only when it changes (onchanges), keeping state application fast on subsequent runs.

# salt/metal/ip_forwarding/init.sls
iptables-persistent:
  pkg.installed: []

bind9-dnsutils:
  pkg.installed: []

netfilter-persistent:
  service.enabled: []

net.ipv4.ip_forward:
  sysctl.present:
    - value: 1

ip_forwarding_script:
  file.managed:
    - name: /usr/local/sbin/apply-forwarding.sh
    - source: salt://metal/ip_forwarding/files/apply-forwarding.sh.jinja
    - template: jinja
    - mode: '0750'
    - user: root
    - group: root
    - require:
      - pkg: iptables-persistent
      - pkg: bind9-dnsutils

apply_ip_forwarding:
  cmd.run:
    - name: /usr/local/sbin/apply-forwarding.sh
    - onchanges:
      - file: ip_forwarding_script
    - require:
      - file: ip_forwarding_script
      - sysctl: net.ipv4.ip_forward

`metal.ipv6_routing` — IPv6 Forwarding and Proxy NDP

Writes /etc/sysctl.d/99-vrack.conf to enable IPv4/IPv6 forwarding and proxy NDP, applies the settings immediately, and installs an if-up.d hook that re-applies VM IPv6 routes whenever a network interface comes up.

# salt/metal/ipv6_routing/init.sls
/etc/sysctl.d/99-vrack.conf:
  file.managed:
    - contents: |
        net.ipv4.ip_forward=1
        net.ipv6.conf.all.forwarding=1
        net.ipv6.conf.all.proxy_ndp=1
    - user: root
    - group: root
    - mode: '0644'

apply_vrack_sysctl:
  cmd.run:
    - name: sysctl -p /etc/sysctl.d/99-vrack.conf
    - onchanges:
      - file: /etc/sysctl.d/99-vrack.conf

/etc/network/if-up.d/ipv6-vm-routing:
  file.managed:
    - source: salt://metal/ipv6_routing/files/ipv6-vm-routing.jinja
    - template: jinja
    - mode: '0755'

apply_ipv6_vm_routing:
  cmd.run:
    - name: /etc/network/if-up.d/ipv6-vm-routing
    - stateful: True
    - require:
      - file: /etc/network/if-up.d/ipv6-vm-routing
      - file: /etc/sysctl.d/99-vrack.conf

`metal.ssh` — SSH Daemon Hardening

Deploys a Jinja-templated sshd_config to the metal host and reloads sshd on changes. The template is rendered with pillar data, allowing per-host SSH policy (e.g. allowed users, port, key types) without separate state files.

# salt/metal/ssh/init.sls
sshd_config:
  file.managed:
    - name: /etc/ssh/sshd_config
    - source: salt://metal/ssh/files/sshd_config.jinja
    - template: jinja
    - user: root
    - group: root
    - mode: '0600'
  cmd.run:
    - name: systemctl reload sshd
    - onchanges:
      - file: sshd_config
    - require:
      - file: sshd_config

`metal.vm_ipv6` — VM IPv6 Address Assignment

This sub-state runs on VM minions (not on the metal hosts). It derives the VM’s Proxmox ID and the metal host’s IPv6 prefix from the proxmox pillar, then assigns a /128 address <prefix>::<vmid> on the VM’s primary interface and sets the default route via the metal host’s link-local gateway.

# salt/metal/vm_ipv6/init.sls (condensed)
{%- set vm = salt['pillar.get']('proxmox:vms:' ~ grains['id'], {}) %}
{%- set vmid = vm.get('vmid') %}
{%- set host_cfg = salt['pillar.get']('proxmox:hosts:' ~ vm.get('metal_host', ''), {}) %}
{%- set prefix = host_cfg.get('ipv6_prefix', '') %}
{%- set gw_ll = host_cfg.get('vm_gateway_ll', '') %}
{%- set iface = vm.get('interface', 'ens18') %}

{%- if vmid is not none %}

/etc/network/interfaces.d/ipv6.conf:
  file.managed:
    - source: salt://metal/vm_ipv6/files/interfaces.jinja
    - template: jinja
    - user: root
    - group: root
    - mode: '0644'

ipv6_addr_{{ vmid }}:
  cmd.run:
    - name: ip -6 addr add {{ prefix }}::{{ vmid }}/128 dev {{ iface }}
    - unless: ip -6 addr show dev {{ iface }} | grep -q '{{ prefix }}::{{ vmid }}/128'

ipv6_default_route_{{ vmid }}:
  cmd.run:
    - name: ip -6 route add default via {{ gw_ll }} dev {{ iface }}
    - unless: ip -6 route show | grep -q 'default via {{ gw_ll }}'
    - require:
      - cmd: ipv6_addr_{{ vmid }}

{%- endif %}

For example, VM mw-us-east-011 with vmid: 160 on metal-us-east-01 (prefix 2604:2dc0:100:295c) receives address 2604:2dc0:100:295c::160/128.

Pillar structure

The metal pillar is split across three files:

init.sls (shared)
metal-us-east-01.sls
metal-us-east-02.sls

# pillar/metal/init.sls — shared across both metal hosts
apt_bootstrap:
  http_cache: http://127.0.0.1:3129

metal_public_ips:
  - 40.160.53.92
  - 40.160.53.94

dns_hosts:
  metal-us-east-01:
    ip: 10.0.1.1
    mac: ~
  metal-us-east-02:
    ip: 10.0.2.1
    mac: ~
  proxy-us-east-011:
    ip: 10.0.1.2
    mac: bc:24:11:7f:59:ff
  db-c1-us-east-021:
    ip: 10.0.2.20
    mac: bc:24:11:89:75:da
  redis-us-east-011.ovvin.wonet:
    ip: 10.0.1.40
    mac: bc:24:11:9f:38:bc
  # ... (full list in pillar/metal/init.sls)

proxmox:
  public_bridge: "vmbr0"
  private_bridge: "vmbr-vrack"
  hosts:
    metal-us-east-01:
      ipv6_prefix: "2604:2dc0:100:295c"
      ipv6_gateway: "2604:2dc0:100:29ff:ff:ff:ff:ff"
      vm_gateway_ll: "fe80::d250:99ff:feda:8f81"
    metal-us-east-02:
      ipv6_prefix: "2604:2dc0:100:295e"
      ipv6_gateway: "2604:2dc0:100:29ff:ff:ff:ff:ff"
      vm_gateway_ll: "fe80::d250:99ff:feda:9205"
  vms:
    mw-us-east-011.ovvin.wonet:
      metal_host: metal-us-east-01
      vmid: 160
      interface: ens18
    db-c1-us-east-021.ovvin.wonet:
      metal_host: metal-us-east-02
      vmid: 220
    # ... (full VM list in pillar/metal/init.sls)

# pillar/metal/metal-us-east-01.sls
public_ip: 40.160.53.92

ip_forwarding:
  - host_port: 2322
    vm_dns: bastion-us-east-011.ovvin.wonet
    vm_ip: 10.0.1.49
    vm_port: 22
    proto: tcp
  - host_port: 80
    vm_dns: proxy-us-east-011.ovvin.wonet
    vm_ip: 10.0.1.2
    vm_port: 80
    proto: tcp
  - host_port: 33060
    vm_dns: db-other-us-east-011.ovvin.wonet
    vm_ip: 10.0.1.20
    vm_port: 3306
    proto: tcp
  - host_port: 33061
    vm_dns: db-pc-us-east-011.ovvin.wonet
    vm_ip: 10.0.1.21
    vm_port: 3306
    proto: tcp

# pillar/metal/metal-us-east-02.sls
public_ip: 40.160.53.94

ip_forwarding:
  - host_port: 2322
    vm_dns: bastion-us-east-021.ovvin.wonet
    vm_ip: 10.0.2.49
    vm_port: 22
    proto: tcp
  - host_port: 80
    vm_dns: proxy-us-east-021.ovvin.wonet
    vm_ip: 10.0.2.2
    vm_port: 80
    proto: tcp
  - host_port: 1234
    vm_dns: apps-us-east-021.ovvin.wonet
    vm_ip: 10.0.2.70
    vm_port: 1234
    proto: tcp
  - host_port: 33060
    vm_dns: db-c1-us-east-021.ovvin.wonet
    vm_ip: 10.0.2.20
    vm_port: 3306
    proto: tcp

Applying the state

# Apply all metal sub-states to both metal hosts
salt 'metal*' state.apply metal

# Apply a single sub-state
salt 'metal*' state.apply metal.apt_proxy
salt 'metal*' state.apply metal.ip_forwarding
salt 'metal*' state.apply metal.ipv6_routing

# Apply vm_ipv6 to VM minions (not metal hosts)
salt 'mw*' state.apply metal.vm_ipv6

# Dry-run first
salt 'metal*' state.apply metal test=True

The metal.ip_forwarding sub-state modifies live iptables rules. Always run with test=True on metal hosts before applying changes to ip_forwarding pillar entries, since an incorrect DNAT rule can redirect live traffic to the wrong VM.

Core

Web Tier

Data & Search

Infrastructure

Monitoring

Metal Salt State: Bare-Metal KVM Host Configuration

Sub-state composition

ip_forwarding

ssh

dns_dhcp

ipv6_routing

apt_proxy

vm_ipv6

`metal.apt_proxy` — Squid APT cache

`metal.dns_dhcp` — Internal DNS and DHCP

`metal.ip_forwarding` — IPv4 Port Forwarding

`metal.ipv6_routing` — IPv6 Forwarding and Proxy NDP

`metal.ssh` — SSH Daemon Hardening

`metal.vm_ipv6` — VM IPv6 Address Assignment

Pillar structure

Applying the state

Build docs developers (and LLMs) love

Core

Web Tier

Data & Search

Infrastructure

Monitoring

Documentation Index

​Sub-state composition

ip_forwarding

ssh

dns_dhcp

ipv6_routing

apt_proxy

vm_ipv6

​metal.apt_proxy — Squid APT cache

​metal.dns_dhcp — Internal DNS and DHCP

​metal.ip_forwarding — IPv4 Port Forwarding

​metal.ipv6_routing — IPv6 Forwarding and Proxy NDP

​metal.ssh — SSH Daemon Hardening

​metal.vm_ipv6 — VM IPv6 Address Assignment

​Pillar structure

​Applying the state

Build docs developers (and LLMs) love

Sub-state composition

`metal.apt_proxy` — Squid APT cache

`metal.dns_dhcp` — Internal DNS and DHCP

`metal.ip_forwarding` — IPv4 Port Forwarding

`metal.ipv6_routing` — IPv6 Forwarding and Proxy NDP

`metal.ssh` — SSH Daemon Hardening

`metal.vm_ipv6` — VM IPv6 Address Assignment

Pillar structure

Applying the state