Database migrations

Always run python tools/dbtool.py update before starting server processes after pulling new commits. Skipping this step can cause crashes or data corruption if the schema has changed.

Overview

LandSandBoat uses a Python-based migration system managed by tools/dbtool.py. Migrations live in tools/migrations/ as individual .py files. They are loaded automatically in alphabetical (sorted) order. Migrations handle transformations that cannot be expressed as a plain SQL INSERT … ON DUPLICATE KEY UPDATE — for example, renaming columns, restructuring character data blobs, or backfilling derived values.

Running migrations

With dbtool update (recommended)

dbtool.py update imports changed SQL files and runs all pending migrations in a single step:

python tools/dbtool.py update

For a full re-import of every SQL file:

python tools/dbtool.py update full

Migrations only

To check and run pending migrations without re-importing SQL files:

python tools/dbtool.py migrate

Express updates

When tools/config.yaml contains a valid db_ver hash, dbtool.py update performs an express update:

Compares the stored hash against the current HEAD using git diff.
Imports only the sql/*.sql files that changed between the two commits.
Runs any pending migrations.
Updates db_ver to the new HEAD hash.

This is significantly faster than a full import on large databases.

# tools/config.yaml
- db_ver: 'abcd1234'   # hash of last successful import

If db_ver is absent or invalid, dbtool.py update falls back to a full import.

Migration internals

Each migration module in tools/migrations/ implements three functions:

Function	Purpose
`migration_name()`	Returns a human-readable name shown in the TUI
`check_preconditions(cur)`	Validates that required tables/columns exist before running
`needs_to_run(cur)`	Returns `True` if the migration has not yet been applied
`migrate(cur, db)`	Performs the actual migration and commits

During run_all_migrations, dbtool calls check_preconditions and needs_to_run for each migration. Only migrations where needs_to_run returns True are executed. Results are printed to the terminal — green [*] for applied, red [ ] for pending. If any migration produces errors, they are written to tools/migration_errors.log. This typically indicates corrupt character data rather than a code defect.

Docker deployments

In Docker Compose, the database-update service runs dbtool.py update and must complete successfully before any server process starts:

database-update:
  image: ghcr.io/landsandboat/server:latest
  command: ["python", "/server/tools/dbtool.py", "update"]
  depends_on:
    database:
      condition: service_healthy

connect:
  # ...
  depends_on:
    database-update:
      condition: service_completed_successfully

To enable express updates in Docker and avoid a full re-import every time the container is recreated, mount a persistent config.yaml:

volumes:
  - ./config.yaml:/server/tools/config.yaml

Create the initial config.yaml with the current git hash:

git rev-parse --short=4 HEAD
# Use the output as db_ver:
cat > config.yaml <<'EOF'
- mysql_bin: ''
- auto_backup: 0
- auto_update_client: true
- db_ver: 'abcd'  # replace with actual hash
EOF

Backup and restore

dbtool.py can create and restore database backups. Backups are stored as .sql files in sql/backups/.

# Full database backup
python tools/dbtool.py backup

# Backup player-data tables only
python tools/dbtool.py backup lite

Restore backups from the interactive TUI:

python tools/dbtool.py
# Select option 4: Restore/Import

If restoring a full backup created by dbtool, update db_ver in config.yaml to the hash embedded in the backup filename (the segment after the timestamp) so that express update works correctly after the restore.

Introduction

Installation

Database Setup

Overview

Running migrations

With dbtool update (recommended)

Migrations only

Express updates

Migration internals

Docker deployments

Backup and restore

​Overview

​Running migrations

​With dbtool update (recommended)

​Migrations only

​Express updates

​Migration internals

​Docker deployments

​Backup and restore

Overview

Running migrations

With dbtool update (recommended)

Migrations only

Express updates

Migration internals

Docker deployments

Backup and restore