Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/msolli/proletarian/llms.txt

Use this file to discover all available pages before exploring further.

Proletarian requires MySQL 8.0.1 or later. The minimum version is enforced by Proletarian’s use of SELECT ... FOR UPDATE SKIP LOCKED, which was added to MySQL in version 8.0.1 and is required for multiple worker threads to dequeue jobs concurrently without contention. The MySQL schema mirrors the PostgreSQL schema in structure and purpose, but uses MySQL-native types — most notably BINARY(16) for the job ID and TIMESTAMP for time columns. It also requires a different job ID strategy in the Proletarian configuration; see the MySQL UUID strategy section below.

​
Schema and Table Definitions

The full DDL is reproduced below. Copy this into a migration file or run it directly against your database with the MySQL client.
database/mysql/tables.sql
--
--
--
CREATE DATABASE IF NOT EXISTS proletarian;

CREATE TABLE IF NOT EXISTS proletarian.job (
    job_id      BINARY(16) PRIMARY KEY, -- job id, generated and returned by proletarian.job/enqueue!
    queue       TEXT      NOT NULL,     -- queue name
    job_type    TEXT      NOT NULL,     -- job type
    payload     TEXT      NOT NULL,     -- Transit-encoded job data
    attempts    INTEGER   NOT NULL,     -- Number of attempts. Starts at 0. Increments when the job is processed.
    enqueued_at TIMESTAMP NOT NULL,     -- When the job was enqueued (never changes)
    process_at  TIMESTAMP NOT NULL,     -- When the job should be run (updates every retry)
    INDEX (queue(256), process_at)
);

CREATE TABLE IF NOT EXISTS proletarian.archived_job (
    job_id      BINARY(16) PRIMARY KEY, -- Copied from job record.
    queue       TEXT      NOT NULL,     -- Copied from job record.
    job_type    TEXT      NOT NULL,     -- Copied from job record.
    payload     TEXT      NOT NULL,     -- Copied from job record.
    attempts    INTEGER   NOT NULL,     -- Copied from job record.
    enqueued_at TIMESTAMP NOT NULL,     -- Copied from job record.
    process_at  TIMESTAMP NOT NULL,     -- Copied from job record (data for the last run only)
    status      TEXT      NOT NULL,     -- success / failure
    finished_at TIMESTAMP NOT NULL      -- When the job was finished (success or failure)
);

​
Column Reference

​
proletarian.job

ColumnTypeDescription
job_idBINARY(16)Primary key. A UUID stored as 16 raw bytes. Generated and returned by proletarian.job/enqueue! when using the MySQL job ID strategy.
queueTEXTQueue name. Defaults to :proletarian/default when not specified.
job_typeTEXTClojure keyword identifying the job type, e.g. ::send-confirmation-email.
payloadTEXTTransit-encoded job data — the map passed as the third argument to enqueue!.
attemptsINTEGERNumber of processing attempts so far. Starts at 0 and increments each time the job is picked up.
enqueued_atTIMESTAMPTimestamp when the job was first enqueued. Never updated. Stored in UTC.
process_atTIMESTAMPEarliest timestamp at which the job should be executed. Updated on each retry according to the retry strategy’s delay.

​
proletarian.archived_job

All columns from proletarian.job are copied verbatim when a job finishes. Two additional columns record the outcome:
ColumnTypeDescription
statusTEXTEither success or failure. Set to failure after all configured retries are exhausted.
finished_atTIMESTAMPTimestamp when the job finished (successfully or after final failure).

​
Key Differences from the PostgreSQL Schema

AspectPostgreSQLMySQL
Job ID typeUUID (native type)BINARY(16) (raw bytes)
Timestamp typeTIMESTAMPTZ (timezone-aware)TIMESTAMP (no timezone)
Index definitionSeparate CREATE INDEX statementInline INDEX (queue(256), process_at) inside CREATE TABLE
Schema creationSeparate CREATE SCHEMA statementInline CREATE DATABASE (MySQL uses databases as schemas)

​
Why BINARY(16) for the Job ID?

MySQL does not have a native UUID column type that stores values as compact binary. Using CHAR(36) to store a UUID string wastes space and index performance. Instead, Proletarian stores UUIDs as 16 raw bytes in a BINARY(16) column. This requires a dedicated job ID strategy in the Clojure code — see below.
All timestamps are stored and compared in UTC regardless of the JVM’s default timezone or the MySQL server’s timezone setting. Proletarian always writes TIMESTAMP values as UTC instants, so your jobs will be processed at the correct wall-clock time even if server or JVM timezones differ.

​
MySQL Job ID Strategy

When using MySQL, you must configure the MySQL UUID strategy on both enqueue! and create-queue-worker. Forgetting to set this option — or setting it on only one side — will cause runtime errors because the default strategy generates PostgreSQL-native UUID objects that cannot be written to a BINARY(16) column.
Proletarian ships the proletarian.job-id-strategies namespace specifically to handle this difference. Require it and pass the MySQL strategy via the :proletarian/job-id-strategy option: 
(require '[proletarian.job :as job]
         '[proletarian.job-id-strategies :as job-id-strategies]
         '[proletarian.worker :as worker])

(def mysql-opts
  {:proletarian/job-id-strategy (job-id-strategies/->mysql-uuid-strategy)})

;; Enqueue a job using the MySQL UUID strategy
(job/enqueue! db-conn ::send-confirmation-email
  {:email "user@example.com"}
  mysql-opts)

;; Create a queue worker that uses the same strategy
(def email-worker
  (worker/create-queue-worker db-conn handle-job! mysql-opts))
The ->mysql-uuid-strategy function returns a strategy object that serialises UUIDs to byte[] before writing them to the database and deserialises them back when reading, matching the BINARY(16) storage format.

​
Installing the Schema

​
Using the Provided Script

The repository ships a Bash install script at database/mysql/install.sh. It creates the database, the proletarian user, applies tables.sql, and grants the necessary privileges. Edit the connection variables at the top of the script to match your MySQL server before running it: 
# Variables to set inside install.sh (defaults shown):
hostname=localhost
port=3306
username=root
password=password
Then run the script: 
# Default settings (database name: proletarian)
bash database/mysql/install.sh

# Override the target database name
DATABASE_NAME=myapp bash database/mysql/install.sh

# Skip database creation if the database already exists
DATABASE_NAME=myapp CREATE_DATABASE=off bash database/mysql/install.sh
The individual SQL files the script applies are: database/mysql/db.sql — creates the database: 
CREATE DATABASE IF NOT EXISTS proletarian;
database/mysql/user.sql — creates the application user: 
CREATE USER IF NOT EXISTS 'proletarian'@'%' IDENTIFIED BY 'password';
database/mysql/privileges.sql — grants the minimum required privileges: 
GRANT USAGE ON proletarian.* TO 'proletarian'@'%';

GRANT SELECT, INSERT, UPDATE, DELETE, DROP ON proletarian.job TO 'proletarian'@'%';
GRANT SELECT, INSERT, DROP ON proletarian.archived_job TO 'proletarian'@'%';

​
Integrating with Migration Tools

If you are already managing schema changes with Flyway or Migratus, copy the contents of tables.sql into a new migration file. No special Proletarian configuration is needed — Proletarian is agnostic about how its tables were created. Flyway example — create src/main/resources/db/migration/V1__proletarian_tables.sql and paste the contents of tables.sql into it. Migratus example — create resources/migrations/20240101000000-proletarian-tables.up.sql and paste the SQL there.

​
Customizing Table and Schema Names

The proletarian database/schema and table names are fully configurable. Pass the :proletarian/job-table and :proletarian/archived-job-table options to both proletarian.job/enqueue! and proletarian.worker/create-queue-worker using the fully-qualified table name as a string. 
(require '[proletarian.job :as job]
         '[proletarian.job-id-strategies :as job-id-strategies]
         '[proletarian.worker :as worker])

(def opts
  {:proletarian/job-id-strategy    (job-id-strategies/->mysql-uuid-strategy)
   :proletarian/job-table          "myapp.proletarian_job"
   :proletarian/archived-job-table "myapp.proletarian_archived_job"})

;; Enqueue with custom table names
(job/enqueue! db-conn ::my-job {:some "payload"} opts)

;; Create a worker that reads from the same custom tables
(def my-worker
  (worker/create-queue-worker db-conn handle-job! opts))
The database/schema and table names can be anything you like, but they must be identical in the options passed to enqueue! and create-queue-worker. If they differ, workers will poll a different table than the one jobs are written to and no jobs will ever be processed.

Build docs developers (and LLMs) love

Get started for freeTalk to us