Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/msolli/proletarian/llms.txt

Use this file to discover all available pages before exploring further.

Proletarian requires PostgreSQL 9.5 or later. The minimum version is enforced by Proletarian’s use of the SELECT ... FOR UPDATE SKIP LOCKED feature, which was introduced in PostgreSQL 9.5 and enables multiple worker threads to poll the job queue concurrently without blocking each other. The install script creates a dedicated proletarian PostgreSQL schema containing two tables — proletarian.job (the live queue) and proletarian.archived_job (the finished-job record) — plus a composite index on the job table that is critical for efficient queue polling.

​
Schema and Table Definitions

The full DDL is reproduced below exactly as shipped in the repository. Copy this into a migration file or run it directly against your database.
database/postgresql/tables.sql
CREATE SCHEMA IF NOT EXISTS proletarian;

CREATE TABLE IF NOT EXISTS proletarian.job
(
    job_id      UUID PRIMARY KEY,     -- job id, generated and returned by proletarian.job/enqueue!
    queue       TEXT        NOT NULL, -- queue name
    job_type    TEXT        NOT NULL, -- job type
    payload     TEXT        NOT NULL, -- Transit-encoded job data
    attempts    INTEGER     NOT NULL, -- Number of attempts. Starts at 0. Increments when the job is processed.
    enqueued_at TIMESTAMPTZ NOT NULL, -- When the job was enqueued (never changes)
    process_at  TIMESTAMPTZ NOT NULL  -- When the job should be run (updates every retry)
);

CREATE TABLE IF NOT EXISTS proletarian.archived_job
(
    job_id      UUID PRIMARY KEY,     -- Copied from job record.
    queue       TEXT        NOT NULL, -- Copied from job record.
    job_type    TEXT        NOT NULL, -- Copied from job record.
    payload     TEXT        NOT NULL, -- Copied from job record.
    attempts    INTEGER     NOT NULL, -- Copied from job record.
    enqueued_at TIMESTAMPTZ NOT NULL, -- Copied from job record.
    process_at  TIMESTAMPTZ NOT NULL, -- Copied from job record (data for the last run only)
    status      TEXT        NOT NULL, -- success / failure
    finished_at TIMESTAMPTZ NOT NULL  -- When the job was finished (success or failure)
);

DROP INDEX IF EXISTS proletarian.job_queue_process_at;

CREATE INDEX job_queue_process_at ON proletarian.job (queue, process_at);

​
Column Reference

​
proletarian.job

ColumnTypeDescription
job_idUUIDPrimary key. Generated by Proletarian and returned by proletarian.job/enqueue!.
queueTEXTQueue name. Defaults to :proletarian/default when not specified.
job_typeTEXTClojure keyword identifying the job type, e.g. ::send-confirmation-email.
payloadTEXTTransit-encoded job data — the map passed as the third argument to enqueue!.
attemptsINTEGERNumber of processing attempts so far. Starts at 0 and increments each time the job is picked up.
enqueued_atTIMESTAMPTZTimestamp when the job was first enqueued. Never updated.
process_atTIMESTAMPTZEarliest timestamp at which the job should be executed. Updated on each retry according to the retry strategy’s delay.

​
proletarian.archived_job

All columns from proletarian.job are copied verbatim when a job is finished. Two additional columns record the outcome:
ColumnTypeDescription
statusTEXTEither success or failure. Set to failure after all configured retries are exhausted.
finished_atTIMESTAMPTZTimestamp when the job finished (successfully or after final failure).

​
The job_queue_process_at Index

CREATE INDEX job_queue_process_at ON proletarian.job (queue, process_at);
This composite index is the performance foundation of the entire queue. Every poll cycle, each worker thread issues a query that:
  1. Filters rows by queue name.
  2. Orders the matching rows by process_at ascending to pick the oldest due job.
  3. Skips any rows already locked by a sibling worker thread (SKIP LOCKED).
Without this index, every poll becomes a sequential scan of the entire proletarian.job table. With it, PostgreSQL resolves the query using an index-only scan over a tiny slice of the B-tree regardless of how many jobs are waiting.

​
Installing the Schema

​
Using the Provided Script

The repository ships a Bash install script at database/postgresql/install.sh. It creates the proletarian role, optionally creates the database, applies tables.sql, and grants the necessary privileges. If you are working inside the Proletarian repository itself, the bundled Makefile exposes two convenience targets: 
# Create the proletarian role, database, schema, tables, and index
make examples.db.install

# Drop the database and role entirely
make examples.db.uninstall
Both targets default to a database named proletarian. You can also invoke the script directly for more control: 
# From the proletarian repository root — default settings (database name: proletarian)
bash database/postgresql/install.sh

# Override the target database name
DATABASE_NAME=myapp bash database/postgresql/install.sh

# Skip database creation if the database already exists
DATABASE_NAME=myapp CREATE_DATABASE=off bash database/postgresql/install.sh
The script respects the standard psql environment variables (PGHOST, PGPORT, PGUSER, PGPASSWORD, etc.) for connecting to the server. The roles and privileges applied by the script are:
database/postgresql/roles.sql
DO
$$
    BEGIN
        CREATE ROLE proletarian WITH LOGIN;
    EXCEPTION
        WHEN duplicate_object THEN
            RAISE NOTICE 'The proletarian role already exists';
    END
$$;
database/postgresql/privileges.sql
GRANT USAGE ON SCHEMA proletarian TO proletarian;

GRANT SELECT, INSERT, UPDATE, DELETE, TRUNCATE ON proletarian.job TO proletarian;
GRANT SELECT, INSERT, TRUNCATE ON proletarian.archived_job TO proletarian;

​
Quick Local Database with Docker

To spin up a throwaway PostgreSQL instance for local development or running the bundled examples: 
docker run -p 55432:5432 -e POSTGRES_PASSWORD=proletarian postgres
Then point your JDBC data source at jdbc:postgresql://localhost:55432/postgres and run the install script (or apply tables.sql directly with psql).

​
Integrating with Migration Tools

If you are already managing schema changes with Flyway or Migratus, simply copy the SQL above into a new migration file. No special configuration is required — Proletarian has no awareness of how the tables were created. Flyway example — create src/main/resources/db/migration/V1__proletarian_tables.sql and paste the contents of tables.sql into it. Migratus example — create resources/migrations/20240101000000-proletarian-tables.up.sql and paste the SQL there.

​
Customizing Table and Schema Names

The proletarian schema and table names are entirely configurable. Pass the :proletarian/job-table and :proletarian/archived-job-table options to both proletarian.job/enqueue! and proletarian.worker/create-queue-worker using the fully-qualified table name as a string. 
(require '[proletarian.job :as job]
         '[proletarian.worker :as worker])

(def table-opts
  {:proletarian/job-table          "myapp.proletarian_job"
   :proletarian/archived-job-table "myapp.proletarian_archived_job"})

;; Enqueue with custom table names
(job/enqueue! db-conn ::my-job {:some "payload"} table-opts)

;; Create a worker that reads from the same custom tables
(def my-worker
  (worker/create-queue-worker db-conn handle-job! table-opts))
The schema and table names can be anything you like, but they must be identical in the options passed to enqueue! and create-queue-worker. If they differ, workers will poll a different table than the one jobs are written to and no jobs will ever be processed.

​
Uninstalling

To drop the database and the proletarian role entirely, use the make target (when inside the repository) or call the uninstall script directly: 
# Using the Makefile shortcut (targets the default 'proletarian' database)
make examples.db.uninstall

# Or invoke the script directly
bash database/postgresql/uninstall.sh

# To target a non-default database name
DATABASE_NAME=myapp bash database/postgresql/uninstall.sh
This drops the named database and then executes DROP ROLE IF EXISTS proletarian; against the postgres maintenance database.

Build docs developers (and LLMs) love

Get started for freeTalk to us