Documentation Index
Fetch the complete documentation index at: https://mintlify.com/msolli/proletarian/llms.txt
Use this file to discover all available pages before exploring further.
Proletarian supports multiple independent queues in the same database, letting you isolate different classes of work, prevent one noisy workload from starving another, and control processing throughput and latency on a per-queue basis.
How queues work
All jobs live in the same proletarian.job database table, differentiated by a queue column. A queue worker is bound to exactly one named queue: it only polls for and processes jobs whose queue column matches its configured queue name.
There is a built-in default queue, :proletarian/default, used by both job/enqueue! and worker/create-queue-worker when no queue is specified.
If your application only has one class of jobs and no priority requirements, you can use the default queue throughout without specifying :proletarian/queue anywhere.
Creating workers for separate queues
Pass :proletarian/queue in the options map of worker/create-queue-worker to bind a worker to a specific named queue:
(ns your-app.workers
(:require [next.jdbc :as jdbc]
[proletarian.worker :as worker]
[your-app.handlers :as handlers]))
(def ds (jdbc/get-datasource "jdbc:postgresql://..."))
;; High-priority queue: 2 worker threads, polls frequently
(def email-worker
(worker/create-queue-worker ds handlers/handle-job!
{:proletarian/queue :emails
:proletarian/worker-threads 2
:proletarian/polling-interval-ms 100}))
;; Low-priority queue: 1 worker thread, polls less often
(def report-worker
(worker/create-queue-worker ds handlers/handle-job!
{:proletarian/queue :reports
:proletarian/worker-threads 1
:proletarian/polling-interval-ms 5000}))
(worker/start! email-worker)
(worker/start! report-worker)
Enqueueing to a specific queue
Pass :proletarian/queue in the options to job/enqueue! to route a job to the correct queue:
(ns your-app.handlers
(:require [next.jdbc :as jdbc]
[proletarian.job :as job]))
(defn some-route-handler [system request]
(jdbc/with-transaction [tx (:db system)]
;; Route this job to the :emails queue
(job/enqueue! tx ::send-welcome-email
{:user-id (:user-id request)}
{:proletarian/queue :emails})
;; Route this job to the :reports queue
(job/enqueue! tx ::generate-monthly-report
{:month "2025-01"}
{:proletarian/queue :reports})))
:proletarian/queue option is omitted from enqueue!, the job is placed on the :proletarian/default queue.
Queue priority via thread count
The number of worker threads directly controls how many jobs on a given queue can run in parallel. More threads means higher throughput:
;; Give the email queue 4 worker threads — it processes 4 jobs simultaneously
(worker/create-queue-worker ds handler {:proletarian/queue :emails
:proletarian/worker-threads 4})
;; The reports queue gets 1 thread — jobs run one at a time
(worker/create-queue-worker ds handler {:proletarian/queue :reports
:proletarian/worker-threads 1})
:emails queue with 4 threads, up to 12 email jobs can run in parallel cluster-wide.
Queue priority via polling interval
The :proletarian/polling-interval-ms option controls how long a worker thread waits after finishing a job (or finding an empty queue) before polling again. A lower polling interval means lower latency from enqueue to execution:
;; Emails: poll every 100 ms — very responsive
(worker/create-queue-worker ds handler {:proletarian/queue :emails
:proletarian/polling-interval-ms 100})
;; Reports: poll every 30 seconds — batch-style processing is fine
(worker/create-queue-worker ds handler {:proletarian/queue :reports
:proletarian/polling-interval-ms 30000})
