Documentation Index
Fetch the complete documentation index at: https://mintlify.com/msolli/proletarian/llms.txt
Use this file to discover all available pages before exploring further.
In production, queue workers need to start when your application boots and stop cleanly when it shuts down. Rather than managing this with bare defs, you should integrate Proletarian into whichever component lifecycle library your application already uses.
The three patterns below cover the most common Clojure component libraries. All of them follow the same idea: worker/start! in the start/init phase, worker/stop! in the stop/halt phase.
When using a component library, set :proletarian/install-jvm-shutdown-hook? to false (the default). The component library controls application lifecycle — having Proletarian install its own JVM shutdown hook alongside the library’s hooks can lead to double-stop races. Let the library call stop!.
Component
Integrant
Mount
Stuart Sierra’s Component uses a Lifecycle protocol with start and stop methods.(ns your-app.workers
(:require [com.stuartsierra.component :as component]
[proletarian.worker :as worker]
[next.jdbc :as jdbc]))
(defrecord QueueWorkerComponent [data-source handler-fn queue-worker]
component/Lifecycle
(start [this]
(let [qw (worker/create-queue-worker data-source handler-fn
{:proletarian/install-jvm-shutdown-hook? false})]
(worker/start! qw)
(assoc this :queue-worker qw)))
(stop [this]
(when queue-worker
(worker/stop! queue-worker))
(assoc this :queue-worker nil)))
(defn new-queue-worker-component [handler-fn]
(map->QueueWorkerComponent {:handler-fn handler-fn}))
data-source:(defn new-system []
(component/system-map
:db (new-database-component)
:worker (component/using
(new-queue-worker-component handlers/handle-job!)
{:data-source :db})))
Integrant uses multimethods keyed on configuration map keys.(ns your-app.system
(:require [integrant.core :as ig]
[proletarian.worker :as worker]))
(defmethod ig/init-key :your-app/queue-worker
[_ {:keys [data-source handler-fn]}]
(doto (worker/create-queue-worker data-source handler-fn
{:proletarian/install-jvm-shutdown-hook? false})
(worker/start!)))
(defmethod ig/halt-key! :your-app/queue-worker
[_ queue-worker]
(worker/stop! queue-worker))
config.edn, depending on the key that provides the data source:{:your-app/db {:jdbc-url "jdbc:postgresql://..."}
:your-app/queue-worker {:data-source #ig/ref :your-app/db
:handler-fn #ig/ref :your-app/handler}}
Mount uses the defstate macro to declare stateful components with :start and :stop expressions.(ns your-app.workers
(:require [mount.core :refer [defstate]]
[proletarian.worker :as worker]
[your-app.db :refer [data-source]]
[your-app.handlers :refer [handle-job!]]))
(defstate queue-worker
:start (doto (worker/create-queue-worker data-source handle-job!
{:proletarian/install-jvm-shutdown-hook? false})
(worker/start!))
:stop (worker/stop! queue-worker))
your-app.db (which defines data-source) is required before your-app.workers in your start sequence. Accessing system state in handlers
Job handlers often need access to stateful resources — database connections, HTTP clients, caches — that live in your component system. There are two idiomatic patterns for making system state available inside a handler.
Closing over the system map
Construct your handler function inside start/init and close over the system components you need:
(defn make-handler [system]
(fn [job-type payload]
;; All components from your system are in scope here
(case job-type
::send-email (email/send! (:mailer system) payload)
::index-doc (search/index! (:search-client system) payload)
::notify-user (db/write! (:db system) payload))))
make-handler the assembled system in your component’s start method:
;; Component example
(start [this]
(let [handler (make-handler this) ;; `this` IS the system map in Component
qw (worker/create-queue-worker (:data-source this) handler)]
(worker/start! qw)
(assoc this :queue-worker qw)))
Using partial for simpler cases
When a handler only needs one or two dependencies, partial keeps things concise:
(defn handle-job! [db mailer job-type payload]
(case job-type
::send-email (email/send! mailer payload)
::write-audit (db/insert! db :audit payload)))
;; In your component start:
(let [handler (partial handle-job! (:db system) (:mailer system))]
(worker/create-queue-worker ds handler))
make-handler (or partial) with stub dependencies and invoke the returned function directly.
