When the builtin backoff strategies on retries are not sufficient, a custom strategy can be defined. Custom backoff strategies are defined by a function on the queue.
Basic Implementation
The backoff function receives:
attemptsMade - Number of attempts already made to process the job
err - The error that the job failed with
The function returns:
Time in milliseconds to delay the retry
0 to retry immediately
-1 to fail the job immediately
const Queue = require ( 'bull' );
const myQueue = new Queue ( 'Server B' , {
settings: {
backoffStrategies: {
jitter : function ( attemptsMade , err ) {
return 5000 + Math . random () * 500 ;
}
}
}
});
Using the Strategy
myQueue . add ({ foo: 'bar' }, {
attempts: 3 ,
backoff: {
type: 'jitter'
}
});
Advanced: Strategy with Options
You can create more flexible strategies that accept configuration options.
const Queue = require ( 'bull' );
const myQueue = new Queue ( 'Server B' , {
settings: {
backoffStrategies: {
// truncated binary exponential backoff
binaryExponential : function ( attemptsMade , err , options ) {
// Options can be undefined, you need to handle it by yourself
if ( ! options ) {
options = {}
}
const delay = options . delay || 1000 ;
const truncate = options . truncate || 1000 ;
console . error ({ attemptsMade , err , options });
return Math . round ( Math . random () * ( Math . pow ( 2 , Math . max ( attemptsMade , truncate )) - 1 ) * delay )
}
}
}
});
myQueue . add ({ foo: 'bar' }, {
attempts: 10 ,
backoff: {
type: 'binaryExponential' ,
options: {
delay: 500 ,
truncate: 5
}
}
});
Error-Based Strategy
You can base your backoff strategy on the specific error type:
const Queue = require ( 'bull' );
function MySpecificError () {};
const myQueue = new Queue ( 'Server C' , {
settings: {
backoffStrategies: {
foo : function ( attemptsMade , err ) {
if ( err instanceof MySpecificError ) {
return 10000 ;
}
return 1000 ;
}
}
}
});
myQueue . process ( function ( job , done ) {
if ( job . data . msg === 'Specific Error' ) {
throw new MySpecificError ();
} else {
throw new Error ();
}
});
myQueue . add ({ msg: 'Hello' }, {
attempts: 3 ,
backoff: {
type: 'foo'
}
});
myQueue . add ({ msg: 'Specific Error' }, {
attempts: 3 ,
backoff: {
type: 'foo'
}
});
Strategy Examples
linearBackoff : function ( attemptsMade ) {
return attemptsMade * 1000 ; // 1s, 2s, 3s, ...
}
exponentialBackoff : function ( attemptsMade ) {
return Math . pow ( 2 , attemptsMade ) * 1000 ; // 2s, 4s, 8s, 16s, ...
}
jitterWithCap : function ( attemptsMade ) {
const baseDelay = Math . min ( Math . pow ( 2 , attemptsMade ) * 1000 , 60000 );
return baseDelay + Math . random () * 1000 ;
}
fixed : function ( attemptsMade ) {
return 5000 ; // Always 5 seconds
}
Return Values
Positive Number Delay retry by this many milliseconds
-1 Fail the job immediately (no more retries)
Use jitter (random variance) in your backoff strategy to prevent the thundering herd problem where many jobs retry at exactly the same time.
Remember to handle the case where options is undefined if your strategy accepts options.