3.1.5 - 3.1.6

3.1.6

• Typescript type defs for deletion config updated via PR.

3.1.5

• Typescript type defs updated for job priority via PR.
• Set default teamConcurrency to 1 when teamSize > 1.

3.1.4

• Typescript type defs updated for static function exports via PR.

3.1.3

• Added support for typeorm with job insertion script via PR.

3.1.2

• Prevented duplicate state completion jobs being created from an expired onComplete() subscription.

3.1.0

Features

• Added wildcard pattern matching for subscriptions. The allows you to have 1 subscription over many queues. For example, the following subscription uses the * placeholder to fetch completed jobs from all queues that start with the text sensor-report-.

    boss.onComplete('sensor-report-*', processSensorReport);

  Wildcards may be placed anywhere in the queue name. The motivation for this feature is adding the capability for an orchestration to use a single subscription to listen to potentially thousands of job processors that just have 1 thing to do via isolated queues.

Changes

• Multiple subscriptions to the same queue are now allowed on the same instance.

  Previously an error was thrown when attempting to subscribe to the same queue more than once on the same instance. This was merely an internal concern with worker tracking. Since teamConcurrency was introduced in 3.0, it blocks polling until the last job in a batch is completed, which may have the side effect of slowing down queue operations if one job is taking a long time to complete. Being able to have multiple subscriptions isn't necessarily something I'd advertise as a feature, but it's something easy I can offer until implementing a more elaborate producer consumer queue pattern that monitors its promises.

  Remember to keep in mind that subscribe() is intended to provide a nice abstraction over fetch() and complete(), which are always there if and when you require a use case that subscribe() cannot provide.

• Internal state job suffixes are now prefixes. The following shows a comparison of completed state jobs for the queue some-job.

  • 3.0: some-job__state__completed
  • 3.1: __state__completed__some-job

  This is a internal implementation detail included here if you happen to have any custom queries written against the job tables. The migration will handle this for the job table (the archive will remain as-is).

Fixes

• Removed connection string parsing and validation. The pg module bundles pg-connection-string which supports everything I was trying to do previously with connection strings. This resolves some existing issues related to conditional connection arguments as well as allowing auto-promotion of any future enhancements that may be provided by these libraries.

3.0.0

Additions and Enhancements

• Retry support added for failed jobs! Pretty much the #1 feature request of all time.
• Retry delay and backoff options added! Expired and failed jobs can now delay a retry by a fixed time, or even a jittered exponential backoff.
  • New publish options: retryDelay (int) and retryBackoff (bool)
  • retryBackoff will use an exponential backoff algorithm with jitter to somewhat randomize the distribution. Inspired by Marc on the AWS blog post Exponential Backoff and Jitter
• Backpressure support added to subscribe()! If your callback returns a promise, it will defer polling and other callbacks until it resolves.
  • Returning a promise replaces the need to use the job.done() callback, as this will be handled automatically. Any errors thrown will also automatically fail the job.
  • A new option teamConcurrency was added that can be used along with teamSize for single job callbacks to control backpressure if a promise is returned.
• subscribe() will now return an array of jobs all at once when batchSize is specified.
• fetch() now returns jobs with a convenience job.done() function like subscribe()
• Reduced polling load by consolidating all state-based completion subscriptions to onComplete()
  • Want to know if the job failed? job.data.failed will be true.
  • Want to know if the job expired? job.data.state will be 'expired'.
  • Want to avoid hard-coding that constant? All state names are now exported in the root module and can be required as needed, like in the following example.

    const {states} = require('pg-boss');
    
    if(job.data.state === states.expired) {
        console.log(`job ${job.data.request.id} in queue ${job.data.request.name} expired`);
        console.log(`createdOn: ${job.data.createdOn}`);
        console.log(`startedOn: ${job.data.startedOn}`);
        console.log(`expiredOn: ${job.data.completedOn}`);
        console.log(`retryCount: ${job.data.retryCount}`);
    }

• Batch failure and completion now create completed state jobs for onComplete(). Previously, if you called complete or fail with an array of job IDs, no state jobs were created.
• Added convenience publish functions that set different configuration options:
  • publishThrottled(name, data, options, seconds, key)
  • publishDebounced(name, data, options, seconds, key)
  • publishAfter(name, data, options, seconds | ISO date string | Date)
  • publishOnce(name, data, options, key)
• Added deleteQueue() and deleteAllQueues() api to clear queues if and when needed.

Semver major items & breaking changes

• Removed all events that emitted jobs, such as failed, expired-job, and job, as these were all instance-bound and pre-dated the distribution-friendly onComplete()
• Removed extra convenience done() argument in subscribe() callback in favor of consolidating to job.done()
• Renamed expired-count event to expired
• Failure and completion results are now wrapped in an object with a value property if they're not an object
• subscribe() with a batchSize property now runs the callback only once with an array of jobs. The teamSize option still calls back once per job.
• Removed onFail(), offFail(), onExpire(), onExpire(), fetchFailed() and fetchExpired(). All job completion subscriptions should now use onComplete() and fetching is consolidated to fetchCompleted(). In order to determine how the job completed, additional helpful properties have been added to data on completed jobs, such as state and failed.
• startIn option has been renamed to startAfter to make its behavior more clear. Previously, this value accepted an integer for the number of seconds of delay, or a PostgreSQL interval string. The interval string has been replaced with an UTC ISO date time string (must end in Z), or you can pass a Date object.
• singletonDays option has been removed
• Dropping node 4 support. All tests in 3.0 have passed in CI on node 4, but during release I removed the Travis CI config for it, so future releases may not work.

Fixes and other items of interest

• The pgcrypto extension is now used internally for uuid generation with onComplete(). It will be added in the database if it's not already added.
• Adjusted indexes to help with fetch performance
• Errors thrown in job handlers will now correctly serialize into the response property of the completion job.

2.5.2

• Typescript defs patch

2.5.1

• Added max constructor option additional to poolSize
• Migration: use pg transaction to avoid inconsistency

2.5.0

• Archive: Existing archive configuration settings now apply to moving jobs into a new table arvhive
  instead of immediate deletion. This allows the concerns of job indexing and job retention to be separated.
• Archive: deleteArchivedJobsEvery and deleteCheckInterval settings added for defining job retention.
  The default retention interval is 7 days.
• Archive: Changed default archive interval to 1 hour from 1 day.
• Monitoring: Updated contract for monitor-states event to add counts by queue, not just totals.
• Monitoring: Adjusted queue size counting to exclude state-based jobs. While these were technically
  correct in regards to physical record count, it was a bit too difficult to explain.
• Downgraded bluebird to a dev dependency. Always nice to have 1 less dependency.

2.4.3

• Typescript defs patch