You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: README.md
+69-90
Original file line number
Diff line number
Diff line change
@@ -1,10 +1,35 @@
1
1
# tress
2
2
3
-
Easy to use asynchronous job queue. It stores jobs in the memory and runs it asynchronously in parallel with a given concurrency. Successor of `caolan/async.queue`.
3
+
Easy to use asynchronous job queue.
4
4
5
5
[![Build Status][travis-image]][travis-url]
6
6
[![NPM version][npm-image]][npm-url]
7
7
8
+
It stores jobs in the memory and runs it asynchronously in parallel with a given concurrency. Successor of [caolan/async.queue](http://caolan.github.io/async/docs.html#queue).
9
+
10
+
Basically `tress` is a clone of `queue` from famous [`caolan/async`](https://github.com/caolan/async) but without all other implements of that Swiss Army Knife of asynchronous code. Although `tress` was intended to be an improved, better maintained and more safe alternative to `caolan/async.queue`, but first and foremost `tress` is backward compatible. It means, that everywhere you use `async.queue` (except undocumented features) you can write `tress` instead and it __must__ work.
11
+
12
+
You can do like this:
13
+
14
+
```js
15
+
// old code:
16
+
varasync=require('async');
17
+
var q =async.queue(function(job, done){/*...*/});
18
+
/*...*/
19
+
20
+
// new code:
21
+
var tress =require('tress');
22
+
var q =tress(function(job, done){/*...*/});
23
+
/*...*/
24
+
25
+
```
26
+
27
+
Every code using `caolan/async.queue`__must__ work with `tress`. If it does not work exactly the same way, please [start the issue](https://github.com/astur/tress/issues).
28
+
29
+
All documentation of `caolan/async.queue` is right for `tress`, but it doesn't describe it completely. Any way, you can use `tress` only with [this](http://caolan.github.io/async/docs.html#queue) documentation (instead of referencs below) and don't even think about any extra features. Only exception - `tress` require `node >=8` and doesn't work in browsers.
30
+
31
+
Main difference between `tress` and `caolan/async.queue` is that in `tress` job not disappear after worker finished. It moves to `failed` or `finished` (depends of `done` first argument) and can be used later. Second difference is that in `tress` fields of queue object are more safe. They are readable/writable only in correct way. Also `tress` has some new fields in queue object (see in reference).
32
+
8
33
## Install
9
34
10
35
```bash
@@ -21,9 +46,9 @@ var q = tress(function(job, done){
21
46
console.log('hello '+job.name);
22
47
someAsyncFunction(job, function(err, data){
23
48
if (err) {
24
-
done(err);
49
+
done(err, 'some message');
25
50
} else {
26
-
done(null, data);
51
+
done(null, 'anything');
27
52
}
28
53
});
29
54
}, 2);
@@ -56,115 +81,69 @@ q.unshift({name: 'Cristobal Jose Junta'});
56
81
57
82
```
58
83
59
-
## Quick Guide
60
-
61
-
Basically `tress` is a clone of [`queue`](http://caolan.github.io/async/docs.html#queue) from famous [`caolan/async`](https://github.com/caolan/async) but without all other implements of that Swiss Army Knife of asynchronous code. Although `tress` was intended to be an extended and more safe alternative of `caolan/async.queue`, but first and foremost `tress` is backward compatible. It means, that everywhere you use `async.queue` (except undocumented features) you can write `tress` instead and it __must__ work.
62
-
63
-
You can do like this:
64
-
65
-
```js
66
-
// old code:
67
-
varasync=require('async');
68
-
var q =async.queue(function(job, done){/*...*/});
69
-
/*...*/
70
-
71
-
// new code:
72
-
var tress =require('tress');
73
-
var q =tress(function(job, done){/*...*/});
74
-
/*...*/
75
-
76
-
```
77
-
78
-
Every code using `caolan/async.queue`__must__ work with `tress`. If it does not work exactly the same way, please [start the issue](https://github.com/astur/tress/issues).
79
-
80
-
All documentation of `caolan/async.queue` is right for `tress`, but it doesn't describe it completely. Any way, you can use `tress` only with [this](http://caolan.github.io/async/docs.html#queue) documentation and don't even think about any extra features.
81
-
82
-
Only exception - `tress` require `Node.js 4+` and doesn't work in browsers.
83
-
84
-
Main difference between `tress` and `caolan/async.queue` is that in `tress` job not disappear after worker finished. It moves to `failed` or `finished` (depends of `done` first argument) and can be used later.
85
-
86
-
Second difference is that in `tress` fields of queue object are more safe. They are readable/writable only in correct way.
87
-
88
-
Also `tress` has some new fields in queue object.
89
-
90
84
## Reference
91
85
92
86
`tress(worker, [concurrency])` creates queue object that will store jobs and process them with `worker` function in parallel (up to the `concurrency` limit).
93
87
94
88
__Arguments:__
95
89
96
-
`worker(job, done)` - An asynchronous function for processing a queued `job`, which must call its `done` argument when finished. Callback `done` may take various argumens, but first argument must be error (if job failed), null/undefined (if job successfully finished) or boolean (if job returned to queue head (if `true`) or to queue tail (if `false`)).
97
-
`concurrency` - An integer for determining how many worker functions should be run in parallel. If omitted, the concurrency defaults to 1. If negative - no parallel and delay between worker functions (concurrency -1000 sets 1 second delay).
90
+
*`worker(job, done)` - An asynchronous function for processing a queued data.
91
+
*`job` - data for processing. It may be primitive or object.
92
+
*`done(err, ...args)` - callback function. `worker` must call `done` when finished. `done` may take various argumens, but first argument must be error (if job failed), null/undefined (if job successfully finished) or boolean (if job need to be returned to queue head (if `true`) or to queue tail (if `false`) for retry).
93
+
*`concurrency` - An integer for determining how many worker functions should be run in parallel. Defaults to 1. If negative - no parallel and delay between worker calls (concurrency -1000 sets 1 second delay).
98
94
99
95
__Queue object properties__
100
96
101
-
`started` - still `false` till any items have been pushed and processed by the queue. Than became `true` and never change in queue lifecycle (Not writable).
102
-
103
-
`concurrency` - This property for alter the concurrency/delay on-the-fly.
104
-
105
-
`buffer` A minimum threshold buffer in order to say that the queue is unsaturated.
106
-
107
-
`paused` - a boolean for determining whether the queue is in a paused state. Not writable (use `pause()` and `resume()` instead).
108
-
109
-
`waiting` (___new___) - array of queued jobs.
110
-
111
-
`active` (___new___) - array of jobs currently being processed.
112
-
113
-
`failed` (___new___) - array of failed jobs (`done` callback was called from worker with error in first argument).
114
-
115
-
`finished` (___new___) - array of correctly finished jobs (`done` callback was called from worker with `null` or `undefined` (or any other `false` equivalent) in first argument).
97
+
*`started` - still `false` till any items have been pushed and processed by the queue. Than became `true` and never change in queue lifecycle (Not writable).
98
+
*`concurrency` - This property for alter the concurrency/delay on-the-fly.
99
+
*`buffer` A minimum threshold buffer in order to say that the queue is unsaturated. Defaults to `concurrency / 4`.
100
+
*`paused` - a boolean for determining whether the queue is in a paused state. Not writable (use `pause()` and `resume()` instead).
101
+
*`waiting` (___new___) - array of queued jobs.
102
+
*`active` (___new___) - array of jobs currently being processed.
103
+
*`failed` (___new___) - array of failed jobs (`done` callback was called from worker with error in first argument).
104
+
*`finished` (___new___) - array of correctly finished jobs (`done` callback was called from worker with `null` or `undefined` in first argument).
116
105
117
106
_Note, that properties `waiting`, `active`, `failed` and `finished` are not writable, but they point to arrays, that you can cahge manually. Do it carefully._
118
107
119
108
__Queue object methods__
120
109
121
110
_Note, that in `tress` you can't rewrite methods._
122
111
123
-
`push(job, [callback])` - add a new job to the queue. Instead of a single job, a jobs array can be submitted.
124
-
125
-
`unshift(job, [callback])` - add a new job to the front of the queue. Instead of a single job, a jobs array can be submitted.
126
-
127
-
_Note, that if you pass callback to `push` or `unshift` as second argument, `tress` calls this callback once the worker has finished processing the job._
128
-
129
-
`pause()` - a function that pauses the processing of jobs until `resume()` is called.
130
-
131
-
`resume()` - a function that resumes the processing of queued jobs when the queue is paused.
132
-
133
-
`kill()` - a function that removes the drain callback and empties remaining jobs from the queue forcing it to go idle.
134
-
135
-
`length()` - a function returning the number of items waiting to be processed.
136
-
137
-
`running()` - a function returning the number of items currently being processed.
138
-
139
-
`workersList()` - a function returning the array of items currently being processed.
140
-
141
-
`idle()` - a function returning false if there are items waiting or being processed, or true if not.
142
-
143
-
`save(callback)` (___new___) - a function that runs a callback with object, that contains arrays of `waiting`, `failed`, and `finished` jobs. If there are any `active` jobs at the moment, they will be concatenated to `waiting` array.
144
-
145
-
`load(data)` (___new___) - a function that loads new arrays from `data` object to `waiting`, `failed`, and `finished` arrays and sets `active` to empty array. Rise error if `started` is `true`.
146
-
147
-
`status(job)` (___new___) a function returning the status of `job` (`'waiting'`, `'running'`, `'finished'`, `'pending'` or `'missing'`).
112
+
*`push(job, [callback])` or `unshift(job, [callback])` - add a new job to the queue. `push` adds job to queue tail and `unshift` to queue head.
113
+
*`job` - Job to add to queue. Instead of a single job, array of jobs can be submitted.
114
+
*`callback(err, ...args)` - optional callback. `tress` calls this callback once the worker has finished processing the job. With same arguments as in `done` callback.
115
+
*`pause()` - a function that pauses the processing of new jobs until `resume()` is called.
116
+
*`resume()` - a function that resumes the processing of queued jobs paused by `pause`.
117
+
*`kill()` - a function that removes the drain callback and empties queued jobs.
118
+
*`remove()` a function that removes job from queue.
119
+
*`length()` - a function returning the number of items waiting to be processed.
120
+
*`running()` - a function returning the number of items currently being processed.
121
+
*`workersList()` - a function returning the array of items currently being processed.
122
+
*`idle()` - a function returning false if there are items waiting or being processed, or true if not.
123
+
*`save(callback)` (___new___) - a function that runs a callback with object, that contains arrays of `waiting`, `failed`, and `finished` jobs. If there are any `active` jobs at the moment, they will be concatenated to `waiting` array.
124
+
*`load(data)` (___new___) - a function that loads new arrays from `data` object to `waiting`, `failed`, and `finished` arrays and sets `active` to empty array. Rise error if `started` is `true`.
125
+
*`status(job)` (___new___) a function returning the status of `job` (`'waiting'`, `'running'`, `'finished'`, `'pending'` or `'missing'`).
148
126
149
127
__Queue objects callbacks__
150
128
151
-
_You can assign callback function to this six properties. Note, you can't call any of that function manually via `tress` property._
152
-
153
-
`saturated` - a callback that is called when the number of running workers hits the concurrency limit, and further jobs will be queued.
154
-
155
-
`unsaturated` - a callback that is called when the number of running workers is less than the concurrency & buffer limits, and further jobs will not be queued.
156
-
157
-
`empty` - a callback that is called when the last item from the queue is given to a worker.
158
-
159
-
`drain` - a callback that is called when the last item from the queue has returned from the worker.
160
-
161
-
`error` (___new___) - a callback that is called when job failed (worker call `done` with error as first argument).
129
+
_You can assign callback function to this six properties. Note, you can't call any of that calbacks manually._
130
+
*`saturated()` - a callback that is called when the number of running workers hits the concurrency limit, and further jobs will be queued.
131
+
*`unsaturated()` - a callback that is called when the number of running workers is less than the concurrency & buffer limits, and further jobs will not be queued.
132
+
*`empty()` - a callback that is called when the last item from the queue is given to a worker.
133
+
*`drain()` - a callback that is called when the last item from the queue has been returned from the worker.
134
+
*`error(...args)` (___new___) - a callback that is called when job failed (worker call `done` with error as first argument).
135
+
*`success(err, ...args)` (___new___) - a callback that is called when job correctly finished (worker call `done` with `null` or `undefined` as first argument).
136
+
*`retry(err, ...args)` (___new___) - a callback that is called when job returned to queue for retry (worker call `done` with boolean as first argument).
162
137
163
-
`success` (___new___) - a callback that is called when job correctly finished (worker call `done` with `null` or `undefined` as first argument).
138
+
__Job lifecycle__
164
139
165
-
`retry` (___new___) - a callback that is called when job returned to queue (worker call `done` with boolean as first argument).
140
+
_After pushing/unshifting to queue every job passes through the following steps:_
166
141
167
-
_Note, that `error`/`success` is called after job has been moved from `active` to `failed`/`finished` and after job callback (from `push`/`unshift`) was called._
142
+
* processed in worker
143
+
* sent to `done` (as `this` or explicitly as one of parameters if you prefere arraw-callbacks)
144
+
* moved from `active` to `failed`/`finished`/`waiting` (wether first parameter of `done` is error, null or boolean)
145
+
* sent to job callback (from `push`/`unshift`)
146
+
* sent to `error`/`success`/`retry` callback (wether first parameter of `done` is error, null or boolean)
0 commit comments