You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -123,41 +129,6 @@ While your application is in [maintenance mode](/docs/{{version}}/configuration#
123
129
124
130
Within Horizon's default configuration file, you will notice a `defaults` configuration option. This configuration option specifies the default values for your application's [supervisors](#supervisors). The supervisor's default configuration values will be merged into the supervisor's configuration for each environment, allowing you to avoid unnecessary repetition when defining your supervisors.
125
131
126
-
<aname="balancing-strategies"></a>
127
-
### Balancing Strategies
128
-
129
-
Unlike Laravel's default queue system, Horizon allows you to choose from three worker balancing strategies: `simple`, `auto`, and `false`. The `simple` strategy splits incoming jobs evenly between worker processes:
130
-
131
-
'balance' => 'simple',
132
-
133
-
The `auto` strategy, which is the configuration file's default, adjusts the number of worker processes per queue based on the current workload of the queue. For example, if your `notifications` queue has 1,000 pending jobs while your `render` queue is empty, Horizon will allocate more workers to your `notifications` queue until the queue is empty.
134
-
135
-
When using the `auto` strategy, you may define the `minProcesses` and `maxProcesses` configuration options to control the minimum number of processes per queue and the maximum number of worker processes in total Horizon should scale up and down to:
136
-
137
-
```php
138
-
'environments' => [
139
-
'production' => [
140
-
'supervisor-1' => [
141
-
'connection' => 'redis',
142
-
'queue' => ['default'],
143
-
'balance' => 'auto',
144
-
'autoScalingStrategy' => 'time',
145
-
'minProcesses' => 1,
146
-
'maxProcesses' => 10,
147
-
'balanceMaxShift' => 1,
148
-
'balanceCooldown' => 3,
149
-
'tries' => 3,
150
-
],
151
-
],
152
-
],
153
-
```
154
-
155
-
The `autoScalingStrategy` configuration value determines if Horizon will assign more worker processes to queues based on the total amount of time it will take to clear the queue (`time` strategy) or by the total number of jobs on the queue (`size` strategy).
156
-
157
-
The `balanceMaxShift` and `balanceCooldown` configuration values determine how quickly Horizon will scale to meet worker demand. In the example above, a maximum of one new process will be created or destroyed every three seconds. You are free to tweak these values as necessary based on your application's needs.
158
-
159
-
When the `balance` option is set to `false`, the default Laravel behavior will be used, wherein queues are processed in the order they are listed in your configuration.
160
-
161
132
<aname="dashboard-authorization"></a>
162
133
### Dashboard Authorization
163
134
@@ -184,6 +155,82 @@ protected function gate(): void
184
155
185
156
Remember that Laravel automatically injects the authenticated user into the gate closure. If your application is providing Horizon security via another method, such as IP restrictions, then your Horizon users may not need to "login". Therefore, you will need to change `function (User $user)` closure signature above to `function (User $user = null)` in order to force Laravel to not require authentication.
186
157
158
+
<aname="max-job-attempts"></a>
159
+
### Max Job Attempts
160
+
161
+
> [!NOTE]
162
+
> Before refining these options, make sure you are familiar with Laravel's default [queue services](/docs/{{version}}/queues#max-job-attempts-and-timeout) and the concept of 'attempts'.
163
+
164
+
You can define the maximum number of attempts a job can consume within a supervisor's configuration:
165
+
166
+
```php
167
+
'environments' => [
168
+
'production' => [
169
+
'supervisor-1' => [
170
+
// ...
171
+
'tries' => 10,
172
+
],
173
+
],
174
+
],
175
+
```
176
+
177
+
> [!NOTE]
178
+
> This option is similar to the `--tries` option when using the Artisan command to process queues.
179
+
180
+
Adjusting the `tries` option is essential when using middlewares such as `WithoutOverlapping` or `RateLimited` because they consume attempts. To handle this, adjust the `tries` configuration value either at the supervisor level or by defining the `$tries` property on the job class.
181
+
182
+
If you don’t set the `tries` option, Horizon defaults to a single attempt, unless the job class defines `$tries`, which takes precedence over the Horizon configuration.
183
+
184
+
Setting `tries` or `$tries` to 0 allows unlimited attempts, which is ideal when the number of attempts is uncertain. To prevent endless failures, you can limit the number of exceptions allowed by setting the `$maxExceptions` property on the job class.
185
+
186
+
<aname="job-timeout"></a>
187
+
### Job Timeout
188
+
189
+
Similarly, you can set a `timeout` value at the supervisor level, which specifies how many seconds a worker process can run a job before it's forcefully terminated. Once terminated, the job will either be retried or marked as failed, depending on your queue configuration:
190
+
191
+
```php
192
+
'environments' => [
193
+
'production' => [
194
+
'supervisor-1' => [
195
+
// ...¨
196
+
'timeout' => 60,
197
+
],
198
+
],
199
+
],
200
+
```
201
+
202
+
> [!WARNING]
203
+
> The `timeout` value should always be at least a few seconds shorter than the `retry_after` value defined in your `config/queue.php` configuration file. Otherwise, your jobs may be processed twice.
204
+
205
+
<aname="job-backoff"></a>
206
+
### Job Backoff
207
+
208
+
You can define the `backoff` value at the supervisor level to specify how long Horizon should wait before retrying a job that encounters an unhandled exception:
209
+
210
+
```php
211
+
'environments' => [
212
+
'production' => [
213
+
'supervisor-1' => [
214
+
// ...
215
+
'backoff' => 10,
216
+
],
217
+
],
218
+
],
219
+
```
220
+
221
+
You may also configure "exponential" backoffs by using an array for the `backoff` value. In this example, the retry delay will be 1 second for the first retry, 5 seconds for the second retry, 10 seconds for the third retry, and 10 seconds for every subsequent retry if there are more attempts remaining:
222
+
223
+
```php
224
+
'environments' => [
225
+
'production' => [
226
+
'supervisor-1' => [
227
+
// ...
228
+
'backoff' => [1, 5, 10],
229
+
],
230
+
],
231
+
],
232
+
```
233
+
187
234
<aname="silenced-jobs"></a>
188
235
### Silenced Jobs
189
236
@@ -208,6 +255,174 @@ class ProcessPodcast implements ShouldQueue, Silenced
208
255
}
209
256
```
210
257
258
+
<aname="balancing-strategies"></a>
259
+
## Balancing Strategies
260
+
261
+
Each supervisor can process one or more queues but unlike Laravel's default queue system, Horizon allows you to choose from three worker balancing strategies: `auto`, `simple`, and `false`.
262
+
263
+
<aname="auto-balancing"></a>
264
+
### Auto Balancing
265
+
266
+
The `auto` strategy, which is the default strategy, adjusts the number of worker processes per queue based on the current workload of the queue. For example, if your `notifications` queue has 1,000 pending jobs while your `default` queue is empty, Horizon will allocate more workers to your `notifications` queue until the queue is empty.
267
+
268
+
When using the `auto` strategy, you may also configure the `minProcesses` and `maxProcesses` configuration options:
269
+
270
+
<divclass="content-list"markdown="1">
271
+
272
+
-`minProcesses` defines the minimum number of worker processes per queue. This value must be greater than or equal to 1.
273
+
-`maxProcesses` defines the maximum total number of worker processes Horizon may scale up to across all queues. This value should typically be greater than the number of queues multiplied by the `minProcesses` value. To prevent the supervisor from spawning any processes, you may set this value to 0.
274
+
275
+
</div>
276
+
277
+
For example, you may configure Horizon to maintain at least one process per queue and scale up to a total of 10 worker processes:
278
+
279
+
```php
280
+
'environments' => [
281
+
'production' => [
282
+
'supervisor-1' => [
283
+
'connection' => 'redis',
284
+
'queue' => ['default', 'notifications'],
285
+
'balance' => 'auto',
286
+
'autoScalingStrategy' => 'time',
287
+
'minProcesses' => 1,
288
+
'maxProcesses' => 10,
289
+
'balanceMaxShift' => 1,
290
+
'balanceCooldown' => 3,
291
+
],
292
+
],
293
+
],
294
+
```
295
+
296
+
The `autoScalingStrategy` configuration option determines how Horizon will assign more worker processes to queues. You can choose between two strategies:
297
+
298
+
<divclass="content-list"markdown="1">
299
+
300
+
- The `time` strategy will assign workers based on the total estimated amount of time it will take to clear the queue.
301
+
- The `size` strategy will assign workers based on the total number of jobs on the queue.
302
+
303
+
</div>
304
+
305
+
The `balanceMaxShift` and `balanceCooldown` configuration values determine how quickly Horizon will scale to meet worker demand. In the example above, a maximum of one new process will be created or destroyed every three seconds. You are free to tweak these values as necessary based on your application's needs.
306
+
307
+
<aname="auto-queue-priorities"></a>
308
+
#### Queue Priorities and Auto Balancing
309
+
310
+
When using the `auto` balancing strategy, Horizon does not enforce strict priority between queues. The order of queues in a supervisor's configuration does not affect how worker processes are assigned. Instead, Horizon relies on the selected `autoScalingStrategy` to dynamically allocate worker processes based on queue load.
311
+
312
+
For example, in the following configuration, the high queue is not prioritized over the default queue, despite appearing first in the list:
313
+
314
+
```php
315
+
'environments' => [
316
+
'production' => [
317
+
'supervisor-1' => [
318
+
// ...
319
+
'queue' => ['high', 'default'],
320
+
'minProcesses' => 1,
321
+
'maxProcesses' => 10,
322
+
],
323
+
],
324
+
],
325
+
```
326
+
327
+
If you need to enforce a relative priority between queues, you may define multiple supervisors and explicitly allocate processing resources:
328
+
329
+
```php
330
+
'environments' => [
331
+
'production' => [
332
+
'supervisor-1' => [
333
+
// ...
334
+
'queue' => ['default'],
335
+
'minProcesses' => 1,
336
+
'maxProcesses' => 10,
337
+
],
338
+
'supervisor-2' => [
339
+
// ...
340
+
'queue' => ['images'],
341
+
'minProcesses' => 1,
342
+
'maxProcesses' => 1,
343
+
],
344
+
],
345
+
],
346
+
```
347
+
348
+
In this example, the default `queue` can scale up to 10 processes, while the `images` queue is limited to one process. This configuration ensures that your queues can scale independently.
349
+
350
+
> [!NOTE]
351
+
> When dispatching resource-intensive jobs, it's sometimes best to assign them to a dedicated queue with a limited `maxProcesses` value. Otherwise, these jobs could consume excessive CPU resources and overload your system.
352
+
353
+
<aname="simple-balancing"></a>
354
+
### Simple Balancing
355
+
356
+
The `simple` strategy distributes worker processes evenly across the specified queues. With this strategy, Horizon does not automatically scale the number of worker processes. Rather, it uses a fixed number of processes:
357
+
358
+
```php
359
+
'environments' => [
360
+
'production' => [
361
+
'supervisor-1' => [
362
+
// ...
363
+
'queue' => ['default', 'notifications'],
364
+
'balance' => 'simple',
365
+
'processes' => 10,
366
+
],
367
+
],
368
+
],
369
+
```
370
+
371
+
In the example above, Horizon will assign 5 processes to each queue, splitting the total of 10 evenly.
372
+
373
+
If you'd like to control the number of worker processes assigned to each queue individually, you can define multiple supervisors:
374
+
375
+
```php
376
+
'environments' => [
377
+
'production' => [
378
+
'supervisor-1' => [
379
+
// ...
380
+
'queue' => ['default'],
381
+
'balance' => 'simple',
382
+
'processes' => 10,
383
+
],
384
+
'supervisor-notifications' => [
385
+
// ...
386
+
'queue' => ['notifications'],
387
+
'balance' => 'simple',
388
+
'processes' => 2,
389
+
],
390
+
],
391
+
],
392
+
```
393
+
394
+
With this configuration, Horizon will assign 10 processes to the `default` queue and 2 processes to the `notifications` queue.
395
+
396
+
<aname="no-balancing"></a>
397
+
### No Balancing
398
+
399
+
When the `balance` option is set to `false`, Horizon processes queues strictly in the order they're listed, similar to Laravel’s default queue system. However, it will still scale the number of worker processes if jobs begin to accumulate:
400
+
401
+
```php
402
+
'environments' => [
403
+
'production' => [
404
+
'supervisor-1' => [
405
+
// ...
406
+
'queue' => ['default', 'notifications'],
407
+
'balance' => false,
408
+
'minProcesses' => 1,
409
+
'maxProcesses' => 10,
410
+
],
411
+
],
412
+
],
413
+
```
414
+
415
+
In the example above, jobs in the `default` queue are always prioritized over jobs in the `notifications` queue. For instance, if there are 1,000 jobs in `default` and only 10 in `notifications`, Horizon will fully process all `default` jobs before handling any from `notifications`.
416
+
417
+
You can control Horizon's ability to scale worker processes using the `minProcesses` and `maxProcesses` options:
418
+
419
+
<divclass="content-list"markdown="1">
420
+
421
+
-`minProcesses` defines the minimum number of worker processes in total. This value must be greater than or equal to 1.
422
+
-`maxProcesses` defines the maximum total number of worker processes Horizon may scale up to.
Copy file name to clipboardExpand all lines: queues.md
+37-1Lines changed: 37 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -607,6 +607,8 @@ public function middleware(): array
607
607
}
608
608
```
609
609
610
+
Releasing an overlapping job back onto the queue will still increment the job's total number of attempts. You may wish to tune your `tries` and `maxExceptions` properties on your job class accordingly. For example, leaving the `tries` property to 1 as it is by default would prevent any overlapping job from being retried later.
611
+
610
612
Any overlapping jobs of the same type will be released back to the queue. You may also specify the number of seconds that must elapse before the released job will be attempted again:
611
613
612
614
```php
@@ -1243,7 +1245,26 @@ class ProcessPodcast implements ShouldQueue
1243
1245
<aname="max-attempts"></a>
1244
1246
#### Max Attempts
1245
1247
1246
-
If one of your queued jobs is encountering an error, you likely do not want it to keep retrying indefinitely. Therefore, Laravel provides various ways to specify how many times or for how long a job may be attempted.
1248
+
Job attempts are a core concept of Laravel's queue system and power many advanced features. While they may seem confusing at first, it's important to understand how they work before modifying the default configuration.
1249
+
1250
+
When a job is dispatched, it is pushed onto the queue. A worker then picks it up and attempts to execute it. This is a job attempt.
1251
+
1252
+
However, an attempt does not necessarily mean the job's `handle` method was executed. Attempts can also be "consumed" in several ways:
1253
+
1254
+
<divclass="content-list"markdown="1">
1255
+
1256
+
- The job encounters an unhandled exception during execution.
1257
+
- The job is manually released back to the queue using `$this->release()`.
1258
+
- Middleware such as `WithoutOverlapping` or `RateLimited` fails to acquire a lock and releases the job.
1259
+
- The job timed out.
1260
+
- The job's `handle` method runs and completes without throwing an exception.
1261
+
1262
+
</div>
1263
+
1264
+
You likely do not want to keep attempting a job indefinitely. Therefore, Laravel provides various ways to specify how many times or for how long a job may be attempted.
1265
+
1266
+
> [!NOTE]
1267
+
> By default, Laravel will only attempt a job once. If your job uses middleware like `WithoutOverlapping` or `RateLimited`, or if you're manually releasing jobs, you will likely need to increase the number of allowed attempts via the `tries` option.
1247
1268
1248
1269
One approach to specifying the maximum number of times a job may be attempted is via the `--tries` switch on the Artisan command line. This will apply to all jobs processed by the worker unless the job being processed specifies the number of times it may be attempted:
1249
1270
@@ -1404,6 +1425,9 @@ If you would like to indicate that a job should be marked as [failed](#dealing-w
1404
1425
public $failOnTimeout = true;
1405
1426
```
1406
1427
1428
+
> [!NOTE]
1429
+
> By default, when a job times out, it consumes one attempt and is released back to the queue (if retries are allowed). However, if you configure the job to fail on timeout, it will not be retried, regardless of the value set for tries.
1430
+
1407
1431
<aname="error-handling"></a>
1408
1432
### Error Handling
1409
1433
@@ -2282,6 +2306,18 @@ class ProcessPodcast implements ShouldQueue
2282
2306
> [!WARNING]
2283
2307
> A new instance of the job is instantiated before invoking the `failed` method; therefore, any class property modifications that may have occurred within the `handle` method will be lost.
2284
2308
2309
+
A failed job is not necessarily one that encountered an unhandled exception. A job may also be considered failed when it has exhausted all of its allowed attempts. These attempts can be consumed in several ways:
2310
+
2311
+
<divclass="content-list"markdown="1">
2312
+
2313
+
- The job timed out.
2314
+
- The job encounters an unhandled exception during execution.
2315
+
- The job is released back to the queue either manually or by a middleware.
2316
+
2317
+
</div>
2318
+
2319
+
If the final attempt fails due to an exception thrown during job execution, that exception will be passed to the job's failed method. However, if the job fails because it has reached the maximum number of allowed attempts, the `$exception` will be an instance of `Illuminate\Queue\MaxAttemptsExceededException`. Similarly, if the job fails due to exceeding the configured timeout, the `$exception` will be an instance of `Illuminate\Queue\TimeoutExceededException`.
0 commit comments