- Author: Nick Baker
- Version: 1.0.0
- License: MIT
- Website: http://www.webtechnick.com
Complete tool to background and schedule your time consuming tasks. Queue up almost anything from CakePHP shells, models, and actions to standard php commands or even shell commands.
- 1.0.0 Initial release
Clone the repository into your app/Plugin/Queue directory:
git clone https://github.com/webtechnick/CakePHP-Queue-Plugin.git app/Plugin/Queue
Or you can install via http://getcomposer.org
Load the plugin in your app/Config/bootstrap.php file:
CakePlugin::load('Queue');
Run the schema into your database to install the required tables.
cake schema create --plugin Queue
Create the file app/Config/queue.php with the defaults from app/Plugin/Queue/Config/queue.php.default:
$config = array(
'Queue' => array(
'log' => true, //logs every task run in a log file.
'limit' => 1, //limit how many queues can run at a time. (default 1).
'allowedTypes' => array( //restrict what type of command can be queued.
1, //model
2, //shell
3, //url
//4, //php_cmd
//5, //shell_cmd
),
'archiveAfterExecute' => true, //will archive the task once finished executing for quicker queues
'cache' => '+5 minute', //how long to cache cpu usage and list. (false disables cache)
'cores' => '8', //number of cpu cores to correctly gauge current CPU load.
)
);
Once you start adding things to your queue you need to process it. You can do it a few ways.
-
By setting up a cron (recommended)
crontab -e */1 * * * * /path/to/app/Plugin/Queue/scripts/process_queue.sh
Note: The cron script assumes Console/cake is executable from the root of your app directory.
-
By processing the Queue via the built in shell
cake Queue.queue process
-
By processing the Queue via the built in library
App::uses('Queue','Queue.Lib'); Queue::process();
-
By navigating to the queue plugin admin web interface and processing the Queue (feature not available yet)
Add a Task to the queue.
$ cake Queue.queue add "Queue.QueueTask::find('first')"
Task succesfully added.
525387a1-2dd0-4100-a48f-4f4be017215a queued model
Command: Queue.QueueTask::find('first')
Priority: 100
Process the Queue.
$ cake Queue.queue process
Processing Queue.
Success.
View the Task added.
$ cake Queue.queue view 525387a1-2dd0-4100-a48f-4f4be017215a
525387a1-2dd0-4100-a48f-4f4be017215a finished model
Command: Queue.QueueTask::find('first')
Priority: 100
Executed on Monday 7th of October 2013 10:19:10 PM. And took 0 ms.
Result: {"QueueTask":{"id":"525387a1-2dd0-4100-a48f-4f4be017215a","user_id":null,"created":"2013-10-07 22:18:41","modified":"2013-10-07 22:19:10","executed":null,"scheduled":null,"scheduled_end":null,"reschedule":null,"start_time":"1381205950","end_time":null,"cpu_limit":null,"is_restricted":false,"priority":"100","status":"2","type":"1","command":"Queue.QueueTask::find('first')","result":null,"execution_time":null,"type_human":"model","status_human":"in progress"}}
Adding a Task to the queue.
App::uses('Queue', 'Queue.Lib');
$task = Queue::add("Queue.QueueTask::find('first')");
/* $task =
'QueueTask' => array(
'priority' => (int) 100,
'command' => 'Queue.QueueTask::find('first')',
'type' => (int) 1,
'scheduled' => null,
'scheduled_end' => null,
'reschedule' => null,
'cpu_limit' => null,
'modified' => '2013-10-07 22:22:36',
'created' => '2013-10-07 22:22:36',
'user_id' => null,
'id' => '5253888c-ae18-4ffd-991a-4436e017215a'
) */
Process the queue.
$result = Queue::process();
/* $result will be boolean true */
View the Task.
$task = Queue::view('5253888c-ae18-4ffd-991a-4436e017215a');
/* $task is the string representation, same as queue view. Not as useful */
$task = Queue::findById('5253888c-ae18-4ffd-991a-4436e017215a');
/* $task is now an associative array, much more useful.
array(
'id' => '52538a36-df1c-4186-a50a-4076e017215a',
'user_id' => null,
'created' => '2013-10-07 22:29:42',
'modified' => '2013-10-07 22:29:42',
'executed' => '2013-10-07 22:29:42',
'scheduled' => null,
'scheduled_end' => null,
'reschedule' => null,
'start_time' => '1381206582',
'end_time' => '1381206582',
'cpu_limit' => null,
'is_restricted' => false,
'priority' => '100',
'status' => '3',
'type' => '1',
'command' => 'Queue.QueueTask::find('first')',
'result' => '{"QueueTask":{"id":"524b0c44-a3a0-4956-8428-dc3ee017215a","user_id":null,"created":"2013-10-01 11:54:12","modified":"2013-10-01 11:54:12","executed":null,"scheduled":null,"scheduled_end":null,"reschedule":null,"start_time":null,"end_time":null,"cpu_limit":null,"is_restricted":false,"priority":"100","status":"1","type":"1","command":"SomeModel::action(\"param\",\"param2\")","result":"","execution_time":null,"type_human":"model","status_human":"queued"}}',
'execution_time' => '0',
'type_human' => 'model',
'status_human' => 'finished'
) */
Once you've decided how you're going to process your queue it's time to start adding tasks to your queue.
To add tasks use the built in shell or library.
Queue::add($command, $type, $options = array());
cake Queue.queue add "command" -t type
There are 5 types of commands you can use. The default is model.
-
Model : Model function to execute. Examples:
#Command strings Model::action() Plugin.Model::action("param1","pararm2")
#Adding the command to the queue. Queue::add("Plugin.Model::action('param1','pararm2')"); cake Queue.queue add "Plugin.Model::action('param1','pararm2')"
Models are setup via ClassRegistry::init() and methods executed on the model object as public method calls. -
Shell : CakePHP shell to execute. Examples:
#Command strings ClearCache.clear_cache shell_command -f flag arg1 arg2
#Adding the command to the queue. Queue::add("shell_command -f flag arg1 arg2", 'shell'); cake Queue.queue add "shell_command -f flag arg1 arg2" -t shell
-
Url : A URL to requestAction. Example:
#Command string /path/to/url
#Adding the command to the queue. Queue::add("/path/to/url", 'url'); cake Queue.queue add "/path/to/url" -t url
-
php_cmd : PHP Command, a simple or complex php command. Examples:
#Command strings 3 + 5 mail('[email protected]','subject','message')
#Adding the command to the queue. Queue::add("mail('[email protected]','subject','message')", 'php_cmd'); cake Queue.queue add "mail('[email protected]','subject','message')" -t php_cmd
-
shell_cmd : Basic Bash shell command. Examples:
#Command string echo 'hello' && echo 'world'
#Adding the command to the queue. Queue::add("echo 'hello' && echo 'world'", 'shell_cmd'); cake Queue.queue add "echo 'hello' && echo 'world'" -t shell_cmd
NOTE php_cmd and shell_cmd are not allowed by default. You have to turn them on in app/Config/queue.php.
You can see what is in the queue at any given time and what is up next by calling Queue::next($limit); or the shell
cake Queue.queue next 10
Retrieving Queue List.
1) 52537d35-95d0-48aa-a48d-416de017215a queued url
Command: /path/to/url
Priority: 100
Restricted By:
Start: 2013-10-11 03:00:00
CPU <= 95%
2) 52537d09-6bf0-4961-94f0-4201e017215a queued model
Command: Model::action()
Priority: 100
3) 52535ab8-6298-4ab3-9fc6-4b49e017215a queued shell_cmd
Command: echo 'hello' && echo 'world'
Priority: 100
Restricted By:
Start: 2013-10-07 23:00:00
4) 52537d35-95d0-48aa-a48d-416de017215a queued url
Command: /path/to/url
Priority: 100
Restricted By:
Start: 2013-10-11 03:00:00
CPU <= 95%
You'll see a list of your upcomming queues including restrictions you set on your tasks.
There are a number of restrictions you can assign to Tasks that will modify their order in the queue and affect when the task executes. Restricted Tasks gain priority over Non-restriction tasks (basic queued tasks). The Queue process will look for restricted tasks that match the current state of the server before it looks for non-restriction tasks as non-restriction tasks are by defination not time sensitive.
-
Scheduled Start Restriction: you can schedule a task to not execute until a certain scheduled time has passed.
Queue::add($command, $type, $options = array( 'start' => 'Friday 11pm', ));
cake Queue.queue add "command" -t type --start "Friday 11pm"
-
Schedule End Restriciton: you can specify an end time to make a "window" of execution time. When using this option you must also specify a reschedule option that will go into affect if the "window" is missed.
Queue::add($command, $type, $options = array( 'start' => 'Friday 11pm', 'end' => 'Saturday 5am', 'reschedule' => '+1 week' //if window is missed, will add '+1 week' to 'Friday 11pm' and 'Saturday 5am' ));
cake Queue.queue add "command" -t type --start "Friday 11pm" --end "Saturday 5am" --reschedule "+1 week"
-
Cpu Load Restriction: you can specify a CPU load restriction to only execute task when CPU is below a certain threshold.
Queue::add($command, $type, $options = array( 'cpu' => '95', ));
cake Queue.queue add "command" -t type --cpu "95"
-
Priority Restriction: by default all Tasks added are given 100 in priority. You can change this priority when adding and will allow you to jump ahead of the line. The lower the number the closer to the top of the queue it becomes.
Queue::add($command, $type, $options = array( 'priority' => '1', ));
cake Queue.queue add "command" -t type -p 1
Mix and match your restrictions to have total control over when your tasks execute and where they're placed in the queue.
You can bypass the queue process and explicicty run a task manually. This will bypass any restrictions execute the command and return the result.
//Queue::run($id);
#cake Queue.queue run <id>
$ cake Queue.queue run 52535ab8-6298-4ab3-9fc6-4b49e017215a
Running 52535ab8-6298-4ab3-9fc6-4b49e017215a
Success.
52535ab8-6298-4ab3-9fc6-4b49e017215a finished shell_cmd
Command: echo 'hello' && echo 'world'
Priority: 100
Restricted By:
Start: 2013-10-07 23:00:00
Executed on Monday 7th of October 2013 10:06:27 PM. And took 0 ms.
Result: "hello\nworld\n"
You can view any task, in the queue or in the queue log (archive after execution) at anytime. You'll see all the meta data and if it's a finished task you'll see the result of the task as well as the execution time and how long it took to complete.
//$task = Queue::view($id);
#cake Queue.queue view <id>
$ cake Queue.queue view 52535ab8-6298-4ab3-9fc6-4b49e017215a
52535ab8-6298-4ab3-9fc6-4b49e017215a finished shell_cmd
Command: echo 'hello' && echo 'world'
Priority: 100
Restricted By:
Start: 2013-10-07 23:00:00
Executed on Monday 7th of October 2013 10:06:27 PM. And took 0 ms.
Result: "hello\nworld\n"
You can view the current tasks in progress via the library or shell.
$queue = Queue::inProgress();
cake Queue.queue in_progress
NOTE: You can also get just the progress count by running cake Queue.queue in_progress_count or Queue::inProgressCount();
Remove tasks from the queue. Note this will not remove tasks that are currently in progress by default.
//Queue::remove($id);
cake Queue.queue remove <id>
NOTE: You can force a task to be removed even if it's in progress by passing true into Queue::remove($id, true); or cake queue.queue remove <id> -f
Enjoy the plugin!