Skip to content

Commit

Permalink
Tweak some PipelineWise documentation (transferwise#950)
Browse files Browse the repository at this point in the history
  • Loading branch information
louis-pie authored Apr 25, 2022
1 parent 4de979f commit 62a0de5
Showing 1 changed file with 5 additions and 10 deletions.
15 changes: 5 additions & 10 deletions docs/user_guide/scheduling.rst
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,15 @@ PipelineWise is tested and can run with at least the following
schedulers:

* `Unix Cron <https://en.wikipedia.org/wiki/Cron/>`_ Unix Cron - This is the simplest option
for a single server installation.
for a single server installation.

* `Cicada <https://github.com/transferwise/cicada/>`_ Cicada Scheduler - A lightweight multi-server
CRON manager

* `Cronicle <https://github.com/jhuckaby/Cronicle/>`_ - Cronicle is a reasonably good and
relatively simple tool to schedule PipelineWise jobs in both Single Server and Multi-Server
cluster installations.

* `Apache Airflow <https://airflow.apache.org/>`_ - Airflow is a robust and mature tool to
schedule and monitor workflows.

Expand All @@ -67,11 +70,3 @@ PipelineWise commands on multiple nodes that operate on the same project, then
every node has to read/write into the same directory, doesn't matter where the nodes are
located. This is typically done by mounting ``${HOME}/.pipelinewise`` on every node to
a shared directory on NFS/EFS.

.. warning::

There are plans to store ``${HOME}/.pipelinewise`` runtime configuration files
optionally on different data stores, like S3, RDBMs or on document stores like
Couchbase or MongoDB. Once it is implemented, Multi-Server Cluster installation
will not require NFS/EFS.

0 comments on commit 62a0de5

Please sign in to comment.