-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Seqerakit docs port #296
base: master
Are you sure you want to change the base?
Seqerakit docs port #296
Changes from 8 commits
17fea83
e16db95
70d9dcf
9782642
8ccc850
0af8b1e
4ad8624
192122a
71965f9
199f110
31523f2
26a4b2f
8f8da47
be69c8d
8fd5458
99dacc7
e765c9b
bfbf602
e6a5c1c
c798662
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,110 @@ | ||||||||||||||||||||||||||||||||||||||||
--- | ||||||||||||||||||||||||||||||||||||||||
title: "Commands" | ||||||||||||||||||||||||||||||||||||||||
description: "seqerakit command options" | ||||||||||||||||||||||||||||||||||||||||
date: "21 Oct 2024" | ||||||||||||||||||||||||||||||||||||||||
tags: [seqerakit, cli, automation, commands] | ||||||||||||||||||||||||||||||||||||||||
--- | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
Use the `--help` or `-h` parameter to list available commands and options: | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
```bash | ||||||||||||||||||||||||||||||||||||||||
seqerakit --help | ||||||||||||||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
### Input | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
`seqerakit` supports input through paths to YAML configuration files or directly from standard input (stdin). | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
- Using file path: | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
```bash | ||||||||||||||||||||||||||||||||||||||||
seqerakit /path/to/file.yaml | ||||||||||||||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
- Using stdin: | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
```console | ||||||||||||||||||||||||||||||||||||||||
$ cat file.yaml | seqerakit - | ||||||||||||||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||||||||||||||
llewellyn-sl marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
See [Define your YAML file using CLI options](#define-your-yaml-file-using-cli-options) for guidance on formatting your input YAML files. | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
### Dryrun | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
Confirm that your configuration and command are correct before creating resources in your Seqera account, particularly when automating the end-to-end creation of multiple entities at once. To print the commands that would be executed with Platform CLI when using a YAML file, run your seqerakit command with the `--dryrun` flag: | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
```bash | ||||||||||||||||||||||||||||||||||||||||
seqerakit file.yaml --dryrun | ||||||||||||||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
### Specify targets | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
When using a YAML file as input that defines multiple resources, use the `--targets` flag to specify which resources to create. This flag accepts a comma-separated list of resource names. | ||||||||||||||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it worth providing the list of possible resource names to be created via YAML config? I've been meaning to add this to the main README as well.
Suggested change
|
||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
For example, given a `test.yml` file that defines the following resources: | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
```yaml | ||||||||||||||||||||||||||||||||||||||||
workspaces: | ||||||||||||||||||||||||||||||||||||||||
- name: 'workspace-1' | ||||||||||||||||||||||||||||||||||||||||
organization: 'seqerakit' | ||||||||||||||||||||||||||||||||||||||||
... | ||||||||||||||||||||||||||||||||||||||||
compute-envs: | ||||||||||||||||||||||||||||||||||||||||
- name: 'compute-env' | ||||||||||||||||||||||||||||||||||||||||
type: 'aws-batch forge' | ||||||||||||||||||||||||||||||||||||||||
workspace: 'seqerakit/workspace-1' | ||||||||||||||||||||||||||||||||||||||||
... | ||||||||||||||||||||||||||||||||||||||||
pipelines: | ||||||||||||||||||||||||||||||||||||||||
- name: "hello-world" | ||||||||||||||||||||||||||||||||||||||||
url: "https://github.com/nextflow-io/hello" | ||||||||||||||||||||||||||||||||||||||||
workspace: 'seqerakit/workspace-1' | ||||||||||||||||||||||||||||||||||||||||
compute-env: "compute-env" | ||||||||||||||||||||||||||||||||||||||||
... | ||||||||||||||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
You can target the creation of `pipelines` only by running: | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
```bash | ||||||||||||||||||||||||||||||||||||||||
seqerakit test.yml --targets pipelines | ||||||||||||||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
This will process only the pipelines block from the YAML file and ignore `workspaces` and `compute-envs`. | ||||||||||||||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
To create both workspaces and pipelines, run: | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
```bash | ||||||||||||||||||||||||||||||||||||||||
seqerakit test.yml --targets workspaces,pipelines | ||||||||||||||||||||||||||||||||||||||||
llewellyn-sl marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
### Delete resources | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
Instead of adding or creating resources, specify the `--delete` flag to recursively delete resources in your YAML file: | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
```bash | ||||||||||||||||||||||||||||||||||||||||
seqerakit file.yml --delete | ||||||||||||||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
For example, if you have a `file.yml` that defines an organization, workspace, team, credentials, and compute environment that have already been created, run `seqerakit file.yml --delete` to recursively delete the same resources. | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
### Use `tw`-specific CLI options | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
Specify `tw`-specific CLI options with the `--cli=` flag: | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
```bash | ||||||||||||||||||||||||||||||||||||||||
seqerakit file.yaml --cli="--arg1 --arg2" | ||||||||||||||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
See [CLI commands](../cli/commands.mdx) or run `tw -h` for the full list of options. | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
:::note | ||||||||||||||||||||||||||||||||||||||||
The `--verbose` option for `tw` CLI is currently not supported by `seqerakit`. | ||||||||||||||||||||||||||||||||||||||||
::: | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
#### Example: HTTP-only connections | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
The Platform CLI expects to connect to a Seqera instance that is secured by a TLS certificate. If your Seqera Enterprise instance does not present a certificate, you must run your `tw` commands with the `--insecure` flag. | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
To use `tw`-specific CLI options such as `--insecure`, use the `--cli=` flag, followed by the options to use enclosed in double quotes: | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
```bash | ||||||||||||||||||||||||||||||||||||||||
seqerakit file.yaml --cli="--insecure" | ||||||||||||||||||||||||||||||||||||||||
``` |
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,117 @@ | ||||||
--- | ||||||
title: "Installation" | ||||||
description: "seqerakit installation options" | ||||||
date: "21 Oct 2024" | ||||||
tags: [seqerakit, cli, automation, installation] | ||||||
--- | ||||||
|
||||||
|
||||||
`seqerakit` is a Python wrapper that sets [Platform CLI](../cli/overview.mdx) command options using YAML configuration files. Individual seqerakit commands and configuration parameters can be chained together to automate the end-to-end creation of all Seqera Platform entities. | ||||||
llewellyn-sl marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
As an extension of the Platform CLI, seqerakit enables: | ||||||
|
||||||
- **Infrastructure as code**: Users manage and provision their infrastructure from the command line. | ||||||
- **Simple configuration**: All Platform CLI command-line options can be defined in simple YAML format. | ||||||
- **Automation**: End-to-end creation of Seqera entities, from adding an organization to launching pipelines. | ||||||
|
||||||
### Installation | ||||||
|
||||||
`seqerakit` has three dependencies: | ||||||
|
||||||
1. [Seqera Platform CLI (`>=0.9.2`)](https://github.com/seqeralabs/tower-cli/releases) | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Wondering if we should bump the minimum version to v0.10.1 here and in the main docs to avoid issues with pipeline name matching and reflection config errors reported by java, amongst other things. |
||||||
2. [Python (`>=3.8`)](https://www.python.org/downloads/) | ||||||
3. [PyYAML](https://pypi.org/project/PyYAML/) | ||||||
|
||||||
#### Pip | ||||||
|
||||||
If you already have [Seqera Platform CLI](../cli/installation.mdx) and Python installed on your system, install `seqerakit` directly from [PyPI](https://pypi.org/project/seqerakit/): | ||||||
|
||||||
```console | ||||||
pip install seqerakit | ||||||
``` | ||||||
|
||||||
Overwrite an existing installation to use the latest version: | ||||||
|
||||||
```console | ||||||
pip install --upgrade --force-reinstall seqerakit | ||||||
``` | ||||||
|
||||||
#### Conda | ||||||
|
||||||
To install `seqerakit` and its dependencies via Conda, first configure the correct channels: | ||||||
|
||||||
```console | ||||||
conda config --add channels bioconda | ||||||
conda config --add channels conda-forge | ||||||
conda config --set channel_priority strict | ||||||
``` | ||||||
|
||||||
Then create a conda environment with `seqerakit` installed: | ||||||
|
||||||
```console | ||||||
conda env create -n seqerakit seqerakit | ||||||
conda activate seqerakit | ||||||
``` | ||||||
|
||||||
#### Local development installation | ||||||
|
||||||
Install the development branch of `seqerakit` on your local machine to test the latest features and updates: | ||||||
llewellyn-sl marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
1. You must have [Python](https://www.python.org/downloads/) and [Git](https://git-scm.com/downloads) installed on your system. | ||||||
1. To install directly from pip: | ||||||
```bash | ||||||
pip install git+https://github.com/seqeralabs/seqera-kit.git@dev | ||||||
``` | ||||||
1. Alternatively, clone the repository locally and install manually: | ||||||
```bash | ||||||
git clone https://github.com/seqeralabs/seqera-kit.git | ||||||
cd seqera-kit | ||||||
git checkout dev | ||||||
pip install . | ||||||
``` | ||||||
1. Verify your installation: | ||||||
```bash | ||||||
pip show seqerakit | ||||||
``` | ||||||
|
||||||
### Configuration | ||||||
|
||||||
Create a [Seqera](https://cloud.seqera.io/) access token via **Your Tokens** in the user menu. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
`seqerakit` reads your access token from the `TOWER_ACCESS_TOKEN` environment variable: | ||||||
|
||||||
```bash | ||||||
export TOWER_ACCESS_TOKEN=<Your Seqera access token> | ||||||
``` | ||||||
|
||||||
For Enterprise installations, specify the custom API endpoint used to connect to Seqera. Export the API endpoint environment variable: | ||||||
|
||||||
```bash | ||||||
export TOWER_API_ENDPOINT=<Seqera Enterprise API URL> | ||||||
``` | ||||||
|
||||||
By default, this is set to `https://api.cloud.seqera.io` to connect to Seqera Cloud. | ||||||
|
||||||
### Usage | ||||||
|
||||||
To confirm the installation of `seqerakit`, configuration of the Platform CLI, and connection to Seqera is working as expected run this command: | ||||||
|
||||||
```bash | ||||||
seqerakit --info | ||||||
``` | ||||||
|
||||||
This runs the `tw info` command under the hood. | ||||||
|
||||||
Use `--version` or `-v` to retrieve the current version of your `seqerakit` installation: | ||||||
|
||||||
```bash | ||||||
seqerakit --version | ||||||
``` | ||||||
|
||||||
Use the `--help` or `-h` parameter to list the available commands and their associated options: | ||||||
|
||||||
```bash | ||||||
seqerakit --help | ||||||
``` | ||||||
|
||||||
See [Commands](./commands.mdx) for detailed instructions to use `seqerakit`. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
--- | ||
title: "Overview" | ||
description: "An overview of seqerakit" | ||
date: "10 Oct 2024" | ||
tags: [seqerakit, cli, automation] | ||
--- | ||
|
||
`seqerakit` is a Python wrapper that allows you to set [Platform CLI](../cli/overview.mdx) command options with YAML configuration files. Individual seqerakit commands and configuration parameters can be chained together to automate the end-to-end creation of all Seqera Platform entities. | ||
|
||
### Key features | ||
|
||
As an extension of the Platform CLI, seqerakit enables: | ||
|
||
- **Infrastructure as code**: Users manage and provision their infrastructure from the command line. | ||
- **Simple configuration**: All Platform CLI command-line options can be defined in simple YAML format. | ||
llewellyn-sl marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- **Automation**: End-to-end creation of Seqera Platform entities, from adding an organization to launching pipelines. | ||
|
||
### Availability | ||
|
||
seqerakit can be installed on macOS, Windows, and Linux. It is compatible with [Seqera Platform Cloud](https://cloud.seqera.io/) and Enterprise versions 21.08 and later. | ||
llewellyn-sl marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
See [Installation](./installation.mdx) to get started. | ||
|
||
:::info | ||
You can also invoke and run `seqerakit` via a [Python script](https://github.com/seqeralabs/seqera-kit#launch-via-a-python-script). | ||
::: |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
--- | ||
title: "Quickstart" | ||
description: "seqerakit Hello World pipeline example" | ||
date: "21 Oct 2024" | ||
tags: [seqerakit, cli, automation, commands, hello world] | ||
--- | ||
|
||
You must provide a YAML file that defines the options for each of the entities you would like to create in Seqera Platform. | ||
|
||
You will need a Seqera account (see [Plans and pricing](https://seqera.io/pricing/)). You will also need access to a workspace and a pre-defined compute environment where you can launch a pipeline. | ||
llewellyn-sl marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
### Launch via YAML | ||
|
||
1. Create a YAML file called `hello-world-config.yml` with the contents below, and customize the `<YOUR_WORKSPACE>` and `<YOUR_COMPUTE_ENVIRONMENT>` entries as needed: | ||
|
||
```yaml | ||
launch: | ||
- name: 'hello-world' # Workflow name | ||
workspace: '<YOUR_WORKSPACE>' # Workspace name | ||
compute-env: '<YOUR_COMPUTE_ENVIRONMENT>' # Compute environment | ||
revision: 'master' # Pipeline revision | ||
pipeline: 'https://github.com/nextflow-io/hello' # Pipeline URL | ||
``` | ||
|
||
2. Launch the pipeline with `seqerakit`: | ||
|
||
```bash | ||
seqerakit hello-world-config.yml | ||
``` | ||
|
||
3. Log in to your Seqera instance and check the **Runs** page in the appropriate workspace for the pipeline you just launched. | ||
|
||
### Launch via a Python script | ||
|
||
You can also launch the same pipeline via a Python script. This allows you to extend Platform CLI functionality by leveraging the flexibility and customization options available in Python. | ||
|
||
1. Download the [`launch_hello_world.py`](./examples/python/launch_hello_world.py) Python script and customize the `<YOUR_WORKSPACE>` and `<YOUR_COMPUTE_ENVIRONMENT>` entries as needed. | ||
|
||
2. Launch the pipeline with `seqerakit`: | ||
|
||
```bash | ||
python launch_hello_world.py | ||
``` | ||
|
||
3. Log in to your Seqera instance and check the **Runs** page in the appropriate workspace for the pipeline you just launched. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't want to be overly pedantic as us SciDev's are guilty of this as well but should we decide on whether we want to use
.yaml
or.yml
? Seems like we should standardize on.yaml
.