Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seqerakit docs port #296

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 110 additions & 0 deletions platform_versioned_docs/version-24.1/seqerakit/commands.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
---
title: "Commands"
description: "seqerakit command options"
date: "21 Oct 2024"
tags: [seqerakit, cli, automation, commands]
---

Use the `--help` or `-h` parameter to list available commands and options:

```bash
seqerakit --help
```

### Input

`seqerakit` supports input through paths to YAML configuration files or directly from standard input (stdin).

- Using file path:

```bash
seqerakit /path/to/file.yaml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't want to be overly pedantic as us SciDev's are guilty of this as well but should we decide on whether we want to use .yaml or .yml? Seems like we should standardize on .yaml.

```

- Using stdin:

```console
$ cat file.yaml | seqerakit -
```
llewellyn-sl marked this conversation as resolved.
Show resolved Hide resolved

See [Define your YAML file using CLI options](#define-your-yaml-file-using-cli-options) for guidance on formatting your input YAML files.

### Dryrun

Confirm that your configuration and command are correct before creating resources in your Seqera account, particularly when automating the end-to-end creation of multiple entities at once. To print the commands that would be executed with Platform CLI when using a YAML file, run your seqerakit command with the `--dryrun` flag:

```bash
seqerakit file.yaml --dryrun
```

### Specify targets

When using a YAML file as input that defines multiple resources, use the `--targets` flag to specify which resources to create. This flag accepts a comma-separated list of resource names.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth providing the list of possible resource names to be created via YAML config? I've been meaning to add this to the main README as well.

Suggested change
When using a YAML file as input that defines multiple resources, use the `--targets` flag to specify which resources to create. This flag accepts a comma-separated list of resource names.
When using a YAML file as input that defines multiple resources, use the `--targets` flag to specify which resources to create. This flag accepts a comma-separated list of resource names.
Supported resource names include:
- actions
- compute-envs
- credentials
- datasets
- labels
- launch
- members
- organizations
- participants
- pipelines
- secrets
- teams
- workspaces


For example, given a `test.yml` file that defines the following resources:

```yaml
workspaces:
- name: 'workspace-1'
organization: 'seqerakit'
...
compute-envs:
- name: 'compute-env'
type: 'aws-batch forge'
workspace: 'seqerakit/workspace-1'
...
pipelines:
- name: "hello-world"
url: "https://github.com/nextflow-io/hello"
workspace: 'seqerakit/workspace-1'
compute-env: "compute-env"
...
```

You can target the creation of `pipelines` only by running:

```bash
seqerakit test.yml --targets pipelines
```

This will process only the pipelines block from the YAML file and ignore `workspaces` and `compute-envs`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This will process only the pipelines block from the YAML file and ignore `workspaces` and `compute-envs`.
This will process only the pipelines block from the YAML file and ignore creation of `workspaces` and `compute-envs`.


To create both workspaces and pipelines, run:

```bash
seqerakit test.yml --targets workspaces,pipelines
llewellyn-sl marked this conversation as resolved.
Show resolved Hide resolved
```

### Delete resources

Instead of adding or creating resources, specify the `--delete` flag to recursively delete resources in your YAML file:

```bash
seqerakit file.yml --delete
```

For example, if you have a `file.yml` that defines an organization, workspace, team, credentials, and compute environment that have already been created, run `seqerakit file.yml --delete` to recursively delete the same resources.

### Use `tw`-specific CLI options

Specify `tw`-specific CLI options with the `--cli=` flag:

```bash
seqerakit file.yaml --cli="--arg1 --arg2"
```

See [CLI commands](../cli/commands.mdx) or run `tw -h` for the full list of options.

:::note
The `--verbose` option for `tw` CLI is currently not supported by `seqerakit`.
:::

#### Example: HTTP-only connections

The Platform CLI expects to connect to a Seqera instance that is secured by a TLS certificate. If your Seqera Enterprise instance does not present a certificate, you must run your `tw` commands with the `--insecure` flag.

To use `tw`-specific CLI options such as `--insecure`, use the `--cli=` flag, followed by the options to use enclosed in double quotes:

```bash
seqerakit file.yaml --cli="--insecure"
```
117 changes: 117 additions & 0 deletions platform_versioned_docs/version-24.1/seqerakit/installation.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
title: "Installation"
description: "seqerakit installation options"
date: "21 Oct 2024"
tags: [seqerakit, cli, automation, installation]
---


`seqerakit` is a Python wrapper that sets [Platform CLI](../cli/overview.mdx) command options using YAML configuration files. Individual seqerakit commands and configuration parameters can be chained together to automate the end-to-end creation of all Seqera Platform entities.
llewellyn-sl marked this conversation as resolved.
Show resolved Hide resolved

As an extension of the Platform CLI, seqerakit enables:

- **Infrastructure as code**: Users manage and provision their infrastructure from the command line.
- **Simple configuration**: All Platform CLI command-line options can be defined in simple YAML format.
- **Automation**: End-to-end creation of Seqera entities, from adding an organization to launching pipelines.

### Installation

`seqerakit` has three dependencies:

1. [Seqera Platform CLI (`>=0.9.2`)](https://github.com/seqeralabs/tower-cli/releases)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if we should bump the minimum version to v0.10.1 here and in the main docs to avoid issues with pipeline name matching and reflection config errors reported by java, amongst other things.

2. [Python (`>=3.8`)](https://www.python.org/downloads/)
3. [PyYAML](https://pypi.org/project/PyYAML/)

#### Pip

If you already have [Seqera Platform CLI](../cli/installation.mdx) and Python installed on your system, install `seqerakit` directly from [PyPI](https://pypi.org/project/seqerakit/):

```console
pip install seqerakit
```

Overwrite an existing installation to use the latest version:

```console
pip install --upgrade --force-reinstall seqerakit
```

#### Conda

To install `seqerakit` and its dependencies via Conda, first configure the correct channels:

```console
conda config --add channels bioconda
conda config --add channels conda-forge
conda config --set channel_priority strict
```

Then create a conda environment with `seqerakit` installed:

```console
conda env create -n seqerakit seqerakit
conda activate seqerakit
```

#### Local development installation

Install the development branch of `seqerakit` on your local machine to test the latest features and updates:
llewellyn-sl marked this conversation as resolved.
Show resolved Hide resolved

1. You must have [Python](https://www.python.org/downloads/) and [Git](https://git-scm.com/downloads) installed on your system.
1. To install directly from pip:
```bash
pip install git+https://github.com/seqeralabs/seqera-kit.git@dev
```
1. Alternatively, clone the repository locally and install manually:
```bash
git clone https://github.com/seqeralabs/seqera-kit.git
cd seqera-kit
git checkout dev
pip install .
```
1. Verify your installation:
```bash
pip show seqerakit
```

### Configuration

Create a [Seqera](https://cloud.seqera.io/) access token via **Your Tokens** in the user menu.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Create a [Seqera](https://cloud.seqera.io/) access token via **Your Tokens** in the user menu.
Create a [Seqera Platform](https://cloud.seqera.io/tokens) access token via **Your Tokens** in the user menu.


`seqerakit` reads your access token from the `TOWER_ACCESS_TOKEN` environment variable:

```bash
export TOWER_ACCESS_TOKEN=<Your Seqera access token>
```

For Enterprise installations, specify the custom API endpoint used to connect to Seqera. Export the API endpoint environment variable:

```bash
export TOWER_API_ENDPOINT=<Seqera Enterprise API URL>
```

By default, this is set to `https://api.cloud.seqera.io` to connect to Seqera Cloud.

### Usage

To confirm the installation of `seqerakit`, configuration of the Platform CLI, and connection to Seqera is working as expected run this command:

```bash
seqerakit --info
```

This runs the `tw info` command under the hood.

Use `--version` or `-v` to retrieve the current version of your `seqerakit` installation:

```bash
seqerakit --version
```

Use the `--help` or `-h` parameter to list the available commands and their associated options:

```bash
seqerakit --help
```

See [Commands](./commands.mdx) for detailed instructions to use `seqerakit`.
26 changes: 26 additions & 0 deletions platform_versioned_docs/version-24.1/seqerakit/overview.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
title: "Overview"
description: "An overview of seqerakit"
date: "10 Oct 2024"
tags: [seqerakit, cli, automation]
---

`seqerakit` is a Python wrapper that allows you to set [Platform CLI](../cli/overview.mdx) command options with YAML configuration files. Individual seqerakit commands and configuration parameters can be chained together to automate the end-to-end creation of all Seqera Platform entities.

### Key features

As an extension of the Platform CLI, seqerakit enables:

- **Infrastructure as code**: Users manage and provision their infrastructure from the command line.
- **Simple configuration**: All Platform CLI command-line options can be defined in simple YAML format.
llewellyn-sl marked this conversation as resolved.
Show resolved Hide resolved
- **Automation**: End-to-end creation of Seqera Platform entities, from adding an organization to launching pipelines.

### Availability

seqerakit can be installed on macOS, Windows, and Linux. It is compatible with [Seqera Platform Cloud](https://cloud.seqera.io/) and Enterprise versions 21.08 and later.
llewellyn-sl marked this conversation as resolved.
Show resolved Hide resolved

See [Installation](./installation.mdx) to get started.

:::info
You can also invoke and run `seqerakit` via a [Python script](https://github.com/seqeralabs/seqera-kit#launch-via-a-python-script).
:::
46 changes: 46 additions & 0 deletions platform_versioned_docs/version-24.1/seqerakit/quickstart.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
title: "Quickstart"
description: "seqerakit Hello World pipeline example"
date: "21 Oct 2024"
tags: [seqerakit, cli, automation, commands, hello world]
---

You must provide a YAML file that defines the options for each of the entities you would like to create in Seqera Platform.

You will need a Seqera account (see [Plans and pricing](https://seqera.io/pricing/)). You will also need access to a workspace and a pre-defined compute environment where you can launch a pipeline.
llewellyn-sl marked this conversation as resolved.
Show resolved Hide resolved

### Launch via YAML

1. Create a YAML file called `hello-world-config.yml` with the contents below, and customize the `<YOUR_WORKSPACE>` and `<YOUR_COMPUTE_ENVIRONMENT>` entries as needed:

```yaml
launch:
- name: 'hello-world' # Workflow name
workspace: '<YOUR_WORKSPACE>' # Workspace name
compute-env: '<YOUR_COMPUTE_ENVIRONMENT>' # Compute environment
revision: 'master' # Pipeline revision
pipeline: 'https://github.com/nextflow-io/hello' # Pipeline URL
```

2. Launch the pipeline with `seqerakit`:

```bash
seqerakit hello-world-config.yml
```

3. Log in to your Seqera instance and check the **Runs** page in the appropriate workspace for the pipeline you just launched.

### Launch via a Python script

You can also launch the same pipeline via a Python script. This allows you to extend Platform CLI functionality by leveraging the flexibility and customization options available in Python.

1. Download the [`launch_hello_world.py`](./examples/python/launch_hello_world.py) Python script and customize the `<YOUR_WORKSPACE>` and `<YOUR_COMPUTE_ENVIRONMENT>` entries as needed.

2. Launch the pipeline with `seqerakit`:

```bash
python launch_hello_world.py
```

3. Log in to your Seqera instance and check the **Runs** page in the appropriate workspace for the pipeline you just launched.

Loading
Loading