From 22677131e01e46722526920f56889b27bae8e145 Mon Sep 17 00:00:00 2001 From: James Sammut Date: Mon, 29 Apr 2024 12:23:13 +1000 Subject: [PATCH 01/20] BB Pipelines docs initial --- docs/Bitbucket.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 docs/Bitbucket.md diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md new file mode 100644 index 00000000..da65a32d --- /dev/null +++ b/docs/Bitbucket.md @@ -0,0 +1,28 @@ +# Bitbucket Pipelines + +The Buildkite Migration tool's currently supported (✅), partially supported (⚠️) and unsupported (❌) properties in translation of Bitbucket pipelines to Buildkite pipelines are listed below. + +### Clone (`clone`) + +| Key | Supported? | Notes | +| --- | --- | --- | + +### Definitions (`definitions`) + +| Key | Supported? | Notes | +| --- | --- | --- | + +### Image (`image`) + +| Key | Supported? | Notes | +| --- | --- | --- | + +### Options (`options`) + +| Key | Supported? | Notes | +| --- | --- | --- | + +### Pipelines (`pipelines`) + +| Key | Supported? | Notes | +| --- | --- | --- | From 77858378123a464c1eff92906e9aa82d2a9785ca Mon Sep 17 00:00:00 2001 From: James Sammut Date: Tue, 30 Apr 2024 08:41:39 +1000 Subject: [PATCH 02/20] BB Pipeline docs fillout #1 --- docs/Bitbucket.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index da65a32d..bb303cd3 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -6,6 +6,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | +| `clone` | ❌ | Clone options for all steps of a Bitbucket Pipeline. These options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. | ### Definitions (`definitions`) @@ -16,6 +17,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | +| `image` | ✅ | The container image that is to be applied for each step within a Bitbucket Pipeline. Images set at this level will be applied to all steps within a Buildkite pipeline, utilising the specified image within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). This has lower precedence over per-step `image` configuration (see `pipelines.default.step.image`). | ### Options (`options`) @@ -26,3 +28,8 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | + +| `pipelines.default.step.clone` | ❌ | Clone options for a specific step of a Bitbucket Pipeline. These options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. | +| `pipelines.default.step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket Pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). | +| `pipelines.default.step.name` | ✅ | The name of a specific step within a Bitbucket Pipeline. Translates to a Buildkite command step's `label`. | +| `pipelines.default.step.script` | ✅ | The individual commands that make up a specific step. Each is translated into a singular command within the `commands` key of a Buildkite command step. | \ No newline at end of file From 71cac95ddd3053b86c89fb6ecae8221605d95cf3 Mon Sep 17 00:00:00 2001 From: James Sammut Date: Tue, 30 Apr 2024 08:58:22 +1000 Subject: [PATCH 03/20] BB Pipeline docs fillout #2 --- docs/Bitbucket.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index bb303cd3..da27df98 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -28,8 +28,12 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | - | `pipelines.default.step.clone` | ❌ | Clone options for a specific step of a Bitbucket Pipeline. These options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. | +| `pipelines.default.step.deployment` | ❌ | The environment set for the Bitbucket Deployments dashboard. This has no translatable equivalent within Buildkite. | +| `pipelines.default.step.fail-fast` | ❌ | Whether a specific step of a Bitbucket Pipeline allows a parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | | `pipelines.default.step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket Pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). | +| `pipelines.default.step.max-time` | ✅ | The maximum allowable time that a step within a Bitbucket Pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. | | `pipelines.default.step.name` | ✅ | The name of a specific step within a Bitbucket Pipeline. Translates to a Buildkite command step's `label`. | -| `pipelines.default.step.script` | ✅ | The individual commands that make up a specific step. Each is translated into a singular command within the `commands` key of a Buildkite command step. | \ No newline at end of file +| `pipelines.default.step.runs-on` | ✅ | Allocating the Bitbucket Pipeline to run on a self-hosted runner with the specific label. All `runs-on` values will be set as agent [tags](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | +| `pipelines.default.step.script` | ✅ | The individual commands that make up a specific step. Each is translated into a singular command within the `commands` key of a Buildkite command step. | +| `pipelines.default.step.size` | ✅ | Allocation of sizing options for the given memory for a specific step within a Bitbucket Pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | \ No newline at end of file From 46998dc99716e0f070890e8d165558f4bf1fae3c Mon Sep 17 00:00:00 2001 From: James Sammut Date: Tue, 30 Apr 2024 10:25:37 +1000 Subject: [PATCH 04/20] BB Pipeline docs fillout #3 --- docs/Bitbucket.md | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index da27df98..ec8a6269 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -6,7 +6,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | -| `clone` | ❌ | Clone options for all steps of a Bitbucket Pipeline. These options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. | +| `clone` | ⚠️ | Clone options for all steps of a Bitbucket Pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property) | ### Definitions (`definitions`) @@ -28,12 +28,27 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | -| `pipelines.default.step.clone` | ❌ | Clone options for a specific step of a Bitbucket Pipeline. These options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. | +| `pipelines.branches` | ✅ | Branch configuration to apply certain Bitbucket Pipeline configuration based for specific branches. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `build.branch`/`BUILDKITE_BRANCH` variable. | +| `pipelines.branches.` | ✅ | The branch name/wildcard to apply specific Bitbucket Pipeline step configuration within. | +| `pipelines.branches..step` | ✅ | Step configuration for a specific branch within a Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported attributes. | +| `pipelines.default.step.after-script` | ❌ | The actions that a Bitbucket Pipeline will undertake after the commands in the `script` key are run. For similar hehaviour, a [repository-level](https://buildkite.com/docs/agent/v3/hooks#hook-locations-repository-hooks) `pre-exit` hook approach will yield similar behaviour - running at the latter end of the [job lifecycle](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks). | +| `pipelines.default.step.artifacts` | ✅ | Build artifacts that will be required for steps later in the Bitbucket Pipeline. Artifacts that are specified (whether one specific file, or multiple) will be set within the generated Buildkite pipeline's command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | +| `pipelines.default.step.caches` | ✅ | Step-level dependencies downloaded from external sources (Docker, Maven, PyPi for example) which will be able to be re-used in later Bitbucket Pipeline steps. Caches that are set at step level are translated in the corresponding Buildkite pipeline utilising the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) to store the downloaded dependencies for re-use between Buildkite builds. | +| `pipelines.default.step.condition` | ✅ | The configuration to prevent a Bitbucket Pipeline step from running unless the specific conditional is met. Translated to an inline conditional (`if`) within the corresponding Buildkite pipelines' command step's `commands` - based on a `git diff` of the base branch.| +| `pipelines.default.step.condition.changeset.includePaths` | ✅ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories. | +| `pipelines.default.step.clone` | ⚠️ | Clone options for a specific step of a Bitbucket Pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property) | +| `pipelines.default.step.clone.sparse-checkout` | ✅ | Sparse-checkout option for a Bitbucket Pipeline step. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin). | +| `pipelines.default.step.clone.sparse-checkout.code-mode` | ✅ | Whether the checkout patterns are considered to be a list of patterns (passed as the `--no-cone` flag to `git sparse-checkout` command). | +| `pipelines.default.step.clone.sparse-checkout.enabled` | ✅ | Whether sparse checkout is enabled for the Bitbucket Pipeline step. | +| `pipelines.default.step.clone.sparse-checkout.patterns` | ✅ | The list of paths to invoke a sparse checkout for. Can be a pattern to a specific file, directory, or wildcard for all files belonging within a certain directory. | | `pipelines.default.step.deployment` | ❌ | The environment set for the Bitbucket Deployments dashboard. This has no translatable equivalent within Buildkite. | +| `pipelines.default.step.docker` | ❌ | The availability of docker in Bitbucket Pipelines steps. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | | `pipelines.default.step.fail-fast` | ❌ | Whether a specific step of a Bitbucket Pipeline allows a parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | | `pipelines.default.step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket Pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). | | `pipelines.default.step.max-time` | ✅ | The maximum allowable time that a step within a Bitbucket Pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. | | `pipelines.default.step.name` | ✅ | The name of a specific step within a Bitbucket Pipeline. Translates to a Buildkite command step's `label`. | +| `pipelines.default.step.oidc` | ✅ | Open ID Connect configuration that will be applied for this Bitbucket Pipeline step. The generated command step in the corresponding Buildkite pipeline will [request](https://buildkite.com/docs/agent/v3/cli-oidc#request-oidc-token) an OIDC token and export it into the job environment as `BITBUCKET_STEP_OIDC_TOKEN` (to be passed to `sts` to assume an AWS role for example) | | `pipelines.default.step.runs-on` | ✅ | Allocating the Bitbucket Pipeline to run on a self-hosted runner with the specific label. All `runs-on` values will be set as agent [tags](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | | `pipelines.default.step.script` | ✅ | The individual commands that make up a specific step. Each is translated into a singular command within the `commands` key of a Buildkite command step. | -| `pipelines.default.step.size` | ✅ | Allocation of sizing options for the given memory for a specific step within a Bitbucket Pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | \ No newline at end of file +| `pipelines.default.step.size` | ✅ | Allocation of sizing options for the given memory for a specific step within a Bitbucket Pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | +| `pipelines.default.step.trigger` | ✅ | The configuration setting the running a Bitbucket Pipeline step manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline before the specified `script` within a further command step. Explicit dependencies with `depends_on` are set between the two steps; requiring manual processing. | \ No newline at end of file From 33276da212e863787bc9b8aab18f47b42b4696f6 Mon Sep 17 00:00:00 2001 From: James Sammut Date: Tue, 30 Apr 2024 11:09:25 +1000 Subject: [PATCH 05/20] BB Pipeline docs fillout #4 --- docs/Bitbucket.md | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index ec8a6269..6f324dc0 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -6,12 +6,20 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | -| `clone` | ⚠️ | Clone options for all steps of a Bitbucket Pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property) | +| `clone` | ⚠️ | Clone options for all steps of a Bitbucket Pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property). `clone` properties at the Bitbucket Pipeline step level take higher precedent over these global properties if set. | +| `clone.sparse-checkout` | ✅ | Sparse-checkout option for all Bitbucket Pipeline steps. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin). | +| `clone.sparse-checkout.code-mode` | ✅ | Whether the checkout patterns are considered to be a list of patterns (passed as the `--no-cone` flag to `git sparse-checkout` command). | +| `clone.sparse-checkout.enabled` | ✅ | Whether sparse checkout is enabled for all Bitbucket Pipeline steps. | +| `clone.sparse-checkout.patterns` | ✅ | The list of paths to invoke a sparse checkout for. Can be a pattern to a specific file, directory, or wildcard for all files belonging within a certain directory. | ### Definitions (`definitions`) | Key | Supported? | Notes | | --- | --- | --- | +| `definitions.caches` | ✅ | Customiserd cache defitions that can be appled to specific Bitbucket Pipeline steps - inclusive of folders, single file - or multifile cache. Targeted into specific steps with the `pipelines.default.step.caches.` property. | +| `definitions.caches.` | ✅ | A customiserd cache name applicable for one or more steps within a Bitbucket Pipeline. | +| `definitions.caches..path` | ✅ | The directory path that is desired to be cached. | +| `definitions.caches..key.files` | ✅ | The list (one or more) files that are monitored for changes - and stored once its hash changes between file versions change. | ### Image (`image`) @@ -23,6 +31,9 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | +| `options.docker` | ❌ | The availability of docker in all Bitbucket Pipelines steps. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | +| `options.max-time` | ✅ | The maximum allowable time that all step within a Bitbucket Pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. Bitbucket Pipeline step-level definition (`pipelines.default.step.max-time`) will override this value. | +| `options.size` | ✅ | Allocation of sizing options for the given memory for all steps within a Bitbucket Pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. Bitbucket Pipeline step-level definition (`pipelines.default.step.size`) will override this value. | ### Pipelines (`pipelines`) @@ -31,9 +42,12 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | `pipelines.branches` | ✅ | Branch configuration to apply certain Bitbucket Pipeline configuration based for specific branches. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `build.branch`/`BUILDKITE_BRANCH` variable. | | `pipelines.branches.` | ✅ | The branch name/wildcard to apply specific Bitbucket Pipeline step configuration within. | | `pipelines.branches..step` | ✅ | Step configuration for a specific branch within a Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported attributes. | +| `pipelines.custom` | ✅ | Bitbucket Pipelines that are only able to be triggered manually. The corresponding Buildkite pipeline is first generated with an [input step](https://buildkite.com/docs/pipelines/input-step) before any command jobs to ensure that triggered builds must be manually processed. | +| `pipelines.custom..step` | ✅ | Step configuration for a custom Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported attributes. | | `pipelines.default.step.after-script` | ❌ | The actions that a Bitbucket Pipeline will undertake after the commands in the `script` key are run. For similar hehaviour, a [repository-level](https://buildkite.com/docs/agent/v3/hooks#hook-locations-repository-hooks) `pre-exit` hook approach will yield similar behaviour - running at the latter end of the [job lifecycle](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks). | | `pipelines.default.step.artifacts` | ✅ | Build artifacts that will be required for steps later in the Bitbucket Pipeline. Artifacts that are specified (whether one specific file, or multiple) will be set within the generated Buildkite pipeline's command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | -| `pipelines.default.step.caches` | ✅ | Step-level dependencies downloaded from external sources (Docker, Maven, PyPi for example) which will be able to be re-used in later Bitbucket Pipeline steps. Caches that are set at step level are translated in the corresponding Buildkite pipeline utilising the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) to store the downloaded dependencies for re-use between Buildkite builds. | +| `pipelines.default.step.caches` | ✅ | Step-level dependencies downloaded from external sources (Docker, Maven, PyPi for example) which will be able to be re-used in later Bitbucket Pipeline steps. Caches that are set at step level (or though the top-level `definition.cache. property) are translated in the corresponding Buildkite pipeline utilising the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) to store the downloaded dependencies for re-use between Buildkite builds. | +| `pipelines.default.step.caches.` | ✅ | Cache names that are reflective of specific tooling required for the specific Bitbucket pipeline. Default caches consist of `docker`, `composer`, `dotnetcore`, `gradle`, `ivy2`, `maven`, `node`, `pip` and `sbt`. Customised caches can be applied within a `definitions.caches. object and applied at this layer. | | `pipelines.default.step.condition` | ✅ | The configuration to prevent a Bitbucket Pipeline step from running unless the specific conditional is met. Translated to an inline conditional (`if`) within the corresponding Buildkite pipelines' command step's `commands` - based on a `git diff` of the base branch.| | `pipelines.default.step.condition.changeset.includePaths` | ✅ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories. | | `pipelines.default.step.clone` | ⚠️ | Clone options for a specific step of a Bitbucket Pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property) | From 7ba80cde1ea1bdf1c303f7d1135c3ea01b0b2633 Mon Sep 17 00:00:00 2001 From: James Sammut Date: Tue, 30 Apr 2024 11:34:00 +1000 Subject: [PATCH 06/20] BB Pipeline docs fillout #5 --- docs/Bitbucket.md | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index 6f324dc0..8da3e877 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -16,8 +16,8 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | -| `definitions.caches` | ✅ | Customiserd cache defitions that can be appled to specific Bitbucket Pipeline steps - inclusive of folders, single file - or multifile cache. Targeted into specific steps with the `pipelines.default.step.caches.` property. | -| `definitions.caches.` | ✅ | A customiserd cache name applicable for one or more steps within a Bitbucket Pipeline. | +| `definitions.caches` | ✅ | Customised cache definitions that can be applied to specific Bitbucket Pipeline steps - inclusive of folders, single file - or multifile cache. Targeted into specific steps with the `pipelines.default.step.caches.` property. | +| `definitions.caches.` | ✅ | A customised cache name applicable for one or more steps within a Bitbucket Pipeline. | | `definitions.caches..path` | ✅ | The directory path that is desired to be cached. | | `definitions.caches..key.files` | ✅ | The list (one or more) files that are monitored for changes - and stored once its hash changes between file versions change. | @@ -41,14 +41,23 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | --- | --- | --- | | `pipelines.branches` | ✅ | Branch configuration to apply certain Bitbucket Pipeline configuration based for specific branches. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `build.branch`/`BUILDKITE_BRANCH` variable. | | `pipelines.branches.` | ✅ | The branch name/wildcard to apply specific Bitbucket Pipeline step configuration within. | -| `pipelines.branches..step` | ✅ | Step configuration for a specific branch within a Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported attributes. | +| `pipelines.branches..step` | ✅ | Individual step configuration for a specific branch within a Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | | `pipelines.custom` | ✅ | Bitbucket Pipelines that are only able to be triggered manually. The corresponding Buildkite pipeline is first generated with an [input step](https://buildkite.com/docs/pipelines/input-step) before any command jobs to ensure that triggered builds must be manually processed. | -| `pipelines.custom..step` | ✅ | Step configuration for a custom Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported attributes. | -| `pipelines.default.step.after-script` | ❌ | The actions that a Bitbucket Pipeline will undertake after the commands in the `script` key are run. For similar hehaviour, a [repository-level](https://buildkite.com/docs/agent/v3/hooks#hook-locations-repository-hooks) `pre-exit` hook approach will yield similar behaviour - running at the latter end of the [job lifecycle](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks). | +| `pipelines.custom..step` | ✅ | Individual step configuration for a custom Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | +| `pipelines.default.parallel` | ✅ | The grouping of multiple steps within a Bitbucket Pipeline that are to be run concurrently. By default, Buildkite executes steps in parallel unless [implicit or explicit](https://buildkite.com/docs/pipelines/dependencies) dependencies are set. Parallel Bitbucket Pipeline steps are transitioned into a [group step](https://buildkite.com/docs/pipelines/group-step) within the generated Buildkite pipeline without explicit dependencies. | +| `pipelines.default.parallel.fail-fast` | ❌ | Whether a Bitbucket Pipeline allows this parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | +| `pipelines.default.parallel.steps` | ❌ | Individual step configuration for a custom Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | +| `pipelines.default.stage` | ✅ | The logical grouping of one or more Bitbucket Pipeline steps. Bitbucket Pipeline stages are translated into the corresponding Buildkite pipeline as a [group step](https://buildkite.com/docs/pipelines/group-step). | +| `pipelines.default.stage.condition.changeset.includePaths` | ✅ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories. | +| `pipelines.default.stage.name` | ✅ | The name of the Bitbucket Pipeline stage. Transitioned to the `group` label of the corresponding Buildkite [group step](https://buildkite.com/docs/pipelines/group-step). | +| `pipelines.default.stage.steps | ✅ | Individual step configuration for a Bitbucket Pipeline stage. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | +| `pipelines.default.stage.trigger | ✅ | The configuration setting the running of a Bitbucket Pipeline stage manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline's group step before the specified `steps` of this stage. An explicit dependencies with `depends_on` are set between the input step and the following steps to ensure manual processing is required for these to run. | +| `pipelines.default.step` | ✅ | The list (one or more) of individual execution units that make up a workflow within a Bitbucket Pipeline. Run from top-to-bottom, with a 100 step limit. | +| `pipelines.default.step.after-script` | ❌ | The actions that a Bitbucket Pipeline will undertake after the commands in the `script` key are run. For similar behaviour, a [repository-level](https://buildkite.com/docs/agent/v3/hooks#hook-locations-repository-hooks) `pre-exit` hook approach will yield similar behaviour - running at the latter end of the [job lifecycle](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks). | | `pipelines.default.step.artifacts` | ✅ | Build artifacts that will be required for steps later in the Bitbucket Pipeline. Artifacts that are specified (whether one specific file, or multiple) will be set within the generated Buildkite pipeline's command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | | `pipelines.default.step.caches` | ✅ | Step-level dependencies downloaded from external sources (Docker, Maven, PyPi for example) which will be able to be re-used in later Bitbucket Pipeline steps. Caches that are set at step level (or though the top-level `definition.cache. property) are translated in the corresponding Buildkite pipeline utilising the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) to store the downloaded dependencies for re-use between Buildkite builds. | | `pipelines.default.step.caches.` | ✅ | Cache names that are reflective of specific tooling required for the specific Bitbucket pipeline. Default caches consist of `docker`, `composer`, `dotnetcore`, `gradle`, `ivy2`, `maven`, `node`, `pip` and `sbt`. Customised caches can be applied within a `definitions.caches. object and applied at this layer. | -| `pipelines.default.step.condition` | ✅ | The configuration to prevent a Bitbucket Pipeline step from running unless the specific conditional is met. Translated to an inline conditional (`if`) within the corresponding Buildkite pipelines' command step's `commands` - based on a `git diff` of the base branch.| +| `pipelines.default.step.condition` | ✅ | The configuration to prevent a Bitbucket Pipeline step from running unless the specific conditional is met. Translated to an inline conditional (`if`) within the corresponding Buildkite pipelines' command step's `commands` - based on a `git diff` of the base branch. | | `pipelines.default.step.condition.changeset.includePaths` | ✅ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories. | | `pipelines.default.step.clone` | ⚠️ | Clone options for a specific step of a Bitbucket Pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property) | | `pipelines.default.step.clone.sparse-checkout` | ✅ | Sparse-checkout option for a Bitbucket Pipeline step. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin). | @@ -65,4 +74,4 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | `pipelines.default.step.runs-on` | ✅ | Allocating the Bitbucket Pipeline to run on a self-hosted runner with the specific label. All `runs-on` values will be set as agent [tags](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | | `pipelines.default.step.script` | ✅ | The individual commands that make up a specific step. Each is translated into a singular command within the `commands` key of a Buildkite command step. | | `pipelines.default.step.size` | ✅ | Allocation of sizing options for the given memory for a specific step within a Bitbucket Pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | -| `pipelines.default.step.trigger` | ✅ | The configuration setting the running a Bitbucket Pipeline step manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline before the specified `script` within a further command step. Explicit dependencies with `depends_on` are set between the two steps; requiring manual processing. | \ No newline at end of file +| `pipelines.default.step.trigger` | ✅ | The configuration setting the running of a Bitbucket Pipeline step manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline before the specified `script` within a further command step. Explicit dependencies with `depends_on` are set between the two steps; requiring manual processing. | \ No newline at end of file From 857a53e5d2a92881a3862a74b224527e950f5145 Mon Sep 17 00:00:00 2001 From: James Sammut Date: Tue, 30 Apr 2024 12:17:06 +1000 Subject: [PATCH 07/20] BB Pipeline docs fillout #6 --- docs/Bitbucket.md | 31 +++++++++++++++++++++++++++---- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index 8da3e877..b2b68737 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -16,10 +16,20 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | +| `definitions` | ⚠️ | Customised definitions utilised throughout a Bitbucket Pipeline. `caches` and `services` are supported for translation within Buildkite Migration tool. | | `definitions.caches` | ✅ | Customised cache definitions that can be applied to specific Bitbucket Pipeline steps - inclusive of folders, single file - or multifile cache. Targeted into specific steps with the `pipelines.default.step.caches.` property. | | `definitions.caches.` | ✅ | A customised cache name applicable for one or more steps within a Bitbucket Pipeline. | | `definitions.caches..path` | ✅ | The directory path that is desired to be cached. | | `definitions.caches..key.files` | ✅ | The list (one or more) files that are monitored for changes - and stored once its hash changes between file versions change. | +| `definitions.pipeline` | ❌ | Pipelines that are exported for re-use within repositories of the same workspace. Like functionality exists within Buildkite as [Pipeline Templates](https://buildkite.com/docs/pipelines/templates). | +| `definitions.services` | ✅ | Defined Docker services that are applied within a Bitbucket Pipeline. Services defined in a corresponding Bitbucket Pipeline step using the `pipelines.default.step.services` property will have its configuration applied with the use of the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin). By default - configuration will be saved to a `compose.yaml` in the job environment. | +| `definitions.services.` | ✅ | The name of an individual Docker service that will be applied within a specific step of the Bitbucket Pipeline. | + +### Export (`export`) + +| Key | Supported? | Notes | +| --- | --- | --- | +| `export` | ❌ | Bitbucket Premium option of sharing pipeline configuration between workspaces. Not applicable within Buildkite as an attribute - like functionality exists within [Pipeline Templates](https://buildkite.com/docs/pipelines/templates). | ### Image (`image`) @@ -31,6 +41,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | +| `options` | ⚠️ | Customised options utilised throughout a Bitbucket Pipeline. `max-time` and `size` are supported for translation within Buildkite Migration tool. | | `options.docker` | ❌ | The availability of docker in all Bitbucket Pipelines steps. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | | `options.max-time` | ✅ | The maximum allowable time that all step within a Bitbucket Pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. Bitbucket Pipeline step-level definition (`pipelines.default.step.max-time`) will override this value. | | `options.size` | ✅ | Allocation of sizing options for the given memory for all steps within a Bitbucket Pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. Bitbucket Pipeline step-level definition (`pipelines.default.step.size`) will override this value. | @@ -39,7 +50,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | -| `pipelines.branches` | ✅ | Branch configuration to apply certain Bitbucket Pipeline configuration based for specific branches. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `build.branch`/`BUILDKITE_BRANCH` variable. | +| `pipelines.branches` | ✅ | Application of specific Bitbucket Pipeline configuration based for specific branches. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `build.branch`/`BUILDKITE_BRANCH` variable. | | `pipelines.branches.` | ✅ | The branch name/wildcard to apply specific Bitbucket Pipeline step configuration within. | | `pipelines.branches..step` | ✅ | Individual step configuration for a specific branch within a Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | | `pipelines.custom` | ✅ | Bitbucket Pipelines that are only able to be triggered manually. The corresponding Buildkite pipeline is first generated with an [input step](https://buildkite.com/docs/pipelines/input-step) before any command jobs to ensure that triggered builds must be manually processed. | @@ -50,8 +61,8 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | `pipelines.default.stage` | ✅ | The logical grouping of one or more Bitbucket Pipeline steps. Bitbucket Pipeline stages are translated into the corresponding Buildkite pipeline as a [group step](https://buildkite.com/docs/pipelines/group-step). | | `pipelines.default.stage.condition.changeset.includePaths` | ✅ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories. | | `pipelines.default.stage.name` | ✅ | The name of the Bitbucket Pipeline stage. Transitioned to the `group` label of the corresponding Buildkite [group step](https://buildkite.com/docs/pipelines/group-step). | -| `pipelines.default.stage.steps | ✅ | Individual step configuration for a Bitbucket Pipeline stage. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | -| `pipelines.default.stage.trigger | ✅ | The configuration setting the running of a Bitbucket Pipeline stage manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline's group step before the specified `steps` of this stage. An explicit dependencies with `depends_on` are set between the input step and the following steps to ensure manual processing is required for these to run. | +| `pipelines.default.stage.steps` | ✅ | Individual step configuration for a Bitbucket Pipeline stage. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | +| `pipelines.default.stage.trigger` | ✅ | The configuration setting the running of a Bitbucket Pipeline stage manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline's group step before the specified `steps` of this stage. An explicit dependencies with `depends_on` are set between the input step and the following steps to ensure manual processing is required for these to run. | | `pipelines.default.step` | ✅ | The list (one or more) of individual execution units that make up a workflow within a Bitbucket Pipeline. Run from top-to-bottom, with a 100 step limit. | | `pipelines.default.step.after-script` | ❌ | The actions that a Bitbucket Pipeline will undertake after the commands in the `script` key are run. For similar behaviour, a [repository-level](https://buildkite.com/docs/agent/v3/hooks#hook-locations-repository-hooks) `pre-exit` hook approach will yield similar behaviour - running at the latter end of the [job lifecycle](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks). | | `pipelines.default.step.artifacts` | ✅ | Build artifacts that will be required for steps later in the Bitbucket Pipeline. Artifacts that are specified (whether one specific file, or multiple) will be set within the generated Buildkite pipeline's command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | @@ -72,6 +83,18 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | `pipelines.default.step.name` | ✅ | The name of a specific step within a Bitbucket Pipeline. Translates to a Buildkite command step's `label`. | | `pipelines.default.step.oidc` | ✅ | Open ID Connect configuration that will be applied for this Bitbucket Pipeline step. The generated command step in the corresponding Buildkite pipeline will [request](https://buildkite.com/docs/agent/v3/cli-oidc#request-oidc-token) an OIDC token and export it into the job environment as `BITBUCKET_STEP_OIDC_TOKEN` (to be passed to `sts` to assume an AWS role for example) | | `pipelines.default.step.runs-on` | ✅ | Allocating the Bitbucket Pipeline to run on a self-hosted runner with the specific label. All `runs-on` values will be set as agent [tags](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | +| `pipelines.default.step.services` | ✅ | The name of one or more services defined at `definitions.services.` that will be applied for this step. Translated to utilise the service configuration with the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin). | | `pipelines.default.step.script` | ✅ | The individual commands that make up a specific step. Each is translated into a singular command within the `commands` key of a Buildkite command step. | | `pipelines.default.step.size` | ✅ | Allocation of sizing options for the given memory for a specific step within a Bitbucket Pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | -| `pipelines.default.step.trigger` | ✅ | The configuration setting the running of a Bitbucket Pipeline step manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline before the specified `script` within a further command step. Explicit dependencies with `depends_on` are set between the two steps; requiring manual processing. | \ No newline at end of file +| `pipelines.default.step.trigger` | ✅ | The configuration setting the running of a Bitbucket Pipeline step manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline before the specified `script` within a further command step. Explicit dependencies with `depends_on` are set between the two steps; requiring manual processing. | +| `pipelines.default.variables` | ✅ | Custom variables that are passed to Bitbucket Pipeline steps. Each variable defined in a Bitbucket Pipeline step is translated to a Buildkite [input step](https://buildkite.com/docs/pipelines/input-step) with/without defaults and allowed values specified below. | +| `pipelines.default.variables.name` | ✅ | The variables' name: translated to the `key` attribute of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step) (text entry).| +| `pipelines.default.variables.default` | ✅ | The default variable value if no value is set. Set as the `default` attribute within the `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | +| `pipelines.default.variables.description` | ✅ | The description of the variable. Translated to the `text` attribute of a specific `field` within the generated [input step](https://buildkite.com/docs/pipelines/input-step). | +| `pipelines.default.variables.allowed-values` | ✅ | The variable's allowed values: each option is translated to a singular `options` oject with given value of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | +| `pipelines.pull_request` | ✅ | Application of specific Bitbucket Pipeline configuration based for pull requests. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `pull_request.id`/`BUILDKITE_PULL_REQUEST` and `build.branch`/`BUILDKITE_BRANCH` variables. | +| `pipelines.pull_request.` | ✅ | The base branch name/wildcard to apply specific Bitbucket Pipeline step configuration within. Apply the configuration for all builds with a `**` wildcard. | +| `pipelines.pull_request..step` | ✅ | Individual step configuration for pull requests builds within a Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | +| `pipelines.tags` | ✅ | Application of specific Bitbucket Pipeline configuration based for triggered tag builds. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `build.tag`/`BUILDKITE_TAG` variable. | +| `pipelines.tags.` | ✅ | The tag name/wildcard to apply specific Bitbucket Pipeline step configuration within. | +| `pipelines.tags..step` | ✅ | Individual step configuration for triggered tag builds within a Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | \ No newline at end of file From 0f87b81ed20f6bc3bff93083460449411682bbb5 Mon Sep 17 00:00:00 2001 From: James Sammut Date: Tue, 30 Apr 2024 12:27:59 +1000 Subject: [PATCH 08/20] BB Pipeline docs fillout #7 --- docs/Bitbucket.md | 99 ++++++++++++++++++++++++----------------------- 1 file changed, 50 insertions(+), 49 deletions(-) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index b2b68737..2e2cf509 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -6,24 +6,24 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | -| `clone` | ⚠️ | Clone options for all steps of a Bitbucket Pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property). `clone` properties at the Bitbucket Pipeline step level take higher precedent over these global properties if set. | -| `clone.sparse-checkout` | ✅ | Sparse-checkout option for all Bitbucket Pipeline steps. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin). | +| `clone` | ⚠️ | Clone options for all steps of a Bitbucket pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property). `clone` properties at the Bitbucket pipeline step level take higher precedent over these global properties if set. | +| `clone.sparse-checkout` | ✅ | Sparse-checkout option for all Bitbucket pipeline steps. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin). | | `clone.sparse-checkout.code-mode` | ✅ | Whether the checkout patterns are considered to be a list of patterns (passed as the `--no-cone` flag to `git sparse-checkout` command). | -| `clone.sparse-checkout.enabled` | ✅ | Whether sparse checkout is enabled for all Bitbucket Pipeline steps. | +| `clone.sparse-checkout.enabled` | ✅ | Whether sparse checkout is enabled for all Bitbucket pipeline steps. | | `clone.sparse-checkout.patterns` | ✅ | The list of paths to invoke a sparse checkout for. Can be a pattern to a specific file, directory, or wildcard for all files belonging within a certain directory. | ### Definitions (`definitions`) | Key | Supported? | Notes | | --- | --- | --- | -| `definitions` | ⚠️ | Customised definitions utilised throughout a Bitbucket Pipeline. `caches` and `services` are supported for translation within Buildkite Migration tool. | -| `definitions.caches` | ✅ | Customised cache definitions that can be applied to specific Bitbucket Pipeline steps - inclusive of folders, single file - or multifile cache. Targeted into specific steps with the `pipelines.default.step.caches.` property. | -| `definitions.caches.` | ✅ | A customised cache name applicable for one or more steps within a Bitbucket Pipeline. | +| `definitions` | ⚠️ | Customised definitions utilised throughout a Bitbucket pipeline. `caches` and `services` are supported for translation within Buildkite Migration tool. | +| `definitions.caches` | ✅ | Customised cache definitions that can be applied to specific Bitbucket pipeline steps - inclusive of folders, single file - or multifile cache. Targeted into specific steps with the `pipelines.default.step.caches.` property. | +| `definitions.caches.` | ✅ | A customised cache name applicable for one or more steps within a Bitbucket pipeline. | | `definitions.caches..path` | ✅ | The directory path that is desired to be cached. | | `definitions.caches..key.files` | ✅ | The list (one or more) files that are monitored for changes - and stored once its hash changes between file versions change. | | `definitions.pipeline` | ❌ | Pipelines that are exported for re-use within repositories of the same workspace. Like functionality exists within Buildkite as [Pipeline Templates](https://buildkite.com/docs/pipelines/templates). | -| `definitions.services` | ✅ | Defined Docker services that are applied within a Bitbucket Pipeline. Services defined in a corresponding Bitbucket Pipeline step using the `pipelines.default.step.services` property will have its configuration applied with the use of the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin). By default - configuration will be saved to a `compose.yaml` in the job environment. | -| `definitions.services.` | ✅ | The name of an individual Docker service that will be applied within a specific step of the Bitbucket Pipeline. | +| `definitions.services` | ✅ | Defined Docker services that are applied within a Bitbucket pipeline. Services defined in a corresponding Bitbucket pipeline step using the `pipelines.default.step.services` property will have its configuration applied with the use of the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin). By default - configuration will be saved to a `compose.yaml` in the job environment. | +| `definitions.services.` | ✅ | The name of an individual Docker service that will be applied within a specific step of the Bitbucket pipeline. | ### Export (`export`) @@ -35,66 +35,67 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | -| `image` | ✅ | The container image that is to be applied for each step within a Bitbucket Pipeline. Images set at this level will be applied to all steps within a Buildkite pipeline, utilising the specified image within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). This has lower precedence over per-step `image` configuration (see `pipelines.default.step.image`). | +| `image` | ✅ | The container image that is to be applied for each step within a Bitbucket pipeline. Images set at this level will be applied to all steps within a Buildkite pipeline, utilising the specified image within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). This has lower precedence over per-step `image` configuration (see `pipelines.default.step.image`). | ### Options (`options`) | Key | Supported? | Notes | | --- | --- | --- | -| `options` | ⚠️ | Customised options utilised throughout a Bitbucket Pipeline. `max-time` and `size` are supported for translation within Buildkite Migration tool. | -| `options.docker` | ❌ | The availability of docker in all Bitbucket Pipelines steps. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | -| `options.max-time` | ✅ | The maximum allowable time that all step within a Bitbucket Pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. Bitbucket Pipeline step-level definition (`pipelines.default.step.max-time`) will override this value. | -| `options.size` | ✅ | Allocation of sizing options for the given memory for all steps within a Bitbucket Pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. Bitbucket Pipeline step-level definition (`pipelines.default.step.size`) will override this value. | +| `options` | ⚠️ | Customised options utilised throughout a Bitbucket pipeline. `max-time` and `size` are supported for translation within Buildkite Migration tool. | +| `options.docker` | ❌ | The availability of docker in all Bitbucket pipeline steps. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | +| `options.max-time` | ✅ | The maximum allowable time that all step within a Bitbucket pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. Bitbucket pipeline step-level definition (`pipelines.default.step.max-time`) will override this value. | +| `options.size` | ✅ | Allocation of sizing options for the given memory for all steps within a Bitbucket pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. Bitbucket pipeline step-level definition (`pipelines.default.step.size`) will override this value. | ### Pipelines (`pipelines`) | Key | Supported? | Notes | | --- | --- | --- | -| `pipelines.branches` | ✅ | Application of specific Bitbucket Pipeline configuration based for specific branches. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `build.branch`/`BUILDKITE_BRANCH` variable. | -| `pipelines.branches.` | ✅ | The branch name/wildcard to apply specific Bitbucket Pipeline step configuration within. | -| `pipelines.branches..step` | ✅ | Individual step configuration for a specific branch within a Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | -| `pipelines.custom` | ✅ | Bitbucket Pipelines that are only able to be triggered manually. The corresponding Buildkite pipeline is first generated with an [input step](https://buildkite.com/docs/pipelines/input-step) before any command jobs to ensure that triggered builds must be manually processed. | -| `pipelines.custom..step` | ✅ | Individual step configuration for a custom Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | -| `pipelines.default.parallel` | ✅ | The grouping of multiple steps within a Bitbucket Pipeline that are to be run concurrently. By default, Buildkite executes steps in parallel unless [implicit or explicit](https://buildkite.com/docs/pipelines/dependencies) dependencies are set. Parallel Bitbucket Pipeline steps are transitioned into a [group step](https://buildkite.com/docs/pipelines/group-step) within the generated Buildkite pipeline without explicit dependencies. | -| `pipelines.default.parallel.fail-fast` | ❌ | Whether a Bitbucket Pipeline allows this parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | -| `pipelines.default.parallel.steps` | ❌ | Individual step configuration for a custom Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | -| `pipelines.default.stage` | ✅ | The logical grouping of one or more Bitbucket Pipeline steps. Bitbucket Pipeline stages are translated into the corresponding Buildkite pipeline as a [group step](https://buildkite.com/docs/pipelines/group-step). | +| `pipelines.branches` | ✅ | Application of specific Bitbucket pipeline configuration based for specific branches. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `build.branch`/`BUILDKITE_BRANCH` variable. | +| `pipelines.branches.` | ✅ | The branch name/wildcard to apply specific Bitbucket pipeline step configuration within. | +| `pipelines.branches..step` | ✅ | Individual step configuration for a specific branch within a Bitbucket pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | +| `pipelines.custom` | ✅ | Bitbucket pipelines that are only able to be triggered manually. The corresponding Buildkite pipeline is first generated with an [input step](https://buildkite.com/docs/pipelines/input-step) before any command jobs to ensure that triggered builds must be manually processed. | +| `pipelines.custom..step` | ✅ | Individual step configuration for a custom Bitbucket pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | +| `pipelines.default.parallel` | ✅ | The grouping of multiple steps within a Bitbucket pipeline that are to be run concurrently. By default, Buildkite executes steps in parallel unless [implicit or explicit](https://buildkite.com/docs/pipelines/dependencies) dependencies are set. Parallel Bitbucket pipeline steps are transitioned into a [group step](https://buildkite.com/docs/pipelines/group-step) within the generated Buildkite pipeline without explicit dependencies. | +| `pipelines.default.parallel.fail-fast` | ❌ | Whether a Bitbucket pipeline allows this parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | +| `pipelines.default.parallel.steps` | ❌ | Individual step configuration for a custom Bitbucket pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | +| `pipelines.default.stage` | ✅ | The logical grouping of one or more Bitbucket pipeline steps. Bitbucket pipeline stages are translated into the corresponding Buildkite pipeline as a [group step](https://buildkite.com/docs/pipelines/group-step). | | `pipelines.default.stage.condition.changeset.includePaths` | ✅ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories. | -| `pipelines.default.stage.name` | ✅ | The name of the Bitbucket Pipeline stage. Transitioned to the `group` label of the corresponding Buildkite [group step](https://buildkite.com/docs/pipelines/group-step). | -| `pipelines.default.stage.steps` | ✅ | Individual step configuration for a Bitbucket Pipeline stage. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | -| `pipelines.default.stage.trigger` | ✅ | The configuration setting the running of a Bitbucket Pipeline stage manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline's group step before the specified `steps` of this stage. An explicit dependencies with `depends_on` are set between the input step and the following steps to ensure manual processing is required for these to run. | -| `pipelines.default.step` | ✅ | The list (one or more) of individual execution units that make up a workflow within a Bitbucket Pipeline. Run from top-to-bottom, with a 100 step limit. | -| `pipelines.default.step.after-script` | ❌ | The actions that a Bitbucket Pipeline will undertake after the commands in the `script` key are run. For similar behaviour, a [repository-level](https://buildkite.com/docs/agent/v3/hooks#hook-locations-repository-hooks) `pre-exit` hook approach will yield similar behaviour - running at the latter end of the [job lifecycle](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks). | -| `pipelines.default.step.artifacts` | ✅ | Build artifacts that will be required for steps later in the Bitbucket Pipeline. Artifacts that are specified (whether one specific file, or multiple) will be set within the generated Buildkite pipeline's command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | -| `pipelines.default.step.caches` | ✅ | Step-level dependencies downloaded from external sources (Docker, Maven, PyPi for example) which will be able to be re-used in later Bitbucket Pipeline steps. Caches that are set at step level (or though the top-level `definition.cache. property) are translated in the corresponding Buildkite pipeline utilising the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) to store the downloaded dependencies for re-use between Buildkite builds. | +| `pipelines.default.stage.name` | ✅ | The name of the Bitbucket pipeline stage. Transitioned to the `group` label of the corresponding Buildkite [group step](https://buildkite.com/docs/pipelines/group-step). | +| `pipelines.default.stage.steps` | ✅ | Individual step configuration for a Bitbucket pipeline stage. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | +| `pipelines.default.stage.trigger` | ✅ | The configuration setting the running of a Bitbucket pipeline stage manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline's group step before the specified `steps` of this stage. An explicit dependencies with `depends_on` are set between the input step and the following steps to ensure manual processing is required for these to run. | +| `pipelines.default.step` | ✅ | The list (one or more) of individual execution units that make up a workflow within a Bitbucket pipeline. Run from top-to-bottom, with a 100 step limit. | +| `pipelines.default.step.after-script` | ❌ | The actions that a Bitbucket pipeline will undertake after the commands in the `script` key are run. For similar behaviour, a [repository-level](https://buildkite.com/docs/agent/v3/hooks#hook-locations-repository-hooks) `pre-exit` hook approach will yield similar behaviour - running at the latter end of the [job lifecycle](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks). | +| `pipelines.default.step.artifacts` | ✅ | Build artifacts that will be required for steps later in the Bitbucket pipeline. Artifacts that are specified (whether one specific file, or multiple) will be set within the generated Buildkite pipeline's command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | +| `pipelines.default.step.caches` | ✅ | Step-level dependencies downloaded from external sources (Docker, Maven, PyPi for example) which will be able to be re-used in later Bitbucket pipeline steps. Caches that are set at step level (or though the top-level `definition.cache. property) are translated in the corresponding Buildkite pipeline utilising the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) to store the downloaded dependencies for re-use between Buildkite builds. | | `pipelines.default.step.caches.` | ✅ | Cache names that are reflective of specific tooling required for the specific Bitbucket pipeline. Default caches consist of `docker`, `composer`, `dotnetcore`, `gradle`, `ivy2`, `maven`, `node`, `pip` and `sbt`. Customised caches can be applied within a `definitions.caches. object and applied at this layer. | -| `pipelines.default.step.condition` | ✅ | The configuration to prevent a Bitbucket Pipeline step from running unless the specific conditional is met. Translated to an inline conditional (`if`) within the corresponding Buildkite pipelines' command step's `commands` - based on a `git diff` of the base branch. | +| `pipelines.default.step.condition` | ✅ | The configuration to prevent a Bitbucket pipeline step from running unless the specific conditional is met. Translated to an inline conditional (`if`) within the corresponding Buildkite pipelines' command step's `commands` - based on a `git diff` of the base branch. | | `pipelines.default.step.condition.changeset.includePaths` | ✅ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories. | -| `pipelines.default.step.clone` | ⚠️ | Clone options for a specific step of a Bitbucket Pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property) | -| `pipelines.default.step.clone.sparse-checkout` | ✅ | Sparse-checkout option for a Bitbucket Pipeline step. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin). | +| `pipelines.default.step.clone` | ⚠️ | Clone options for a specific step of a Bitbucket pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property) | +| `pipelines.default.step.clone.sparse-checkout` | ✅ | Sparse-checkout option for a Bitbucket pipeline step. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin). | | `pipelines.default.step.clone.sparse-checkout.code-mode` | ✅ | Whether the checkout patterns are considered to be a list of patterns (passed as the `--no-cone` flag to `git sparse-checkout` command). | -| `pipelines.default.step.clone.sparse-checkout.enabled` | ✅ | Whether sparse checkout is enabled for the Bitbucket Pipeline step. | +| `pipelines.default.step.clone.sparse-checkout.enabled` | ✅ | Whether sparse checkout is enabled for the Bitbucket pipeline step. | | `pipelines.default.step.clone.sparse-checkout.patterns` | ✅ | The list of paths to invoke a sparse checkout for. Can be a pattern to a specific file, directory, or wildcard for all files belonging within a certain directory. | | `pipelines.default.step.deployment` | ❌ | The environment set for the Bitbucket Deployments dashboard. This has no translatable equivalent within Buildkite. | -| `pipelines.default.step.docker` | ❌ | The availability of docker in Bitbucket Pipelines steps. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | -| `pipelines.default.step.fail-fast` | ❌ | Whether a specific step of a Bitbucket Pipeline allows a parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | -| `pipelines.default.step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket Pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). | -| `pipelines.default.step.max-time` | ✅ | The maximum allowable time that a step within a Bitbucket Pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. | -| `pipelines.default.step.name` | ✅ | The name of a specific step within a Bitbucket Pipeline. Translates to a Buildkite command step's `label`. | -| `pipelines.default.step.oidc` | ✅ | Open ID Connect configuration that will be applied for this Bitbucket Pipeline step. The generated command step in the corresponding Buildkite pipeline will [request](https://buildkite.com/docs/agent/v3/cli-oidc#request-oidc-token) an OIDC token and export it into the job environment as `BITBUCKET_STEP_OIDC_TOKEN` (to be passed to `sts` to assume an AWS role for example) | -| `pipelines.default.step.runs-on` | ✅ | Allocating the Bitbucket Pipeline to run on a self-hosted runner with the specific label. All `runs-on` values will be set as agent [tags](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | +| `pipelines.default.step.docker` | ❌ | The availability of docker in a specific Bitbucket pipeline step. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | +| `pipelines.default.step.fail-fast` | ❌ | Whether a specific step of a Bitbucket pipeline allows a parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | +| `pipelines.default.step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). | +| `pipelines.default.step.max-time` | ✅ | The maximum allowable time that a step within a Bitbucket pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. | +| `pipelines.default.step.name` | ✅ | The name of a specific step within a Bitbucket pipeline. Translates to a Buildkite command step's `label`. | +| `pipelines.default.step.oidc` | ✅ | Open ID Connect configuration that will be applied for this Bitbucket pipeline step. The generated command step in the corresponding Buildkite pipeline will [request](https://buildkite.com/docs/agent/v3/cli-oidc#request-oidc-token) an OIDC token and export it into the job environment as `BITBUCKET_STEP_OIDC_TOKEN` (to be passed to `sts` to assume an AWS role for example) | +| `pipelines.default.step.runs-on` | ✅ | Allocating the Bitbucket pipeline to run on a self-hosted runner with the specific label. All `runs-on` values will be set as agent [tags](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | | `pipelines.default.step.services` | ✅ | The name of one or more services defined at `definitions.services.` that will be applied for this step. Translated to utilise the service configuration with the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin). | | `pipelines.default.step.script` | ✅ | The individual commands that make up a specific step. Each is translated into a singular command within the `commands` key of a Buildkite command step. | -| `pipelines.default.step.size` | ✅ | Allocation of sizing options for the given memory for a specific step within a Bitbucket Pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | -| `pipelines.default.step.trigger` | ✅ | The configuration setting the running of a Bitbucket Pipeline step manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline before the specified `script` within a further command step. Explicit dependencies with `depends_on` are set between the two steps; requiring manual processing. | -| `pipelines.default.variables` | ✅ | Custom variables that are passed to Bitbucket Pipeline steps. Each variable defined in a Bitbucket Pipeline step is translated to a Buildkite [input step](https://buildkite.com/docs/pipelines/input-step) with/without defaults and allowed values specified below. | +| `pipelines.default.step.script.pipe` | ❌ | Re-usable modules to make configuration in Bitbucet pipelines easier and modular. Consider using [Buildkte Plugins](https://buildkite.com/docs/plugins) and/or [Pipeline Templates](https://buildkite.com/docs/pipelines/templates) in configuring re-usable configuration accross a Buildkite organization. | +| `pipelines.default.step.size` | ✅ | Allocation of sizing options for the given memory for a specific step within a Bitbucket pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | +| `pipelines.default.step.trigger` | ✅ | The configuration setting the running of a Bitbucket pipeline step manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline before the specified `script` within a further command step. Explicit dependencies with `depends_on` are set between the two steps; requiring manual processing. | +| `pipelines.default.variables` | ✅ | Custom variables that are passed to Bitbucket pipeline steps. Each variable defined in a Bitbucket pipeline step is translated to a Buildkite [input step](https://buildkite.com/docs/pipelines/input-step) with/without defaults and allowed values specified below. | | `pipelines.default.variables.name` | ✅ | The variables' name: translated to the `key` attribute of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step) (text entry).| | `pipelines.default.variables.default` | ✅ | The default variable value if no value is set. Set as the `default` attribute within the `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | | `pipelines.default.variables.description` | ✅ | The description of the variable. Translated to the `text` attribute of a specific `field` within the generated [input step](https://buildkite.com/docs/pipelines/input-step). | | `pipelines.default.variables.allowed-values` | ✅ | The variable's allowed values: each option is translated to a singular `options` oject with given value of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | -| `pipelines.pull_request` | ✅ | Application of specific Bitbucket Pipeline configuration based for pull requests. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `pull_request.id`/`BUILDKITE_PULL_REQUEST` and `build.branch`/`BUILDKITE_BRANCH` variables. | -| `pipelines.pull_request.` | ✅ | The base branch name/wildcard to apply specific Bitbucket Pipeline step configuration within. Apply the configuration for all builds with a `**` wildcard. | -| `pipelines.pull_request..step` | ✅ | Individual step configuration for pull requests builds within a Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | -| `pipelines.tags` | ✅ | Application of specific Bitbucket Pipeline configuration based for triggered tag builds. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `build.tag`/`BUILDKITE_TAG` variable. | -| `pipelines.tags.` | ✅ | The tag name/wildcard to apply specific Bitbucket Pipeline step configuration within. | -| `pipelines.tags..step` | ✅ | Individual step configuration for triggered tag builds within a Bitbucket Pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | \ No newline at end of file +| `pipelines.pull_request` | ✅ | Application of specific Bitbucket pipeline configuration based for pull requests. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `pull_request.id`/`BUILDKITE_PULL_REQUEST` and `build.branch`/`BUILDKITE_BRANCH` variables. | +| `pipelines.pull_request.` | ✅ | The base branch name/wildcard to apply specific Bitbucket pipeline step configuration within. Apply the configuration for all builds with a `**` wildcard. | +| `pipelines.pull_request..step` | ✅ | Individual step configuration for pull requests builds within a Bitbucket pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | +| `pipelines.tags` | ✅ | Application of specific Bitbucket pipeline configuration based for triggered tag builds. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `build.tag`/`BUILDKITE_TAG` variable. | +| `pipelines.tags.` | ✅ | The tag name/wildcard to apply specific Bitbucket pipeline step configuration within. | +| `pipelines.tags..step` | ✅ | Individual step configuration for triggered tag builds within a Bitbucket pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | \ No newline at end of file From bce542708c3c6fec075a79992a20ba2dff7c3aca Mon Sep 17 00:00:00 2001 From: James Sammut Date: Tue, 30 Apr 2024 12:40:30 +1000 Subject: [PATCH 09/20] BB Pipeline docs fillout #8, Readme link --- Readme.md | 3 ++- docs/Bitbucket.md | 15 ++++++++++----- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/Readme.md b/Readme.md index d8d8e00c..58067990 100644 --- a/Readme.md +++ b/Readme.md @@ -79,5 +79,6 @@ Buildkite has its own suggested best practices, these may differ to those from o Further information on the currently supported attributes of CI provider pipeline translation to Buildkite pipelines can be found below (within the `/docs` directory): -- [GitHub Actions](/docs/GHA.md) +- [Bitbucket Pipelines](/docs/Bitbucket.md) - [CircleCI](/docs/CircleCI.md) +- [GitHub Actions](/docs/GHA.md) \ No newline at end of file diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index 2e2cf509..063d3afc 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -35,8 +35,10 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | -| `image` | ✅ | The container image that is to be applied for each step within a Bitbucket pipeline. Images set at this level will be applied to all steps within a Buildkite pipeline, utilising the specified image within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). This has lower precedence over per-step `image` configuration (see `pipelines.default.step.image`). | - +| `image` | ✅ | The container image that is to be applied for each step within a Bitbucket pipeline, utilising the specified image within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). This has lower precedence over per-step `image` configuration (see `pipelines.default.step.image`). The `name`, `username` and `password` properties below are the alternate way of defining all Bitbucket pipeline steps to use a specific image (with authentication). | +| `image.name` | ✅ | The name of the image to utilise within all Bitbucket pipeline steps. Translated to the `image` property within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin) (if the image name was already not set in the `image` property above itself). | +| `image.username` | ✅ | The username of the user for private Docker images hosted on a given registry. Translated to the `username` property within the [docker-login-buildkite-plugin](https://github.com/buildkite-plugins/docker-login-buildkite-plugin) | +| `image.password` | ✅ | The password of the user for private Docker images hosted on a given registry. Translated to the `password` property within the [docker-login-buildkite-plugin](https://github.com/buildkite-plugins/docker-login-buildkite-plugin). Be sure to utilise [best practices](https://buildkite.com/docs/pipelines/secrets) for storing and accessing secrets within a Buildkite pipeline! | ### Options (`options`) | Key | Supported? | Notes | @@ -78,21 +80,24 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | `pipelines.default.step.deployment` | ❌ | The environment set for the Bitbucket Deployments dashboard. This has no translatable equivalent within Buildkite. | | `pipelines.default.step.docker` | ❌ | The availability of docker in a specific Bitbucket pipeline step. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | | `pipelines.default.step.fail-fast` | ❌ | Whether a specific step of a Bitbucket pipeline allows a parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | -| `pipelines.default.step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). | +| `pipelines.default.step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). The `name`, `username` and `password` properties below are the alternate way of defining a Bitbucket pipeline step to use a specific image (with authentication). | +| `pipelines.default.step.image.name` | ✅ | The name of the image to utilise within the Bitbucket pipeline step. Translated to the `image` property within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin) (if the image name was already not set in the `pipelines.default.step.image` property above itself.) | +| `pipelines.default.step.image.username` | ✅ | The username of the user for private Docker images hosted on a given registry. Translated to the `username` property within the [docker-login-buildkite-plugin](https://github.com/buildkite-plugins/docker-login-buildkite-plugin) | +| `pipelines.default.step.image.password` | ✅ | The password of the user for private Docker images hosted on a given registry. Translated to the `password` property within the [docker-login-buildkite-plugin](https://github.com/buildkite-plugins/docker-login-buildkite-plugin). Be sure to utilise [best practices](https://buildkite.com/docs/pipelines/secrets) for storing and accessing secrets within a Buildkite pipeline! | | `pipelines.default.step.max-time` | ✅ | The maximum allowable time that a step within a Bitbucket pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. | | `pipelines.default.step.name` | ✅ | The name of a specific step within a Bitbucket pipeline. Translates to a Buildkite command step's `label`. | | `pipelines.default.step.oidc` | ✅ | Open ID Connect configuration that will be applied for this Bitbucket pipeline step. The generated command step in the corresponding Buildkite pipeline will [request](https://buildkite.com/docs/agent/v3/cli-oidc#request-oidc-token) an OIDC token and export it into the job environment as `BITBUCKET_STEP_OIDC_TOKEN` (to be passed to `sts` to assume an AWS role for example) | | `pipelines.default.step.runs-on` | ✅ | Allocating the Bitbucket pipeline to run on a self-hosted runner with the specific label. All `runs-on` values will be set as agent [tags](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | | `pipelines.default.step.services` | ✅ | The name of one or more services defined at `definitions.services.` that will be applied for this step. Translated to utilise the service configuration with the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin). | | `pipelines.default.step.script` | ✅ | The individual commands that make up a specific step. Each is translated into a singular command within the `commands` key of a Buildkite command step. | -| `pipelines.default.step.script.pipe` | ❌ | Re-usable modules to make configuration in Bitbucet pipelines easier and modular. Consider using [Buildkte Plugins](https://buildkite.com/docs/plugins) and/or [Pipeline Templates](https://buildkite.com/docs/pipelines/templates) in configuring re-usable configuration accross a Buildkite organization. | +| `pipelines.default.step.script.pipe` | ❌ | Re-usable modules to make configuration in Bitbucket pipelines easier and modular. Consider using [Buildkite Plugins](https://buildkite.com/docs/plugins) and/or [Pipeline Templates](https://buildkite.com/docs/pipelines/templates) in configuring re-usable configuration accross a Buildkite organization. | | `pipelines.default.step.size` | ✅ | Allocation of sizing options for the given memory for a specific step within a Bitbucket pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | | `pipelines.default.step.trigger` | ✅ | The configuration setting the running of a Bitbucket pipeline step manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline before the specified `script` within a further command step. Explicit dependencies with `depends_on` are set between the two steps; requiring manual processing. | | `pipelines.default.variables` | ✅ | Custom variables that are passed to Bitbucket pipeline steps. Each variable defined in a Bitbucket pipeline step is translated to a Buildkite [input step](https://buildkite.com/docs/pipelines/input-step) with/without defaults and allowed values specified below. | | `pipelines.default.variables.name` | ✅ | The variables' name: translated to the `key` attribute of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step) (text entry).| | `pipelines.default.variables.default` | ✅ | The default variable value if no value is set. Set as the `default` attribute within the `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | | `pipelines.default.variables.description` | ✅ | The description of the variable. Translated to the `text` attribute of a specific `field` within the generated [input step](https://buildkite.com/docs/pipelines/input-step). | -| `pipelines.default.variables.allowed-values` | ✅ | The variable's allowed values: each option is translated to a singular `options` oject with given value of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | +| `pipelines.default.variables.allowed-values` | ✅ | The variable's allowed values: each option is translated to a singular `options` object with given value of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | | `pipelines.pull_request` | ✅ | Application of specific Bitbucket pipeline configuration based for pull requests. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `pull_request.id`/`BUILDKITE_PULL_REQUEST` and `build.branch`/`BUILDKITE_BRANCH` variables. | | `pipelines.pull_request.` | ✅ | The base branch name/wildcard to apply specific Bitbucket pipeline step configuration within. Apply the configuration for all builds with a `**` wildcard. | | `pipelines.pull_request..step` | ✅ | Individual step configuration for pull requests builds within a Bitbucket pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | From 40c489c59f0590a75b177d6de5e61545d262ed12 Mon Sep 17 00:00:00 2001 From: James Sammut Date: Tue, 30 Apr 2024 12:44:54 +1000 Subject: [PATCH 10/20] Spacing for options heading --- docs/Bitbucket.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index 063d3afc..394b097c 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -39,6 +39,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | `image.name` | ✅ | The name of the image to utilise within all Bitbucket pipeline steps. Translated to the `image` property within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin) (if the image name was already not set in the `image` property above itself). | | `image.username` | ✅ | The username of the user for private Docker images hosted on a given registry. Translated to the `username` property within the [docker-login-buildkite-plugin](https://github.com/buildkite-plugins/docker-login-buildkite-plugin) | | `image.password` | ✅ | The password of the user for private Docker images hosted on a given registry. Translated to the `password` property within the [docker-login-buildkite-plugin](https://github.com/buildkite-plugins/docker-login-buildkite-plugin). Be sure to utilise [best practices](https://buildkite.com/docs/pipelines/secrets) for storing and accessing secrets within a Buildkite pipeline! | + ### Options (`options`) | Key | Supported? | Notes | From 2c0c958f344756bdaa5ebf48e16b7891a3f62870 Mon Sep 17 00:00:00 2001 From: James Sammut Date: Wed, 1 May 2024 08:38:09 +1000 Subject: [PATCH 11/20] Bitbucket docs adjustment #1 --- docs/Bitbucket.md | 57 +++++++++++++++-------------------------------- 1 file changed, 18 insertions(+), 39 deletions(-) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index 394b097c..3f550dd2 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -2,28 +2,26 @@ The Buildkite Migration tool's currently supported (✅), partially supported (⚠️) and unsupported (❌) properties in translation of Bitbucket pipelines to Buildkite pipelines are listed below. +> [!NOTE] +> The Bitbucket Pipeline configuration that is described in the various sections below is specified in the central `bitbucket-pipelines.yml` within a specific Bitbucket workspace [repository](https://support.atlassian.com/bitbucket-cloud/docs/what-is-a-workspace/). In Buildkite - pipeline configuration can be set in a singular `pipeline.yml` within a repository - but can also be set and uploaded dynamically through the use of [Dynamic Pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines). Additionally, control and governance of Buildkite pipelines can be achieved by the use of [Pipeline Templates](https://buildkite.com/docs/pipelines/templates) to set shared pipeline configuration within a Buildkite organization. + ### Clone (`clone`) | Key | Supported? | Notes | | --- | --- | --- | -| `clone` | ⚠️ | Clone options for all steps of a Bitbucket pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property). `clone` properties at the Bitbucket pipeline step level take higher precedent over these global properties if set. | -| `clone.sparse-checkout` | ✅ | Sparse-checkout option for all Bitbucket pipeline steps. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin). | -| `clone.sparse-checkout.code-mode` | ✅ | Whether the checkout patterns are considered to be a list of patterns (passed as the `--no-cone` flag to `git sparse-checkout` command). | -| `clone.sparse-checkout.enabled` | ✅ | Whether sparse checkout is enabled for all Bitbucket pipeline steps. | -| `clone.sparse-checkout.patterns` | ✅ | The list of paths to invoke a sparse checkout for. Can be a pattern to a specific file, directory, or wildcard for all files belonging within a certain directory. | +| `clone` | ⚠️ | Clone options for all steps of a Bitbucket pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook.

Sparse-checkout properties of `code-mode`, `enabled` and `patterns` will be translated to the respective properties within the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin).

clone` properties at the Bitbucket pipeline, if set, have higher precedence over these global properties. | ### Definitions (`definitions`) | Key | Supported? | Notes | | --- | --- | --- | | `definitions` | ⚠️ | Customised definitions utilised throughout a Bitbucket pipeline. `caches` and `services` are supported for translation within Buildkite Migration tool. | -| `definitions.caches` | ✅ | Customised cache definitions that can be applied to specific Bitbucket pipeline steps - inclusive of folders, single file - or multifile cache. Targeted into specific steps with the `pipelines.default.step.caches.` property. | +| `definitions.caches` | ⚠️ | Customised cache definitions that can be applied to specific Bitbucket pipeline steps - inclusive of folders, single file - or multifile cache. Targeted into specific steps with the `pipelines.default.step.caches.` property, and in which the translation will utilise the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) that may require further setup/tweaking around specific caching strategies. | | `definitions.caches.` | ✅ | A customised cache name applicable for one or more steps within a Bitbucket pipeline. | | `definitions.caches..path` | ✅ | The directory path that is desired to be cached. | -| `definitions.caches..key.files` | ✅ | The list (one or more) files that are monitored for changes - and stored once its hash changes between file versions change. | -| `definitions.pipeline` | ❌ | Pipelines that are exported for re-use within repositories of the same workspace. Like functionality exists within Buildkite as [Pipeline Templates](https://buildkite.com/docs/pipelines/templates). | -| `definitions.services` | ✅ | Defined Docker services that are applied within a Bitbucket pipeline. Services defined in a corresponding Bitbucket pipeline step using the `pipelines.default.step.services` property will have its configuration applied with the use of the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin). By default - configuration will be saved to a `compose.yaml` in the job environment. | -| `definitions.services.` | ✅ | The name of an individual Docker service that will be applied within a specific step of the Bitbucket pipeline. | +| `definitions.caches..key.files` | ⚠️ | The list (one or more) files that are monitored for changes - and stored once its hash changes between file versions change. If multiple files are specified - multiple cache-plugin definitions are set on the resultant Buildkite command step (differing `manifest` properties between each). | +| `definitions.pipeline` | ⚠️ | Pipelines that are exported for re-use within repositories of the same workspace. Like functionality exists within Buildkite as [Pipeline Templates](https://buildkite.com/docs/pipelines/templates). | +| `definitions.services` | ✅ | Defined Docker services that are applied within a Bitbucket pipeline. Services defined in a corresponding Bitbucket pipeline step using the `pipelines.default.step.services` property will have its configuration applied with the use of the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin).

Generated configuration will need to be saved to a `compose.yaml` file within the repository, and the image utilised with the Buildkite command step as `app`.

Refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/databases-and-service-containers/) for more details on service containers and configuration references. | ### Export (`export`) @@ -35,19 +33,13 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | -| `image` | ✅ | The container image that is to be applied for each step within a Bitbucket pipeline, utilising the specified image within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). This has lower precedence over per-step `image` configuration (see `pipelines.default.step.image`). The `name`, `username` and `password` properties below are the alternate way of defining all Bitbucket pipeline steps to use a specific image (with authentication). | -| `image.name` | ✅ | The name of the image to utilise within all Bitbucket pipeline steps. Translated to the `image` property within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin) (if the image name was already not set in the `image` property above itself). | -| `image.username` | ✅ | The username of the user for private Docker images hosted on a given registry. Translated to the `username` property within the [docker-login-buildkite-plugin](https://github.com/buildkite-plugins/docker-login-buildkite-plugin) | -| `image.password` | ✅ | The password of the user for private Docker images hosted on a given registry. Translated to the `password` property within the [docker-login-buildkite-plugin](https://github.com/buildkite-plugins/docker-login-buildkite-plugin). Be sure to utilise [best practices](https://buildkite.com/docs/pipelines/secrets) for storing and accessing secrets within a Buildkite pipeline! | +| `image` | ✅ | The container image that is to be applied for each step within a Bitbucket pipeline, utilising the specified image within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). This has lower precedence over per-step `image` configuration (see `pipelines.default.step.image`).

The `aws`, `aws.oidc`, `name`, `username` and `password` sub-properties are supported - refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/docker-image-options/) for more information. | ### Options (`options`) | Key | Supported? | Notes | | --- | --- | --- | -| `options` | ⚠️ | Customised options utilised throughout a Bitbucket pipeline. `max-time` and `size` are supported for translation within Buildkite Migration tool. | -| `options.docker` | ❌ | The availability of docker in all Bitbucket pipeline steps. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | -| `options.max-time` | ✅ | The maximum allowable time that all step within a Bitbucket pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. Bitbucket pipeline step-level definition (`pipelines.default.step.max-time`) will override this value. | -| `options.size` | ✅ | Allocation of sizing options for the given memory for all steps within a Bitbucket pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. Bitbucket pipeline step-level definition (`pipelines.default.step.size`) will override this value. | +| `options` | ⚠️ | Customised options utilised throughout a Bitbucket pipeline. The `max-time` and `size` sub-properties are supported for translation within Buildkite Migration tool through to the generated Buildkite command step's `timeout_in_minutes` and agent tag respectfully.

The `docker` sub-property is not supported: and will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available.

Both supported properties at Bitbucket pipeline step-level definition (`pipelines.default.step.max-time` and `pipelines.default.step.size`) will have higher precedences than the two values set at `options` level. | ### Pipelines (`pipelines`) @@ -57,41 +49,28 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | `pipelines.branches.` | ✅ | The branch name/wildcard to apply specific Bitbucket pipeline step configuration within. | | `pipelines.branches..step` | ✅ | Individual step configuration for a specific branch within a Bitbucket pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | | `pipelines.custom` | ✅ | Bitbucket pipelines that are only able to be triggered manually. The corresponding Buildkite pipeline is first generated with an [input step](https://buildkite.com/docs/pipelines/input-step) before any command jobs to ensure that triggered builds must be manually processed. | -| `pipelines.custom..step` | ✅ | Individual step configuration for a custom Bitbucket pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | -| `pipelines.default.parallel` | ✅ | The grouping of multiple steps within a Bitbucket pipeline that are to be run concurrently. By default, Buildkite executes steps in parallel unless [implicit or explicit](https://buildkite.com/docs/pipelines/dependencies) dependencies are set. Parallel Bitbucket pipeline steps are transitioned into a [group step](https://buildkite.com/docs/pipelines/group-step) within the generated Buildkite pipeline without explicit dependencies. | +| `pipelines.default.parallel` | ✅ | The grouping of multiple steps within a Bitbucket pipeline that are to be run concurrently. By default, Buildkite executes steps in parallel unless [implicit or explicit dependencies](https://buildkite.com/docs/pipelines/dependencies) are set. Parallel Bitbucket pipeline steps are transitioned into a [group step](https://buildkite.com/docs/pipelines/group-step) within the generated Buildkite pipeline without explicit dependencies. | | `pipelines.default.parallel.fail-fast` | ❌ | Whether a Bitbucket pipeline allows this parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | -| `pipelines.default.parallel.steps` | ❌ | Individual step configuration for a custom Bitbucket pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | | `pipelines.default.stage` | ✅ | The logical grouping of one or more Bitbucket pipeline steps. Bitbucket pipeline stages are translated into the corresponding Buildkite pipeline as a [group step](https://buildkite.com/docs/pipelines/group-step). | -| `pipelines.default.stage.condition.changeset.includePaths` | ✅ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories. | -| `pipelines.default.stage.name` | ✅ | The name of the Bitbucket pipeline stage. Transitioned to the `group` label of the corresponding Buildkite [group step](https://buildkite.com/docs/pipelines/group-step). | -| `pipelines.default.stage.steps` | ✅ | Individual step configuration for a Bitbucket pipeline stage. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | -| `pipelines.default.stage.trigger` | ✅ | The configuration setting the running of a Bitbucket pipeline stage manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline's group step before the specified `steps` of this stage. An explicit dependencies with `depends_on` are set between the input step and the following steps to ensure manual processing is required for these to run. | -| `pipelines.default.step` | ✅ | The list (one or more) of individual execution units that make up a workflow within a Bitbucket pipeline. Run from top-to-bottom, with a 100 step limit. | +| `pipelines.default.stage.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

In cases where many paths are needed to be polled for changes - utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin) is recommended; watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | | `pipelines.default.step.after-script` | ❌ | The actions that a Bitbucket pipeline will undertake after the commands in the `script` key are run. For similar behaviour, a [repository-level](https://buildkite.com/docs/agent/v3/hooks#hook-locations-repository-hooks) `pre-exit` hook approach will yield similar behaviour - running at the latter end of the [job lifecycle](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks). | -| `pipelines.default.step.artifacts` | ✅ | Build artifacts that will be required for steps later in the Bitbucket pipeline. Artifacts that are specified (whether one specific file, or multiple) will be set within the generated Buildkite pipeline's command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | +| `pipelines.default.step.artifacts` | ⚠ | Build artifacts that will be required for steps later in the Bitbucket pipeline (by default, not obtained unless an explicit `buildkite-agent artifact download` [command](https://buildkite.com/docs/agent/v3/cli-artifact#downloading-artifacts) is run beforehand within the generated Buildkite command step). Artifacts that are specified (whether one specific file, or multiple) will be set within the generated command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | | `pipelines.default.step.caches` | ✅ | Step-level dependencies downloaded from external sources (Docker, Maven, PyPi for example) which will be able to be re-used in later Bitbucket pipeline steps. Caches that are set at step level (or though the top-level `definition.cache. property) are translated in the corresponding Buildkite pipeline utilising the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) to store the downloaded dependencies for re-use between Buildkite builds. | -| `pipelines.default.step.caches.` | ✅ | Cache names that are reflective of specific tooling required for the specific Bitbucket pipeline. Default caches consist of `docker`, `composer`, `dotnetcore`, `gradle`, `ivy2`, `maven`, `node`, `pip` and `sbt`. Customised caches can be applied within a `definitions.caches. object and applied at this layer. | | `pipelines.default.step.condition` | ✅ | The configuration to prevent a Bitbucket pipeline step from running unless the specific conditional is met. Translated to an inline conditional (`if`) within the corresponding Buildkite pipelines' command step's `commands` - based on a `git diff` of the base branch. | -| `pipelines.default.step.condition.changeset.includePaths` | ✅ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories. | +| `pipelines.default.step.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

In cases where many paths are needed to be polled for changes - utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin) is recommended; watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | | `pipelines.default.step.clone` | ⚠️ | Clone options for a specific step of a Bitbucket pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property) | -| `pipelines.default.step.clone.sparse-checkout` | ✅ | Sparse-checkout option for a Bitbucket pipeline step. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin). | -| `pipelines.default.step.clone.sparse-checkout.code-mode` | ✅ | Whether the checkout patterns are considered to be a list of patterns (passed as the `--no-cone` flag to `git sparse-checkout` command). | -| `pipelines.default.step.clone.sparse-checkout.enabled` | ✅ | Whether sparse checkout is enabled for the Bitbucket pipeline step. | -| `pipelines.default.step.clone.sparse-checkout.patterns` | ✅ | The list of paths to invoke a sparse checkout for. Can be a pattern to a specific file, directory, or wildcard for all files belonging within a certain directory. | +| `pipelines.default.step.clone.sparse-checkout` | ✅ | Sparse-checkout option for a Bitbucket pipeline step. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin).

Sparse-checkout properties of `code-mode`, `enabled` and `patterns` will be trasnlated to the respective properties within the mentinoed plugin's configuration. | | `pipelines.default.step.deployment` | ❌ | The environment set for the Bitbucket Deployments dashboard. This has no translatable equivalent within Buildkite. | | `pipelines.default.step.docker` | ❌ | The availability of docker in a specific Bitbucket pipeline step. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | | `pipelines.default.step.fail-fast` | ❌ | Whether a specific step of a Bitbucket pipeline allows a parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | -| `pipelines.default.step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). The `name`, `username` and `password` properties below are the alternate way of defining a Bitbucket pipeline step to use a specific image (with authentication). | -| `pipelines.default.step.image.name` | ✅ | The name of the image to utilise within the Bitbucket pipeline step. Translated to the `image` property within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin) (if the image name was already not set in the `pipelines.default.step.image` property above itself.) | -| `pipelines.default.step.image.username` | ✅ | The username of the user for private Docker images hosted on a given registry. Translated to the `username` property within the [docker-login-buildkite-plugin](https://github.com/buildkite-plugins/docker-login-buildkite-plugin) | -| `pipelines.default.step.image.password` | ✅ | The password of the user for private Docker images hosted on a given registry. Translated to the `password` property within the [docker-login-buildkite-plugin](https://github.com/buildkite-plugins/docker-login-buildkite-plugin). Be sure to utilise [best practices](https://buildkite.com/docs/pipelines/secrets) for storing and accessing secrets within a Buildkite pipeline! | +| `pipelines.default.step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin).

The `aws`, `aws.oidc`, `name`, `username` and `password` sub-properties are supported - refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/docker-image-options/) for more information. | | `pipelines.default.step.max-time` | ✅ | The maximum allowable time that a step within a Bitbucket pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. | | `pipelines.default.step.name` | ✅ | The name of a specific step within a Bitbucket pipeline. Translates to a Buildkite command step's `label`. | | `pipelines.default.step.oidc` | ✅ | Open ID Connect configuration that will be applied for this Bitbucket pipeline step. The generated command step in the corresponding Buildkite pipeline will [request](https://buildkite.com/docs/agent/v3/cli-oidc#request-oidc-token) an OIDC token and export it into the job environment as `BITBUCKET_STEP_OIDC_TOKEN` (to be passed to `sts` to assume an AWS role for example) | | `pipelines.default.step.runs-on` | ✅ | Allocating the Bitbucket pipeline to run on a self-hosted runner with the specific label. All `runs-on` values will be set as agent [tags](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | -| `pipelines.default.step.services` | ✅ | The name of one or more services defined at `definitions.services.` that will be applied for this step. Translated to utilise the service configuration with the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin). | +| `pipelines.default.step.services` | ⚠️ | The name of one or more services defined at `definitions.services.` that will be applied for this step. Translated to utilise the service configuration with the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin).

Generated configuration will need to be saved to a `compose.yaml` file within the repository, and the image utilised with the Buildkite command step as `app`.

Refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/databases-and-service-containers/) for more details on service containers and configuration references. | | `pipelines.default.step.script` | ✅ | The individual commands that make up a specific step. Each is translated into a singular command within the `commands` key of a Buildkite command step. | -| `pipelines.default.step.script.pipe` | ❌ | Re-usable modules to make configuration in Bitbucket pipelines easier and modular. Consider using [Buildkite Plugins](https://buildkite.com/docs/plugins) and/or [Pipeline Templates](https://buildkite.com/docs/pipelines/templates) in configuring re-usable configuration accross a Buildkite organization. | +| `pipelines.default.step.script.pipe` | ❌ | Re-usable modules to make configuration in Bitbucket pipelines easier and modular. Consider exploring the suite of available [Buildkite Plugins](https://buildkite.com/plugins) for corresponding functionality that is required. | | `pipelines.default.step.size` | ✅ | Allocation of sizing options for the given memory for a specific step within a Bitbucket pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | | `pipelines.default.step.trigger` | ✅ | The configuration setting the running of a Bitbucket pipeline step manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline before the specified `script` within a further command step. Explicit dependencies with `depends_on` are set between the two steps; requiring manual processing. | | `pipelines.default.variables` | ✅ | Custom variables that are passed to Bitbucket pipeline steps. Each variable defined in a Bitbucket pipeline step is translated to a Buildkite [input step](https://buildkite.com/docs/pipelines/input-step) with/without defaults and allowed values specified below. | From 4d44ba57fbf59e463098dec86ebf637f436ce035 Mon Sep 17 00:00:00 2001 From: James Sammut Date: Wed, 1 May 2024 11:53:45 +1000 Subject: [PATCH 12/20] Bitbucket docs adjustment #2 --- docs/Bitbucket.md | 157 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 119 insertions(+), 38 deletions(-) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index 3f550dd2..43cb0bd4 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -5,13 +5,13 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( > [!NOTE] > The Bitbucket Pipeline configuration that is described in the various sections below is specified in the central `bitbucket-pipelines.yml` within a specific Bitbucket workspace [repository](https://support.atlassian.com/bitbucket-cloud/docs/what-is-a-workspace/). In Buildkite - pipeline configuration can be set in a singular `pipeline.yml` within a repository - but can also be set and uploaded dynamically through the use of [Dynamic Pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines). Additionally, control and governance of Buildkite pipelines can be achieved by the use of [Pipeline Templates](https://buildkite.com/docs/pipelines/templates) to set shared pipeline configuration within a Buildkite organization. -### Clone (`clone`) +## Clone (`clone`) | Key | Supported? | Notes | | --- | --- | --- | | `clone` | ⚠️ | Clone options for all steps of a Bitbucket pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook.

Sparse-checkout properties of `code-mode`, `enabled` and `patterns` will be translated to the respective properties within the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin).

clone` properties at the Bitbucket pipeline, if set, have higher precedence over these global properties. | -### Definitions (`definitions`) +## Definitions (`definitions`) | Key | Supported? | Notes | | --- | --- | --- | @@ -23,64 +23,145 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | `definitions.pipeline` | ⚠️ | Pipelines that are exported for re-use within repositories of the same workspace. Like functionality exists within Buildkite as [Pipeline Templates](https://buildkite.com/docs/pipelines/templates). | | `definitions.services` | ✅ | Defined Docker services that are applied within a Bitbucket pipeline. Services defined in a corresponding Bitbucket pipeline step using the `pipelines.default.step.services` property will have its configuration applied with the use of the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin).

Generated configuration will need to be saved to a `compose.yaml` file within the repository, and the image utilised with the Buildkite command step as `app`.

Refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/databases-and-service-containers/) for more details on service containers and configuration references. | -### Export (`export`) +## Export (`export`) | Key | Supported? | Notes | | --- | --- | --- | | `export` | ❌ | Bitbucket Premium option of sharing pipeline configuration between workspaces. Not applicable within Buildkite as an attribute - like functionality exists within [Pipeline Templates](https://buildkite.com/docs/pipelines/templates). | -### Image (`image`) +## Image (`image`) | Key | Supported? | Notes | | --- | --- | --- | | `image` | ✅ | The container image that is to be applied for each step within a Bitbucket pipeline, utilising the specified image within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). This has lower precedence over per-step `image` configuration (see `pipelines.default.step.image`).

The `aws`, `aws.oidc`, `name`, `username` and `password` sub-properties are supported - refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/docker-image-options/) for more information. | -### Options (`options`) +## Options (`options`) | Key | Supported? | Notes | | --- | --- | --- | -| `options` | ⚠️ | Customised options utilised throughout a Bitbucket pipeline. The `max-time` and `size` sub-properties are supported for translation within Buildkite Migration tool through to the generated Buildkite command step's `timeout_in_minutes` and agent tag respectfully.

The `docker` sub-property is not supported: and will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available.

Both supported properties at Bitbucket pipeline step-level definition (`pipelines.default.step.max-time` and `pipelines.default.step.size`) will have higher precedences than the two values set at `options` level. | +| `options` | ⚠️ | Customised options utilised throughout a Bitbucket pipeline. The `max-time` and `size` sub-properties are supported for translation within Buildkite Migration tool through to the generated Buildkite command step's `timeout_in_minutes` and agent tag respectfully.

The `docker` sub-property is not supported: and will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available.

Both supported properties at Bitbucket pipeline step-level definition (`pipelines..step.max-time` and `pipelines..step.size`) will have higher precedences than the two values set at `options` level. | -### Pipelines (`pipelines`) +## Pipeline Starting Conditions (`pipelines.`) + +> [!NOTE] +> Bitbucket Pipelines allows the configuration of various pipeline start conditions: each supporting different configuration and permissible properties: +> - `branches`: Defines the branch-specific configuration of a Bitbucket pipeline. +> - `custom`: A customised starting condition whereby only manual triggering is allowed. +> - `default`: The default starting configuration of a Bitbucket pipeline if it does not fall into one of the other conditions. +> - `pull_request`: Defines the pull-request specific configuration of a Bitbucket pipeline. +> - `tags`: Defines the tag specific configuration of a Bitbucket pipeline. +> +> For information on each of these individual starting conditions, refer to the reference within the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/pipeline-start-conditions/#Custom--manual--pipeline-variables). + +### Branches (`pipelines.branches`) | Key | Supported? | Notes | | --- | --- | --- | | `pipelines.branches` | ✅ | Application of specific Bitbucket pipeline configuration based for specific branches. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `build.branch`/`BUILDKITE_BRANCH` variable. | | `pipelines.branches.` | ✅ | The branch name/wildcard to apply specific Bitbucket pipeline step configuration within. | +| `pipelines.branches..parallel` | ✅ | Parallel (concurrent step) configuration for a specific branch within a Bitbucket pipeline. View the available parallel [properties](#parallel-pipelinestypeparallel) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/parallel-step-options/#Parallel). | | `pipelines.branches..step` | ✅ | Individual step configuration for a specific branch within a Bitbucket pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | +| `pipelines.branches..stage` | ✅ | Stage configuration for a specific branch within a Bitbucket pipeline. View the available stage [properties](#stage-pipelinestypestage) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/stage-options/#Stage). | + +### Custom (`pipelines.custom`) + +| Key | Supported? | Notes | +| --- | --- | --- | | `pipelines.custom` | ✅ | Bitbucket pipelines that are only able to be triggered manually. The corresponding Buildkite pipeline is first generated with an [input step](https://buildkite.com/docs/pipelines/input-step) before any command jobs to ensure that triggered builds must be manually processed. | -| `pipelines.default.parallel` | ✅ | The grouping of multiple steps within a Bitbucket pipeline that are to be run concurrently. By default, Buildkite executes steps in parallel unless [implicit or explicit dependencies](https://buildkite.com/docs/pipelines/dependencies) are set. Parallel Bitbucket pipeline steps are transitioned into a [group step](https://buildkite.com/docs/pipelines/group-step) within the generated Buildkite pipeline without explicit dependencies. | -| `pipelines.default.parallel.fail-fast` | ❌ | Whether a Bitbucket pipeline allows this parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | -| `pipelines.default.stage` | ✅ | The logical grouping of one or more Bitbucket pipeline steps. Bitbucket pipeline stages are translated into the corresponding Buildkite pipeline as a [group step](https://buildkite.com/docs/pipelines/group-step). | -| `pipelines.default.stage.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

In cases where many paths are needed to be polled for changes - utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin) is recommended; watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | -| `pipelines.default.step.after-script` | ❌ | The actions that a Bitbucket pipeline will undertake after the commands in the `script` key are run. For similar behaviour, a [repository-level](https://buildkite.com/docs/agent/v3/hooks#hook-locations-repository-hooks) `pre-exit` hook approach will yield similar behaviour - running at the latter end of the [job lifecycle](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks). | -| `pipelines.default.step.artifacts` | ⚠ | Build artifacts that will be required for steps later in the Bitbucket pipeline (by default, not obtained unless an explicit `buildkite-agent artifact download` [command](https://buildkite.com/docs/agent/v3/cli-artifact#downloading-artifacts) is run beforehand within the generated Buildkite command step). Artifacts that are specified (whether one specific file, or multiple) will be set within the generated command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | -| `pipelines.default.step.caches` | ✅ | Step-level dependencies downloaded from external sources (Docker, Maven, PyPi for example) which will be able to be re-used in later Bitbucket pipeline steps. Caches that are set at step level (or though the top-level `definition.cache. property) are translated in the corresponding Buildkite pipeline utilising the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) to store the downloaded dependencies for re-use between Buildkite builds. | -| `pipelines.default.step.condition` | ✅ | The configuration to prevent a Bitbucket pipeline step from running unless the specific conditional is met. Translated to an inline conditional (`if`) within the corresponding Buildkite pipelines' command step's `commands` - based on a `git diff` of the base branch. | -| `pipelines.default.step.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

In cases where many paths are needed to be polled for changes - utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin) is recommended; watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | -| `pipelines.default.step.clone` | ⚠️ | Clone options for a specific step of a Bitbucket pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property) | -| `pipelines.default.step.clone.sparse-checkout` | ✅ | Sparse-checkout option for a Bitbucket pipeline step. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin).

Sparse-checkout properties of `code-mode`, `enabled` and `patterns` will be trasnlated to the respective properties within the mentinoed plugin's configuration. | -| `pipelines.default.step.deployment` | ❌ | The environment set for the Bitbucket Deployments dashboard. This has no translatable equivalent within Buildkite. | -| `pipelines.default.step.docker` | ❌ | The availability of docker in a specific Bitbucket pipeline step. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | -| `pipelines.default.step.fail-fast` | ❌ | Whether a specific step of a Bitbucket pipeline allows a parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | -| `pipelines.default.step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin).

The `aws`, `aws.oidc`, `name`, `username` and `password` sub-properties are supported - refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/docker-image-options/) for more information. | -| `pipelines.default.step.max-time` | ✅ | The maximum allowable time that a step within a Bitbucket pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. | -| `pipelines.default.step.name` | ✅ | The name of a specific step within a Bitbucket pipeline. Translates to a Buildkite command step's `label`. | -| `pipelines.default.step.oidc` | ✅ | Open ID Connect configuration that will be applied for this Bitbucket pipeline step. The generated command step in the corresponding Buildkite pipeline will [request](https://buildkite.com/docs/agent/v3/cli-oidc#request-oidc-token) an OIDC token and export it into the job environment as `BITBUCKET_STEP_OIDC_TOKEN` (to be passed to `sts` to assume an AWS role for example) | -| `pipelines.default.step.runs-on` | ✅ | Allocating the Bitbucket pipeline to run on a self-hosted runner with the specific label. All `runs-on` values will be set as agent [tags](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | -| `pipelines.default.step.services` | ⚠️ | The name of one or more services defined at `definitions.services.` that will be applied for this step. Translated to utilise the service configuration with the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin).

Generated configuration will need to be saved to a `compose.yaml` file within the repository, and the image utilised with the Buildkite command step as `app`.

Refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/databases-and-service-containers/) for more details on service containers and configuration references. | -| `pipelines.default.step.script` | ✅ | The individual commands that make up a specific step. Each is translated into a singular command within the `commands` key of a Buildkite command step. | -| `pipelines.default.step.script.pipe` | ❌ | Re-usable modules to make configuration in Bitbucket pipelines easier and modular. Consider exploring the suite of available [Buildkite Plugins](https://buildkite.com/plugins) for corresponding functionality that is required. | -| `pipelines.default.step.size` | ✅ | Allocation of sizing options for the given memory for a specific step within a Bitbucket pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | -| `pipelines.default.step.trigger` | ✅ | The configuration setting the running of a Bitbucket pipeline step manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline before the specified `script` within a further command step. Explicit dependencies with `depends_on` are set between the two steps; requiring manual processing. | -| `pipelines.default.variables` | ✅ | Custom variables that are passed to Bitbucket pipeline steps. Each variable defined in a Bitbucket pipeline step is translated to a Buildkite [input step](https://buildkite.com/docs/pipelines/input-step) with/without defaults and allowed values specified below. | -| `pipelines.default.variables.name` | ✅ | The variables' name: translated to the `key` attribute of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step) (text entry).| -| `pipelines.default.variables.default` | ✅ | The default variable value if no value is set. Set as the `default` attribute within the `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | -| `pipelines.default.variables.description` | ✅ | The description of the variable. Translated to the `text` attribute of a specific `field` within the generated [input step](https://buildkite.com/docs/pipelines/input-step). | -| `pipelines.default.variables.allowed-values` | ✅ | The variable's allowed values: each option is translated to a singular `options` object with given value of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | +| `pipelines.custom.` | ❌ | The name of the custom Bitbucket pipeline | +| `pipelines.custom..parallel` | ❌ | Parallel (concurrent step) configuration for custom Bitbucket pipelines. | +| `pipelines.custom..step` | ✅ | Individual step configuration for a custom Bitbucket pipeline. View the available step [properties](#step-pipelinestypestep) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/step-options/#The-Step-property). | +| `pipelines.custom..stage` | ❌ | Stage configuration for custom Bitbucket pipelines. | +| `pipelines.custom..variables` | ❌ | Variable configuration for custom Bitbucket pipelines. | + +### Default (`pipelines.default`) + +| Key | Supported? | Notes | +| --- | --- | --- | +| `pipelines.default` | ✅ | Bitbucket pipeline configuration that does not meet a specific configuration. Additional details can be found on this pipeline type's [documentation](https://support.atlassian.com/bitbucket-cloud/docs/pipeline-start-conditions/#Default) | +| `pipelines.default.parallel` | ✅ | Parallel (concurrent step) configuration for default Bitbucket pipelines. View the available parallel [properties](#parallel-pipelinestypeparallel) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/parallel-step-options/#Parallel). | +| `pipelines.default.step` | ✅ | Individual step configuration for default Bitbucket pipelines. View the available step [properties](#step-pipelinestypestep) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/step-options/#The-Step-property). | +| `pipelines.default.stage` | ✅ | Stage configuration for default Bitbucket pipelines. View the available stage [properties](#stage-pipelinestypestage) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/stage-options/#Stage). | + +### Pull Request (`pipelines.pull_request`) + +| Key | Supported? | Notes | +| --- | --- | --- | | `pipelines.pull_request` | ✅ | Application of specific Bitbucket pipeline configuration based for pull requests. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `pull_request.id`/`BUILDKITE_PULL_REQUEST` and `build.branch`/`BUILDKITE_BRANCH` variables. | | `pipelines.pull_request.` | ✅ | The base branch name/wildcard to apply specific Bitbucket pipeline step configuration within. Apply the configuration for all builds with a `**` wildcard. | -| `pipelines.pull_request..step` | ✅ | Individual step configuration for pull requests builds within a Bitbucket pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | +| `pipelines.pull_request..parallel` | ✅ | Parallel (concurrent step) configuration for pull request builds of a Bitbucket pipeline. View the available parallel [properties](#parallel-pipelinestypeparallel) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/parallel-step-options/#Parallel). | +| `pipelines.pull_request..step` | ✅ | Individual step configuration for pull requests builds within a Bitbucket pipeline. View the available step [properties](#step-pipelinestypestep) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/step-options/#The-Step-property). | +| `pipelines.default.stage` | ✅ | Stage configuration for pull request builds of a Bitbucket pipelines. View the available stage [properties](#stage-pipelinestypestage) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/stage-options/#Stage). | + +### Tags (`pipelines.tags`) + +| Key | Supported? | Notes | +| --- | --- | --- | | `pipelines.tags` | ✅ | Application of specific Bitbucket pipeline configuration based for triggered tag builds. Translated to a [step conditional](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-steps) in the corresponding Buildkite pipeline utilising the `build.tag`/`BUILDKITE_TAG` variable. | | `pipelines.tags.` | ✅ | The tag name/wildcard to apply specific Bitbucket pipeline step configuration within. | -| `pipelines.tags..step` | ✅ | Individual step configuration for triggered tag builds within a Bitbucket pipeline. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | \ No newline at end of file +| `pipelines.tags..parallel` | ✅ | Parallel (concurrent step) configuration for tag builds of a Bitbucket pipeline. View the available parallel [properties](#parallel-pipelinestypeparallel) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/parallel-step-options/#Parallel). | +| `pipelines.tags..step` | ✅ | Individual step configuration for tag builds within a Bitbucket pipeline. View the available step [properties](#step-pipelinestypestep) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/step-options/#The-Step-property). | +| `pipelines.tags..stage` | ✅ | Stage configuration for tag builds of a Bitbucket pipelines. View the available stage [properties](#stage-pipelinestypestage) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/stage-options/#Stage). | + +## Pipeline Properties (`pipelines..`) + +> Each starting pipeline condition listed [above](#pipeline-starting-conditions-pipelinesstart-condition) can support various pipeline properties: +> - `parallel`: The grouping of multiple steps within a Bitbucket pipeline to run concurrently. +> - `step`: A logical execution unit that makes up a specific workflow within a Bitbucket pipeline. +> - `stage`: The grouping of multiple Bitbucket pipeline steps with shared properties. +> - `variables`: Customized variable definition to utilise within a custom Bitbucket pipeline starting condition. +> +> For information on each of these individual properties, refer to the reference within the Bitbucket Pipelines documentation for [parallel](https://support.atlassian.com/bitbucket-cloud/docs/parallel-step-options/#Parallel), [step](https://support.atlassian.com/bitbucket-cloud/docs/step-options/#The-Step-property), [stage](https://support.atlassian.com/bitbucket-cloud/docs/stage-options/#Stage) and [variable](https://support.atlassian.com/bitbucket-cloud/docs/pipeline-start-conditions/#Custom--manual--pipeline-variables) properties. + +### Parallel (`pipelines..parallel`) + +| Key | Supported? | Notes | +| --- | --- | --- | +| `pipelines..parallel` | ✅ | The grouping of multiple steps within a Bitbucket pipeline that are to be run concurrently. By default, Buildkite executes steps in parallel unless [implicit or explicit dependencies](https://buildkite.com/docs/pipelines/dependencies) are set. Parallel Bitbucket pipeline steps are transitioned into a [group step](https://buildkite.com/docs/pipelines/group-step) within the generated Buildkite pipeline without explicit dependencies. | +| `pipelines..parallel.fail-fast` | ❌ | Whether a Bitbucket pipeline allows this parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | + +### Step (`pipelines..step`) + +| Key | Supported? | Notes | +| --- | --- | --- | +| `pipelines..step.after-script` | ❌ | The actions that a Bitbucket pipeline will undertake after the commands in the `script` key are run. For similar behaviour, a [repository-level](https://buildkite.com/docs/agent/v3/hooks#hook-locations-repository-hooks) `pre-exit` hook approach will yield similar behaviour - running at the latter end of the [job lifecycle](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks). | +| `pipelines..step.artifacts` | ⚠ | Build artifacts that will be required for steps later in the Bitbucket pipeline (by default, not obtained unless an explicit `buildkite-agent artifact download` [command](https://buildkite.com/docs/agent/v3/cli-artifact#downloading-artifacts) is run beforehand within the generated Buildkite command step). Artifacts that are specified (whether one specific file, or multiple) will be set within the generated command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | +| `pipelines..step.caches` | ✅ | Step-level dependencies downloaded from external sources (Docker, Maven, PyPi for example) which will be able to be re-used in later Bitbucket pipeline steps. Caches that are set at step level (or though the top-level `definition.cache. property) are translated in the corresponding Buildkite pipeline utilising the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) to store the downloaded dependencies for re-use between Buildkite builds. | +| `pipelines..step.condition` | ✅ | The configuration to prevent a Bitbucket pipeline step from running unless the specific conditional is met. Translated to an inline conditional (`if`) within the corresponding Buildkite pipelines' command step's `commands` - based on a `git diff` of the base branch. | +| `pipelines..step.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

In cases where many paths are needed to be polled for changes - utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin) is recommended; watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | +| `pipelines..step.clone` | ⚠️ | Clone options for a specific step of a Bitbucket pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property) | +| `pipelines..step.clone.sparse-checkout` | ✅ | Sparse-checkout option for a Bitbucket pipeline step. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin).

Sparse-checkout properties of `code-mode`, `enabled` and `patterns` will be trasnlated to the respective properties within the mentinoed plugin's configuration. | +| `pipelines..step.deployment` | ❌ | The environment set for the Bitbucket Deployments dashboard. This has no translatable equivalent within Buildkite. | +| `pipelines..step.docker` | ❌ | The availability of docker in a specific Bitbucket pipeline step. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | +| `pipelines..step.fail-fast` | ❌ | Whether a specific step of a Bitbucket pipeline allows a parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | +| `pipelines..step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin).

The `aws`, `aws.oidc`, `name`, `username` and `password` sub-properties are supported - refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/docker-image-options/) for more information. | +| `pipelines..step.max-time` | ✅ | The maximum allowable time that a step within a Bitbucket pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. | +| `pipelines..step.name` | ✅ | The name of a specific step within a Bitbucket pipeline. Translates to a Buildkite command step's `label`. | +| `pipelines..step.oidc` | ✅ | Open ID Connect configuration that will be applied for this Bitbucket pipeline step. The generated command step in the corresponding Buildkite pipeline will [request](https://buildkite.com/docs/agent/v3/cli-oidc#request-oidc-token) an OIDC token and export it into the job environment as `BITBUCKET_STEP_OIDC_TOKEN` (to be passed to `sts` to assume an AWS role for example) | +| `pipelines..step.runs-on` | ✅ | Allocating the Bitbucket pipeline to run on a self-hosted runner with the specific label. All `runs-on` values will be set as agent [tags](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | +| `pipelines..step.services` | ⚠️ | The name of one or more services defined at `definitions.services.` that will be applied for this step. Translated to utilise the service configuration with the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin).

Generated configuration will need to be saved to a `compose.yaml` file within the repository, and the image utilised with the Buildkite command step as `app`.

Refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/databases-and-service-containers/) for more details on service containers and configuration references. | +| `pipelines..step.script` | ✅ | The individual commands that make up a specific step. Each is translated into a singular command within the `commands` key of a Buildkite command step. | +| `pipelines..step.script.pipe` | ❌ | Re-usable modules to make configuration in Bitbucket pipelines easier and modular. Consider exploring the suite of available [Buildkite Plugins](https://buildkite.com/plugins) for corresponding functionality that is required. | +| `pipelines..step.size` | ✅ | Allocation of sizing options for the given memory for a specific step within a Bitbucket pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | +| `pipelines..step.trigger` | ✅ | The configuration setting the running of a Bitbucket pipeline step manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline before the specified `script` within a further command step. Explicit dependencies with `depends_on` are set between the two steps; requiring manual processing. | + +### Stage (`pipelines..stage`) + +| Key | Supported? | Notes | +| --- | --- | --- | +| `pipelines..stage` | ✅ | The logical grouping of one or more Bitbucket pipeline steps. Bitbucket pipeline stages are translated into the corresponding Buildkite pipeline as a [group step](https://buildkite.com/docs/pipelines/group-step). | +| `pipelines..stage.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

In cases where many paths are needed to be polled for changes - utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin) is recommended; watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | +| `pipelines.default.stage.name` | ✅ | The name of the Bitbucket pipeline stage. Transitioned to the `group` label of the corresponding Buildkite [group step](https://buildkite.com/docs/pipelines/group-step). | +| `pipelines.default.stage.steps` | ✅ | Individual step configuration for a Bitbucket pipeline stage. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | +| `pipelines.default.stage.trigger` | ✅ | The configuration setting the running of a Bitbucket pipeline stage manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline's group step before the specified `steps` of this stage. An explicit dependencies with `depends_on` are set between the input step and the following steps to ensure manual processing is required for these to run. | + +### Variables (`pipelines..variables`) + +| Key | Supported? | Notes | +| --- | --- | --- | +| `pipelines..variables` | ✅ | Custom variables that are passed to Bitbucket pipeline steps. Each variable defined in a Bitbucket pipeline step is translated to a Buildkite [input step](https://buildkite.com/docs/pipelines/input-step) with/without defaults and allowed values specified below. | +| `pipelines..variables.name` | ✅ | The variables' name: translated to the `key` attribute of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step) (text entry).| +| `pipelines..variables.default` | ✅ | The default variable value if no value is set. Set as the `default` attribute within the `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | +| `pipelines..variables.description` | ✅ | The description of the variable. Translated to the `text` attribute of a specific `field` within the generated [input step](https://buildkite.com/docs/pipelines/input-step). | +| `pipelines..variables.allowed-values` | ✅ | The variable's allowed values: each option is translated to a singular `options` object with given value of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | + From 2d79b25d428891511ed90e8c0ce0323d53dfc42e Mon Sep 17 00:00:00 2001 From: James Sammut Date: Wed, 1 May 2024 11:57:45 +1000 Subject: [PATCH 13/20] Bitbucket docs adjustment #3 --- docs/Bitbucket.md | 78 +++++++++++++++++++++++------------------------ 1 file changed, 39 insertions(+), 39 deletions(-) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index 43cb0bd4..24e0c7d8 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -39,7 +39,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | -| `options` | ⚠️ | Customised options utilised throughout a Bitbucket pipeline. The `max-time` and `size` sub-properties are supported for translation within Buildkite Migration tool through to the generated Buildkite command step's `timeout_in_minutes` and agent tag respectfully.

The `docker` sub-property is not supported: and will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available.

Both supported properties at Bitbucket pipeline step-level definition (`pipelines..step.max-time` and `pipelines..step.size`) will have higher precedences than the two values set at `options` level. | +| `options` | ⚠️ | Customised options utilised throughout a Bitbucket pipeline. The `max-time` and `size` sub-properties are supported for translation within Buildkite Migration tool through to the generated Buildkite command step's `timeout_in_minutes` and agent tag respectfully.

The `docker` sub-property is not supported: and will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available.

Both supported properties at Bitbucket pipeline [step-level definition](#pipeline-properties-pipelinestypeproperty) will have higher precedences than the two values set at `options` level. | ## Pipeline Starting Conditions (`pipelines.`) @@ -103,7 +103,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | `pipelines.tags..step` | ✅ | Individual step configuration for tag builds within a Bitbucket pipeline. View the available step [properties](#step-pipelinestypestep) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/step-options/#The-Step-property). | | `pipelines.tags..stage` | ✅ | Stage configuration for tag builds of a Bitbucket pipelines. View the available stage [properties](#stage-pipelinestypestage) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/stage-options/#Stage). | -## Pipeline Properties (`pipelines..`) +## Pipeline Properties (`pipelines..`) > Each starting pipeline condition listed [above](#pipeline-starting-conditions-pipelinesstart-condition) can support various pipeline properties: > - `parallel`: The grouping of multiple steps within a Bitbucket pipeline to run concurrently. @@ -113,55 +113,55 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( > > For information on each of these individual properties, refer to the reference within the Bitbucket Pipelines documentation for [parallel](https://support.atlassian.com/bitbucket-cloud/docs/parallel-step-options/#Parallel), [step](https://support.atlassian.com/bitbucket-cloud/docs/step-options/#The-Step-property), [stage](https://support.atlassian.com/bitbucket-cloud/docs/stage-options/#Stage) and [variable](https://support.atlassian.com/bitbucket-cloud/docs/pipeline-start-conditions/#Custom--manual--pipeline-variables) properties. -### Parallel (`pipelines..parallel`) +### Parallel (`pipelines..parallel`) | Key | Supported? | Notes | | --- | --- | --- | -| `pipelines..parallel` | ✅ | The grouping of multiple steps within a Bitbucket pipeline that are to be run concurrently. By default, Buildkite executes steps in parallel unless [implicit or explicit dependencies](https://buildkite.com/docs/pipelines/dependencies) are set. Parallel Bitbucket pipeline steps are transitioned into a [group step](https://buildkite.com/docs/pipelines/group-step) within the generated Buildkite pipeline without explicit dependencies. | -| `pipelines..parallel.fail-fast` | ❌ | Whether a Bitbucket pipeline allows this parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | +| `pipelines..parallel` | ✅ | The grouping of multiple steps within a Bitbucket pipeline that are to be run concurrently. By default, Buildkite executes steps in parallel unless [implicit or explicit dependencies](https://buildkite.com/docs/pipelines/dependencies) are set. Parallel Bitbucket pipeline steps are transitioned into a [group step](https://buildkite.com/docs/pipelines/group-step) within the generated Buildkite pipeline without explicit dependencies. | +| `pipelines..parallel.fail-fast` | ❌ | Whether a Bitbucket pipeline allows this parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | -### Step (`pipelines..step`) +### Step (`pipelines..step`) | Key | Supported? | Notes | | --- | --- | --- | -| `pipelines..step.after-script` | ❌ | The actions that a Bitbucket pipeline will undertake after the commands in the `script` key are run. For similar behaviour, a [repository-level](https://buildkite.com/docs/agent/v3/hooks#hook-locations-repository-hooks) `pre-exit` hook approach will yield similar behaviour - running at the latter end of the [job lifecycle](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks). | -| `pipelines..step.artifacts` | ⚠ | Build artifacts that will be required for steps later in the Bitbucket pipeline (by default, not obtained unless an explicit `buildkite-agent artifact download` [command](https://buildkite.com/docs/agent/v3/cli-artifact#downloading-artifacts) is run beforehand within the generated Buildkite command step). Artifacts that are specified (whether one specific file, or multiple) will be set within the generated command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | -| `pipelines..step.caches` | ✅ | Step-level dependencies downloaded from external sources (Docker, Maven, PyPi for example) which will be able to be re-used in later Bitbucket pipeline steps. Caches that are set at step level (or though the top-level `definition.cache. property) are translated in the corresponding Buildkite pipeline utilising the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) to store the downloaded dependencies for re-use between Buildkite builds. | -| `pipelines..step.condition` | ✅ | The configuration to prevent a Bitbucket pipeline step from running unless the specific conditional is met. Translated to an inline conditional (`if`) within the corresponding Buildkite pipelines' command step's `commands` - based on a `git diff` of the base branch. | -| `pipelines..step.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

In cases where many paths are needed to be polled for changes - utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin) is recommended; watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | -| `pipelines..step.clone` | ⚠️ | Clone options for a specific step of a Bitbucket pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property) | -| `pipelines..step.clone.sparse-checkout` | ✅ | Sparse-checkout option for a Bitbucket pipeline step. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin).

Sparse-checkout properties of `code-mode`, `enabled` and `patterns` will be trasnlated to the respective properties within the mentinoed plugin's configuration. | -| `pipelines..step.deployment` | ❌ | The environment set for the Bitbucket Deployments dashboard. This has no translatable equivalent within Buildkite. | -| `pipelines..step.docker` | ❌ | The availability of docker in a specific Bitbucket pipeline step. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | -| `pipelines..step.fail-fast` | ❌ | Whether a specific step of a Bitbucket pipeline allows a parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | -| `pipelines..step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin).

The `aws`, `aws.oidc`, `name`, `username` and `password` sub-properties are supported - refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/docker-image-options/) for more information. | -| `pipelines..step.max-time` | ✅ | The maximum allowable time that a step within a Bitbucket pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. | -| `pipelines..step.name` | ✅ | The name of a specific step within a Bitbucket pipeline. Translates to a Buildkite command step's `label`. | -| `pipelines..step.oidc` | ✅ | Open ID Connect configuration that will be applied for this Bitbucket pipeline step. The generated command step in the corresponding Buildkite pipeline will [request](https://buildkite.com/docs/agent/v3/cli-oidc#request-oidc-token) an OIDC token and export it into the job environment as `BITBUCKET_STEP_OIDC_TOKEN` (to be passed to `sts` to assume an AWS role for example) | -| `pipelines..step.runs-on` | ✅ | Allocating the Bitbucket pipeline to run on a self-hosted runner with the specific label. All `runs-on` values will be set as agent [tags](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | -| `pipelines..step.services` | ⚠️ | The name of one or more services defined at `definitions.services.` that will be applied for this step. Translated to utilise the service configuration with the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin).

Generated configuration will need to be saved to a `compose.yaml` file within the repository, and the image utilised with the Buildkite command step as `app`.

Refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/databases-and-service-containers/) for more details on service containers and configuration references. | -| `pipelines..step.script` | ✅ | The individual commands that make up a specific step. Each is translated into a singular command within the `commands` key of a Buildkite command step. | -| `pipelines..step.script.pipe` | ❌ | Re-usable modules to make configuration in Bitbucket pipelines easier and modular. Consider exploring the suite of available [Buildkite Plugins](https://buildkite.com/plugins) for corresponding functionality that is required. | -| `pipelines..step.size` | ✅ | Allocation of sizing options for the given memory for a specific step within a Bitbucket pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | -| `pipelines..step.trigger` | ✅ | The configuration setting the running of a Bitbucket pipeline step manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline before the specified `script` within a further command step. Explicit dependencies with `depends_on` are set between the two steps; requiring manual processing. | - -### Stage (`pipelines..stage`) +| `pipelines..step.after-script` | ❌ | The actions that a Bitbucket pipeline will undertake after the commands in the `script` key are run. For similar behaviour, a [repository-level](https://buildkite.com/docs/agent/v3/hooks#hook-locations-repository-hooks) `pre-exit` hook approach will yield similar behaviour - running at the latter end of the [job lifecycle](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks). | +| `pipelines..step.artifacts` | ⚠ | Build artifacts that will be required for steps later in the Bitbucket pipeline (by default, not obtained unless an explicit `buildkite-agent artifact download` [command](https://buildkite.com/docs/agent/v3/cli-artifact#downloading-artifacts) is run beforehand within the generated Buildkite command step). Artifacts that are specified (whether one specific file, or multiple) will be set within the generated command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | +| `pipelines..step.caches` | ✅ | Step-level dependencies downloaded from external sources (Docker, Maven, PyPi for example) which will be able to be re-used in later Bitbucket pipeline steps. Caches that are set at step level (or though the top-level `definition.cache. property) are translated in the corresponding Buildkite pipeline utilising the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) to store the downloaded dependencies for re-use between Buildkite builds. | +| `pipelines..step.condition` | ✅ | The configuration to prevent a Bitbucket pipeline step from running unless the specific conditional is met. Translated to an inline conditional (`if`) within the corresponding Buildkite pipelines' command step's `commands` - based on a `git diff` of the base branch. | +| `pipelines..step.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

In cases where many paths are needed to be polled for changes - utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin) is recommended; watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | +| `pipelines..step.clone` | ⚠️ | Clone options for a specific step of a Bitbucket pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property) | +| `pipelines..step.clone.sparse-checkout` | ✅ | Sparse-checkout option for a Bitbucket pipeline step. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin).

Sparse-checkout properties of `code-mode`, `enabled` and `patterns` will be trasnlated to the respective properties within the mentinoed plugin's configuration. | +| `pipelines..step.deployment` | ❌ | The environment set for the Bitbucket Deployments dashboard. This has no translatable equivalent within Buildkite. | +| `pipelines..step.docker` | ❌ | The availability of docker in a specific Bitbucket pipeline step. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | +| `pipelines..step.fail-fast` | ❌ | Whether a specific step of a Bitbucket pipeline allows a parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | +| `pipelines..step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin).

The `aws`, `aws.oidc`, `name`, `username` and `password` sub-properties are supported - refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/docker-image-options/) for more information. | +| `pipelines..step.max-time` | ✅ | The maximum allowable time that a step within a Bitbucket pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. | +| `pipelines..step.name` | ✅ | The name of a specific step within a Bitbucket pipeline. Translates to a Buildkite command step's `label`. | +| `pipelines..step.oidc` | ✅ | Open ID Connect configuration that will be applied for this Bitbucket pipeline step. The generated command step in the corresponding Buildkite pipeline will [request](https://buildkite.com/docs/agent/v3/cli-oidc#request-oidc-token) an OIDC token and export it into the job environment as `BITBUCKET_STEP_OIDC_TOKEN` (to be passed to `sts` to assume an AWS role for example) | +| `pipelines..step.runs-on` | ✅ | Allocating the Bitbucket pipeline to run on a self-hosted runner with the specific label. All `runs-on` values will be set as agent [tags](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | +| `pipelines..step.services` | ⚠️ | The name of one or more services defined at `definitions.services.` that will be applied for this step. Translated to utilise the service configuration with the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin).

Generated configuration will need to be saved to a `compose.yaml` file within the repository, and the image utilised with the Buildkite command step as `app`.

Refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/databases-and-service-containers/) for more details on service containers and configuration references. | +| `pipelines..step.script` | ✅ | The individual commands that make up a specific step. Each is translated into a singular command within the `commands` key of a Buildkite command step. | +| `pipelines..step.script.pipe` | ❌ | Re-usable modules to make configuration in Bitbucket pipelines easier and modular. Consider exploring the suite of available [Buildkite Plugins](https://buildkite.com/plugins) for corresponding functionality that is required. | +| `pipelines..step.size` | ✅ | Allocation of sizing options for the given memory for a specific step within a Bitbucket pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | +| `pipelines..step.trigger` | ✅ | The configuration setting the running of a Bitbucket pipeline step manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline before the specified `script` within a further command step. Explicit dependencies with `depends_on` are set between the two steps; requiring manual processing. | + +### Stage (`pipelines..stage`) | Key | Supported? | Notes | | --- | --- | --- | -| `pipelines..stage` | ✅ | The logical grouping of one or more Bitbucket pipeline steps. Bitbucket pipeline stages are translated into the corresponding Buildkite pipeline as a [group step](https://buildkite.com/docs/pipelines/group-step). | -| `pipelines..stage.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

In cases where many paths are needed to be polled for changes - utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin) is recommended; watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | -| `pipelines.default.stage.name` | ✅ | The name of the Bitbucket pipeline stage. Transitioned to the `group` label of the corresponding Buildkite [group step](https://buildkite.com/docs/pipelines/group-step). | -| `pipelines.default.stage.steps` | ✅ | Individual step configuration for a Bitbucket pipeline stage. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | -| `pipelines.default.stage.trigger` | ✅ | The configuration setting the running of a Bitbucket pipeline stage manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline's group step before the specified `steps` of this stage. An explicit dependencies with `depends_on` are set between the input step and the following steps to ensure manual processing is required for these to run. | +| `pipelines..stage` | ✅ | The logical grouping of one or more Bitbucket pipeline steps. Bitbucket pipeline stages are translated into the corresponding Buildkite pipeline as a [group step](https://buildkite.com/docs/pipelines/group-step). | +| `pipelines..stage.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

In cases where many paths are needed to be polled for changes - utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin) is recommended; watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | +| `pipelines..stage.name` | ✅ | The name of the Bitbucket pipeline stage. Transitioned to the `group` label of the corresponding Buildkite [group step](https://buildkite.com/docs/pipelines/group-step). | +| `pipelines..stage.steps` | ✅ | Individual step configuration for a Bitbucket pipeline stage. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | +| `pipelines..stage.trigger` | ✅ | The configuration setting the running of a Bitbucket pipeline stage manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline's group step before the specified `steps` of this stage. An explicit dependencies with `depends_on` are set between the input step and the following steps to ensure manual processing is required for these to run. | -### Variables (`pipelines..variables`) +### Variables (`pipelines..variables`) | Key | Supported? | Notes | | --- | --- | --- | -| `pipelines..variables` | ✅ | Custom variables that are passed to Bitbucket pipeline steps. Each variable defined in a Bitbucket pipeline step is translated to a Buildkite [input step](https://buildkite.com/docs/pipelines/input-step) with/without defaults and allowed values specified below. | -| `pipelines..variables.name` | ✅ | The variables' name: translated to the `key` attribute of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step) (text entry).| -| `pipelines..variables.default` | ✅ | The default variable value if no value is set. Set as the `default` attribute within the `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | -| `pipelines..variables.description` | ✅ | The description of the variable. Translated to the `text` attribute of a specific `field` within the generated [input step](https://buildkite.com/docs/pipelines/input-step). | -| `pipelines..variables.allowed-values` | ✅ | The variable's allowed values: each option is translated to a singular `options` object with given value of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | +| `pipelines..variables` | ✅ | Custom variables that are passed to Bitbucket pipeline steps. Each variable defined in a Bitbucket pipeline step is translated to a Buildkite [input step](https://buildkite.com/docs/pipelines/input-step) with/without defaults and allowed values specified below. | +| `pipelines..variables.name` | ✅ | The variables' name: translated to the `key` attribute of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step) (text entry).| +| `pipelines..variables.default` | ✅ | The default variable value if no value is set. Set as the `default` attribute within the `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | +| `pipelines..variables.description` | ✅ | The description of the variable. Translated to the `text` attribute of a specific `field` within the generated [input step](https://buildkite.com/docs/pipelines/input-step). | +| `pipelines..variables.allowed-values` | ✅ | The variable's allowed values: each option is translated to a singular `options` object with given value of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | From 5193699805ebb91d16f23015f235f4f8f08072be Mon Sep 17 00:00:00 2001 From: James Sammut Date: Wed, 1 May 2024 11:59:15 +1000 Subject: [PATCH 14/20] =?UTF-8?q?Incorrect=20rendering=20=E2=9A=A0?= =?UTF-8?q?=EF=B8=8F?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/Bitbucket.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index 24e0c7d8..0e8ca684 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -125,7 +125,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | | `pipelines..step.after-script` | ❌ | The actions that a Bitbucket pipeline will undertake after the commands in the `script` key are run. For similar behaviour, a [repository-level](https://buildkite.com/docs/agent/v3/hooks#hook-locations-repository-hooks) `pre-exit` hook approach will yield similar behaviour - running at the latter end of the [job lifecycle](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks). | -| `pipelines..step.artifacts` | ⚠ | Build artifacts that will be required for steps later in the Bitbucket pipeline (by default, not obtained unless an explicit `buildkite-agent artifact download` [command](https://buildkite.com/docs/agent/v3/cli-artifact#downloading-artifacts) is run beforehand within the generated Buildkite command step). Artifacts that are specified (whether one specific file, or multiple) will be set within the generated command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | +| `pipelines..step.artifacts` | ⚠️ | Build artifacts that will be required for steps later in the Bitbucket pipeline (by default, not obtained unless an explicit `buildkite-agent artifact download` [command](https://buildkite.com/docs/agent/v3/cli-artifact#downloading-artifacts) is run beforehand within the generated Buildkite command step). Artifacts that are specified (whether one specific file, or multiple) will be set within the generated command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | | `pipelines..step.caches` | ✅ | Step-level dependencies downloaded from external sources (Docker, Maven, PyPi for example) which will be able to be re-used in later Bitbucket pipeline steps. Caches that are set at step level (or though the top-level `definition.cache. property) are translated in the corresponding Buildkite pipeline utilising the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) to store the downloaded dependencies for re-use between Buildkite builds. | | `pipelines..step.condition` | ✅ | The configuration to prevent a Bitbucket pipeline step from running unless the specific conditional is met. Translated to an inline conditional (`if`) within the corresponding Buildkite pipelines' command step's `commands` - based on a `git diff` of the base branch. | | `pipelines..step.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

In cases where many paths are needed to be polled for changes - utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin) is recommended; watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | From 4353db517ad82e7e37c1e686c4ee970e4f99c154 Mon Sep 17 00:00:00 2001 From: James Sammut Date: Wed, 1 May 2024 12:08:34 +1000 Subject: [PATCH 15/20] Added import property (custom starting condition) row --- docs/Bitbucket.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index 0e8ca684..64eb9a68 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -69,6 +69,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | --- | --- | --- | | `pipelines.custom` | ✅ | Bitbucket pipelines that are only able to be triggered manually. The corresponding Buildkite pipeline is first generated with an [input step](https://buildkite.com/docs/pipelines/input-step) before any command jobs to ensure that triggered builds must be manually processed. | | `pipelines.custom.` | ❌ | The name of the custom Bitbucket pipeline | +| `pipelines.custom..import-pipeline.import` | ❌ | The specification of importing a pipeline from within a specific repository defined in top-level `definitions`. | | `pipelines.custom..parallel` | ❌ | Parallel (concurrent step) configuration for custom Bitbucket pipelines. | | `pipelines.custom..step` | ✅ | Individual step configuration for a custom Bitbucket pipeline. View the available step [properties](#step-pipelinestypestep) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/step-options/#The-Step-property). | | `pipelines.custom..stage` | ❌ | Stage configuration for custom Bitbucket pipelines. | From eb83f28d0ae847b2ceffb9005f85e7deae46af9d Mon Sep 17 00:00:00 2001 From: James Sammut Date: Wed, 1 May 2024 13:44:25 +1000 Subject: [PATCH 16/20] Added depends_on to GroupSteps, variables swotch to custom starting condition --- app/lib/bk/compat/pipeline/step.rb | 6 ++-- .../bitbucket/examples/variables.yml.snap | 7 ++++- .../compat/bitbucket/examples/variables.yml | 31 ++++++++++--------- 3 files changed, 26 insertions(+), 18 deletions(-) diff --git a/app/lib/bk/compat/pipeline/step.rb b/app/lib/bk/compat/pipeline/step.rb index d637b52c..1c68602e 100644 --- a/app/lib/bk/compat/pipeline/step.rb +++ b/app/lib/bk/compat/pipeline/step.rb @@ -206,17 +206,19 @@ def recurse_to_string(value, block, *, **) # group step class GroupStep - attr_accessor :label, :key, :steps, :conditional + attr_accessor :label, :key, :steps, :depends_on, :conditional - def initialize(label: '~', key: nil, steps: [], conditional: nil) + def initialize(label: '~', key: nil, steps: [], depends_on: [], conditional: nil) @label = label @key = key + @depends_on = depends_on @steps = steps @conditional = conditional end def to_h { group: @label, key: @key, steps: @steps.map(&:to_h) }.tap do |h| + h[:depends_on] = @depends_on unless @depends_on.empty? h[:if] = @conditional unless @conditional.nil? end.compact end diff --git a/app/spec/lib/bk/compat/bitbucket/__snapshots__/spec/lib/bk/compat/bitbucket/examples/variables.yml.snap b/app/spec/lib/bk/compat/bitbucket/__snapshots__/spec/lib/bk/compat/bitbucket/examples/variables.yml.snap index 7bbd6bd9..8e49cb21 100644 --- a/app/spec/lib/bk/compat/bitbucket/__snapshots__/spec/lib/bk/compat/bitbucket/examples/variables.yml.snap +++ b/app/spec/lib/bk/compat/bitbucket/__snapshots__/spec/lib/bk/compat/bitbucket/examples/variables.yml.snap @@ -1,6 +1,9 @@ --- steps: -- group: default +- key: execute-34fda39a698739f93a76af1583cf0b9e937f3561 + prompt: Execute step 34fda39a698739f93a76af1583cf0b9e937f3561? + input: execute-34fda39a698739f93a76af1583cf0b9e937f3561 +- group: deployer steps: - key: custom-vars fields: @@ -23,3 +26,5 @@ steps: - commands: - echo "$Username manually triggered for a build for $Region as $Role!" label: Script step + depends_on: + - execute-34fda39a698739f93a76af1583cf0b9e937f3561 diff --git a/app/spec/lib/bk/compat/bitbucket/examples/variables.yml b/app/spec/lib/bk/compat/bitbucket/examples/variables.yml index 0346eebb..48699af2 100644 --- a/app/spec/lib/bk/compat/bitbucket/examples/variables.yml +++ b/app/spec/lib/bk/compat/bitbucket/examples/variables.yml @@ -1,18 +1,19 @@ # from https://support.atlassian.com/bitbucket-cloud/docs/pipeline-start-conditions/#Example-%E2%80%94-using-the-variables-property-to-define-custom-pipeline-variables pipelines: - default: - - variables: - - name: Username - - name: Role - default: "admin" # optionally provide a default variable value - description: "Add user role" - - name: Region - default: "us-east-1" - allowed-values: # optionally restrict variable values - - "ap-southeast-2" - - "us-east-1" - - "us-west-2" - - step: - script: - - echo "$Username manually triggered for a build for $Region as $Role!" \ No newline at end of file + custom: + deployer: # The name that is displayed in the list in the Bitbucket Cloud GUI + - variables: + - name: Username + - name: Role + default: "admin" # optionally provide a default variable value + description: "Add user role" + - name: Region + default: "us-east-1" + allowed-values: # optionally restrict variable values + - "ap-southeast-2" + - "us-east-1" + - "us-west-2" + - step: + script: + - echo "$Username manually triggered for a build for $Region as $Role!" \ No newline at end of file From f1d7b8138e7a7747166bebdd7100a24c1975d8da Mon Sep 17 00:00:00 2001 From: James Sammut Date: Wed, 1 May 2024 14:01:43 +1000 Subject: [PATCH 17/20] Variables property tweak (custom starting condition aligned) --- docs/Bitbucket.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index 64eb9a68..55aa944f 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -73,7 +73,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | `pipelines.custom..parallel` | ❌ | Parallel (concurrent step) configuration for custom Bitbucket pipelines. | | `pipelines.custom..step` | ✅ | Individual step configuration for a custom Bitbucket pipeline. View the available step [properties](#step-pipelinestypestep) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/step-options/#The-Step-property). | | `pipelines.custom..stage` | ❌ | Stage configuration for custom Bitbucket pipelines. | -| `pipelines.custom..variables` | ❌ | Variable configuration for custom Bitbucket pipelines. | +| `pipelines.custom..variables` | ⚠️ | Variable configuration for custom Bitbucket pipelines. View the available variable [properties](#variables-pipelinesstart-conditionvariables) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/pipeline-start-conditions/#Custom--manual--pipeline-variables). | ### Default (`pipelines.default`) @@ -160,7 +160,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | -| `pipelines..variables` | ✅ | Custom variables that are passed to Bitbucket pipeline steps. Each variable defined in a Bitbucket pipeline step is translated to a Buildkite [input step](https://buildkite.com/docs/pipelines/input-step) with/without defaults and allowed values specified below. | +| `pipelines..variables` | ⚠️ | Custom variables that are passed to Bitbucket pipeline steps. Each variable defined in a Bitbucket pipeline step is translated to a Buildkite [input step](https://buildkite.com/docs/pipelines/input-step) with/without defaults and allowed values specified below.

Variables that are translated into the corresponding [input step](https://buildkite.com/docs/pipelines/input-step) within the generated Buildkite pipeline will require to be fetched in subsequent steps through a `buildkite-agent meta-data get` [commmand](https://buildkite.com/docs/agent/v3/cli-meta-data#getting-data). | | `pipelines..variables.name` | ✅ | The variables' name: translated to the `key` attribute of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step) (text entry).| | `pipelines..variables.default` | ✅ | The default variable value if no value is set. Set as the `default` attribute within the `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | | `pipelines..variables.description` | ✅ | The description of the variable. Translated to the `text` attribute of a specific `field` within the generated [input step](https://buildkite.com/docs/pipelines/input-step). | From 4543bdd0969ff6e000595e4b512fe84f6524aa61 Mon Sep 17 00:00:00 2001 From: James Sammut Date: Wed, 1 May 2024 14:04:32 +1000 Subject: [PATCH 18/20] [!NOTE] for Pipeline Properties --- docs/Bitbucket.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index 55aa944f..8dc39b12 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -106,6 +106,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( ## Pipeline Properties (`pipelines..`) +> [!NOTE] > Each starting pipeline condition listed [above](#pipeline-starting-conditions-pipelinesstart-condition) can support various pipeline properties: > - `parallel`: The grouping of multiple steps within a Bitbucket pipeline to run concurrently. > - `step`: A logical execution unit that makes up a specific workflow within a Bitbucket pipeline. From d1fd080c3650bc57d8a80b192c30c120f03363c2 Mon Sep 17 00:00:00 2001 From: James Sammut Date: Wed, 1 May 2024 14:27:53 +1000 Subject: [PATCH 19/20] Some more supported custom starting condition properties --- docs/Bitbucket.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index 8dc39b12..23acf089 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -68,12 +68,12 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | | `pipelines.custom` | ✅ | Bitbucket pipelines that are only able to be triggered manually. The corresponding Buildkite pipeline is first generated with an [input step](https://buildkite.com/docs/pipelines/input-step) before any command jobs to ensure that triggered builds must be manually processed. | -| `pipelines.custom.` | ❌ | The name of the custom Bitbucket pipeline | +| `pipelines.custom.` | ✅ | The name of the custom Bitbucket pipeline. | | `pipelines.custom..import-pipeline.import` | ❌ | The specification of importing a pipeline from within a specific repository defined in top-level `definitions`. | -| `pipelines.custom..parallel` | ❌ | Parallel (concurrent step) configuration for custom Bitbucket pipelines. | +| `pipelines.custom..parallel` | ✅ | Parallel (concurrent step) configuration for a custom Bitbucket pipeline. View the available parallel [properties](#parallel-pipelinestypeparallel) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/parallel-step-options/#Parallel). | | `pipelines.custom..step` | ✅ | Individual step configuration for a custom Bitbucket pipeline. View the available step [properties](#step-pipelinestypestep) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/step-options/#The-Step-property). | -| `pipelines.custom..stage` | ❌ | Stage configuration for custom Bitbucket pipelines. | -| `pipelines.custom..variables` | ⚠️ | Variable configuration for custom Bitbucket pipelines. View the available variable [properties](#variables-pipelinesstart-conditionvariables) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/pipeline-start-conditions/#Custom--manual--pipeline-variables). | +| `pipelines.custom..stage` | ✅ | Stage configuration for a custom Bitbucket pipeline. View the available stage [properties](#stage-pipelinestypestage) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/stage-options/#Stage). | +| `pipelines.custom..variables` | ⚠️ | Variable configuration for a custom Bitbucket pipeline. View the available variable [properties](#variables-pipelinesstart-conditionvariables) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/pipeline-start-conditions/#Custom--manual--pipeline-variables). | ### Default (`pipelines.default`) From 78fb10526154e5d1f07e61ad947ffbd3dd1b50e6 Mon Sep 17 00:00:00 2001 From: James Sammut Date: Mon, 6 May 2024 11:26:20 +1000 Subject: [PATCH 20/20] Improvements --- docs/Bitbucket.md | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/docs/Bitbucket.md b/docs/Bitbucket.md index 23acf089..995001b9 100644 --- a/docs/Bitbucket.md +++ b/docs/Bitbucket.md @@ -9,7 +9,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | -| `clone` | ⚠️ | Clone options for all steps of a Bitbucket pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook.

Sparse-checkout properties of `code-mode`, `enabled` and `patterns` will be translated to the respective properties within the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin).

clone` properties at the Bitbucket pipeline, if set, have higher precedence over these global properties. | +| `clone` | ⚠️ | Clone options for all steps of a Bitbucket pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook.

Sparse-checkout properties of `code-mode`, `enabled` and `patterns` will be translated to the respective properties within the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin).

`clone` properties at the Bitbucket pipeline, if set, have higher precedence over these global properties. | ## Definitions (`definitions`) @@ -19,9 +19,9 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | `definitions.caches` | ⚠️ | Customised cache definitions that can be applied to specific Bitbucket pipeline steps - inclusive of folders, single file - or multifile cache. Targeted into specific steps with the `pipelines.default.step.caches.` property, and in which the translation will utilise the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) that may require further setup/tweaking around specific caching strategies. | | `definitions.caches.` | ✅ | A customised cache name applicable for one or more steps within a Bitbucket pipeline. | | `definitions.caches..path` | ✅ | The directory path that is desired to be cached. | -| `definitions.caches..key.files` | ⚠️ | The list (one or more) files that are monitored for changes - and stored once its hash changes between file versions change. If multiple files are specified - multiple cache-plugin definitions are set on the resultant Buildkite command step (differing `manifest` properties between each). | +| `definitions.caches..key.files` | ⚠️ | The list (one or more) files that are monitored for changes - and stored once its hash changes between file versions change. If multiple files are specified - multiple cache-plugin definitions are set on the resultant Buildkite command step (differing `manifest` properties between each).

Note this may cause issues if the same folder is being maintained by each cache definition! | | `definitions.pipeline` | ⚠️ | Pipelines that are exported for re-use within repositories of the same workspace. Like functionality exists within Buildkite as [Pipeline Templates](https://buildkite.com/docs/pipelines/templates). | -| `definitions.services` | ✅ | Defined Docker services that are applied within a Bitbucket pipeline. Services defined in a corresponding Bitbucket pipeline step using the `pipelines.default.step.services` property will have its configuration applied with the use of the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin).

Generated configuration will need to be saved to a `compose.yaml` file within the repository, and the image utilised with the Buildkite command step as `app`.

Refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/databases-and-service-containers/) for more details on service containers and configuration references. | +| `definitions.services` | ⚠️ | Defined Docker services that are applied within a Bitbucket pipeline. Services defined in a corresponding Bitbucket pipeline step using the `pipelines.default.step.services` property will have its configuration applied with the use of the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin).

Generated configuration will need to be saved to a `compose.yaml` file within the repository, and the image utilised with the Buildkite command step as `app`.

Refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/databases-and-service-containers/) for more details on service containers and configuration references.

Authentication based parameters will not be translated to the corresponding Buildkite pipeline if defined. | ## Export (`export`) @@ -33,7 +33,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | -| `image` | ✅ | The container image that is to be applied for each step within a Bitbucket pipeline, utilising the specified image within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). This has lower precedence over per-step `image` configuration (see `pipelines.default.step.image`).

The `aws`, `aws.oidc`, `name`, `username` and `password` sub-properties are supported - refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/docker-image-options/) for more information. | +| `image` | ✅ | The container image that is to be applied for each step within a Bitbucket pipeline, utilising the specified image within the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin). This has lower precedence over per-step `image` configuration (see `pipelines.default.step.image`).

The `aws`, `aws.oidc`, `name`, `username` and `password` sub-properties are supported through the use of the corresponding plugin ([Docker Login](https://github.com/buildkite-plugins/docker-login-buildkite-plugin/) or [ECR](https://github.com/buildkite-plugins/ecr-buildkite-plugin/)). | ## Options (`options`) @@ -69,7 +69,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | --- | --- | --- | | `pipelines.custom` | ✅ | Bitbucket pipelines that are only able to be triggered manually. The corresponding Buildkite pipeline is first generated with an [input step](https://buildkite.com/docs/pipelines/input-step) before any command jobs to ensure that triggered builds must be manually processed. | | `pipelines.custom.` | ✅ | The name of the custom Bitbucket pipeline. | -| `pipelines.custom..import-pipeline.import` | ❌ | The specification of importing a pipeline from within a specific repository defined in top-level `definitions`. | +| `pipelines.custom..import-pipeline.import` | ❌ | The specification of importing a pipeline from within a specific repository defined in top-level `definitions`. Consider using [trigger steps](https://buildkite.com/docs/pipelines/trigger-step) to invoke builds on a specific Buildkite pipeline. | | `pipelines.custom..parallel` | ✅ | Parallel (concurrent step) configuration for a custom Bitbucket pipeline. View the available parallel [properties](#parallel-pipelinestypeparallel) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/parallel-step-options/#Parallel). | | `pipelines.custom..step` | ✅ | Individual step configuration for a custom Bitbucket pipeline. View the available step [properties](#step-pipelinestypestep) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/step-options/#The-Step-property). | | `pipelines.custom..stage` | ✅ | Stage configuration for a custom Bitbucket pipeline. View the available stage [properties](#stage-pipelinestypestage) supported by the Migrations Tool - and additional property information in the Bitbucket Pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/stage-options/#Stage). | @@ -114,6 +114,8 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( > - `variables`: Customized variable definition to utilise within a custom Bitbucket pipeline starting condition. > > For information on each of these individual properties, refer to the reference within the Bitbucket Pipelines documentation for [parallel](https://support.atlassian.com/bitbucket-cloud/docs/parallel-step-options/#Parallel), [step](https://support.atlassian.com/bitbucket-cloud/docs/step-options/#The-Step-property), [stage](https://support.atlassian.com/bitbucket-cloud/docs/stage-options/#Stage) and [variable](https://support.atlassian.com/bitbucket-cloud/docs/pipeline-start-conditions/#Custom--manual--pipeline-variables) properties. +> +> Additionally, implementation of these pipeline properties can be enhanced with best practices through the use of [Dynamic Pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) to generate and upload pipeline configuration dynamically, using [conditionals](https://buildkite.com/docs/pipelines/conditionals#conditionals-in-pipelines) at both pipelines/step level to apply jobs only on certain conditions and setting [trigger steps](https://buildkite.com/docs/pipelines/trigger-step) with required attributes/environment variable configuration passed through to triggered builds. ### Parallel (`pipelines..parallel`) @@ -128,20 +130,19 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | --- | --- | --- | | `pipelines..step.after-script` | ❌ | The actions that a Bitbucket pipeline will undertake after the commands in the `script` key are run. For similar behaviour, a [repository-level](https://buildkite.com/docs/agent/v3/hooks#hook-locations-repository-hooks) `pre-exit` hook approach will yield similar behaviour - running at the latter end of the [job lifecycle](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks). | | `pipelines..step.artifacts` | ⚠️ | Build artifacts that will be required for steps later in the Bitbucket pipeline (by default, not obtained unless an explicit `buildkite-agent artifact download` [command](https://buildkite.com/docs/agent/v3/cli-artifact#downloading-artifacts) is run beforehand within the generated Buildkite command step). Artifacts that are specified (whether one specific file, or multiple) will be set within the generated command step within the `artifact_paths` [key](https://buildkite.com/docs/pipelines/command-step). Each file found matching (or via glob syntax) will be uploaded to Buildkite's [Artifact storage](https://buildkite.com/docs/agent/v3/cli-artifact) that can be obtained in later steps. | -| `pipelines..step.caches` | ✅ | Step-level dependencies downloaded from external sources (Docker, Maven, PyPi for example) which will be able to be re-used in later Bitbucket pipeline steps. Caches that are set at step level (or though the top-level `definition.cache. property) are translated in the corresponding Buildkite pipeline utilising the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) to store the downloaded dependencies for re-use between Buildkite builds. | -| `pipelines..step.condition` | ✅ | The configuration to prevent a Bitbucket pipeline step from running unless the specific conditional is met. Translated to an inline conditional (`if`) within the corresponding Buildkite pipelines' command step's `commands` - based on a `git diff` of the base branch. | -| `pipelines..step.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

In cases where many paths are needed to be polled for changes - utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin) is recommended; watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | +| `pipelines..step.caches` | ✅ | Step-level dependencies downloaded from external sources (Docker, Maven, PyPi for example) which will be able to be re-used in later Bitbucket pipeline steps. Caches that are set at step level (or through the top-level `definition.cache.` property) are translated in the corresponding Buildkite pipeline utilising the [cache-buildkite-plugin](https://github.com/buildkite-plugins/cache-buildkite-plugin) to store the downloaded dependencies for re-use between Buildkite builds. | +| `pipelines..step.condition` | ⚠️ | The configuration to prevent a Bitbucket pipeline step from running unless the specific conditional is met. Translated to an inline conditional (`if`) within the corresponding Buildkite pipelines' command step's `commands` - based on a `git diff` of the base branch. | +| `pipelines..step.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

Translated to a script that will review the changed files through git. This means that the step itself will actually run and just be marked as passed, which may not be what you want or need.

You may want to consider utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin); watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | | `pipelines..step.clone` | ⚠️ | Clone options for a specific step of a Bitbucket pipeline. The majority of these options should be set on a Buildkite agent itself via [configuration](https://buildkite.com/docs/agent/v3/configuration) of properties such as the clone flags (`git-clone-flags`, `git-clone-mirror-flags` if utilising a Git mirror), fetch flags (`git-fetch-flags`) - or changing the entire checkout process in a customised [plugin](https://buildkite.com/docs/plugins/writing) overriding the default agent `checkout` hook. Sparse checkout options are supported (with the `sparse-checkout` sub-property) | -| `pipelines..step.clone.sparse-checkout` | ✅ | Sparse-checkout option for a Bitbucket pipeline step. Translated to utilising the [sparse-checkout-buildkite-plugin](https://github.com/buildkite-plugins/sparse-checkout-buildkite-plugin).

Sparse-checkout properties of `code-mode`, `enabled` and `patterns` will be trasnlated to the respective properties within the mentinoed plugin's configuration. | | `pipelines..step.deployment` | ❌ | The environment set for the Bitbucket Deployments dashboard. This has no translatable equivalent within Buildkite. | | `pipelines..step.docker` | ❌ | The availability of docker in a specific Bitbucket pipeline step. This will depend on the agent configuration that the corresponding Buildkite command step is being targeted to run said job has available. Consider [tagging](https://buildkite.com/docs/agent/v3/cli-start#tags) agents with `docker=true` to ensure Buildkite command steps requiring hosts with Docker installed and configured to accept/run specific jobs. | | `pipelines..step.fail-fast` | ❌ | Whether a specific step of a Bitbucket pipeline allows a parallel step to fail entirely if it fails (set as `true`), or allows failures (set as `false`). Consider using a combination of `soft_fail` and/or `cancel_on_build_failing` in the corresponding Buildkite command steps' [attributes](https://buildkite.com/docs/pipelines/command-step#command-step-attributes) for a similar [approach](https://buildkite.com/docs/pipelines/command-step#fail-fast). | -| `pipelines..step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin).

The `aws`, `aws.oidc`, `name`, `username` and `password` sub-properties are supported - refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/docker-image-options/) for more information. | +| `pipelines..step.image` | ✅ | The container image that is to be applied for a specific step within a Bitbucket pipeline. Images set at this level will be applied irrespective of the pipeline-level `image` key that is set, and will be applied in the corresponding Buildkite pipeline using the [docker-buildkite-plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin).

The `aws`, `aws.oidc`, `name`, `username` and `password` sub-properties are supported through the use of the corresponding plugin ([Docker Login](https://github.com/buildkite-plugins/docker-login-buildkite-plugin/) or [ECR](https://github.com/buildkite-plugins/ecr-buildkite-plugin/)). | | `pipelines..step.max-time` | ✅ | The maximum allowable time that a step within a Bitbucket pipeline is able to run for. Translates to the corresponding Buildkite pipelines' command step `timeout_in_minutes` attribute. | | `pipelines..step.name` | ✅ | The name of a specific step within a Bitbucket pipeline. Translates to a Buildkite command step's `label`. | | `pipelines..step.oidc` | ✅ | Open ID Connect configuration that will be applied for this Bitbucket pipeline step. The generated command step in the corresponding Buildkite pipeline will [request](https://buildkite.com/docs/agent/v3/cli-oidc#request-oidc-token) an OIDC token and export it into the job environment as `BITBUCKET_STEP_OIDC_TOKEN` (to be passed to `sts` to assume an AWS role for example) | | `pipelines..step.runs-on` | ✅ | Allocating the Bitbucket pipeline to run on a self-hosted runner with the specific label. All `runs-on` values will be set as agent [tags](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | -| `pipelines..step.services` | ⚠️ | The name of one or more services defined at `definitions.services.` that will be applied for this step. Translated to utilise the service configuration with the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin).

Generated configuration will need to be saved to a `compose.yaml` file within the repository, and the image utilised with the Buildkite command step as `app`.

Refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/databases-and-service-containers/) for more details on service containers and configuration references. | +| `pipelines..step.services` | ⚠️ | The name of one or more services defined at `definitions.services.` that will be applied for this step. Translated to utilise the service configuration with the [docker-compose-buildkite-plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin).

Generated configuration will need to be saved to a `compose.yaml` file within the repository, and the image utilised with the Buildkite command step as `app`.

Refer to the Bitbucket pipelines [documentation](https://support.atlassian.com/bitbucket-cloud/docs/databases-and-service-containers/) for more details on service containers and configuration references.

Authentication based parameters will not be translated to the corresponding Buildkite pipeline if defined. | | `pipelines..step.script` | ✅ | The individual commands that make up a specific step. Each is translated into a singular command within the `commands` key of a Buildkite command step. | | `pipelines..step.script.pipe` | ❌ | Re-usable modules to make configuration in Bitbucket pipelines easier and modular. Consider exploring the suite of available [Buildkite Plugins](https://buildkite.com/plugins) for corresponding functionality that is required. | | `pipelines..step.size` | ✅ | Allocation of sizing options for the given memory for a specific step within a Bitbucket pipeline. The `size` value will be set as an agent [tag](https://buildkite.com/docs/pipelines/defining-steps#targeting-specific-agents) in the Buildkite command step for targeting on specific Buildkite agents within an organization. | @@ -152,7 +153,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | | `pipelines..stage` | ✅ | The logical grouping of one or more Bitbucket pipeline steps. Bitbucket pipeline stages are translated into the corresponding Buildkite pipeline as a [group step](https://buildkite.com/docs/pipelines/group-step). | -| `pipelines..stage.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

In cases where many paths are needed to be polled for changes - utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin) is recommended; watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | +| `pipelines..stage.condition.changeset.includePaths` | ⚠️ | The specific file (or files) that need to be detected as changed for the `condition` to apply based. This can be set as specific files - or wildcards that match multiple files in a specific directory/directories.

Translated to a script that will review the changed files through git. This means that the step itself will actually run and just be marked as passed, which may not be what you want or need.

You may want to consider utilising the [monorepo-diff-buildkite-plugin](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin); watching for specific folders/files and uploading resultant [Dynamic pipelines](https://buildkite.com/docs/pipelines/defining-steps#dynamic-pipelines) upon diff detection. | | `pipelines..stage.name` | ✅ | The name of the Bitbucket pipeline stage. Transitioned to the `group` label of the corresponding Buildkite [group step](https://buildkite.com/docs/pipelines/group-step). | | `pipelines..stage.steps` | ✅ | Individual step configuration for a Bitbucket pipeline stage. See configuration options in this section (`pipelines.default.step.`) for supported/unsupported properties. | | `pipelines..stage.trigger` | ✅ | The configuration setting the running of a Bitbucket pipeline stage manually or automatically (latter being defaulted). For `manual` triggers - an [input step](https://buildkite.com/docs/pipelines/input-step) is inserted into the generated Buildkite pipeline's group step before the specified `steps` of this stage. An explicit dependencies with `depends_on` are set between the input step and the following steps to ensure manual processing is required for these to run. | @@ -161,7 +162,7 @@ The Buildkite Migration tool's currently supported (✅), partially supported ( | Key | Supported? | Notes | | --- | --- | --- | -| `pipelines..variables` | ⚠️ | Custom variables that are passed to Bitbucket pipeline steps. Each variable defined in a Bitbucket pipeline step is translated to a Buildkite [input step](https://buildkite.com/docs/pipelines/input-step) with/without defaults and allowed values specified below.

Variables that are translated into the corresponding [input step](https://buildkite.com/docs/pipelines/input-step) within the generated Buildkite pipeline will require to be fetched in subsequent steps through a `buildkite-agent meta-data get` [commmand](https://buildkite.com/docs/agent/v3/cli-meta-data#getting-data). | +| `pipelines..variables` | ⚠️ | Custom variables that are passed to Bitbucket pipeline steps. Each variable defined in a Bitbucket pipeline step is translated to a Buildkite [input step](https://buildkite.com/docs/pipelines/input-step) with/without defaults and allowed values specified below.

Variables that are translated into the corresponding [input step](https://buildkite.com/docs/pipelines/input-step) within the generated Buildkite pipeline will require to be fetched in subsequent steps through a `buildkite-agent meta-data get` [command](https://buildkite.com/docs/agent/v3/cli-meta-data#getting-data). | | `pipelines..variables.name` | ✅ | The variables' name: translated to the `key` attribute of a specific `field` of an [input step](https://buildkite.com/docs/pipelines/input-step) (text entry).| | `pipelines..variables.default` | ✅ | The default variable value if no value is set. Set as the `default` attribute within the `field` of an [input step](https://buildkite.com/docs/pipelines/input-step). | | `pipelines..variables.description` | ✅ | The description of the variable. Translated to the `text` attribute of a specific `field` within the generated [input step](https://buildkite.com/docs/pipelines/input-step). |