From da3a619d9d8059ed82b7d52944fb45f1cb7b3b2b Mon Sep 17 00:00:00 2001 From: Kyongsik Yun Date: Tue, 12 Nov 2024 16:11:38 -0800 Subject: [PATCH 1/9] Create readme.md --- .../continuous-delivery/readme.md | 217 ++++++++++++++++++ 1 file changed, 217 insertions(+) create mode 100644 docs/guides/software-lifecycle/continuous-delivery/readme.md diff --git a/docs/guides/software-lifecycle/continuous-delivery/readme.md b/docs/guides/software-lifecycle/continuous-delivery/readme.md new file mode 100644 index 000000000..9afaa346c --- /dev/null +++ b/docs/guides/software-lifecycle/continuous-delivery/readme.md @@ -0,0 +1,217 @@ +--- +title: Continuous Delivery +description: A comprehensive guide to continuous delivery in software development. +sidebar_label: Continuous Delivery +--- + +# Continuous Delivery + +
A comprehensive guide to developing a continuous delivery plan, implementation, and automation approach for your project.
+ +![continuous-delivery-representative-image](/img/continuous-delivery-image.png) + +## Introduction + +### Background + +Continuous Delivery (CD) is the practice of automatically building, testing, and preparing code changes for release to production. It extends Continuous Integration by ensuring that all code changes after passing automated tests are immediately ready for deployment. This guide provides a streamlined approach to implementing CD through standardized repository choices, naming conventions, and automation processes. + +### Approach + +Our recommendation is to follow a 4-step plan for continuous delivery: (1) select appropriate repositories, (2) implement standardized naming, (3) automate publishing, and (4) maintain delivery pipeline. + +```mermaid +graph TD + Repository[Select Repository] --> Naming[Implement Naming Convention] --> Automate[Automate Publishing] + Automate --> Maintain[Maintain Pipeline] + Maintain --> Repository +``` + +### Use Cases + +- Automatically publishing built artifacts to package managers +- Standardizing package naming across repositories +- Implementing versioned releases with semantic versioning +- Managing test data distribution +- Automating container image publishing +- Facilitating infrastructure as code deployment + +## Quick Start + +The most important step in setting up continuous delivery is choosing the right repositories and implementing proper naming conventions. + +**Key Concepts to Get Started:** + +**[⬇️ Choose a Package Repository](#package-repositories)** based on your artifact type: +- PyPI for Python packages +- Maven Central for Java +- NPM Registry for NodeJS +- ECR/DockerHub for Containers + +**📝 Implement [Standardized Naming Conventions](#naming-conventions):** +- `nasa-[project-org]-[module-name]` for Python +- `gov.nasa.[project-org].[module-name]` for Java +- `@nasa-[project-org]/[module-name]` for NodeJS + +**🚀 Set up [Automated Publishing](#automated-publishing)** using GitHub Actions + +## Step-by-Step Guide + +### 1. Select Package Repositories + +Choose appropriate repositories based on your artifact type: + +#### 1.1 Code Packages + +##### Python Packages +- **Repository**: PyPI +- **Size Limit**: 60MB +- **Cost**: Free +- **Best For**: Python libraries and tools +- **Setup Steps**: + 1. Create account on PyPI + 2. Set up project with `setup.py` or `pyproject.toml` + 3. Configure automated publishing + +##### Java Packages +- **Repository**: Maven Central +- **Size Limit**: No specific limit +- **Cost**: Free +- **Best For**: Java libraries and frameworks +- **Setup Steps**: + 1. Create Sonatype account + 2. Configure Maven settings + 3. Set up GPG signing + +##### NodeJS Packages +- **Repository**: NPM Registry +- **Size Limit**: No specific limit +- **Cost**: Free +- **Best For**: JavaScript/TypeScript packages +- **Setup Steps**: + 1. Create NPM account + 2. Configure package.json + 3. Set up automated publishing + +#### 1.2 Container Images + +##### Public Containers +- **Repository**: GitHub Packages/GitLab Registry +- **Best For**: Open source projects +- **Limitations**: Higher latency for runtime + +##### Private Containers +- **Repository**: Amazon ECR +- **Best For**: Production deployments +- **Features**: Low-latency pulls, private repos + +#### 1.3 Test Data + +##### Small Datasets (<2GB) +- **Repository**: GitHub/GitLab Releases +- **Naming**: `[project-org]-[project-module]-test-dataset` +- **Best For**: Unit test data, small samples + +##### Medium Datasets (2GB-100GB) +- **Repository**: Amazon S3 +- **Features**: Pre-signed URLs, bandwidth control +- **Best For**: Integration test data + +##### Large Datasets (>100GB) +- **Repository**: EOSDIS DAAC (Earth data) or PDS (Planetary data) +- **Best For**: Mission data, large-scale testing + +### 2. Implement Naming Conventions + +#### 2.1 Package Naming +Follow standard naming conventions for each repository type: + +```mermaid +graph TD + A[Package Type] --> B{Language?} + B -->|Python| C[nasa-project-module] + B -->|Java| D[gov.nasa.project.module] + B -->|NodeJS| E[@nasa-project/module] +``` + +#### 2.2 Version Naming +Use semantic versioning (MAJOR.MINOR.PATCH): +- MAJOR: Breaking changes +- MINOR: New features, backward compatible +- PATCH: Bug fixes + +### 3. Automate Publishing + +#### 3.1 GitHub Actions Workflow + +```yaml +name: Publish Package + +on: + release: + types: [published] + +jobs: + publish: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - name: Set up environment + uses: actions/setup-python@v3 + with: + python-version: '3.x' + - name: Build and publish + env: + TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }} + TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }} + run: | + python -m pip install build twine + python -m build + python -m twine upload dist/* +``` + +#### 3.2 Automated Testing Integration + +```mermaid +graph TD + A[Code Change] --> B[Run Tests] + B --> C{Tests Pass?} + C -->|Yes| D[Build Artifact] + C -->|No| E[Fix Issues] + D --> F[Publish to Repository] +``` + +### 4. Maintain Delivery Pipeline + +Regular maintenance tasks: +1. Update repository credentials +2. Monitor publishing success rates +3. Verify artifact integrity +4. Review and update workflows +5. Clean up old artifacts + +## Frequently Asked Questions (FAQ) + +**Q: How do I handle dependencies between packages?** +A: Use semantic versioning and dependency ranges to manage package relationships. + +**Q: What about handling sensitive data in artifacts?** +A: Use private repositories and encrypted secrets in CI/CD pipelines. + +**Q: How often should artifacts be published?** +A: Publish on every tagged release for stable versions, and optionally for development versions. + +**Q: How to manage large binary artifacts?** +A: Use specialized repositories like Amazon S3 for large artifacts and reference them in package metadata. + +## Credits + +**Authorship**: +- [Kyongsik Yun](https://github.com/yunks128) + +**Acknowledgements**: +- Thanks to the SLIM team for providing guidance + +## Feedback and Contributions + +We welcome feedback and contributions to help improve and grow this page. Please see our [contribution guidelines](https://nasa-ammos.github.io/slim/docs/contribute/contributing/). From 865a93616980c6eb7cee20bd278bd7eaec4bc3a4 Mon Sep 17 00:00:00 2001 From: Kyongsik Yun Date: Tue, 12 Nov 2024 16:13:45 -0800 Subject: [PATCH 2/9] Update readme.md --- docs/guides/software-lifecycle/continuous-delivery/readme.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/guides/software-lifecycle/continuous-delivery/readme.md b/docs/guides/software-lifecycle/continuous-delivery/readme.md index 9afaa346c..7ffb81d99 100644 --- a/docs/guides/software-lifecycle/continuous-delivery/readme.md +++ b/docs/guides/software-lifecycle/continuous-delivery/readme.md @@ -193,15 +193,19 @@ Regular maintenance tasks: ## Frequently Asked Questions (FAQ) **Q: How do I handle dependencies between packages?** + A: Use semantic versioning and dependency ranges to manage package relationships. **Q: What about handling sensitive data in artifacts?** + A: Use private repositories and encrypted secrets in CI/CD pipelines. **Q: How often should artifacts be published?** + A: Publish on every tagged release for stable versions, and optionally for development versions. **Q: How to manage large binary artifacts?** + A: Use specialized repositories like Amazon S3 for large artifacts and reference them in package metadata. ## Credits From c3e7aa3df1f8bf97aaafbbc65d4845d09a2b386c Mon Sep 17 00:00:00 2001 From: Kyongsik Yun Date: Tue, 12 Nov 2024 16:19:47 -0800 Subject: [PATCH 3/9] Update readme.md --- docs/guides/software-lifecycle/continuous-delivery/readme.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/software-lifecycle/continuous-delivery/readme.md b/docs/guides/software-lifecycle/continuous-delivery/readme.md index 7ffb81d99..adc7fc2a2 100644 --- a/docs/guides/software-lifecycle/continuous-delivery/readme.md +++ b/docs/guides/software-lifecycle/continuous-delivery/readme.md @@ -46,7 +46,7 @@ The most important step in setting up continuous delivery is choosing the right - PyPI for Python packages - Maven Central for Java - NPM Registry for NodeJS -- ECR/DockerHub for Containers +- ECR (Amazon Elastic Container Registry)/DockerHub for Containers **📝 Implement [Standardized Naming Conventions](#naming-conventions):** - `nasa-[project-org]-[module-name]` for Python From 9fc3032c9ed2caaab5d71adf3f354b1d07c4cd78 Mon Sep 17 00:00:00 2001 From: Kyongsik Yun Date: Thu, 12 Dec 2024 08:18:39 -0800 Subject: [PATCH 4/9] Update readme.md --- docs/guides/software-lifecycle/continuous-delivery/readme.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/software-lifecycle/continuous-delivery/readme.md b/docs/guides/software-lifecycle/continuous-delivery/readme.md index adc7fc2a2..bbfc5f069 100644 --- a/docs/guides/software-lifecycle/continuous-delivery/readme.md +++ b/docs/guides/software-lifecycle/continuous-delivery/readme.md @@ -22,7 +22,7 @@ Our recommendation is to follow a 4-step plan for continuous delivery: (1) selec ```mermaid graph TD - Repository[Select Repository] --> Naming[Implement Naming Convention] --> Automate[Automate Publishing] + Repository[Select Repository] --> Naming[Implement Naming
Convention] --> Automate[Automate Publishing] Automate --> Maintain[Maintain Pipeline] Maintain --> Repository ``` From 40a0d7e53440c46fd5c698941f29edf63fa3329e Mon Sep 17 00:00:00 2001 From: Kyongsik Yun Date: Sun, 5 Jan 2025 21:39:46 -0800 Subject: [PATCH 5/9] Update readme.md --- docs/guides/software-lifecycle/continuous-delivery/readme.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/guides/software-lifecycle/continuous-delivery/readme.md b/docs/guides/software-lifecycle/continuous-delivery/readme.md index bbfc5f069..83c03c8e3 100644 --- a/docs/guides/software-lifecycle/continuous-delivery/readme.md +++ b/docs/guides/software-lifecycle/continuous-delivery/readme.md @@ -8,7 +8,9 @@ sidebar_label: Continuous Delivery 
A comprehensive guide to developing a continuous delivery plan, implementation, and automation approach for your project.
+ ## Introduction From 417d6f7daa1496b9af301ae8698ffac310ea5bfe Mon Sep 17 00:00:00 2001 From: Kyongsik Yun Date: Thu, 9 Jan 2025 08:49:32 -0800 Subject: [PATCH 6/9] Update readme.md --- .../continuous-delivery/readme.md | 29 ++++++++++--------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/docs/guides/software-lifecycle/continuous-delivery/readme.md b/docs/guides/software-lifecycle/continuous-delivery/readme.md index 83c03c8e3..4b4a0916a 100644 --- a/docs/guides/software-lifecycle/continuous-delivery/readme.md +++ b/docs/guides/software-lifecycle/continuous-delivery/readme.md @@ -1,12 +1,11 @@ --- title: Continuous Delivery -description: A comprehensive guide to continuous delivery in software development. -sidebar_label: Continuous Delivery +description: A streamlined guide to setting up a robust continuous delivery pipeline, automating releases, and maintaining best practices. --- # Continuous Delivery -
A comprehensive guide to developing a continuous delivery plan, implementation, and automation approach for your project.
+
A streamlined guide to setting up a robust continuous delivery pipeline, automating releases, and maintaining best practices.
Naming[Implement Naming
Convention] --> Automate[Automate Publishing] + Repository[Choose Repositories] --> Naming[Adopt Naming
Conventions] --> Automate[Automate Publishing] Automate --> Maintain[Maintain Pipeline] Maintain --> Repository ``` -### Use Cases +### Key Use Cases -- Automatically publishing built artifacts to package managers -- Standardizing package naming across repositories -- Implementing versioned releases with semantic versioning -- Managing test data distribution -- Automating container image publishing -- Facilitating infrastructure as code deployment +- Auto-publishing built artifacts to package managers. +- Standardizing naming conventions across repositories. +- Versioning releases using semantic versioning. +- Distributing test data automatically. +- Automating container image publication. +- Enabling infrastructure-as-code deployment. ## Quick Start From 1a2cebfa6f4fb90e29c53a6c0d3cc0ec96190670 Mon Sep 17 00:00:00 2001 From: Kyongsik Yun Date: Thu, 9 Jan 2025 10:48:27 -0800 Subject: [PATCH 7/9] Update readme.md --- .../continuous-delivery/readme.md | 44 +++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/docs/guides/software-lifecycle/continuous-delivery/readme.md b/docs/guides/software-lifecycle/continuous-delivery/readme.md index 4b4a0916a..b1ab6f9fb 100644 --- a/docs/guides/software-lifecycle/continuous-delivery/readme.md +++ b/docs/guides/software-lifecycle/continuous-delivery/readme.md @@ -195,6 +195,50 @@ Regular maintenance tasks: 4. Review and update workflows 5. Clean up old artifacts +### 5. GitHub Actions Workflow Example for PyPI Project Continuous Delivery +Create a `.github/workflows/pypi-cd.yml` file in your GitHub repository with the following content: + +```yaml +name: Continuous Delivery for PyPI Project + +on: + push: + branches: + - main # Trigger on push to the 'main' branch + tags: + - 'v*.*.*' # Trigger on tags matching semantic versioning (v1.0.0) + +jobs: + # Job to set up the environment, install dependencies, and publish to PyPI + publish-to-pypi: + runs-on: ubuntu-latest + + steps: + - name: Checkout repository + uses: actions/checkout@v3 + + - name: Set up Python + uses: actions/setup-python@v3 + with: + python-version: '3.x' # Use a specific Python version, e.g., '3.8', '3.9', etc. + + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install build twine # Required for building and publishing to PyPI + + - name: Build the package + run: | + python -m build # This creates the distribution files under the 'dist' directory + + - name: Publish package to PyPI + env: + TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }} # Store PyPI credentials as GitHub secrets + TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }} + run: | + python -m twine upload dist/* # Uploads the package to PyPI +``` + ## Frequently Asked Questions (FAQ) **Q: How do I handle dependencies between packages?** From 22bf4e80fee80092b2bd7e72cb0dcab0835d50d4 Mon Sep 17 00:00:00 2001 From: Kyongsik Yun Date: Thu, 9 Jan 2025 12:40:00 -0800 Subject: [PATCH 8/9] Create pypi-cd-template.yml --- .../continuous-delivery/pypi-cd-template.yml | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100644 docs/guides/software-lifecycle/continuous-delivery/pypi-cd-template.yml diff --git a/docs/guides/software-lifecycle/continuous-delivery/pypi-cd-template.yml b/docs/guides/software-lifecycle/continuous-delivery/pypi-cd-template.yml new file mode 100644 index 000000000..cf497007d --- /dev/null +++ b/docs/guides/software-lifecycle/continuous-delivery/pypi-cd-template.yml @@ -0,0 +1,38 @@ +name: Continuous Delivery for PyPI Project + +on: + push: + branches: + - main # Trigger on push to the 'main' branch + tags: + - 'v*.*.*' # Trigger on tags matching semantic versioning (v1.0.0) + +jobs: + # Job to set up the environment, install dependencies, and publish to PyPI + publish-to-pypi: + runs-on: ubuntu-latest + + steps: + - name: Checkout repository + uses: actions/checkout@v3 + + - name: Set up Python + uses: actions/setup-python@v3 + with: + python-version: '3.x' # Use a specific Python version, e.g., '3.8', '3.9', etc. + + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install build twine # Required for building and publishing to PyPI + + - name: Build the package + run: | + python -m build # This creates the distribution files under the 'dist' directory + + - name: Publish package to PyPI + env: + TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }} # Store PyPI credentials as GitHub secrets + TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }} + run: | + python -m twine upload dist/* # Uploads the package to PyPI From c51ece910e5935306f42c2cbdb91515fe984e9cb Mon Sep 17 00:00:00 2001 From: Kyongsik Yun Date: Thu, 9 Jan 2025 12:41:22 -0800 Subject: [PATCH 9/9] Update readme.md --- docs/guides/software-lifecycle/continuous-delivery/readme.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/software-lifecycle/continuous-delivery/readme.md b/docs/guides/software-lifecycle/continuous-delivery/readme.md index b1ab6f9fb..d30689452 100644 --- a/docs/guides/software-lifecycle/continuous-delivery/readme.md +++ b/docs/guides/software-lifecycle/continuous-delivery/readme.md @@ -196,7 +196,7 @@ Regular maintenance tasks: 5. Clean up old artifacts ### 5. GitHub Actions Workflow Example for PyPI Project Continuous Delivery -Create a `.github/workflows/pypi-cd.yml` file in your GitHub repository with the following content: +Create a `.github/workflows/pypi-cd.yml` file in your GitHub repository with the following [content](pypi-cd-template.yml): ```yaml name: Continuous Delivery for PyPI Project