feat: Integrate Pixi for improved CI/CD and update project configuration #64
Conversation
📝 WalkthroughWalkthroughThe changes update repository configuration, CI workflows, and code formatting. The configuration files now include entries to treat certain generated files appropriately. Both release and test GitHub workflows have been refactored to replace Poetry commands with Pixi commands, update job names and checkout actions, and adjust steps for building and publishing distributions to PyPI. Minor code changes also refactor assertion statements into a concise, single-line format without altering functionality. Changes
Sequence Diagram(s)Release WorkflowsequenceDiagram
participant CI as CI System
participant Repo as Repository
participant Pixi as Pixi Tool
participant PyPI as PyPI Registry
CI->>Repo: Checkout repository (actions/checkout@v4)
CI->>Pixi: Install Pixi (prefix-dev/[email protected])
CI->>Pixi: Run build command ("pixi run --environment publish check-build")
CI->>PyPI: Publish distribution (pypa/gh-action-pypi-publish@release/v1)
Test WorkflowsequenceDiagram
participant CI as CI System
participant Repo as Repository
participant Pixi as Pixi Tool
%% Quality-Control Job
CI->>Repo: Checkout repository (actions/checkout@v4)
CI->>Pixi: Install Pixi
CI->>Pixi: Run formatting and linting commands ("pixi run" commands)
%% Testing Job
CI->>Repo: Checkout repository (actions/checkout@v4)
CI->>Pixi: Install Pixi
CI->>Pixi: Execute test suite ("pixi run" command)
Assessment against linked issues
Tip ⚡🧪 Multi-step agentic review comment chat (experimental)
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
.gitignore (1)
162-165: Review Pixi-Related Ignore EntriesThe addition of the new entries to ignore
.pixidirectories and*.egg-infofiles is appropriate for the move to Pixi. One point to verify is whether keeping apoetry.lockentry here is still necessary given the intended migration; if not, consider removing it or documenting its role.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
pixi.lockis excluded by!**/*.lockpyproject.tomlis excluded by!pyproject.toml
📒 Files selected for processing (6)
.gitattributes(1 hunks).github/workflows/release-please.yml(1 hunks).github/workflows/test.yml(1 hunks).gitignore(1 hunks)snakemake_interface_storage_plugins/storage_object.py(1 hunks)snakemake_interface_storage_plugins/tests.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
snakemake_interface_storage_plugins/storage_object.pysnakemake_interface_storage_plugins/tests.py
🔇 Additional comments (14)
.gitattributes (1)
1-2: Validate pixi.lock ConfigurationThe new entry for
pixi.lockis well configured—marking it as a binary file, assigning YAML syntax highlighting, and indicating that it is generated. This setup should help avoid merge conflicts and unexpected diffs in CI/CD contexts.snakemake_interface_storage_plugins/tests.py (1)
59-61: Refined Assertion StatementThe updated assertion for checking that
store_onlyandretrieve_onlyare not both set simultaneously is clear and concise. Ensure that the test suite includes cases to trigger this assertion when misconfiguration occurs.snakemake_interface_storage_plugins/storage_object.py (1)
95-97: Optimized Assertion in cache_key MethodThe reformatting of the assertion in the
cache_keymethod improves readability without altering functionality. The concise structure makes it easier to confirm that no cache key is generated when a custom local path is set..github/workflows/test.yml (5)
10-15: CI Permissions SetupThe newly added permissions for contents, checks, issues, and pull-requests ensure that the workflow has the appropriate access rights. This setup is clear and aligns well with the requirements for integrated CI/CD.
16-29: Streamlined Quality-Control Job with Pixi IntegrationThe
quality-controljob now uses checkout@v4 and installs Pixi with the specified version and environment. Switching from Poetry commands to Pixi for formatting and linting is well executed and simplifies the overall CI configuration.
30-43: Efficient Quality-Control StepsThe steps for running Ruff format and linter via Pixi (with the proper environment flag) are a good example of streamlining the CI process. This update removes redundancy and aligns with the pixi-driven workflow paradigm.
44-46: Concise Quality-Control ReportingThe final step that echoes a success message ("All quality control checks passed") provides a clear, simple indicator of job success without unnecessary complexity.
47-61: Updated Testing Job Using PixiThe testing job mirrors the quality-control job by installing Pixi and executing tests with the new Pixi command (
pixi run --environment dev test --show-capture=all -s -vv). This consistency helps maintain a unified CI/CD approach. Make sure that any dependencies or environment-specific settings needed by the tests are covered by the Pixi installation step..github/workflows/release-please.yml (6)
18-18: Update Package Name to Correct Typo
The package name has been corrected from the misspelled version "snakemake-interface-stroage-plugins" to "snakemake-interface-storage-plugins". This ensures that the downstream publishing process targets the correct package.
20-24: Introduce New Job "publish-pypi" with Conditional Execution
A new job namedpublish-pypihas been added. It is configured to run onubuntu-latest, depends on therelease-pleasejob, and conditionally executes only when a release is created. This aligns well with the updated CI/CD flow.
26-27: Upgrade Checkout Action to v4
The checkout step now usesactions/checkout@v4instead of v3. This update should provide improved performance and additional features. Ensure that this version is stable for your project’s workflows.
28-32: Integrate Pixi Installation Step
The new step installs Pixi usingprefix-dev/[email protected]with the parametersenvironments: publishandpixi-version: v0.42.1. This configuration correctly sets up the environment for subsequent build and check tasks in the CI/CD pipeline.
34-37: Refactor Build Step to Use Pixi Command
The build step has been updated to "Build source and wheel distribution + check build" by replacing the Poetry command withpixi run --environment publish check-build. This refactor aligns with the migration away from Poetry and streamlines the distribution build process.
39-43: Adopt PyPI Publish Action for Secure Distribution Release
The final step now employs thepypa/gh-action-pypi-publish@release/v1action to publish the distribution to PyPI. This method is secure and leverages the GitHub Action ecosystem to automate the publishing process.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
.github/workflows/test.yml (2)
38-42: Mypy Step Commented Out: ReminderThe mypy type-checking step is currently commented out. As noted in the PR description, this is temporary until the mypy issues are resolved in the interface-common repository. Remember to re-enable this step once type-check fixes are in place to maintain comprehensive CI coverage.
43-45: Collect QC Step: Optional Logging ConsiderationThe “Collect QC” step simply echoes a success message. While this can improve clarity in logs, consider whether this step is necessary if success is otherwise determined by the Pixi commands’ exit statuses. It might be redundant if logging isn’t critical.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
pyproject.tomlis excluded by!pyproject.toml
📒 Files selected for processing (4)
.github/workflows/test.yml(1 hunks).gitignore(1 hunks)snakemake_interface_storage_plugins/storage_object.py(1 hunks)snakemake_interface_storage_plugins/tests.py(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
- snakemake_interface_storage_plugins/tests.py
- snakemake_interface_storage_plugins/storage_object.py
- .gitignore
🔇 Additional comments (7)
.github/workflows/test.yml (7)
10-15: Permissions Block VerificationThe newly added permissions block grants explicit access (contents: read, checks: write, issues: write, pull-requests: write) for the workflow. Please verify that these permission levels meet the minimum required for the Pixi integration and do not exceed your project's security policies.
16-28: Quality-Control Job: Pixi SetupThe quality-control job now uses the updated checkout action (v4) and installs Pixi (via
prefix-dev/[email protected]) with the specified environment and version parameters. This setup appears consistent with your CI/CD modernization effort. Please confirm that the Pixi version (v0.42.1) and the environment setting align with your project requirements.
29-33: Ruff Format Step: Formatting CheckThe “Ruff Format” step now uses
pixi run --environment dev format --checkto validate formatting. The use ofif: always()ensures this check is executed regardless of earlier steps, which can be beneficial for continuous feedback. Ensure that the command wraps up the intended formatter settings.
34-38: Ruff Lint Step: Linting CheckThe “Ruff lint” step leverages Pixi to run linting with the
--diffflag, enabling a clear identification of code issues. Confirm that this diff output aligns with your review process and that the linting configuration meets your project’s standards.
46-50: Testing Job: Checkout ConsistencyIn the testing job, the checkout step uses actions/checkout@v4, matching the quality-control job’s version. This consistency is good for reproducibility. Just verify that the updated version does not introduce unexpected behaviors.
51-56: Testing Job: Pixi InstallationThe testing job again installs Pixi using the same action and parameters as in the quality-control job. This ensures consistency across CI tasks. Confirm that these settings satisfy both quality and testing requirements.
57-59: Run Tests Step: Command Flags VerificationThe “Run tests” step uses
pixi run --environment dev test --show-capture=all -s -vvto execute tests with comprehensive output. Check that these flags provide the necessary verbosity for debugging, and that they do not lead to overly verbose logs or performance overhead during CI runs.
similar to snakemake/snakemake-interface-common#58
closed #62 in favor of smaller, more focussed PRs to ensure no breaking changes
This PR: Setup pixi and workspace for now.
Future PRs: Iterate over type issues after the the interface-common mypy issues are addressed
type-checkstep in the GHA will be uncommented once the type errors are fixedNote: the two modified python files are just formatting
Summary by CodeRabbit