Update dry run for atmos vendor pull to support ssh + detailed SCP urls alignment #1076
Conversation
Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]>
Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]>
Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]>
Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]>
Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]>
Listener430
changed the title
Listener430
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
… only in dry mode
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (2)
9-18: Basic HTTPS Source Configuration Review
The first source entry clearly documents a basic HTTPS URL (with token injection expected), and the structure is correct. Note that line 18 has trailing spaces that should be removed to satisfy YAML lint rules.- # Basic HTTPS default (token injection expected)␣␣ + # Basic HTTPS default (token injection expected)🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 18-18: trailing spaces
(trailing-spaces)
19-27: Direct Credentials URL Configuration Caution
The second source entry uses direct credentials in the URL. While this may be intentional for testing or specific scenarios, please verify that exposing credentials in configuration is acceptable for your environment.tests/test-cases/demo-vendoring.yaml (2)
61-79: Environment Variable Formatting Consistency
The "atmos vendor pull ssh component" test case is sound in its intent. However, notice that theATMOS_GITHUB_TOKENvalue is provided without quotes here while quoted in the SSH test case. For maintainability and consistency, consider using the same quoting style for environment variables across test cases.
96-96: Remove Trailing Whitespace
Static analysis has detected trailing spaces on these lines. Removing these extraneous spaces helps maintain clean YAML formatting.- - "Injecting token" + - "Injecting token" - - "!supersecret" + - "!supersecret"Also applies to: 98-98
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 96-96: trailing spaces
(trailing-spaces)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
internal/exec/vendor_utils.go(1 hunks)tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_custom_detector_credentials_leakage.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden(1 hunks)tests/test-cases/demo-vendoring.yaml(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- internal/exec/vendor_utils.go
🚧 Files skipped from review as they are similar to previous changes (2)
- tests/snapshots/TestCLICommands_atmos_vendor_pull_custom_detector_credentials_leakage.stderr.golden
- tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden
🧰 Additional context used
🪛 YAMLlint (1.35.1)
tests/test-cases/demo-vendoring.yaml
[error] 96-96: trailing spaces
(trailing-spaces)
[error] 98-98: trailing spaces
(trailing-spaces)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml
[error] 18-18: trailing spaces
(trailing-spaces)
⏰ Context from checks skipped due to timeout of 90000ms (4)
- GitHub Check: [localstack] demo-localstack
- GitHub Check: Acceptance Tests (macos-latest, macos)
- GitHub Check: Acceptance Tests (windows-latest, windows)
- GitHub Check: Summary
🔇 Additional comments (4)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (2)
1-8: File Header Configuration Looks Good
The top-level keys such asapiVersion,kind, andmetadata(withnameanddescription) are clearly defined and follow the expected schema for an Atmos vendoring manifest.
28-36: SSH Implicit Source Configuration Validated
The third source entry correctly represents an SSH implicit method, aligning with the enhanced SCP and SSH URL support highlighted in the PR objectives. The structure is consistent with the other components, ensuring that the custom detectors in the codebase can correctly process these URLs.tests/test-cases/demo-vendoring.yaml (2)
43-60: SSH Dry-Run Test Case Validation
This test case correctly exercises the dry-run mode for vendor pulls using an SSH-style URL. The use of detailed logging (via--logs-level=Debug) and the dry-run flag ensures that no actual file downloads occur. The work directory and environment variable setup align with the PR objectives.
80-101: Credential Masking Verification
This test case does an excellent job ensuring that injected credentials do not appear in log outputs by checking for a masked output (e.g., "!supersecret") and an indicative log message ("Injecting token"). The expected stdout and stderr values properly cover the security requirement regarding credential leakage.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 96-96: trailing spaces
(trailing-spaces)
[error] 98-98: trailing spaces
(trailing-spaces)
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (13)
tests/fixtures/scenarios/vendor-creds-sanitize/atmos.yaml (1)
13-13: Remove trailing whitespace.There's trailing whitespace on this line that should be removed.
- +🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 13-13: trailing spaces
(trailing-spaces)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (1)
1-25: Minor typo in comment.There's a small typo in line 17's comment.
- ## Explicit ssh vednoring (the schema is explicitly spcified along with a username, no custom detector is invoked) + ## Explicit ssh vendoring (the schema is explicitly specified along with a username, no custom detector is invoked)tests/fixtures/scenarios/vendoring-dry-run/atmos.yaml (1)
1-23: Fix YAML formatting issues.There are a few formatting issues in this YAML file that should be addressed.
- base_path: "./" settings: inject_github_token: true components: terraform: base_path: "components/terraform" apply_auto_approve: false deploy_run_init: true init_run_reconfigure: true auto_generate_backend_file: false - + stacks: base_path: "stacks" included_paths: - "deploy/**/*" excluded_paths: - "**/_defaults.yaml" name_pattern: "{stage}" - - +🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
[error] 13-13: trailing spaces
(trailing-spaces)
[warning] 22-22: too many blank lines
(2 > 0) (empty-lines)
tests/fixtures/scenarios/vendor-pulls-ssh/atmos.yaml (1)
1-20: Add newline at end of file.The YAML file is missing a newline character at the end.
logs: file: "/dev/stderr" level: Info +🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 20-20: no new line character at the end of file
(new-line-at-end-of-file)
tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml (2)
18-18: Fix trailing whitespaceRemove the trailing spaces at the end of this line.
- # Direct credentials provided in the URL (token injection should be skipped) + # Direct credentials provided in the URL (token injection should be skipped)🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 18-18: trailing spaces
(trailing-spaces)
28-35: Add newline at end of fileAdd a newline character at the end of the file to comply with YAML standards.
tags: - demo +🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 35-35: no new line character at the end of file
(new-line-at-end-of-file)
website/docs/cli/commands/vendor/vendor-pull.mdx (1)
216-218: Consider refining the wording and punctuation.You might strengthen the verb choice in "When resolving Git sources, Atmos follows these rules" by saying “When processing Git sources, Atmos follows these rules.” Also, using a typographical ellipsis character “…” rather than “...” can be more stylistically consistent.
🧰 Tools
🪛 LanguageTool
[style] ~216-~216: Consider using a different verb to strengthen your wording.
Context: ...lved When resolving Git sources, Atmos follows these rules: 1. If a *full HTTPS URL...(FOLLOW_OBEY)
[style] ~218-~218: Consider using the typographical ellipsis character here instead.
Context: ...1. If a full HTTPS URL is provided (git::https://github.com/...), it is used as-is. No token data is i...(ELLIPSIS)
tests/test-cases/vendoring-ssh-dryrun.yaml (1)
57-59: Remove trailing spaces and add a newline at EOF.YAMLLint flagged trailing spaces on line 57 and requires a newline at the end of the file on line 59. Please ensure both are fixed:
- !not 'supersecret' - - !not 'ATMOS_GITHUB_TOKEN' \ No newline at end of file + - !not 'ATMOS_GITHUB_TOKEN' +🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 57-57: trailing spaces
(trailing-spaces)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
website/docs/cli/configuration/configuration.mdx (2)
681-683: Minor grammar fixes.
- Lowercase "Requests" for consistency in “Unauthenticated requests”.
- Possibly clarify “avoided” vs. “used” regarding rate limits.
- | ATMOS_BITBUCKET_TOKEN | N/A | App password for Bitbucket API requests is set to avoid rate limits. Unauthenticated Requests are limited to 60 requests per hour across all API resources. | + | ATMOS_BITBUCKET_TOKEN | N/A | App password for Bitbucket API requests, helping avoid rate limits. Unauthenticated requests are limited to 60 requests per hour across all API resources. |
699-699: Typographical correction.Fix the spelling from “comamnds” to “commands”:
- | ATMOS_TERRAFORM_WORKSPACE | The name of the Terraform workspace in which Terraform comamnds should be run | + | ATMOS_TERRAFORM_WORKSPACE | The name of the Terraform workspace in which Terraform commands should be run |internal/exec/go_getter_utils_test.go (3)
118-137: Token injection test could be more robust.While the test correctly verifies the basic token injection, consider adding cases for empty tokens or different host scenarios to ensure comprehensive coverage.
func TestInjectToken(t *testing.T) { os.Setenv("GITHUB_TOKEN", "testtoken") defer os.Unsetenv("GITHUB_TOKEN") config := fakeAtmosConfig(true) detector := &CustomGitDetector{AtmosConfig: &config} uObj, err := url.Parse("https://github.com/user/repo.git") if err != nil { t.Fatalf("Failed to parse URL: %v", err) } detector.injectToken(uObj, hostGitHub) if uObj.User == nil { t.Error("Expected token to be injected into URL") } else { user := uObj.User.Username() if user != getDefaultUsername(hostGitHub) { t.Errorf("Expected username %s, got %s", getDefaultUsername(hostGitHub), user) } } + + // Test with empty token + os.Unsetenv("GITHUB_TOKEN") + uObj2, _ := url.Parse("https://github.com/user/repo.git") + detector.injectToken(uObj2, hostGitHub) + if uObj2.User != nil { + t.Error("Expected no token injection with empty token") + } + + // Test with non-GitHub host + os.Setenv("GITLAB_TOKEN", "gltoken") + defer os.Unsetenv("GITLAB_TOKEN") + uObj3, _ := url.Parse("https://gitlab.com/user/repo.git") + detector.injectToken(uObj3, hostGitLab) + if uObj3.User == nil { + t.Error("Expected token to be injected for GitLab URL") + } else if uObj3.User.Username() != getDefaultUsername(hostGitLab) { + t.Errorf("Expected username %s, got %s", getDefaultUsername(hostGitLab), uObj3.User.Username()) + } }
300-310: Consider consolidating redundant URI validation tests.These test cases duplicate checks already present in the main TestValidateURI function (lines 37-42). Consider consolidating into a single comprehensive test to improve maintainability.
312-319: Consider consolidating redundant SCP URL tests.This test duplicates functionality already tested in TestRewriteSCPURL (lines 98-102). Consolidating these would make the test suite more maintainable.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (25)
.github/workflows/test.yml(3 hunks).golangci.yml(1 hunks)codecov.yml(1 hunks)internal/exec/go_getter_utils.go(4 hunks)internal/exec/go_getter_utils_test.go(1 hunks)internal/exec/vendor_model.go(2 hunks)internal/exec/vendor_utils.go(2 hunks)internal/exec/yaml_func_include.go(1 hunks)pkg/utils/url_utils.go(1 hunks)pkg/utils/url_utils_test.go(1 hunks)tests/fixtures/scenarios/vendor-creds-sanitize/atmos.yaml(1 hunks)tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml(1 hunks)tests/fixtures/scenarios/vendor-pulls-ssh/atmos.yaml(1 hunks)tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml(1 hunks)tests/fixtures/scenarios/vendoring-dry-run/atmos.yaml(1 hunks)tests/fixtures/scenarios/vendoring-dry-run/components/terraform/ipinfo/component.yaml(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_component_using_SSH.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_custom_detector_credentials_leakage.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh_component.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_using_SSH.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_with_custom_detector_and_handling_credentials_leakage.stderr.golden(1 hunks)tests/test-cases/vendoring-ssh-dryrun.yaml(1 hunks)website/docs/cli/commands/vendor/vendor-pull.mdx(2 hunks)website/docs/cli/configuration/configuration.mdx(2 hunks)
🧰 Additional context used
🧠 Learnings (6)
internal/exec/vendor_model.go (1)
Learnt from: samtholiya
PR: cloudposse/atmos#1077
File: pkg/downloader/custom_github_detector.go:0-0
Timestamp: 2025-02-21T20:56:05.539Z
Learning: The `github.com/charmbracelet/log` package should be imported with the alias `log` according to the project's import alias configuration.
tests/snapshots/TestCLICommands_atmos_vendor_pull_component_using_SSH.stderr.golden (1)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:8-8
Timestamp: 2025-03-12T18:52:22.003Z
Learning: Test snapshots in the Atmos project, particularly for dry run scenarios, may be updated during the development process, and temporary inconsistencies in their content should not be flagged as issues.
tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden (1)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:8-8
Timestamp: 2025-03-12T18:52:22.003Z
Learning: Test snapshots in the Atmos project, particularly for dry run scenarios, may be updated during the development process, and temporary inconsistencies in their content should not be flagged as issues.
tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh_component.stderr.golden (1)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:8-8
Timestamp: 2025-03-12T18:52:22.003Z
Learning: Test snapshots in the Atmos project, particularly for dry run scenarios, may be updated during the development process, and temporary inconsistencies in their content should not be flagged as issues.
tests/snapshots/TestCLICommands_atmos_vendor_pull_using_SSH.stderr.golden (1)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:8-8
Timestamp: 2025-03-12T18:52:22.003Z
Learning: Test snapshots in the Atmos project, particularly for dry run scenarios, may be updated during the development process, and temporary inconsistencies in their content should not be flagged as issues.
internal/exec/go_getter_utils.go (2)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: internal/exec/go_getter_utils.go:74-75
Timestamp: 2025-03-12T18:52:31.814Z
Learning: In the `CustomGitDetector.Detect` method of `internal/exec/go_getter_utils.go`, verbose debug logging of raw URLs is intentionally kept for debugging purposes, despite potential credential exposure risks.
Learnt from: samtholiya
PR: cloudposse/atmos#1077
File: pkg/downloader/custom_github_detector.go:0-0
Timestamp: 2025-02-21T20:56:05.539Z
Learning: The `github.com/charmbracelet/log` package should be imported with the alias `log` according to the project's import alias configuration.
🪛 YAMLlint (1.35.1)
tests/fixtures/scenarios/vendor-creds-sanitize/atmos.yaml
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
[error] 13-13: trailing spaces
(trailing-spaces)
tests/fixtures/scenarios/vendor-pulls-ssh/atmos.yaml
[error] 20-20: no new line character at the end of file
(new-line-at-end-of-file)
tests/fixtures/scenarios/vendoring-dry-run/atmos.yaml
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
[error] 13-13: trailing spaces
(trailing-spaces)
[warning] 22-22: too many blank lines
(2 > 0) (empty-lines)
tests/test-cases/vendoring-ssh-dryrun.yaml
[error] 57-57: trailing spaces
(trailing-spaces)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml
[error] 18-18: trailing spaces
(trailing-spaces)
[error] 35-35: no new line character at the end of file
(new-line-at-end-of-file)
🪛 LanguageTool
website/docs/cli/commands/vendor/vendor-pull.mdx
[style] ~216-~216: Consider using a different verb to strengthen your wording.
Context: ...lved When resolving Git sources, Atmos follows these rules: 1. If a *full HTTPS URL...
(FOLLOW_OBEY)
[style] ~218-~218: Consider using the typographical ellipsis character here instead.
Context: ...1. If a full HTTPS URL is provided (git::https://github.com/...), it is used as-is. No token data is i...
(ELLIPSIS)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Summary
🔇 Additional comments (42)
.golangci.yml (1)
173-173: Increased nestif complexity thresholdRaising the min-complexity threshold from 4 to 6 will reduce noise in linting output by only flagging more complex nested if statements.
codecov.yml (1)
16-16: Improved boolean type representationChanged from string representation
yesto proper boolean valuetrue. This is a good practice for YAML configuration files.internal/exec/yaml_func_include.go (1)
50-50: Changed parameter passing to use pointerUpdated to pass
atmosConfigby reference instead of by value, which aligns with the function signature ingo_getter_utils.gothat expects a pointer parameter.tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden (1)
1-10: New test snapshot for SSH vendor pull dry runThe snapshot captures the expected CLI output for the vendor pull command when using SSH, including debug logs, warnings, and status messages. This supports the PR objective of enhancing dry run output with more detailed information.
I note this snapshot shows proper sanitized output including detailed information about vendored components without downloading files, which directly addresses the objectives of this PR.
internal/exec/vendor_model.go (2)
15-15: Good job fixing the import alias.The
charmbracelet/logpackage is now correctly imported with thelogalias, following the project's import alias configuration.
337-337: Improved debugging visibility.Adding this debug log provides better visibility into the package installation process, which is valuable for troubleshooting and aligns with the PR's goal to enhance output information.
tests/fixtures/scenarios/vendor-creds-sanitize/atmos.yaml (1)
1-21: Good test configuration for credential sanitization.This configuration file properly sets up a test scenario with GitHub token injection enabled, which will be useful for testing the sanitization features in the dry run output.
🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
[error] 13-13: trailing spaces
(trailing-spaces)
tests/snapshots/TestCLICommands_atmos_vendor_pull_component_using_SSH.stderr.golden (1)
1-11: Good test snapshot for SSH vendoring.This snapshot correctly captures the expected CLI output for the SSH vendoring dry run scenario, showing proper debug messages, warnings, and confirmation of the dry run completion.
pkg/utils/url_utils.go (2)
8-8: Good constant definition for masking secrets.Using a named constant for the masked value follows good practice and addresses the previous feedback to move the string to a constant.
10-26: Well-implemented URL sanitization function.The
MaskBasicAuthfunction properly handles URL parsing, credential detection, and masking. It correctly differentiates between URLs with username-only and username-password combinations.internal/exec/vendor_utils.go (2)
22-23: LGTM - Adding dedicated stderr logger helps separate detailed messaging.The new StderrLogger ensures that detailed messaging, specifically for files vendoring, doesn't pollute stdout. This aligns well with improving the dry run output clarity.
514-514: Clean implementation of stderr logging.Good replacement of standard log with the dedicated StderrLogger for including paths. This ensures detailed file inclusion messages go to stderr, keeping stdout clean as intended.
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (2)
9-16: Good implementation of implicit SSH vendoring.The implicit SSH vendoring configuration correctly uses SCP-style Git URL format (
[email protected]:cloudposse/terraform-null-label.git). This addresses the need for SCP URL alignment in the dry run output as mentioned in the PR objectives.
17-24: Explicit SSH vendoring looks correct.The explicit SSH vendoring configuration uses the proper schema (
ssh://) with a username. This provides a good comparison case to the implicit SSH vendoring and ensures comprehensive testing of SSH support.tests/fixtures/scenarios/vendoring-dry-run/atmos.yaml (1)
4-4: Good addition of GitHub token injection setting.The
inject_github_token: truesetting is important for the vendoring functionality being tested, especially for dry run output that needs to show sanitized credentials.tests/fixtures/scenarios/vendor-pulls-ssh/atmos.yaml (1)
18-20: Appropriate log configuration for testing.Setting logs to stderr with Info level is a good choice for this scenario. It ensures that logs don't interfere with test outputs while maintaining visibility of important information during tests.
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 20-20: no new line character at the end of file
(new-line-at-end-of-file)
tests/snapshots/TestCLICommands_atmos_vendor_pull_using_SSH.stderr.golden (1)
11-13: Dry run output structure looks goodThe output clearly shows what would happen in actual execution (vendoring 2 components) after confirming no actual components were vendored during dry run. This separation helps users understand the expected actions without making changes to the filesystem.
tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml (1)
9-35: Well-structured test scenarios for credential handlingGood job defining three distinct scenarios for credential handling:
- Default HTTPS with token injection
- Direct credentials in URL (skipping token injection)
- Pre-existing credentials (skipping token injection)
This comprehensive test file will effectively validate the credential sanitization features mentioned in the PR objectives.
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 18-18: trailing spaces
(trailing-spaces)
[error] 35-35: no new line character at the end of file
(new-line-at-end-of-file)
tests/snapshots/TestCLICommands_atmos_vendor_pull_with_custom_detector_and_handling_credentials_leakage.stderr.golden (1)
8-12: Excellent credential handling and loggingThe debug logs provide detailed visibility into the URL transformation process while properly masking sensitive credentials ("xxx:xxx"). This achieves the PR objective of showing detailed information about injected tokens and URL transformations without exposing sensitive data.
tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh_component.stderr.golden (2)
8-12: Successfully implements SCP-style SSH URL rewritingThe logs clearly demonstrate the key feature of this PR - rewriting SCP-style SSH URLs (github.com:cloudposse/atmos.git) to standard SSH URLs (ssh://github.com/cloudposse/atmos.git). The credential injection and masking are also correctly handled, fulfilling the PR's objective of supporting SSH in vendor pull commands.
14-16: Dry run output structure is clearSimilar to the first file, the output effectively separates what actually happened (dry run with no actual vendoring) from what would happen in a real execution (vendoring 1 component). This structure helps users understand the expected actions without making filesystem changes.
.github/workflows/test.yml (3)
49-50: Added full Git history checkout to ensure complete repository access.The change to
fetch-depth: 0ensures the entire Git history is available during workflow execution, which is essential for comprehensive Git-related operations like the vendor pull command with SSH support.
88-88: Increased timeout to allow for more complex Git operations.The timeout increase from 15 to 20 minutes provides adequate time for the enhanced vendor pull functionality to complete successfully, particularly with SSH and URL processing.
159-159: Extended acceptance test timeout to accommodate new vendor pull functionality.Increasing the test timeout to 20 minutes ensures tests have sufficient time to execute, especially when testing various Git URL formats and SSH handling.
tests/snapshots/TestCLICommands_atmos_vendor_pull_custom_detector_credentials_leakage.stderr.golden (3)
7-11: Properly masks credentials in Git URLs during dry run.The logs correctly show the custom Git detector in action with appropriate credential masking. The URL transformation process is well-documented in the logs:
- Detection of Git URL
- Default scheme application
- Token injection
- Credential masking (showing "xxx:xxx" instead of actual tokens)
This is exactly what we want to see for secure credential handling in dry run mode.
32-32: Log entry incorrectly identifies directory as file.The debug message refers to "docs" as a file, but it's actually a directory.
87-92: Clear success indicators for vendored components.The output properly indicates successful vendoring of each component with version information, making it easy to verify the operation's results.
pkg/utils/url_utils_test.go (1)
1-49: Comprehensive test coverage for URL credential masking.The tests thoroughly cover all important scenarios for the MaskBasicAuth function:
- URLs with both username and password
- URLs with username only
- URLs without credentials
- Invalid URL handling
Each test verifies the expected output and proper error handling, ensuring the URL masking functionality is reliable.
tests/fixtures/scenarios/vendoring-dry-run/components/terraform/ipinfo/component.yaml (2)
7-9: SSH URL format correctly implemented for Git source.The configuration properly uses the SSH format (
[email protected]) for the Git URI, which aligns with the PR objective of supporting SSH URLs in the vendor pull command.
10-13: Appropriate file inclusion patterns for Terraform components.The included paths cover all essential Terraform files (.tf), variable files (.tfvars), and documentation (*.md), ensuring that all necessary component files are vendored.
website/docs/cli/commands/vendor/vendor-pull.mdx (2)
111-120: Good addition for SSH vendoring!The new "Vendoring from SSH" section is clear and helps users understand how SSH-based repository access is configured.
150-156: Clarity improvement acknowledged.The explanation about invalid URLs and Atmos defaulting to HTTPS is straightforward and addresses common user pitfalls.
internal/exec/go_getter_utils.go (3)
66-70: Struct definition looks good.This new
CustomGitDetectorneatly encapsulates the Atmos config reference for advanced Git URL handling.
74-124: Useful custom detection logic.The rewriting of SCP-style URLs and the fallback to
https://help unify various URL forms. Masking the source URL in debug logs further protects credentials.
345-359: Symlink removal approach is straightforward.Recursively walking the downloaded directory and removing symlinks reduces security risks. This appears correct for ensuring only genuine files remain after the clone.
internal/exec/go_getter_utils_test.go (7)
30-52: Great job with comprehensive URI validation tests.The test cases cover all essential validation paths: empty URIs, excessively long URIs, path traversal sequences, spaces, and various valid URI formats including OCI URIs. This thorough approach will help catch validation issues early.
55-65: Well-structured scheme validation tests.You've properly tested both valid schemes (HTTP, HTTPS, Git, SSH, Git+HTTPS, Git+SSH) and invalid schemes (FTP). This ensures the function handles all expected inputs correctly.
68-86: Good coverage of URL scheme enforcement scenarios.This test effectively verifies three key behaviors:
- Preserving existing schemes
- Rewriting SCP-style URLs to use SSH
- Defaulting to HTTPS when no scheme is present
These align well with the PR objective to support SSH and handle SCP URL alignment.
89-103: Solid test for SCP URL rewriting.The test correctly verifies both positive and negative cases for SCP URL rewriting, which is central to the PR's goals of supporting SSH and SCP URL alignment.
194-221: Good platform-aware symlink testing.The test correctly skips symlink tests on Windows where symlink behavior differs, creates a proper test environment, and verifies both the removal of symlinks and preservation of regular files.
223-258: Comprehensive file download test with proper cleanup.This test thoroughly validates the file download functionality by creating temporary directories, writing test content, performing the download, and verifying the result. The Windows skip and proper cleanup with defer are good practices.
338-342: Good test cleanup practice.Properly restoring original detectors in TestMain ensures tests don't interfere with each other or leave the environment in an altered state.
| @@ -13,7 +13,7 @@ comment: | |||
| layout: "reach,diff,flags,tree" # Display different coverage views | |||
| behavior: default # Default PR comment behavior | |||
| require_changes: true # Only post if coverage changes | |||
There was a problem hiding this comment.
| require_changes: true # Only post if coverage changes | |
| require_changes: true # Only post if coverage changes |
| @@ -13,7 +13,7 @@ comment: | |||
| layout: "reach,diff,flags,tree" # Display different coverage views | |||
| behavior: default # Default PR comment behavior | |||
| require_changes: true # Only post if coverage changes | |||
| require_base: yes # Compare against base branch coverage | |||
| require_base: true # Compare against base branch coverage | |||
There was a problem hiding this comment.
| require_base: true # Compare against base branch coverage | |
| require_base: true # Compare against base branch coverage |
| @@ -19,6 +19,9 @@ import ( | |||
| u "github.com/cloudposse/atmos/pkg/utils" | |||
| ) | |||
|
|
|||
| // Dedicated logger for stderr to keep stdout clean of detailed messaging, e.g. for files vendoring. | |||
| var StderrLogger = log.New(os.Stderr) | |||
There was a problem hiding this comment.
We do not need to introduce a new logger. We use log.* everywhere now without this.
| @@ -508,7 +511,7 @@ func generateSkipFunction(tempDir string, s *schema.AtmosVendorSource) func(os.F | |||
| } | |||
|
|
|||
| // If 'included_paths' is not provided, include all files that were not excluded | |||
| log.Debug("Including", u.TrimBasePathFromPath(tempDir+"/", src)) | |||
| StderrLogger.Debug("Including", "path", u.TrimBasePathFromPath(tempDir+"/", src)) | |||
There was a problem hiding this comment.
| StderrLogger.Debug("Including", "path", u.TrimBasePathFromPath(tempDir+"/", src)) | |
| log.Debug("Including", "path", u.TrimBasePathFromPath(tempDir+"/", src)) |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Listener430
force-pushed
the
DEV-2964_vendoring-dry-run
branch
from
9dd664e to
f222283
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
📝 WalkthroughWalkthroughThis pull request updates several configurations and code modules. The GitHub Actions workflow now includes a deeper checkout and extended timeouts, while linter and code coverage configurations have been refined. Internal logging has been adjusted with a new dedicated logger, and a new URL utility function ( Changes
Sequence Diagram(s)sequenceDiagram
participant Caller as Function Caller
participant MaskUtil as MaskBasicAuth
participant Parser as URL Parser
Caller->>MaskUtil: Call MaskBasicAuth(rawURL)
MaskUtil->>Parser: Parse raw URL
Parser-->>MaskUtil: Parsed URL or error
MaskUtil-->>Caller: Return masked URL (credentials replaced) or error
Assessment against linked issues
Possibly related PRs
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (8)
.golangci.yml (1)
173-173: Increased nestif complexity thresholdThe minimum complexity for the nestif linter has been increased to 6, which will cause fewer complex nested if statements to be flagged.
This change should be evaluated against the team's code quality standards. Consider documenting this decision in a comment or commit message to explain why the threshold was raised.
tests/fixtures/scenarios/vendoring-dry-run/atmos.yaml (1)
1-23: Configuration looks good with minor formatting issuesThe configuration follows a consistent pattern with other similar files. The structure is well-organized with clear sections for base path, settings, components, and stacks.
Consider fixing the following formatting issues:
- Remove extra blank lines at start and end
- Remove trailing spaces on line 13
- base_path: "./" settings: inject_github_token: true components: terraform: base_path: "components/terraform" apply_auto_approve: false deploy_run_init: true init_run_reconfigure: true auto_generate_backend_file: false - + stacks: base_path: "stacks" included_paths: - "deploy/**/*" excluded_paths: - "**/_defaults.yaml" name_pattern: "{stage}" -🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
[error] 13-13: trailing spaces
(trailing-spaces)
[warning] 22-22: too many blank lines
(2 > 0) (empty-lines)
tests/fixtures/scenarios/vendor-creds-sanitize/atmos.yaml (1)
1-21: Configuration looks good with minor formatting issuesThe configuration structure follows a consistent pattern with other files, containing the necessary sections for deployments.
Consider fixing the following formatting issues:
- Remove extra blank line at the start
- Remove trailing spaces on line 13
- base_path: "./" settings: inject_github_token: true components: terraform: base_path: "components/terraform" apply_auto_approve: false deploy_run_init: true init_run_reconfigure: true auto_generate_backend_file: false - + stacks: base_path: "stacks" included_paths: - "deploy/**/*" excluded_paths: - "**/_defaults.yaml" name_pattern: "{stage}"🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
[error] 13-13: trailing spaces
(trailing-spaces)
tests/fixtures/scenarios/vendor-pulls-ssh/atmos.yaml (1)
1-20: Configuration looks good but needs a newline at end of fileThe configuration is well-structured and includes additional logging settings. This is the only file of the three with logs configuration, which makes sense for SSH vendoring scenarios.
Add a newline at the end of the file:
logs: file: "/dev/stderr" level: Info +🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 20-20: no new line character at the end of file
(new-line-at-end-of-file)
tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml (1)
1-25: Clear SSH Vendoring Configuration with Minor Typos
This configuration file nicely illustrates two SSH vendoring scenarios:
• An implicit SSH vendoring case where the custom detector parses an SCP-style URL.
• An explicit case where the SSH scheme is clearly provided, bypassing the custom detection.Nitpick: In the comments on lines 9–17, there are minor typos (“vednoring” should be “vendoring” and “spcified” should be “specified”). While they don’t affect functionality, cleaning these up would improve clarity.
tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml (1)
1-35: Comprehensive HTTPS Vendoring Setup with Formatting Suggestions
The YAML file effectively covers three distinct scenarios:
• Basic HTTPS with token injection expected.
• Direct credentials embedded in the URL (where token injection is skipped).
• HTTPS with pre-existing credentials (again, without token injection).The structure is easy to follow and aligns with the enhanced dry run outputs.
Formatting Note: There are trailing spaces detected (e.g., on line 18) and the file is missing a newline at the end. It would be beneficial to remove these trailing spaces and ensure the file ends with a newline.A possible diff for cleanup:
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 18-18: trailing spaces
(trailing-spaces)
[error] 35-35: no new line character at the end of file
(new-line-at-end-of-file)
website/docs/cli/commands/vendor/vendor-pull.mdx (1)
218-218: Use proper ellipsis character.In the example URL, replace the three dots with a proper ellipsis character for typographical correctness.
-If a **full HTTPS URL** is provided (`git::https://github.com/...`), it is used as-is. No token data is injected, even if environment variables are set. +If a **full HTTPS URL** is provided (`git::https://github.com/…`), it is used as-is. No token data is injected, even if environment variables are set.🧰 Tools
🪛 LanguageTool
[style] ~218-~218: Consider using the typographical ellipsis character here instead.
Context: ...1. If a full HTTPS URL is provided (git::https://github.com/...), it is used as-is. No token data is i...(ELLIPSIS)
tests/test-cases/vendoring-ssh-dryrun.yaml (1)
41-59: Test Case: Handling Credentials LeakageThis test ensures that sensitive credentials (e.g.,
ATMOS_GITHUB_TOKEN) are not leaked in the logs by using YAML’s!nottag. This is a neat method for verifying log sanitization.A couple of minor refinements are needed based on static analysis hints:
- Trailing Spaces (Line 57): There are trailing spaces that should be removed.
- Newline at EOF (Line 59): Ensure the file ends with a newline.
Below is a suggested diff for these fixes:
- - !not 'supersecret' + - !not 'supersecret'- exit_code: 0 +[...] + exit_code: 0 +Please remove the trailing spaces and add a newline at the end to comply with YAML standards.
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 57-57: trailing spaces
(trailing-spaces)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (22)
.github/workflows/test.yml(3 hunks).golangci.yml(1 hunks)codecov.yml(1 hunks)internal/exec/vendor_model.go(2 hunks)internal/exec/vendor_utils.go(2 hunks)pkg/utils/url_utils.go(1 hunks)pkg/utils/url_utils_test.go(1 hunks)tests/fixtures/scenarios/vendor-creds-sanitize/atmos.yaml(1 hunks)tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml(1 hunks)tests/fixtures/scenarios/vendor-pulls-ssh/atmos.yaml(1 hunks)tests/fixtures/scenarios/vendor-pulls-ssh/vendor.yaml(1 hunks)tests/fixtures/scenarios/vendoring-dry-run/atmos.yaml(1 hunks)tests/fixtures/scenarios/vendoring-dry-run/components/terraform/ipinfo/component.yaml(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_component_using_SSH.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_custom_detector_credentials_leakage.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh_component.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_using_SSH.stderr.golden(1 hunks)tests/snapshots/TestCLICommands_atmos_vendor_pull_with_custom_detector_and_handling_credentials_leakage.stderr.golden(1 hunks)tests/test-cases/vendoring-ssh-dryrun.yaml(1 hunks)website/docs/cli/commands/vendor/vendor-pull.mdx(2 hunks)website/docs/cli/configuration/configuration.mdx(2 hunks)
🧰 Additional context used
🧠 Learnings (7)
tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden (2)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:8-8
Timestamp: 2025-03-18T00:20:29.425Z
Learning: Test snapshots in the Atmos project, particularly for dry run scenarios, may be updated during the development process, and temporary inconsistencies in their content should not be flagged as issues.
Learnt from: Listener430
PR: cloudposse/atmos#1149
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:7-7
Timestamp: 2025-03-18T12:26:25.329Z
Learning: In the Atmos project, typos or inconsistencies in test snapshot files (such as "terrafrom" instead of "terraform") may be intentional as they capture the exact output of commands and should not be flagged as issues requiring correction.
tests/snapshots/TestCLICommands_atmos_vendor_pull_with_custom_detector_and_handling_credentials_leakage.stderr.golden (2)
Learnt from: Listener430
PR: cloudposse/atmos#1149
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:7-7
Timestamp: 2025-03-18T12:26:25.329Z
Learning: In the Atmos project, typos or inconsistencies in test snapshot files (such as "terrafrom" instead of "terraform") may be intentional as they capture the exact output of commands and should not be flagged as issues requiring correction.
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:8-8
Timestamp: 2025-03-18T00:20:29.425Z
Learning: Test snapshots in the Atmos project, particularly for dry run scenarios, may be updated during the development process, and temporary inconsistencies in their content should not be flagged as issues.
tests/snapshots/TestCLICommands_atmos_vendor_pull_component_using_SSH.stderr.golden (2)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:8-8
Timestamp: 2025-03-18T00:20:29.425Z
Learning: Test snapshots in the Atmos project, particularly for dry run scenarios, may be updated during the development process, and temporary inconsistencies in their content should not be flagged as issues.
Learnt from: Listener430
PR: cloudposse/atmos#1149
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:7-7
Timestamp: 2025-03-18T12:26:25.329Z
Learning: In the Atmos project, typos or inconsistencies in test snapshot files (such as "terrafrom" instead of "terraform") may be intentional as they capture the exact output of commands and should not be flagged as issues requiring correction.
tests/snapshots/TestCLICommands_atmos_vendor_pull_custom_detector_credentials_leakage.stderr.golden (1)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:8-8
Timestamp: 2025-03-18T00:20:29.425Z
Learning: Test snapshots in the Atmos project, particularly for dry run scenarios, may be updated during the development process, and temporary inconsistencies in their content should not be flagged as issues.
tests/snapshots/TestCLICommands_atmos_vendor_pull_using_SSH.stderr.golden (2)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:8-8
Timestamp: 2025-03-18T00:20:29.425Z
Learning: Test snapshots in the Atmos project, particularly for dry run scenarios, may be updated during the development process, and temporary inconsistencies in their content should not be flagged as issues.
Learnt from: Listener430
PR: cloudposse/atmos#1149
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:7-7
Timestamp: 2025-03-18T12:26:25.329Z
Learning: In the Atmos project, typos or inconsistencies in test snapshot files (such as "terrafrom" instead of "terraform") may be intentional as they capture the exact output of commands and should not be flagged as issues requiring correction.
tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh_component.stderr.golden (2)
Learnt from: Listener430
PR: cloudposse/atmos#1061
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:8-8
Timestamp: 2025-03-18T00:20:29.425Z
Learning: Test snapshots in the Atmos project, particularly for dry run scenarios, may be updated during the development process, and temporary inconsistencies in their content should not be flagged as issues.
Learnt from: Listener430
PR: cloudposse/atmos#1149
File: tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden:7-7
Timestamp: 2025-03-18T12:26:25.329Z
Learning: In the Atmos project, typos or inconsistencies in test snapshot files (such as "terrafrom" instead of "terraform") may be intentional as they capture the exact output of commands and should not be flagged as issues requiring correction.
internal/exec/vendor_model.go (1)
Learnt from: samtholiya
PR: cloudposse/atmos#1077
File: pkg/downloader/custom_github_detector.go:0-0
Timestamp: 2025-02-21T20:56:05.539Z
Learning: The `github.com/charmbracelet/log` package should be imported with the alias `log` according to the project's import alias configuration.
🧬 Code Definitions (1)
pkg/utils/url_utils_test.go (1)
pkg/utils/url_utils.go (1) (1)
MaskBasicAuth(10:26)
🪛 YAMLlint (1.35.1)
tests/fixtures/scenarios/vendoring-dry-run/atmos.yaml
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
[error] 13-13: trailing spaces
(trailing-spaces)
[warning] 22-22: too many blank lines
(2 > 0) (empty-lines)
tests/fixtures/scenarios/vendor-pulls-ssh/atmos.yaml
[error] 20-20: no new line character at the end of file
(new-line-at-end-of-file)
tests/fixtures/scenarios/vendor-creds-sanitize/atmos.yaml
[warning] 1-1: too many blank lines
(1 > 0) (empty-lines)
[error] 13-13: trailing spaces
(trailing-spaces)
tests/fixtures/scenarios/vendor-creds-sanitize/vendor.yaml
[error] 18-18: trailing spaces
(trailing-spaces)
[error] 35-35: no new line character at the end of file
(new-line-at-end-of-file)
tests/test-cases/vendoring-ssh-dryrun.yaml
[error] 57-57: trailing spaces
(trailing-spaces)
[error] 59-59: no new line character at the end of file
(new-line-at-end-of-file)
🪛 LanguageTool
website/docs/cli/commands/vendor/vendor-pull.mdx
[style] ~216-~216: Consider using a different verb to strengthen your wording.
Context: ...lved When resolving Git sources, Atmos follows these rules: 1. If a *full HTTPS URL...
(FOLLOW_OBEY)
[style] ~218-~218: Consider using the typographical ellipsis character here instead.
Context: ...1. If a full HTTPS URL is provided (git::https://github.com/...), it is used as-is. No token data is i...
(ELLIPSIS)
⏰ Context from checks skipped due to timeout of 90000ms (9)
- GitHub Check: [mock-macos] tests/fixtures/scenarios/complete
- GitHub Check: [mock-macos] examples/demo-vendoring
- GitHub Check: [mock-windows] tests/fixtures/scenarios/complete
- GitHub Check: [mock-windows] examples/demo-vendoring
- GitHub Check: Acceptance Tests (macos-latest, macos)
- GitHub Check: Acceptance Tests (windows-latest, windows)
- GitHub Check: Acceptance Tests (ubuntu-latest, linux)
- GitHub Check: [localstack] demo-localstack
- GitHub Check: Summary
🔇 Additional comments (30)
internal/exec/vendor_utils.go (1)
22-23: Consider the necessity of a dedicated stderr loggerThe addition of a dedicated logger for stderr seems to diverge from the project's convention. Based on previous review comments, the project generally uses
log.*directly without dedicated instances.Are there specific requirements driving the need for a separate stderr logger? If you need to direct output to stderr, consider if this could be configured globally instead of introducing a new logger instance.
internal/exec/vendor_model.go (1)
337-337: Added debug logging for package installationThis additional debug logging improves the traceability of the download and install process.
The log statement appropriately includes structured key-value pairs as per the project convention.
codecov.yml (1)
16-16: Improved YAML syntax for boolean valueChanged from string representation
yesto proper booleantrue. This improves configuration clarity and follows standard YAML conventions.This change maintains the same functionality while using more standard syntax.
pkg/utils/url_utils.go (2)
8-8: Good choice making MaskedSecret a constantThe constant definition improves maintainability and addresses the previous suggestion by osterman. This is cleaner than hardcoding "xxx" in the implementation.
10-26: The MaskBasicAuth function implementation looks solidThe function correctly:
- Handles URL parsing errors
- Checks for user information presence
- Differentiates between username+password and username-only scenarios
- Uses the MaskedSecret constant consistently
The code correctly wraps errors with additional context and follows Go's error handling pattern well.
tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh.stderr.golden (1)
1-10: Snapshot content confirms enhanced dry run output.The snapshot demonstrates successful implementation of the enhanced dry run output for SSH-based vendor pulling, showing clear logging with debug information and successful component handling for
terrafrom-null-label. The final messages properly indicate this was a dry run with appropriate summary information.tests/snapshots/TestCLICommands_atmos_vendor_pull_component_using_SSH.stderr.golden (1)
1-11: SSH component vendoring snapshot looks good.The snapshot correctly captures the behavior of component-specific SSH vendoring in dry run mode, with appropriate debug and information messages. The output shows proper handling of the SSH-based
ipinfocomponent from the main branch.tests/fixtures/scenarios/vendoring-dry-run/components/terraform/ipinfo/component.yaml (1)
7-9: SSH URI format is correctly implemented.The component configuration properly uses an SSH Git URI format (
[email protected]) which demonstrates support for SSH vendoring. This correctly implements the SSH URL support mentioned in the PR objectives.tests/snapshots/TestCLICommands_atmos_vendor_pull_custom_detector_credentials_leakage.stderr.golden (3)
9-11: Effective credential sanitization implemented.The debug output now properly shows:
- The token injection process
- Credentials sanitized as "xxx:xxx" to prevent leakage
- Clear indication of the final URL construction with depth parameter
This successfully implements the PR objective to enhance dry run output with token injection information in a secure manner.
32-32: Path vs file designation in debug output.The debug statement uses "file" to refer to "docs" which is actually a directory.
87-92: Console output formatting issue.As previously discussed, these component success messages are formatted as INFO logs but might be better as plain stderr output to maintain a clean stdout for other tools to consume.
tests/snapshots/TestCLICommands_atmos_vendor_pull_with_custom_detector_and_handling_credentials_leakage.stderr.golden (1)
1-20: Enhanced Debug Log with Credential Sanitization
The golden file demonstrates detailed logging for the dry run flow. It clearly shows the setting of log levels, the processing of store configurations, and—most importantly—the secure handling of credentials by sanitizing the injected tokens and rewriting URLs. This output aligns well with the PR objective of providing detailed, secure dry run outputs.tests/snapshots/TestCLICommands_atmos_vendor_pull_ssh_component.stderr.golden (1)
1-18: Review of SSH Component Dry Run Output - Verify Consistency
The log clearly details the dry run process for component vendoring (noting the rewriting of SCP-style SSH URLs and injection of tokens). However, there’s a potential inconsistency: line 14 reports "Done! Dry run completed. No components vendored." while line 16 states "INFO Vendored 1 components." This difference might be confusing for users. Please verify if this mixed messaging is intended for dry-run reporting or if clarification is needed.tests/snapshots/TestCLICommands_atmos_vendor_pull_using_SSH.stderr.golden (1)
1-15: Concise Dry Run Output for SSH Scenario
The output for the SSH vendoring flow is clear and includes the expected debug and info messages, such as processing the vendor configuration, downloading packages, and confirming token injection. Similar to the previous snapshot, verify that the final summary—even though it states "No components vendored" in the dry run—is intentional when followed by a count (line 13: "INFO Vendored 2 components."). Consistency here is key to avoid misleading users about the outcomes of the dry run..github/workflows/test.yml (3)
49-50: Good addition for Git history access.Adding
fetch-depth: 0ensures the checkout action retrieves the complete Git history, which is necessary for certain operations that need historical context.
88-88: Appropriate timeout extension.Increasing the test job timeout from 15 to 20 minutes provides more headroom for completing the acceptance tests, which aligns with the needs mentioned in the PR objectives.
159-159: Consistent timeout configuration.The increase from 10 to 20 minutes for the Acceptance tests step matches the parent job timeout, ensuring tests have adequate time to complete, particularly when testing vendor pull functionality.
website/docs/cli/commands/vendor/vendor-pull.mdx (3)
111-159: Well-structured SSH vendoring documentation.This new section clearly explains how Atmos supports SSH for accessing non-public Git repositories, which directly addresses the PR objectives for enhancing the vendor pull command.
The documentation properly covers:
- SSH key authentication
- SCP-style source formatting
- URL rewriting for standard SSH format
- Automatic username injection for GitHub
160-221: Comprehensive authentication documentation.This section thoroughly documents the HTTPS vendoring options and authentication mechanisms for GitHub, Bitbucket, and GitLab. The explanation of token priority and default username behavior is particularly valuable.
The definition lists provide a clean, organized presentation of the environment variables.
🧰 Tools
🪛 LanguageTool
[style] ~216-~216: Consider using a different verb to strengthen your wording.
Context: ...lved When resolving Git sources, Atmos follows these rules: 1. If a *full HTTPS URL...(FOLLOW_OBEY)
[style] ~218-~218: Consider using the typographical ellipsis character here instead.
Context: ...1. If a full HTTPS URL is provided (git::https://github.com/...), it is used as-is. No token data is i...(ELLIPSIS)
216-216:✅ Verification successful
Consider rewording for stronger phrasing.
The sentence currently reads "When resolving Git sources, Atmos follows these rules:" - consider a more assertive phrasing like "Atmos resolves Git sources according to these rules:" to strengthen the wording.
-When resolving Git sources, Atmos follows these rules: +Atmos resolves Git sources according to these rules:
🌐 Web query:
What are best practices for technical documentation phrasing?💡 Result:
Effective technical documentation phrasing relies on clear, concise, and user-centered communication. Here are the core principles and actionable strategies to optimize your technical writing:
Clarity and Simplicity
- Use plain language: Avoid jargon and complex terminology. Opt for everyday words to ensure accessibility for both technical and non-technical audiences[1][2][4][10].
- Prioritize active voice: Clearly state who performs actions (e.g., "The system generates a report" vs. "A report is generated")[4][13].
- Eliminate ambiguity: Replace pronouns like "it" or "they" with specific nouns when context is unclear[4][13].
- Break down complex ideas: Use short sentences (ideally <25 words) and concise paragraphs focused on a single concept[7][9].
Consistency
- Standardize terminology: Use the same terms for concepts throughout the document (e.g., stick to "user agent" instead of alternating with "browser")[4][11].
- Adopt uniform formatting: Maintain consistent heading styles, code syntax highlighting, and punctuation (e.g., Oxford commas, hyphens)[8][12][13].
- Leverage templates: Create reusable templates for common document types (API references, tutorials) to streamline structure and styling[1][11].
Organization and Structure
- Logical flow: Group related tasks thematically and use numbered lists for sequential steps[2][4][11].
- Scannable hierarchy:
- Start sections with key takeaways (e.g., "Key principle: Always validate inputs")[9].
- Use headings (H2/H3) and bullet points to break up walls of text[10][11].
- Include tables for comparisons (e.g., feature A vs. feature B)[15].
- Code documentation specifics:
- Structure inline comments with titles for quick scanning[3].
- Explain non-obvious decisions (e.g., "Chose X algorithm for O(n) time complexity")[3].
- Pair code snippets with prerequisite context (dependencies, use cases)[6][11].
User-Centric Approach
- Define audience personas: Tailor depth and tone to user expertise (e.g., developers vs. end-users)[1][8][14].
- Incorporate examples:
- Real-world scenarios ("When processing large datasets, enable compression")[2][9].
- Visual aids like annotated screenshots or workflow diagrams[6][10].
- Test usability: Conduct user interviews to identify gaps and confusion points[5][11].
Maintenance and Collaboration
- Sync documentation with code: Update docs in the same pull request as feature changes[13].
- Audit regularly: Schedule quarterly reviews to remove outdated content and refresh examples[3][11].
- Collaborate with SMEs: Use interviews or questionnaires to validate technical accuracy[1][5][14].
Common Challenges and Solutions
Challenge Solution Information overload Trim redundant details; use progressive disclosure (e.g., expandable sections)[3][9] Documentation decay Implement version control and automated link checks[6][11] Inconsistent style Create a shared style guide covering terminology, formatting, and voice[11][13] By integrating these practices, technical documentation becomes a living resource that evolves with your product and empowers users. Focus on iterative improvement—even small enhancements in clarity or structure significantly boost usability over time[4][11][13].
Citations:
- 1: https://scribehow.com/library/technical-documentation-best-practices
- 2: https://www.indeed.com/career-advice/career-development/technical-writing-tips
- 3: https://document360.com/blog/code-documentation/
- 4: https://developer.mozilla.org/en-US/blog/technical-writing/
- 5: https://www.adoc-studio.app/blog/technical-writing-guide
- 6: https://swimm.io/blog/get-started-with-technical-documentation-writing-best-practices
- 7: https://www.scilife.io/blog/best-practices-tips-technical-writing
- 8: https://www.everythingtechnicalwriting.com/the-technical-writing-process/
- 9: https://wellshapedwords.com/essentials/practices/
- 10: https://www.stc.org/notebook/2021/11/17/10-tips-to-enhance-your-technical-writing-skills/
- 11: https://www.archbee.com/blog/technical-documentation-best-practices
- 12: https://blog.colinbreck.com/techniques-for-improving-your-technical-writing/
- 13: https://google.github.io/styleguide/docguide/best_practices.html
- 14: https://extendedstudies.ucsd.edu/news-events/extended-studies-blog/a-beginner-s-guide-to-understanding-technical-writing
- 15: https://forum.uipath.com/t/best-practice-writing-documentation/531430
Actionable Improvement: Adopt an Assertive, Active Tone
The revised phrasing "Atmos resolves Git sources according to these rules:" better reflects best practices in technical documentation—clear, concise, and using active voice. This change ensures the statement is more direct and user-centered.
-When resolving Git sources, Atmos follows these rules: +Atmos resolves Git sources according to these rules:🧰 Tools
🪛 LanguageTool
[style] ~216-~216: Consider using a different verb to strengthen your wording.
Context: ...lved When resolving Git sources, Atmos follows these rules: 1. If a *full HTTPS URL...(FOLLOW_OBEY)
pkg/utils/url_utils_test.go (4)
7-17: Good test coverage for username+password case.This test properly verifies that the MaskBasicAuth function correctly masks both username and password in a URL, which is essential for the PR's goal of displaying sanitized URLs in dry run output.
19-29: Proper testing of username-only case.This test ensures the function works correctly when a URL only contains a username without a password, validating another important use case.
31-41: Important pass-through validation.This test confirms that URLs without credentials remain unchanged, ensuring the function doesn't modify URLs that don't need sanitization.
43-49: Good error handling verification.Testing the error case ensures the function properly handles invalid URLs, which is essential for robust error reporting in the CLI.
website/docs/cli/configuration/configuration.mdx (2)
681-683: Clear documentation for new authentication variables.The addition of these three new environment variables (ATMOS_BITBUCKET_TOKEN, ATMOS_BITBUCKET_USERNAME, and ATMOS_GITLAB_TOKEN) provides the necessary documentation for users to authenticate with these services when using vendor pull functionality.
These additions directly support the PR objectives of enhancing the vendor pull command to work with various authentication methods.
697-697: Good shell environment context documentation.The ATMOS_SHLVL and ATMOS_TERRAFORM_WORKSPACE variables provide important context for users working in a shell environment, helping them understand when they're in an Atmos-spawned shell and which Terraform workspace is active.
Also applies to: 699-699
tests/test-cases/vendoring-ssh-dryrun.yaml (4)
1-2: Schema Declaration and File HeaderThe file starts with a clear schema declaration for the YAML language server. Ensure that the referenced
schema.jsonis available in the expected location to facilitate proper validation.
3-3: Collection of TestsThe tests are defined under the root
testskey, establishing a straightforward structure for multiple test cases.
4-20: Test Case: atmos vendor pull using SSHThis test case is well defined. It clearly sets the working directory, command, arguments, and environment variables for an SSH-style dry-run vendoring operation. The expectation of no diff and a
0exit code is appropriate.
22-40: Test Case: atmos vendor pull component using SSHThis case focuses on component-specific testing for vendoring (targeting "ipinfo"). The configuration, including disabling TTY and a clear snapshot expectation, is solid. One minor note: the environment variable value isn’t quoted (line 36), which is acceptable in YAML if interpreted as a string.
| @@ -508,7 +511,7 @@ func generateSkipFunction(tempDir string, s *schema.AtmosVendorSource) func(os.F | |||
| } | |||
|
|
|||
| // If 'included_paths' is not provided, include all files that were not excluded | |||
| log.Debug("Including", u.TrimBasePathFromPath(tempDir+"/", src)) | |||
| StderrLogger.Debug("Including", "path", u.TrimBasePathFromPath(tempDir+"/", src)) | |||
There was a problem hiding this comment.
Log formatting change needs justification
Changed the log call to use the new StderrLogger instead of the default logger. While the format with key-value pairs is correct, this change should be evaluated against whether we actually need a separate stderr logger.
If the primary goal is to keep stdout clean, consider if there are alternative approaches in line with the project's conventions.
what
This PR upgrades the output of dry run mode for vendor pull command.
After the change is applied, the dry run mode shows
yet there's no actual files download.
Before ( component vendoring dry-run output)
After ( component vendoring dry-run output)
component.yaml, refer to the respective test case for details
Before ( generic vendoring dry-run output)
After ( generic vendoring dry-run output)
vendor.yaml, refer to the respective test case for detaails
why
Received feedback from in the parent PR
references
Upscopes from #1061 and
Summary by CodeRabbit
Improvements
New Features
Documentation