Supported the atmos.d, allowing automatic inclusion of configuration files from the atmos.d directory #808
Conversation
haitham911
requested review from
johncblandii,
RoseSecurity,
aknysh and
osterman
📝 Walkthrough📝 WalkthroughWalkthroughThis pull request introduces several new configuration files for the Atmos CLI, enhancing its configuration management capabilities. Key additions include two Changes
Assessment against linked issues
Possibly related PRs
Suggested labels
Suggested reviewers
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
🧹 Outside diff range and nitpick comments (7)
examples/demo-atmos.d/atmos.yaml (3)
4-11: Document security-sensitive settingsCritical settings like
apply_auto_approveandauto_generate_backend_filewould benefit from inline documentation explaining their security implications.Consider adding comments:
components: terraform: base_path: "components/terraform" + # Disable auto-approve for safety in production environments apply_auto_approve: false deploy_run_init: true init_run_reconfigure: true + # Disable auto-generation of backend files to prevent unauthorized state access auto_generate_backend_file: false
12-19: Consider expanding name pattern flexibilityThe current name pattern only supports {stage}. Consider supporting additional variables for better organization.
Consider expanding:
- name_pattern: "{stage}" + name_pattern: "{org}-{stage}-{component}"
20-29: Clean up formatting and enhance documentationThe vendor configuration contains helpful examples but needs formatting cleanup.
Apply these changes:
-vendor: +vendor: # Single file - base_path: "./vendor.yaml" + base_path: "./vendor.yaml" # Default configuration # Directory with multiple files #base_path: "./vendor" # Absolute path #base_path: "vendor.d/vendor1.yaml"🧰 Tools
🪛 yamllint (1.29.0-1)
[error] 20-20: trailing spaces
(trailing-spaces)
[error] 23-23: trailing spaces
(trailing-spaces)
[error] 26-26: trailing spaces
(trailing-spaces)
examples/demo-atmos.d/custom-import/atmos.yaml (2)
26-35: Consider moving configuration examples to documentationWhile the commented examples are helpful, they might be better suited in the documentation, keeping the configuration file cleaner.
- # Single file base_path: "./vendor.yaml" - - # Directory with multiple files - #base_path: "./vendor" - - # Absolute path - #base_path: "vendor.d/vendor1.yaml"🧰 Tools
🪛 yamllint (1.29.0-1)
[error] 26-26: trailing spaces
(trailing-spaces)
[error] 29-29: trailing spaces
(trailing-spaces)
[error] 32-32: trailing spaces
(trailing-spaces)
36-38: Consider environment-specific log configurationsThe current logging setup is good for development, but you might want to add environment-specific configurations for production use.
Example addition:
logs: file: "${ATMOS_LOG_FILE:-/dev/stderr}" level: "${ATMOS_LOG_LEVEL:-Info}"pkg/schema/schema.go (1)
29-29: Add field documentation.Consider adding a documentation comment to describe the purpose and usage of the
Importfield, following Go's documentation conventions.Add this documentation above the field:
+ // Import specifies a list of paths from which to import additional configurations. + // Supports local files, directories (using glob patterns), and remote URLs. Import []string `yaml:"import" json:"import" mapstructure:"import"`pkg/config/config.go (1)
231-232: Combine the return statement for clarityThe return statement is unnecessarily split across two lines.
Consider writing it on a single line:
-return cliConfig, - err +return cliConfig, err
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (10)
examples/demo-atmos.d/atmos.d/configs.d/config1.yaml(1 hunks)examples/demo-atmos.d/atmos.d/configs.d/sub/config2.yaml(1 hunks)examples/demo-atmos.d/atmos.yaml(1 hunks)examples/demo-atmos.d/custom-import/atmos.yaml(1 hunks)examples/demo-atmos.d/custom-import/configs.d/config1.yaml(1 hunks)examples/demo-atmos.d/custom-import/configs.d/sub/config2.yaml(1 hunks)examples/demo-atmos.d/custom-import/extra-config.yaml(1 hunks)pkg/config/config.go(3 hunks)pkg/schema/schema.go(1 hunks)pkg/utils/glob_utils.go(2 hunks)
✅ Files skipped from review due to trivial changes (5)
- examples/demo-atmos.d/atmos.d/configs.d/config1.yaml
- examples/demo-atmos.d/atmos.d/configs.d/sub/config2.yaml
- examples/demo-atmos.d/custom-import/configs.d/config1.yaml
- examples/demo-atmos.d/custom-import/configs.d/sub/config2.yaml
- examples/demo-atmos.d/custom-import/extra-config.yaml
🧰 Additional context used
🪛 yamllint (1.29.0-1)
examples/demo-atmos.d/atmos.yaml
[error] 20-20: trailing spaces
(trailing-spaces)
[error] 23-23: trailing spaces
(trailing-spaces)
[error] 26-26: trailing spaces
(trailing-spaces)
[warning] 38-38: wrong indentation: expected 2 but found 0
(indentation)
[warning] 41-41: wrong indentation: expected 4 but found 2
(indentation)
examples/demo-atmos.d/custom-import/atmos.yaml
[error] 26-26: trailing spaces
(trailing-spaces)
[error] 29-29: trailing spaces
(trailing-spaces)
[error] 32-32: trailing spaces
(trailing-spaces)
[warning] 44-44: wrong indentation: expected 2 but found 0
(indentation)
[warning] 47-47: wrong indentation: expected 4 but found 2
(indentation)
🔇 Additional comments (6)
examples/demo-atmos.d/atmos.yaml (1)
30-33: Add validation for log levels
Consider adding validation to ensure only valid log levels (Debug, Info, Warning, Error) are accepted.
examples/demo-atmos.d/custom-import/atmos.yaml (3)
7-9: Consider adding URL validation for remote configurations
While loading from remote URLs provides flexibility, it could pose security risks. Consider implementing URL validation and HTTPS enforcement.
Would you like me to propose a validation pattern for secure URL handling?
18-24: Strong stack configuration, warrior! 💪
The stack configuration is well-structured with clear inclusion/exclusion patterns and a simple naming convention.
11-16: Verify backend file management strategy
Setting auto_generate_backend_file: false means you'll need to manage backend configurations manually. This is a valid choice but requires careful consideration.
Run this check to ensure consistent backend configuration across components:
✅ Verification successful
Setting auto_generate_backend_file: false is appropriate for this setup
The configuration choice is valid as there's no existing Terraform component directory at the specified path, indicating this is likely a new project setup. Manual backend management is a reasonable approach when starting fresh, allowing for more controlled and explicit backend configuration as the project evolves.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check for existing backend configurations
fd -e tf -e hcl . components/terraform -x grep -l 'backend'
Length of output: 167
Script:
#!/bin/bash
# First, let's verify if the components/terraform directory exists and check the directory structure
fd --type d terraform
# Then, let's search for backend configurations across all Terraform files in the repository
fd -e tf -e hcl . -x grep -l 'backend'
# Also check for any Terraform component references in the atmos configuration
fd -g "*.yaml" -x grep -A 5 -B 5 "terraform:"
Length of output: 33961
Script:
#!/bin/bash
# Let's check for backend configurations in the actual Terraform components
fd -t f -e tf . examples/demo-atmos.d/custom-import/components/terraform -x grep -l 'backend'
# Also check for any backend configuration files
fd -t f -g "*.backend.tf" -g "backend.tf" examples/demo-atmos.d/custom-import/components/terraform
# Check if there are any existing backend configurations in the component directory
ls -la examples/demo-atmos.d/custom-import/components/terraform/
Length of output: 709
pkg/utils/glob_utils.go (2)
7-7: LGTM! Clean import addition, warrior!
The sort package import is well-placed and necessary for the new functionality.
Line range hint 19-48: Verify caching behavior for dynamic configurations, brave one!
The caching mechanism using sync.Map could prevent picking up new configuration files added to atmos.d during runtime. Additionally, we should verify that the error handling aligns with the requirement to log non-existent paths as warnings.
Let's verify the implementation:
✅ Verification successful
Cache invalidation is not a concern, brave warrior!
The caching mechanism in GetGlobMatches is pattern-based, which means:
- Each unique glob pattern gets its own cache entry
- When a new file is added that matches an existing pattern, it will be discovered on the next process restart
- The cache is process-scoped (using
sync.Map), so it's automatically cleared between runs - Error handling correctly returns both the error and logs the non-existent paths
The implementation aligns perfectly with the requirements:
- Cache is scoped to the current process, preventing stale state between runs
- Error handling returns descriptive errors for non-existent paths:
failed to find a match for the import '%s' ('%s' + '%s')
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for warning logs related to non-existent paths and cache invalidation
# Search for warning log implementations
rg -A 2 "failed to find a match for the import" .
# Check if there's any cache invalidation mechanism
rg -A 5 "getGlobMatchesSyncMap" .
# Look for any warning level log statements
rg -A 2 "Warning|WARN|warn" .
Length of output: 47400
There was a problem hiding this comment.
Actionable comments posted: 3
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (2)
pkg/config/config.go(3 hunks)pkg/schema/schema.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- pkg/schema/schema.go
🔇 Additional comments (1)
pkg/config/config.go (1)
231-232: LGTM! Clean error handling.
The multi-line error return improves readability.
…correct command structure
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (3)
examples/demo-atmos.d/custom-import/atmos.yaml (2)
26-34: Clean up trailing spacesRemove trailing spaces from lines 26, 29, and 32 for better maintainability.
-vendor: +vendor: # Single file -base_path: "./vendor.yaml" +base_path: "./vendor.yaml" # Directory with multiple files -#base_path: "./vendor" +#base_path: "./vendor"🧰 Tools
🪛 yamllint (1.29.0-1)
[error] 26-26: trailing spaces
(trailing-spaces)
[error] 29-29: trailing spaces
(trailing-spaces)
[error] 32-32: trailing spaces
(trailing-spaces)
36-38: Logging configuration is good but could be enhancedCurrent configuration is solid. Consider adding:
- Log rotation settings
- Format specification
- Additional log destinations for production environments
pkg/config/config.go (1)
401-444: Consider adding rate limiting and path validationThe import processing is robust, but consider these enhancements:
- Add rate limiting for remote downloads to prevent abuse
- Validate import paths before processing to ensure they're safe
func processImports(cliConfig schema.CliConfiguration, v *viper.Viper) error { + // Add rate limiting for remote downloads + rateLimiter := time.NewTicker(time.Second) + defer rateLimiter.Stop() + for _, importPath := range cliConfig.Import { if importPath == "" { continue } + // Basic path validation + if strings.Contains(importPath, "..") { + u.LogWarning(cliConfig, fmt.Sprintf("Warning: skipping potentially unsafe import path '%s'", importPath)) + continue + }
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (4)
examples/demo-atmos.d/atmos.d/config1.yaml(1 hunks)examples/demo-atmos.d/atmos.d/sub/config2.yaml(1 hunks)examples/demo-atmos.d/custom-import/atmos.yaml(1 hunks)pkg/config/config.go(3 hunks)
✅ Files skipped from review due to trivial changes (2)
- examples/demo-atmos.d/atmos.d/config1.yaml
- examples/demo-atmos.d/atmos.d/sub/config2.yaml
🧰 Additional context used
🪛 yamllint (1.29.0-1)
examples/demo-atmos.d/custom-import/atmos.yaml
[error] 26-26: trailing spaces
(trailing-spaces)
[error] 29-29: trailing spaces
(trailing-spaces)
[error] 32-32: trailing spaces
(trailing-spaces)
🔇 Additional comments (7)
examples/demo-atmos.d/custom-import/atmos.yaml (4)
11-16: Terraform security settings look good!
The Terraform component configuration follows security best practices:
apply_auto_approve: falseprevents unintended infrastructure changesauto_generate_backend_file: falseensures explicit backend configuration
18-24: Stack configuration structure is well-organized!
Good use of:
- Explicit path inclusions/exclusions
- Clear naming pattern
- Logical directory structure
43-47: Command configuration looks good!
The test command is well-defined with clear description and steps.
7-9: Verify security measures for remote configuration imports
The remote URL import capability could pose security risks. Ensure that:
- HTTPS is enforced for remote URLs
- URL validation is implemented
- Content verification is performed before applying configurations
Consider implementing:
- URL allowlist/blocklist
- SSL certificate validation
- Content integrity checks (e.g., checksums)
pkg/config/config.go (3)
231-232: Clean error handling format!
The multi-line error return improves readability while maintaining functionality.
239-250: Validate path to prevent directory traversal
While the implementation correctly handles the atmos.d directory existence, consider adding path validation as suggested in the past review to prevent potential path traversal attacks.
446-464: Add safeguards to remote downloads
The download function needs the safeguards mentioned in the past review:
- Context timeout
- File size limits
- Temporary directory cleanup
…onfig and processImports
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (4)
pkg/config/config.go (4)
231-232: Consider using idiomatic Go error handling.While splitting the return statement improves readability, consider using the more idiomatic Go style:
- return cliConfig, - err + return cliConfig, fmt.Errorf("failed to unmarshal config: %w", err)
239-256: Strong implementation with robust security measures!The atmos.d directory support is well-implemented with proper path validation and error handling. The path traversal prevention is particularly noteworthy.
Consider adding more context to the error message:
- return cliConfig, fmt.Errorf("invalid atmos.d path: attempted directory traversal") + return cliConfig, fmt.Errorf("invalid atmos.d path '%s': attempted directory traversal outside base path '%s'", atmosDPath, basePath)
257-269: Consider adding debug logging for configuration merging.The import processing logic is sound, but adding debug logs would help troubleshoot configuration issues.
err = v.Unmarshal(&cliConfig) if err != nil { + u.LogTrace(cliConfig, fmt.Sprintf("Failed to unmarshal merged config: %v", err)) return cliConfig, err } + u.LogTrace(cliConfig, "Successfully merged and unmarshaled imported configurations")
407-459: Consider parallel processing for multiple imports.The implementation is robust with proper error handling and path validation. However, processing imports sequentially might be slow for many imports.
Consider using a worker pool for parallel processing:
func processImports(cliConfig schema.CliConfiguration, v *viper.Viper) error { + type importResult struct { + path string + err error + } + workers := make(chan struct{}, 4) // Limit concurrent downloads + results := make(chan importResult) + for _, importPath := range cliConfig.Import { - if importPath == "" { - continue - } + go func(imp string) { + workers <- struct{}{} // Acquire worker + defer func() { <-workers }() // Release worker + + if imp == "" { + results <- importResult{"", nil} + return + } + // ... rest of the import processing logic ... + }(importPath) }
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (1)
pkg/config/config.go(3 hunks)
🔇 Additional comments (1)
pkg/config/config.go (1)
461-479: Past review comment about remote download safeguards is still applicable.
The previous review comment suggesting context timeout, file size limits, and temporary directory cleanup remains valid.
…andle cleanup on error
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (1)
pkg/config/config.go (1)
232-233: Combine the return statement into a single lineThe return statement is split across two lines. Merging it into one line enhances readability.
Apply this diff:
- return cliConfig, - err + return cliConfig, err
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (1)
pkg/config/config.go(3 hunks)
🔇 Additional comments (4)
pkg/config/config.go (4)
240-257: Good handling of the 'import' key and path security
The logic for setting the default Import path when it's empty is well-implemented. Ensuring the path doesn't escape the intended directory adds a valuable security check.
259-270: Import processing added effectively
The addition of import processing and re-unmarshalling is correctly handled. Error checks are appropriately placed.
417-426: Consider validating remote URLs for security
Currently, any URL starting with "http://" or "https://" is accepted. Validating these URLs against an allowlist can prevent unintended or malicious imports.
464-484: Add a file size limit check in downloadRemoteConfig
While the context timeout is set, the function lacks a file size limit check. This could allow the download of excessively large files, leading to resource exhaustion.
|
Update documentation for Atmos CLI Configuration schema and examples |
|
Update documentation for Atmos CLI Configuration |
|
@coderabbitai full review |
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Outside diff range and nitpick comments (4)
pkg/config/config.go (1)
232-233: Consider merging the return statement into a single line for clarity.Splitting the return statement across two lines may reduce readability. Merging it into a single line enhances clarity.
examples/demo-atmos.d/atmos.yaml (1)
20-29: CLEAN THE BATTLEFIELD OF TRAILING SPACES!Remove trailing spaces after
vendor:andbase_path:for a cleaner configuration.-vendor: +vendor: # Single file - base_path: "./vendor.yaml" + base_path: "./vendor.yaml"🧰 Tools
🪛 yamllint (1.35.1)
[error] 20-20: trailing spaces
(trailing-spaces)
[error] 23-23: trailing spaces
(trailing-spaces)
[error] 26-26: trailing spaces
(trailing-spaces)
examples/demo-atmos.d/custom-import/atmos.yaml (1)
26-47: MAINTAIN FORMATION DISCIPLINE!Remove trailing spaces after
vendor:for consistent formatting.-vendor: +vendor:🧰 Tools
🪛 yamllint (1.35.1)
[error] 26-26: trailing spaces
(trailing-spaces)
[error] 29-29: trailing spaces
(trailing-spaces)
[error] 32-32: trailing spaces
(trailing-spaces)
pkg/schema/schema.go (1)
29-29: Consider adding documentation and validation for the Import field.To improve maintainability and usability:
- Add a comment describing the field's purpose and expected format
- Consider adding validation for import paths
Example documentation:
+// Import specifies a list of paths from which to import additional configurations. +// Supports local files, directories (using glob patterns), and remote URLs. Import []string `yaml:"import" json:"import" mapstructure:"import"`
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (10)
examples/demo-atmos.d/atmos.d/config1.yaml(1 hunks)examples/demo-atmos.d/atmos.d/sub/config2.yaml(1 hunks)examples/demo-atmos.d/atmos.yaml(1 hunks)examples/demo-atmos.d/custom-import/atmos.yaml(1 hunks)examples/demo-atmos.d/custom-import/configs.d/config1.yaml(1 hunks)examples/demo-atmos.d/custom-import/configs.d/sub/config2.yaml(1 hunks)examples/demo-atmos.d/custom-import/extra-config.yaml(1 hunks)pkg/config/config.go(3 hunks)pkg/schema/schema.go(1 hunks)pkg/utils/glob_utils.go(2 hunks)
🧰 Additional context used
🪛 yamllint (1.35.1)
examples/demo-atmos.d/atmos.yaml
[error] 20-20: trailing spaces
(trailing-spaces)
[error] 23-23: trailing spaces
(trailing-spaces)
[error] 26-26: trailing spaces
(trailing-spaces)
[warning] 38-38: wrong indentation: expected 2 but found 0
(indentation)
[warning] 41-41: wrong indentation: expected 4 but found 2
(indentation)
examples/demo-atmos.d/custom-import/atmos.yaml
[error] 26-26: trailing spaces
(trailing-spaces)
[error] 29-29: trailing spaces
(trailing-spaces)
[error] 32-32: trailing spaces
(trailing-spaces)
🔇 Additional comments (14)
examples/demo-atmos.d/atmos.d/config1.yaml (2)
3-4: Verify the validity of the retries configuration
The settings.retries configuration needs clarification:
- Is this a supported configuration option?
- What operations does this retry mechanism control?
- What's the recommended range of values?
Let's check the schema and documentation for this setting:
#!/bin/bash
# Search for schema definitions or documentation of retries setting
rg -A 5 "retries.*schema"
# Check for existing usage of this setting
ast-grep --pattern 'settings:
$$$
retries: $_
$$$'1-2: Consider security implications of Debug logging in production
Setting the log level to Debug could expose sensitive information in production environments. Consider using INFO as the default level and enabling Debug only for development/testing environments.
Let's check if there are any sensitive data patterns in debug logs:
✅ Verification successful
The previous search was too specific. Let's broaden our scope to understand the logging implementation and potential security implications better.
Debug logging level is safe to use in this context
Based on the codebase analysis, the Debug log level in this application is used primarily for:
- Displaying configuration information
- Showing command execution details
- Tracking workflow and component processing
No sensitive data patterns (passwords, tokens, secrets) are logged at the Debug level. The codebase has proper log level handling with sensitive operations using appropriate levels.
Key findings:
- Default log level is "Info" across all configuration files
- Debug logs mainly show infrastructure-related metadata and execution paths
- Sensitive operations use proper log level gates (Info/Warning)
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for potential sensitive data patterns in debug log statements
rg -i "debug.*password|debug.*secret|debug.*key|debug.*token"
Length of output: 63
Script:
#!/bin/bash
# Let's broaden our search to find any debug logging patterns
rg -i "debug.*log|log.*debug" -A 2
# Also search for logging configuration patterns
rg -i "log.*level|loglevel|logging.*level" -A 2
# Check for any logging-related files
fd -i "log" --type f
Length of output: 118987
examples/demo-atmos.d/custom-import/extra-config.yaml (1)
1-2: Verify if 'logs.file' is a valid configuration parameter.
Ensure that 'logs.file' is supported in the atmos configuration schema.
examples/demo-atmos.d/custom-import/configs.d/config1.yaml (1)
3-4: Confirm that 'settings.retries' is a recognized configuration option.
It's important to use parameters that are defined in the atmos schema.
examples/demo-atmos.d/custom-import/configs.d/sub/config2.yaml (1)
3-4: Ensure 'settings.timeout' is a valid configuration setting.
Please verify that 'settings.timeout' is supported by atmos.
examples/demo-atmos.d/atmos.d/sub/config2.yaml (2)
2-2: Consider enhancing the example with meaningful configurations.
Organizing configurations by keeping 'integrations', 'terraform', and 'helmfile' settings separate could improve clarity.
4-4: Check if 'settings.timeout' is a valid parameter in atmos.
Ensure that this parameter aligns with the atmos configuration schema.
examples/demo-atmos.d/atmos.yaml (3)
1-3: WARRIOR! Let's make the atmos.d support explicit!
The comment mentions "./atmos.d" as the default import path, but this configuration would be clearer with explicit import declarations.
Add this configuration to embrace the atmos.d convention:
base_path: "./"
+import:
+ - "./atmos.d/*.yaml"4-19: EXCELLENT STRUCTURE, WARRIOR!
The components and stacks configuration is well-organized and follows best practices.
34-41: ALIGN YOUR RANKS, WARRIOR!
The command structure needs proper indentation.
commands:
- - name: "test"
- description: "Run all tests"
- steps:
- - atmos vendor pull
+ - name: "test"
+ description: "Run all tests"
+ steps:
+ - atmos vendor pull🧰 Tools
🪛 yamllint (1.35.1)
[warning] 38-38: wrong indentation: expected 2 but found 0
(indentation)
[warning] 41-41: wrong indentation: expected 4 but found 2
(indentation)
examples/demo-atmos.d/custom-import/atmos.yaml (2)
10-25: STRONG FORMATION, WARRIOR!
The components and stacks configuration maintains consistency with the base configuration.
8-8:
SECURE YOUR SUPPLY LINES, WARRIOR!
Loading configuration from a remote URL without HTTPS validation could pose security risks. Consider adding validation for HTTPS URLs and implementing SSL certificate verification.
pkg/utils/glob_utils.go (1)
42-43: STRONG IMPLEMENTATION, WARRIOR!
The lexicographical sorting of matches is correctly implemented on the fullMatches slice, ensuring consistent ordering of configuration files.
✅ Verification successful
VICTORY! YOUR SORTING IMPLEMENTATION STANDS STRONG!
The lexicographical sorting of matches is a worthy addition that brings consistency to all consumers of GetGlobMatches. The function is used across critical parts of the codebase:
- Configuration processing in
pkg/config/ - Stack processing in
internal/exec/
The sorted output will ensure deterministic behavior in these components, making the system more reliable and predictable.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Verify that GetGlobMatches is used consistently across the codebase
ast-grep --pattern 'GetGlobMatches($pattern)'
Length of output: 47
Script:
#!/bin/bash
# Let's try a different approach to find GetGlobMatches usage
rg "GetGlobMatches" -A 2
# Also check the implementation
rg "func.*GetGlobMatches" -A 5
Length of output: 2731
pkg/schema/schema.go (1)
29-29: LGTM! The Import field is well-structured.
The field follows the established pattern with consistent tag naming and appropriate type for handling multiple import paths.
| if strings.HasPrefix(importPath, "http://") || strings.HasPrefix(importPath, "https://") { | ||
| // Handle remote URLs | ||
| tempDir, tempFile, err := downloadRemoteConfig(importPath) | ||
| if err != nil { | ||
| u.LogWarning(cliConfig, fmt.Sprintf("Warning: failed to download remote config '%s': %v", importPath, err)) | ||
| continue | ||
| } | ||
| resolvedPaths = []string{tempFile} |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Add URL validation before downloading remote configurations.
Validating URLs ensures that only trusted sources are used, enhancing security.
pkg/config/config.go
Outdated
| if err != nil { | ||
| return cliConfig, err | ||
| } | ||
| atmosDPath := path.Join(basePath, "atmos.d") |
There was a problem hiding this comment.
I would prefer we support multiple paths for atmos.d configuration files, allowing for flexibility and adherence to Linux CLI conventions. The search order should be:
-
Environment variable overrides:
$ATMOS_CONFIG_PATH— If defined, this should be the first and only path checked.$XDG_CONFIG_HOME/atmos/atmos.d— If$XDG_CONFIG_HOMEis set, use it as the base for user-specific configuration (default:~/.config).
-
Current working directory (project-specific configuration):
./atmos.d./.atmos.d
-
User-specific configuration:
~/.config/atmos/atmos.d~/.atmos.d
-
System-wide configuration:
/etc/atmos.d
The first directory or file found in the search order should be used. This approach allows for:
- Explicit overrides via environment variables (
$ATMOS_CONFIG_PATH). - Adherence to XDG standards for user-specific paths.
- Fallbacks to project-specific and system-wide configurations.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (6)
examples/demo-atmos.d/atmos.d/sub/config2.yaml (2)
1-2: Consider documenting log level precedenceThis configuration sets logging to
Warninglevel, while other files use different levels. For a demo, it would be helpful to explain how these different log levels are merged or which takes precedence.
17-19: Clean up extra empty linesConsider maintaining a single newline at the end of the file, removing the extra empty lines.
cluster_name_pattern: "{namespace}-{tenant}-{environment}-{stage}-eks-cluster" - -examples/demo-atmos.d/atmos.d/config1.yaml (1)
4-7: Consider enhancing test command with validation steps.The test command could be more comprehensive by including additional validation steps.
commands: - name: "test" description: "Run all tests with custom command" steps: - atmos vendor pull + - atmos validate + - atmos lintexamples/demo-atmos.d/atmos.yaml (3)
1-3: Consider explicit atmos.d configuration.The comment mentions "./atmos.d" as the default import path, but this isn't reflected in the configuration.
base_path: "./" +import: + - "./atmos.d/*.yaml"
36-44: Fix trailing whitespace and improve vendor configuration clarity.The vendor section contains trailing whitespace and could benefit from clearer documentation.
-vendor: +vendor: # Single file base_path: "./vendor.yaml" - - # Directory with multiple files #base_path: "./vendor" - # Absolute path #base_path: "vendor.d/vendor1.yaml"🧰 Tools
🪛 yamllint (1.35.1)
[error] 36-36: trailing spaces
(trailing-spaces)
[error] 39-39: trailing spaces
(trailing-spaces)
[error] 42-42: trailing spaces
(trailing-spaces)
53-57: Fix indentation in commands section.The commands section needs proper indentation.
commands: -- name: "test" - description: "Run all tests" - steps: - - atmos vendor pull + - name: "test" + description: "Run all tests" + steps: + - atmos vendor pull🧰 Tools
🪛 yamllint (1.35.1)
[warning] 54-54: wrong indentation: expected 2 but found 0
(indentation)
[warning] 57-57: wrong indentation: expected 4 but found 2
(indentation)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (6)
examples/demo-atmos.d/atmos.d/config1.yaml(1 hunks)examples/demo-atmos.d/atmos.d/sub/config2.yaml(1 hunks)examples/demo-atmos.d/atmos.yaml(1 hunks)examples/demo-atmos.d/custom-import/configs.d/config1.yaml(1 hunks)examples/demo-atmos.d/custom-import/configs.d/sub/config2.yaml(1 hunks)pkg/config/config.go(3 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- examples/demo-atmos.d/custom-import/configs.d/sub/config2.yaml
- examples/demo-atmos.d/custom-import/configs.d/config1.yaml
🧰 Additional context used
📓 Learnings (1)
pkg/config/config.go (1)
Learnt from: osterman
PR: cloudposse/atmos#808
File: pkg/config/config.go:478-483
Timestamp: 2024-12-02T21:26:32.337Z
Learning: In the 'atmos' project, when reviewing Go code like `pkg/config/config.go`, avoid suggesting file size checks after downloading remote configs if such checks aren't implemented elsewhere in the codebase.
🪛 yamllint (1.35.1)
examples/demo-atmos.d/atmos.yaml
[error] 36-36: trailing spaces
(trailing-spaces)
[error] 39-39: trailing spaces
(trailing-spaces)
[error] 42-42: trailing spaces
(trailing-spaces)
[warning] 54-54: wrong indentation: expected 2 but found 0
(indentation)
[warning] 57-57: wrong indentation: expected 4 but found 2
(indentation)
🔇 Additional comments (7)
examples/demo-atmos.d/atmos.d/sub/config2.yaml (2)
5-12: Well-documented environment variable overrides!
Excellent job documenting the environment variable alternatives for each configuration option. This makes the configuration very flexible and helps users understand all available options.
3-16: Consider reorganizing example configurations by purpose
While the configuration is technically sound, consider restructuring the demo to show how different aspects (integrations, terraform, helmfile) can be organized into separate files for better maintainability.
examples/demo-atmos.d/atmos.d/config1.yaml (1)
1-2: Consider logging level for production use.
Debug level logging in production can impact performance and expose sensitive information.
pkg/config/config.go (4)
240-259: LGTM! Robust handling of atmos.d directory.
The implementation correctly:
- Checks for empty imports
- Validates path traversal
- Handles file system errors appropriately
466-486: LGTM! Well-implemented remote configuration download.
The implementation includes:
- Proper temporary directory handling
- Context timeout for downloads
- Cleanup on error
440-445: 🛠️ Refactor suggestion
Replace 'path.Join' with 'filepath.Join' for cross-platform compatibility.
Using 'path.Join' can cause issues on Windows systems.
-imp := path.Join(basePath, impWithExt)
+imp := filepath.Join(basePath, impWithExt)Likely invalid or redundant comment.
419-428: 🛠️ Refactor suggestion
Consider adding URL validation for remote configurations.
While the implementation handles remote downloads well, adding URL validation would enhance security.
if strings.HasPrefix(importPath, "http://") || strings.HasPrefix(importPath, "https://") {
+ // Validate URL
+ parsedURL, err := url.Parse(importPath)
+ if err != nil || (parsedURL.Scheme != "http" && parsedURL.Scheme != "https") {
+ u.LogWarning(cliConfig, fmt.Sprintf("Warning: invalid URL '%s': %v", importPath, err))
+ continue
+ }
// Handle remote URLs
tempDir, tempFile, err := downloadRemoteConfig(importPath)Likely invalid or redundant comment.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (2)
pkg/config/config.go (2)
409-463: Add validation for import paths.The function handles imports well, but consider adding:
- Maximum depth limit for recursive imports to prevent infinite loops
- Validation for allowed file extensions beyond just .yaml
- Deduplication of import paths to prevent processing the same file multiple times
465-485: Add URL validation and consider increasing the timeout.The function handles downloads well, but consider:
- Adding URL validation to restrict to allowed domains
- Increasing the timeout from 5 seconds for slower connections
- Adding retry logic with backoff for transient failures
Example implementation:
func downloadRemoteConfig(url string) (string, string, error) { + if err := validateURL(url); err != nil { + return "", "", fmt.Errorf("invalid URL: %w", err) + } tempDir, err := os.MkdirTemp("", "atmos-import-*") if err != nil { return "", "", err } tempFile := filepath.Join(tempDir, "config.yaml") - ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) defer cancel() + + // Add retry logic with backoff + backoff := time.Second + for retries := 3; retries > 0; retries-- { client := &getter.Client{ Ctx: ctx, Src: url, Dst: tempFile, Mode: getter.ClientModeFile, } err = client.Get() - if err != nil { + if err == nil { + break + } + if retries > 1 { + time.Sleep(backoff) + backoff *= 2 + continue + } os.RemoveAll(tempDir) return "", "", fmt.Errorf("failed to download remote config: %w", err) - } + } return tempDir, tempFile, nil }
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (1)
pkg/config/config.go(10 hunks)
🧰 Additional context used
📓 Learnings (1)
pkg/config/config.go (1)
Learnt from: osterman
PR: cloudposse/atmos#808
File: pkg/config/config.go:478-483
Timestamp: 2024-12-02T21:26:32.337Z
Learning: In the 'atmos' project, when reviewing Go code like `pkg/config/config.go`, avoid suggesting file size checks after downloading remote configs if such checks aren't implemented elsewhere in the codebase.
🔇 Additional comments (2)
pkg/config/config.go (2)
231-232: LGTM! Clean error handling.
The error handling has been improved by moving the error to a new line, which enhances readability.
239-271: Consider implementing multi-path support for atmos.d.
While the current implementation is solid, consider implementing the multi-path support as suggested in the past review comments. The search order should include:
$ATMOS_CONFIG_PATH$XDG_CONFIG_HOME/atmos/atmos.d./atmos.dand./.atmos.d~/.config/atmos/atmos.d~/.atmos.d/etc/atmos.d
|
|
||
| _, err = os.Stat(atmosDPath) | ||
| if err == nil { | ||
| cliConfig.Import = []string{"atmos.d/**/*.yaml"} |
There was a problem hiding this comment.
We should support .yml and .yaml everywhere.
| if err != nil { | ||
| return "", "", err | ||
| } | ||
| tempFile := filepath.Join(tempDir, "config.yaml") |
There was a problem hiding this comment.
| tempFile := filepath.Join(tempDir, "config.yaml") | |
| tempFile := filepath.Join(tempDir, "atmos.yaml") |
There was a problem hiding this comment.
It's more clear that we're downloading an "atmos.yaml" CLI config
| - "configs.d/**/*" # Recursively load configurations from a directory | ||
| - "https://example.com/config.yaml" # Load from a remote URL | ||
| - "./extra-config.yaml" # Load a specific file |
There was a problem hiding this comment.
| - "configs.d/**/*" # Recursively load configurations from a directory | |
| - "https://example.com/config.yaml" # Load from a remote URL | |
| - "./extra-config.yaml" # Load a specific file | |
| - "configs.d/**/*" # Recursively load configurations from a directory | |
| - "https://example.com/config.yaml" # Load from a remote URL | |
| - "./extra-config.yaml" # Load a specific file |
| @@ -0,0 +1,19 @@ | |||
| logs: | |||
There was a problem hiding this comment.
Rename this file from config2.yaml to helmfile.yaml
| @@ -0,0 +1,7 @@ | |||
| logs: | |||
There was a problem hiding this comment.
Rename this file from config1.yaml to commands.yaml
| base_path: "./" | ||
| import: | ||
| - "configs.d/**/*" # Recursively load configurations from a directory | ||
| - "https://example.com/config.yaml" # Load from a remote URL |
| @@ -0,0 +1,2 @@ | |||
| logs: | |||
There was a problem hiding this comment.
Rename this file from extra-config.yaml to logging.yaml and move all log-levels here.
TODO
|
what
Supported the atmos.d convention for atmos.yaml configuration, allowing automatic inclusion of configuration files from the atmos.d directory.
Made the path to atmos.d configurable within atmos.yaml, enabling users to specify custom directories for additional configurations.
Implemented deep merging of configurations in lexicographical order, recursively processing files within specified directories to ensure consistent and predictable configuration outcomes.
Added support for the import key inside atmos.yaml, allowing users to define a list of locations (local files, directories using glob patterns, and remote URLs) to import configurations from.
why
Simplifies configuration management by adopting the atmos.d directory convention, making it easier to include additional configurations without explicit declarations.
references
https://linear.app/cloudposse/issue/DEV-1534/support-atmosd-convention-for-atmosyaml-configuration
Summary by CodeRabbit
New Features
Improvements
Documentation