Fix Vendoring Issues with Globs and Symlinks

[Listener430]

what

Done:

1. append //. at the end to github repo url (in order to clone entire repo)
2. Removed any symlinks inside go-getter
3. Fixed support for double star globs in excluded_paths and included_paths:

   included_paths:
     - "**/{demo-library,demo-stacks}/**/*.{tf,md}"
   excluded_paths:
     - "**/demo-library/**/*.{tfvars,tf}"
   

4. Added test to vendoring scenario in fixtures
5. Added depth=1 for all github downloads through custom detector
6. Breaking change: * now correctly matches a single segment. Anyone using a single star to match multiple segments should change it to **. This should never have matched multiple segments so long as double star was supposed to work.

why

• double star globs were not correctly matching multiple segments in all cases
• vendoring without a shallow depth is 2x slower
• the //. is an esoteric expression to copy all files from the root. Rather than expect users to know this, we default it where it makes sense.

references

• DEV-2964
• DEV-2813
• Closes Vendor pull does not support double-star glob for exclusions or templating on the include/exclude list #835
• Closes atmos vendor pull fails when source repository contains symlinks #698
• Closes Vendoring: Do shallow clones per default #1025

Summary by CodeRabbit

• New Features

  • Enhanced vendor pull operations with robust handling of repository URLs, token injection, and safe removal of directory symlinks.
  • Improved file copying with advanced pattern matching for flexible inclusion and exclusion of files.
  • Introduced configurable options for vendor file management.
  • Added a new function for masking user credentials in URLs.

• Documentation

  • Added a new guide on using glob patterns in vendor configurations for better file selection.

• Tests

  • Expanded test coverage to verify accurate file inclusion and exclusion during vendoring operations.
  • Introduced new test cases to validate the functionality of the atmos vendor pull command with glob patterns.

[osterman]

Please add a test for this type of vendoring. It can be in the vendoring scenario we already have.

[coderabbitai]

Actionable comments posted: 0

♻️ Duplicate comments (1)

internal/exec/copy_glob.go (1)

55-55: ⚠️ Potential issue

Fix logger usage to use charmbracelet logger correctly.

The current logger usage is causing compilation errors. Based on past review comments, we should use the charmbracelet logger directly.

Apply this fix:

-	l.LogDebug("Error computing relative path", 'srcPath', srcPath, 'err', err)
+	l.LogDebug("Error computing relative path", "srcPath", srcPath, "err", err)

🧰 Tools

🪛 golangci-lint (1.62.2)

55-55: illegal rune literal

(typecheck)

🪛 GitHub Check: Build (macos-latest, macos)

[failure] 55-55:
undefined: l

---

[failure] 55-55:
more than one character in rune literal

🪛 GitHub Check: Build (ubuntu-latest, linux)

[failure] 55-55:
undefined: l

---

[failure] 55-55:
more than one character in rune literal

🪛 GitHub Check: autofix

[failure] 55-55:
illegal rune literal

---

[failure] 55-55:
illegal rune literal

🪛 GitHub Actions: autofix.ci

[error] 55-55: illegal rune literal

🧹 Nitpick comments (3)

internal/exec/copy_glob.go (3)

15-45: Consider enhancing error wrapping for better debugging.

The error handling is thorough, but we could make it more consistent with Go 1.13+ error wrapping conventions.

Consider this improvement:

-		return fmt.Errorf("opening source file %q: %w", src, err)
+		return fmt.Errorf("failed to open source file %q: %w", src, err)
-		return fmt.Errorf("creating destination directory for %q: %w", dst, err)
+		return fmt.Errorf("failed to create destination directory for %q: %w", dst, err)
-		return fmt.Errorf("creating destination file %q: %w", dst, err)
+		return fmt.Errorf("failed to create destination file %q: %w", dst, err)
-		return fmt.Errorf("copying content from %q to %q: %w", src, dst, err)
+		return fmt.Errorf("failed to copy content from %q to %q: %w", src, dst, err)
-		return fmt.Errorf("getting file info for %q: %w", src, err)
+		return fmt.Errorf("failed to get file info for %q: %w", src, err)
-		return fmt.Errorf("setting permissions on %q: %w", dst, err)
+		return fmt.Errorf("failed to set permissions on %q: %w", dst, err)

---

139-166: Consider extracting pattern suffix as a constant.

The pattern suffix "/*" is used multiple times and could be made more maintainable.

Consider this improvement:

+const (
+	singleLevelGlobSuffix = "/*"
+	recursiveGlobSuffix   = "/**"
+)

 func getMatchesForPattern(atmosConfig schema.AtmosConfiguration, sourceDir, pattern string) ([]string, error) {
 	fullPattern := filepath.Join(sourceDir, pattern)
 	matches, err := u.GetGlobMatches(fullPattern)
 	if err != nil {
 		return nil, fmt.Errorf("error getting glob matches for %q: %w", fullPattern, err)
 	}
 	if len(matches) == 0 {
-		if strings.HasSuffix(pattern, "/*") {
-			recursivePattern := strings.TrimSuffix(pattern, "/*") + "/**"
+		if strings.HasSuffix(pattern, singleLevelGlobSuffix) {
+			recursivePattern := strings.TrimSuffix(pattern, singleLevelGlobSuffix) + recursiveGlobSuffix

---

168-255: Consider breaking down the pattern matching logic into smaller functions.

The function is well-implemented but could be more maintainable if split into smaller, focused functions.

Consider extracting these parts into separate functions:

1. Pattern matching logic (lines 196-211)
2. File copying with exclusions (lines 212-247)
3. Fallback copying logic (lines 248-253)

This would improve readability and make the code easier to test.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 06f5f3b and 82caa05.

📒 Files selected for processing (1)

• internal/exec/copy_glob.go (1 hunks)

🧰 Additional context used

🪛 golangci-lint (1.62.2)

internal/exec/copy_glob.go

55-55: illegal rune literal

(typecheck)

🪛 GitHub Check: Build (macos-latest, macos)

internal/exec/copy_glob.go

[failure] 55-55:
undefined: l

---

[failure] 55-55:
more than one character in rune literal

🪛 GitHub Check: Build (ubuntu-latest, linux)

internal/exec/copy_glob.go

[failure] 55-55:
undefined: l

---

[failure] 55-55:
more than one character in rune literal

🪛 GitHub Check: autofix

internal/exec/copy_glob.go

[failure] 55-55:
illegal rune literal

---

[failure] 55-55:
illegal rune literal

🪛 GitHub Actions: autofix.ci

internal/exec/copy_glob.go

[error] 55-55: illegal rune literal

⏰ Context from checks skipped due to timeout of 90000ms (2)

• GitHub Check: Build (windows-latest, windows)
• GitHub Check: Summary

🔇 Additional comments (2)

internal/exec/copy_glob.go (2)

1-13: Well-structured imports and package organization!

Good choice using the optimized otiai10/copy library for unfiltered copying operations.

---

94-137: Solid implementation of recursive directory copying!

Good job handling symlinks and maintaining proper error propagation.

[osterman]

@Listener430 please resolve merge conflicts

[mergify]

💥 This pull request now has conflicts. Could you fix it @Listener430? 🙏

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (18)

internal/exec/vendor_utils_test.go (5)

115-119: Redundant error handling.

There is a require.NoError check followed by an if err != nil check. One is sufficient:

-    require.NoError(t, err)
-    if err != nil {
-        t.Errorf("Failed to execute vendor pull command: %v", err)
-    }
+    require.NoError(t, err, "Failed to execute vendor pull command")

---

153-158: Dry-run flag usage is verified.

Again, there's a redundant error check around require.NoError. Revisit for clarity.

---

159-163: Verifying dry-run command execution.

Same redundant error check pattern as noted before.

---

164-168: Setting tags for demonstration.

Same note regarding require.NoError plus extra check.

---

169-173: Testing pull command with 'demo' tag.

Repeating the same dual error check pattern.

pkg/config/config.go (1)

414-447: Manually parsing double-dash flags with parseFlags.

Using a manual parser is workable, but relying on Cobra or viper could reduce complexity and potential edge cases.

internal/exec/error.go (1)

28-30: Check for message consistency in error variables.

The errors ErrFailedToInitializeTUIModelWithDetails, ErrValidPackage, and ErrTUIModel appear somewhat redundant. Consider unifying them or clarifying distinct scenarios to avoid confusion. Also, ensure the message "no valid installer package provided for" is complete.

internal/exec/oci_utils.go (1)

37-37: Consider passing configuration by pointer.

defer removeTempDir(*atmosConfig, tempDir) copies a potentially large struct. If performance is a concern, pass atmosConfig by pointer to avoid copies.

internal/exec/go_getter_utils.go (1)

356-356: Use pointer for Atmos configuration.

GoGetterGet(*atmosConfig, file, f, ...) passes atmosConfig by value. To avoid copying a large struct, consider passing a pointer consistently throughout.

internal/exec/vendor_utils.go (4)

60-89: Check for potential concurrency handles.

ReadAndProcessVendorConfigFile merges multiple vendoring configs sequentially. In future, consider concurrency if multiple config merges get expensive. Currently, this is likely acceptable, but be mindful of potential performance overhead in large-scale, multi-file merges.

---

183-220: Potential improvement to logging usage.

ExecuteAtmosVendorInternal logs an initial message and uses the same logger for warnings and errors. Consider whether structured logs at each step would help debugging when processing multiple imports or vendor specs in a single run.

---

398-404: Log message logic is straightforward.

logInitialMessage provides clarity on vendoring operation context. In the future, adding more context (e.g., component or source counts) could enhance usability.

---

463-479: Refine approach for local paths vs. single files.

copyToTarget currently adjusts local file target paths if there's no extension. This effectively avoids naming collisions. However, if some single-file sources do have an extension, watch for potential naming conflicts in target directories.

internal/exec/vendor_model.go (1)

331-354: downloadAndInstall concurrency.

This function is triggered via TUI messages. If concurrency arises in the future, be aware of potential race conditions for temp directories. Currently, the single-thread approach is simpler and stable.

pkg/schema/schema.go (1)

130-146: processManifestSchemas effectively re-marshals to SchemaRegistry.

Re-marshalling to JSON is a straightforward way to unify data shapes. Just watch out for performance if the schema set grows significantly.

internal/exec/vendor.go (2)

74-113: Preferences-based flag parsing.

The approach systematically retrieves dry-run, component, stack, tags, everything, and type. This is logically sound.

Consider logging or documenting default values (like --everything) to clarify how they are set for user reference.

---

139-172: Vendor config handling.

1. Good approach to detect a config file, if any, and fallback to direct component vendoring otherwise.
2. Consider partial dryness checks or further warnings if stack vendoring is invoked.

internal/exec/vendor_component_utils.go (1)

491-545: Mixin installation logic.

1. The structured approach for remote vs. OCI vs. local is consistent with the rest of the system.
2. The 10-minute timeout for fetching large packages may be adequate, but consider user customization.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8b0a3b0 and b468fb2.

📒 Files selected for processing (17)

• internal/exec/error.go (1 hunks)
• internal/exec/go_getter_utils.go (4 hunks)
• internal/exec/oci_utils.go (1 hunks)
• internal/exec/tar_utils.go (1 hunks)
• internal/exec/validate_component.go (2 hunks)
• internal/exec/validate_stacks.go (5 hunks)
• internal/exec/vendor.go (1 hunks)
• internal/exec/vendor_component_utils.go (6 hunks)
• internal/exec/vendor_model.go (8 hunks)
• internal/exec/vendor_model_component.go (0 hunks)
• internal/exec/vendor_utils.go (6 hunks)
• internal/exec/vendor_utils_test.go (3 hunks)
• pkg/config/config.go (5 hunks)
• pkg/config/utils.go (2 hunks)
• pkg/schema/schema.go (2 hunks)
• pkg/vender/component_vendor_test.go (2 hunks)
• pkg/vender/vendor_config_test.go (4 hunks)

💤 Files with no reviewable changes (1)

• internal/exec/vendor_model_component.go

🧰 Additional context used

🧠 Learnings (2)

internal/exec/validate_stacks.go (2)

Learnt from: haitham911
PR: cloudposse/atmos#731
File: internal/exec/validate_stacks.go:0-0
Timestamp: 2025-03-19T20:50:48.312Z
Learning: In `internal/exec/validate_stacks.go`, when downloading the Atmos JSON Schema file to the temp directory, the temporary file is overwritten each time, so explicit removal is not necessary.

Learnt from: haitham911
PR: cloudposse/atmos#731
File: internal/exec/validate_stacks.go:93-98
Timestamp: 2025-03-19T20:50:42.838Z
Learning: When downloading schema files in `internal/exec/validate_stacks.go`, use a consistent temporary file name to overwrite the file each time and avoid creating multiple temporary files.

internal/exec/go_getter_utils.go (2)

Learnt from: Listener430
PR: cloudposse/atmos#1061
File: internal/exec/go_getter_utils.go:74-75
Timestamp: 2025-03-19T20:50:48.313Z
Learning: In the `CustomGitDetector.Detect` method of `internal/exec/go_getter_utils.go`, verbose debug logging of raw URLs is intentionally kept for debugging purposes, despite potential credential exposure risks.

Learnt from: osterman
PR: cloudposse/atmos#984
File: internal/exec/go_getter_utils.go:103-109
Timestamp: 2025-03-19T20:50:48.312Z
Learning: When checking for subdirectories in GitHub URLs, use `parsedURL.Path` to check for "//" instead of the entire URL, as the scheme portion (e.g., "https://") will always contain "//".

🧬 Code Definitions (12)

pkg/vender/component_vendor_test.go (1)

internal/exec/vendor_component_utils.go (2) (2)

• ReadAndProcessComponentVendorConfigFile (58-105)
• ExecuteComponentVendorInternal (222-283)

internal/exec/vendor_utils_test.go (7)

internal/exec/validate_stacks.go (1) (1)

• err (221-221)

internal/exec/vendor_utils.go (2) (2)

• err (185-185)
• ReadAndProcessVendorConfigFile (61-89)

internal/exec/validate_component.go (1) (1)

• err (119-119)

internal/exec/vendor.go (1) (1)

• err (76-76)

pkg/config/config.go (2) (2)

• err (126-126)
• atmosConfig (125-125)

pkg/utils/markdown_utils.go (1) (1)

• err (111-111)

internal/exec/vendor_model.go (1) (1)

• cmd (204-204)

internal/exec/validate_component.go (1)

pkg/utils/file_utils.go (2) (2)

• err (296-296)
• JoinAbsolutePathWithPaths (72-80)

pkg/config/config.go (2)

pkg/schema/schema.go (2) (2)

• AtmosConfiguration (13-44)
• Logs (290-293)

pkg/config/utils.go (1) (1)

• processCommandLineArgs (405-432)

pkg/vender/vendor_config_test.go (2)

internal/exec/vendor.go (1) (1)

• err (76-76)

internal/exec/vendor_component_utils.go (2) (2)

• componentConfig (64-64)
• ReadAndProcessComponentVendorConfigFile (58-105)

internal/exec/validate_stacks.go (1)

pkg/schema/schema.go (2) (2)

• SchemaRegistry (607-610)
• AtmosConfiguration (13-44)

internal/exec/go_getter_utils.go (2)

pkg/schema/schema.go (2) (2)

• AtmosConfiguration (13-44)
• Settings (694-698)

pkg/utils/url_utils.go (1) (1)

• MaskBasicAuth (9-20)

pkg/config/utils.go (1)

pkg/schema/schema.go (12) (12)

• ResourcePath (603-605)
• SchemaRegistry (607-610)
• AtmosConfiguration (13-44)
• ConfigAndStacksInfo (356-420)
• Components (272-275)
• Terraform (247-257)
• Command (505-515)
• Helmfile (263-270)
• Stacks (277-283)
• Workflows (285-288)
• Logs (290-293)
• Settings (694-698)

internal/exec/oci_utils.go (6)

internal/exec/aws_eks_update_kubeconfig.go (3) (3)

• atmosConfig (126-126)
• err (127-127)
• err (155-155)

cmd/root.go (2) (2)

• atmosConfig (26-26)
• err (199-199)

pkg/schema/schema.go (1) (1)

• AtmosConfiguration (13-44)

internal/exec/atlantis_generate_repo_config.go (1) (1)

• err (154-154)

internal/exec/file_utils.go (1) (1)

• removeTempDir (16-21)

internal/exec/tar_utils.go (1) (1)

• extractTarball (18-21)

internal/exec/vendor.go (2)

pkg/config/config.go (3) (3)

• err (126-126)
• atmosConfig (125-125)
• InitCliConfig (117-394)

internal/exec/vendor_utils.go (4) (4)

• err (185-185)
• vendorConfig (66-66)
• vendorConfig (146-146)
• ReadAndProcessVendorConfigFile (61-89)

internal/exec/vendor_utils.go (1)

pkg/schema/schema.go (5) (5)

• AtmosVendorSource (720-729)
• Version (317-319)
• AtmosVendorSpec (731-734)
• AtmosVendorConfig (741-746)
• Vendor (748-752)

internal/exec/vendor_component_utils.go (4)

pkg/schema/schema.go (5) (5)

• AtmosConfiguration (13-44)
• Components (272-275)
• VendorComponentSpec (486-489)
• Version (317-319)
• VendorComponentMixins (479-484)

internal/exec/vendor_model.go (5) (5)

• executeVendorModel (98-126)
• pkgComponentVendor (71-82)
• pkgType (23-23)
• p (47-53)
• p (356-387)

internal/exec/go_getter_utils.go (1) (1)

• GoGetterGet (280-318)

internal/exec/oci_utils.go (1) (1)

• processOciImage (32-76)

🪛 golangci-lint (1.64.8)

internal/exec/error.go

[error] 9-9: const progressWidth is unused

(unused)

---

[error] 10-10: const getterTimeout is unused

(unused)

---

[error] 11-11: const componentTempDirPermissions is unused

(unused)

---

[error] 12-12: const wrapErrFmtWithDetails is unused

(unused)

---

[error] 13-13: const timeFormatBase is unused

(unused)

⏰ Context from checks skipped due to timeout of 90000ms (1)

• GitHub Check: Summary

🔇 Additional comments (128)

pkg/config/utils.go (10)

329-356: Updated environment variable handling with improved logging.

The transition from u.LogDebug to the structured log.Debug method from Charmbracelet provides better logging context by explicitly naming environment variables. The schema configuration is now properly set using a map approach with explicit types.

---

403-432: Good refactoring of command line argument processing.

Breaking down the large function into smaller, focused helper functions improves maintainability and readability. The function signature change to pass configAndStacksInfo by reference is a good optimization for memory usage.

---

434-440: Base path configuration looks good.

Simple helper function with proper error handling pattern.

---

442-452: Terraform configuration handling looks good.

Clear logging with context improves debuggability.

---

454-464: Helmfile configuration handling looks good.

Follows consistent pattern with the other configuration helpers.

---

466-472: Stacks configuration handling looks good.

Maintains the consistent pattern established in other helper functions.

---

474-504: Feature flags configuration handling looks good.

Proper error handling for boolean parsing and consistent logging pattern.

---

506-526: Schema directories configuration looks good.

The use of explicit schema structs improves type safety and clarity.

---

528-542: Logging configuration handling looks good.

Validates log level before setting it, showing good defensive programming.

---

544-551: Settings configuration handling looks good.

Follows consistent pattern with other configuration helpers.

pkg/vender/component_vendor_test.go (4)

33-33: Passing atmosConfig by reference improves consistency.

Changed to pass the configuration object by reference instead of by value, consistent with the updates in other parts of the codebase.

---

38-38: Passing componentConfig.Spec by reference maintains consistency.

Now passing the component spec by reference which aligns with the parameter changes in the function definition.

---

53-53: Consistent reference passing for second test case.

Ensures consistency in how configuration is passed between functions.

---

58-58: Consistent reference passing for component execution.

Maintains the pattern of passing configuration objects by reference.

pkg/vender/vendor_config_test.go (5)

55-55: Updated to pass atmosConfig by reference.

Changed to use the address-of operator (&) which matches the updated function signature.

---

92-92: Consistent parameter passing in component config test.

Ensures that the atmosConfig is passed by reference throughout the test.

---

106-106: Reference passing in no-config test case.

Maintains consistency with the updated function signatures.

---

111-111: Reference passing in component test.

Follows the pattern established across the codebase of passing configuration by reference.

---

134-134: Consistent reference passing in final test case.

Ensures all test cases follow the same pattern of passing configuration by reference.

internal/exec/validate_component.go (3)

209-209: Updated schema path access to use GetResourcePath method.

Now uses the GetResourcePath method to retrieve the base path for JSON schema validation, which is more flexible than direct struct field access.

---

213-213: Consistent schema path access for OPA validation.

Uses the same GetResourcePath method for OPA schema paths, maintaining consistency with the JSON schema case.

---

240-240: Updated module paths to use GetResourcePath for OPA.

Ensures consistent path resolution for OPA module paths using the GetResourcePath method.

internal/exec/vendor_utils_test.go (13)

4-4: Importing errors package.

This import is standard for error handling. No issues here.

---

10-10: Adding testify/require dependency.

This is a good practice for writing clean and readable tests.

---

23-27: Cleaning up environment variables after tests.

This deferred cleanup is a solid move to ensure test isolation and avoid side effects.

---

65-65: Using the pointer version of ReadAndProcessVendorConfigFile.

Pointer usage promotes consistency with other signatures. Looks fine.

---

69-73: Added documentation for TestExecuteVendorPull.

Clear docstrings are helpful to understand test coverage and intentions.

---

74-85: Unset environment variables conditionally.

It's good to ensure the environment is clean before proceeding.

---

86-97: Capturing and restoring working directory.

Preserving the original working directory is a neat solution to avoid residual changes after tests.

---

99-104: Switching to a target directory.

This step is straightforward and effectively sets up the test environment.

---

105-114: Flag initialization and command execution.

Registering flags and invoking the vendor pull command is straightforward.

---

120-147: List of expected files.

Defining file paths in one place is helpful for clarity.

---

148-152: Verifying file existence and deleting state files.

Test calls are consistent and help validate vendoring outcomes.

---

176-185: verifyFileExists function.

Implementation is straightforward, returning early if any file is missing.

---

187-193: deleteStateFiles function.

Simple approach for cleanup of generated vendor files.

pkg/config/config.go (4)

10-10: Imported strings package.

This enables efficient string handling below.

---

83-90: Converting Schemas to a map.

This flexible map-based schema definition better accommodates multiple registry entries.

---

263-263: Calling atmosConfig.ProcessSchemas().

Ensures schemas are processed immediately after unmarshal, aiding consistency.

---

396-412: setLogConfig function.

Adjusts log settings from environment variables and flags. The logic is straightforward and consistent with typical override patterns.

internal/exec/validate_stacks.go (13)

14-14: Introducing structured logging.

Using charmbracelet/log for better clarity.

---

58-60: Overriding the 'atmos' schema registry.

This helps maintain user-provided schema paths at runtime.

---

108-109: Beginning switch statement for manifest checks.

Improves clarity over multiple if-else conditions.

---

116-117: Assigning the embedded schema path.

A fallback path for the manifest is a handy approach.

---

118-118: Logging default schema usage.

Good trace message for diagnosing schema resolution.

---

119-119: Direct file existence check.

Simple approach to confirm the user-specified manifest.

---

120-120: Relative path check.

Verifies the combined path from base config.

---

122-122: Handling remote schema with isURL.

Ensures we fetch from the network if specified.

---

127-127: Error fallback for missing schemas.

Direct feedback for misconfigurations or typos.

---

133-133: User instructions in error.

Guidance on possible ways to configure the schema manifest.

---

390-390: downloadSchemaFromURL signature uses pointer.

Maintains consistency with the rest of the codebase's pointer usage.

---

391-391: Extracting manifestURL from config.

In line with the central approach to read from the 'atmos' registry.

---

392-392: manifestSchema retrieval from atmosConfig.

Follows the standardized pattern for fetching registry entries.

internal/exec/tar_utils.go (11)

11-12: Adding structured logger and errors.

This aligns with the rest of the logging approach and standard error usage.

---

15-15: Defining ErrInvalidFilePath.

Helpful for identifying unsafe extractions.

---

17-21: Refactoring extractTarball signature.

Accepting an io.Reader broadens input options (e.g., streaming data).

---

23-26: untar routine.

Centralizing tar extraction logic for clarity.

---

28-35: Reading tar headers.

Clear error handling with structured logging. This approach handles partial extractions gracefully.

---

37-38: Detecting potential directory traversal.

Skipping suspicious paths helps mitigate security risks.

---

40-43: Delegating to processTarHeader.

Separating logic into functions enhances readability.

---

45-45: Returning upon successful untar.

No major concerns.

---

49-70: processTarHeader with security checks.

Validates and cleans file paths, skipping or halting on invalid attempts.

---

72-78: createDirectory function.

Minimal approach to ensure directories exist.

---

80-105: createFileFromTar handles file creation and permissions.

Permissions are adjusted safely. Consider logging if setuid/setgid bits are encountered and stripped. Otherwise, all good.

internal/exec/error.go (1)

9-14: Review the unused constants.

Static analysis flags these constants (progressWidth, getterTimeout, componentTempDirPermissions, wrapErrFmtWithDetails, timeFormatBase) as unused. Confirm whether you still need them or remove them if they are no longer relevant.

🧰 Tools

🪛 golangci-lint (1.64.8)

[error] 9-9: const progressWidth is unused

(unused)

---

[error] 10-10: const getterTimeout is unused

(unused)

---

[error] 11-11: const componentTempDirPermissions is unused

(unused)

---

[error] 12-12: const wrapErrFmtWithDetails is unused

(unused)

---

[error] 13-13: const timeFormatBase is unused

(unused)

internal/exec/oci_utils.go (2)

24-25: Error declaration looks good.

Defining ErrNoLayers here improves code readability and consistency for layer-check scenarios.

---

56-57: Confirm artifact mismatch handling.

checkArtifactType logs a warning but does not return an error on mismatch. Verify if the code should fail when the artifact type deviates from the expected value.

internal/exec/go_getter_utils.go (3)

73-123: Confirm shallow clone behavior.

The code automatically sets depth=1 unless an explicit value is present. Ensure this does not break workflows needing a full clone.

---

257-265: Validate subdir adjustment logic.

adjustSubdir appends //. if no subdirectory is detected. Confirm there are no edge cases where multiple slashes or alternative paths might bypass or misapply this logic.

---

335-349: Refine symlink removal to handle nested structures.

Removing symlinks in nested directories may require skipping subdirectories once a symlink is removed. Consider using a deeper check or WalkDir.

 func removeSymlinks(root string) error {
-	return filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
+	return filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error {
 		if err != nil {
 			return err
 		}
-		if info.Mode()&os.ModeSymlink != 0 {
+		if d.Type()&os.ModeSymlink != 0 {
 			log.Debug("Removing symlink", "path", path)
 			return os.Remove(path)
 		}
 		return nil
 	})
 }

internal/exec/vendor_utils.go (12)

22-40: StderrLogger initialization and new error variables look good.

These additions provide more descriptive error handling and a clean separation of stderr logging. This approach helps keep stdout streams free of diagnostic messages, which is a valuable design choice for vendoring processes.

---

42-50: Struct naming clarifies usage.

Using processTargetsParams and executeVendorOptions effectively encapsulates parameters for source processing and vendor execution. This fosters modular design and readability.

---

113-142: Double-star glob usage is correct.

Here, doublestar.Glob is a good choice for matching files in directories. The check for empty matches and sorting them ensures stable ordering for merges. Keep this approach for consistency with your vendoring logic.

---

145-181: Avoid duplicate components.

mergeVendorConfigFiles checks for duplicates by component name, throwing early errors if encountered. This is good practice. Just ensure you also consider or handle case sensitivity if that emerges as a future requirement.

---

222-254: Tag-based filtering logic is sound.

validateTagsAndComponents ensures that only specified tags or components pass through. This effectively guards against extraneous sources. Be sure to keep the intersection logic thoroughly tested for new tag behavior.

---

323-348: processTargets function is concise and readable.

Shared logic for creating the pkgAtmosVendor struct is well-organized. The parameter struct fosters maintainability, and the function adheres to single-responsibility nicely.

---

350-396: Recursively processing imports is well-structured.

processVendorImports helpfully prevents circular imports with the allImports check. Good use of error messages to clarify the cause of duplicate imports.

---

406-418: Guard against missing fields.

validateSourceFields checks for required fields: Source and Targets. The explicit error strings help. Consider verifying even more potential user mistakes, like empty strings in component.

---

427-461: Solid path traversal checks.

determineSourceType carefully handles .. in URIs and ensures local paths are joined properly. This helps mitigate path traversal risks. Keep verifying that other allowable schemes are handled as expected.

---

481-517: Skip function usage is constructive.

Using generateSkipFunction to unify exclusion/inclusion logic is wise. This ensures consistent behavior across vendoring operations. Just be cautious when patterns overlap in both included and excluded sets.

---

523-535: Exclusion pattern logic is correct.

shouldExcludeFile checks each excluded path pattern with the helpful debug statement. The code is straightforward. If additional patterns or advanced matching are introduced, ensure performance remains acceptable.

---

537-559: Inclusion logic complements exclusion checks.

shouldIncludeFile ensures that if there are any included_paths, a file must match at least one. The fallback of including all if no patterns are specified is a sensible default.

internal/exec/vendor_model.go (13)

14-14: New logger import is consistent with existing usage.

Importing charmbracelet/log aligns well with the chosen logging strategy. It helps ensure a uniform logging style.

---

26-40: Readable constants and styling blocks.

Defining tempDirPermissions, progressBarWidth, and maxWidth as constants clarifies usage. The style variables (checkMark, xMark, etc.) keep UI code centralized and consistent.

---

42-46: installedPkgMsg transition is straightforward.

Encapsulating installation result details in installedPkgMsg fosters easier message handling. The approach to pass package name and error is clear and maintainable.

---

98-126: Consider a fallback logging statement for the TTY check.

executeVendorModel handles TTY vs non-TTY scenarios gracefully, defaulting to a fallback mode. This keeps normal usage interactive and still logs essential info if TTY is unavailable.

---

128-178: Well-structured generic factory.

newModelVendor uses a type parameter to distinguish pkgComponentVendor from pkgAtmosVendor. This approach is a strong example of generics usage for code reuse, simplifying the creation of unified models.

---

192-194: Window size logic is safe.

Ensuring m.width never exceeds maxWidth helps preserve layout consistency when rendering progress in narrower terminals. This is a handy boundary check.

---

197-199: Non-blocking exit with ctrl+c.

handleKeyPress gracefully quits on typical terminal exit keys. The code is easy to follow and aligns with standard TUI patterns.

---

225-269: Comprehensive install result handling.

handleInstalledPkgMsg increments failedPkg if errors occur. The logic to print mark, maintain progress, and decide next steps is a well-defined workflow. Good job ensuring logs are used if TTY is absent.

---

271-292: logNonNTYFinalStatus is a useful helper.

It avoids repetition in non-TTY contexts, summarizing final results succinctly. Keep an eye on the formatting options if users desire more verbose logs.

---

356-387: Installer function covers all package types.

(*pkgAtmosVendor).installer does a nice job branching by pkgType. The go-getter usage for remote packages is clearly separated from local copies. If more package types appear, consider a plugin approach to keep this code lean.

---

398-411: Secure temporary directory creation.

createTempDir with os.MkdirTemp is best practice, and applying restricted permissions is an important security measure. Great job.

---

413-418: newInstallError helps unify error reporting.

Keeping error creation in a single place promotes consistency for failed installs, including naming the package in the error message.

---

420-437: Overall robust fallback in ExecuteInstall.

If neither atmosPackage nor componentPackage is set, the code returns a clear error. This guards against unexpected usage and keeps the contract explicit.

pkg/schema/schema.go (8)

4-5: New imports are consistent with existing structure.

Using encoding/json and gopkg.in/yaml.v3 is aligned with the approach for schema marshalling/unmarshalling. This unifies standard Go libraries for JSON tasks.

Also applies to: 7-7

---

12-44: Expanded AtmosConfiguration fields facilitate flexible stack definitions.

The introduction and reorganization of fields, alongside the Schemas map, allow more dynamic CLI configuration. Ensure that each new field is properly tested in typical user scenarios.

---

46-57: GetSchemaRegistry method ensures safe retrieval.

This avoids panics by returning a zero struct if not found. Good pattern for user-friendly API calls.

---

59-70: Graceful fallback in GetResourcePath.

Similar to GetSchemaRegistry, returning an empty ResourcePath if not found is consistent. This fosters error-free usage in dependent code.

---

72-118: UnmarshalYAML carefully handles multiple structure possibilities.

The fallback to storing raw YAML node is an inclusive approach. This ensures minimal data loss if certain structures don't match strongly typed fields.

---

120-128: ProcessSchemas systematically processes known keys.

Separating resource-based keys (cue, opa, jsonschema) from manifest-based keys keeps logic simpler and maintainable. This is a clean design choice.

---

148-165: processResourceSchema also reuses JSON marshalling.

Consistent approach with processManifestSchemas. Good to keep resource path logic separate from manifest logic for clarity.

---

603-611: Schema types unify resource references.

ResourcePath and SchemaRegistry add structure around different schema styles. If owners want to store more metadata, these types are well-placed for extension in the future.

internal/exec/vendor.go (9)

5-9: Imports look concise.

All newly introduced imports appear broadly relevant to the vendoring logic (e.g. strings, errors, cobra, pflag, etc.). This is coherent and doesn't present immediate concerns.

Also applies to: 12-13

---

15-21: Clear and descriptive error grouping.

Declaring these error variables here facilitates maintainability and provides tailored messages for vendor operations.

---

24-25: Minimal function wrapper.

The function simply delegates to ExecuteVendorPullCommand, which is good for clarity and discoverability.

---

30-32: Gracefully handling not implemented command.

Returning a structured error here helps communicate the missing functionality.

---

34-41: Well-defined flags struct.

Encapsulating parameters into a VendorFlags struct improves code organization and clarity for command parsing.

---

43-72: Robust command execution flow.

1. The function integrates CLI configuration, parses flags, and handles distinct scenarios (component vs. stack vs. config file).
2. Very readable, but ensure consistent test coverage across paths (e.g., stack usage).

Would you like me to provide a script that checks references to ExecuteVendorPullCommand to confirm that all code paths are tested and no extraneous calls exist?

---

115-121: Thoughtful fallback for 'Everything' flag.

Automatically setting Everything if no other flags were provided is a nice convenience.

---

123-137: Flag validation is straightforward.

Checks are well-ordered and produce clear error messages that guide the user.

---

174-196: Single-entry function for component vendoring.

Pulling config and then delegating to component vendoring flow is cohesive. Ensure you handle edge cases when the folder or config is missing.

internal/exec/vendor_component_utils.go (17)

13-14: Improved error definitions.

Centralizing error variables (e.g., ErrMissingMixinURI, ErrFolderNotFound, etc.) clarifies failure modes for mixins, directories, etc.

Also applies to: 18-19, 29-41

---

44-55: findComponentConfigFile: descriptive error usage.

Returning a combined error with contextual path fosters debugging clarity.

---

57-72: Component vendoring config reading.

The switch on componentType is straightforward, though consider gracefully handling future expansions for additional component types.

Suggest verifying that all references to “helmfile” and “terraform” in your codebase are enumerated. Would you like a script to confirm that no references to other component types exist?

---

82-83: Directory existence check.

Clear error if the target directory is missing. This is a good developer experience.

---

100-101: Kind validation for vendor config.

Strictly checking for "ComponentVendorConfig" guards against accidental usage of the wrong file type.

---

107-115: Descriptive documentation for internal vendor functions.

The docstrings provide a thorough overview of supported protocols and relevant references. This fosters maintainability.

---

125-130: Configurable copy options.

Your usage of Skip and the skip function approach is flexible for future expansions.

---

158-181: Skip function design.

1. The logic covers .git directories, excluded paths, and included paths, combining them with clarity.
2. Good usage of double-star and ignoring matched patterns.

---

184-197: Pattern matching for excludes.

Your design is consistent with typical glob usage, returning early upon a single match.

---

199-220: Pattern matching for includes.

1. Checking any match to “include” a file is intuitive.
2. The debug logs are descriptive.

---

223-283: Component vendoring logic.

Process for building a list of pkgComponentVendor items and executing them is well-structured. This promotes a flexible pipeline for main component + mixins.

---

285-309: handleLocalFileScheme logic.

Falling back to a local path if it can be resolved is helpful. Clean path usage prevents confusion on different OS.

---

311-366: Mixin processing.

1. The function gracefully checks for essential fields (Uri, Filename).
2. Good approach skipping mixin installation when a local file is discovered.

---

368-384: Templated URI approach.

Dynamic building of mixin URIs is flexible. Minimal error references ensure clarity.

---

386-427: Download flow with TUI state handling.

The approach to handle both IsComponent and IsMixins is well-structured, returning an appropriate message.

---

429-467: Install component sequence.

Temporary directory usage is practical for clean file operations. The fallback to GoGetterGet or processOciImage is consistent.

---

469-489: Local system handling.

This function clearly identifies local file copying while preserving symlinks.