Support Multiple Schema Validations #1066

samtholiya · 2025-02-14T22:43:10Z

Why Draft

I want to refactor the code to be more beautiful
add some trace and debug logs.
Adding test cases
The atmos schema i downloaded from our prod website is not updated. We need to udpate that as well
add a way to validate only partial yaml schema's based on keys provided by users
Find a better UX than the bellow.

what

Added a new validator command to the CLI.
This command verifies the YAML structure of the atmos CLI.
f a schema key is present in the YAML, it validates the structure against the schema.

why

Ensures that configuration files follow the correct YAML structure.
Prevents misconfigurations and errors caused by invalid or malformed YAML.
Adds an extra layer of validation by enforcing schema compliance when applicable.
Improves reliability and usability of the CLI by catching issues early.

references

DEV-3030

internal/exec/validate_schema.go

osterman · 2025-02-15T01:19:52Z

pkg/validator/atmos_fetcher.go

+var ErrAtmosSchemaNotFound = errors.New("atmos schema not found")
+
+var atmosData = map[string][]byte{
+	"atmos://schema": []byte(atmosSchema),


Please see the ticket this is not right

pkg/validator/url_fetcher.go

mergify · 2025-02-26T21:18:47Z

💥 This pull request now has conflicts. Could you fix it @samtholiya? 🙏

…-for-vendoring-and-atmos

…ing-and-atmos' of https://github.com/cloudposse/atmos into feature/dev-3030-implement-schema-validation-for-vendoring-and-atmos

…-for-vendoring-and-atmos

mergify · 2025-02-28T22:45:35Z

Warning

This PR exceeds the recommended limit of 1,000 lines.

Large PRs are difficult to review and may be rejected due to their size.

Please verify that this PR does not address multiple issues.
Consider refactoring it into smaller, more focused PRs to facilitate a smoother review process.

…-for-vendoring-and-atmos

mergify · 2025-03-04T20:24:18Z

💥 This pull request now has conflicts. Could you fix it @samtholiya? 🙏

mergify · 2025-03-04T20:24:19Z

Important

Cloud Posse Engineering Team Review Required

This pull request modifies files that require Cloud Posse's review. Please be patient, and a core maintainer will review your changes.

To expedite this process, reach out to us on Slack in the #pr-reviews channel.

…estability-and' into feature/dev-3030-implement-schema-validation-for-vendoring-and-atmos

codecov · 2025-03-04T20:41:34Z

Codecov Report

Attention: Patch coverage is 41.11675% with 116 lines in your changes missing coverage. Please review.

Project coverage is 19.95%. Comparing base (dd3ed57) to head (bbda1d9).

Files with missing lines	Patch %	Lines
pkg/datafetcher/mock_fetch_schema.go	0.00%	30 Missing ⚠️
pkg/validator/schema_validator.go	55.35%	19 Missing and 6 partials ⚠️
cmd/validate_schema.go	16.00%	21 Missing ⚠️
pkg/validator/mock_schema_validator.go	0.00%	15 Missing ⚠️
internal/exec/validate_schema.go	60.86%	8 Missing and 1 partial ⚠️
pkg/downloader/mock_file_downloader_interface.go	0.00%	9 Missing ⚠️
pkg/datafetcher/fetch_schema.go	80.95%	3 Missing and 1 partial ⚠️
pkg/downloader/file_downloader.go	57.14%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@                                            Coverage Diff                                            @@
##           feature/dev-3056-refactor-gogetter-utility-for-better-testability-and    #1066      +/-   ##
=========================================================================================================
+ Coverage                                                                  19.72%   19.95%   +0.22%     
=========================================================================================================
  Files                                                                        174      182       +8     
  Lines                                                                      18885    19082     +197     
=========================================================================================================
+ Hits                                                                        3726     3807      +81     
- Misses                                                                     14443    14550     +107     
- Partials                                                                     716      725       +9

Flag	Coverage Δ
unittests	`19.95% <41.11%> (+0.22%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

mergify · 2025-03-04T20:53:37Z

💥 This pull request now has conflicts. Could you fix it @samtholiya? 🙏

…estability-and' into feature/dev-3030-implement-schema-validation-for-vendoring-and-atmos

samtholiya · 2025-03-14T21:29:04Z

New pr with rebased things: #1147

mergify bot added the triage Needs triage label Feb 14, 2025

osterman reviewed Feb 15, 2025

View reviewed changes

osterman changed the title ~~add validate schema~~ Support Multiple Schema Validations Feb 15, 2025

coderabbitai bot previously approved these changes Feb 18, 2025

View reviewed changes

mergify bot removed the triage Needs triage label Feb 18, 2025

samtholiya dismissed coderabbitai[bot]’s stale review via a9bb5bc February 19, 2025 20:21

mergify bot added the triage Needs triage label Feb 19, 2025

samtholiya and others added 4 commits February 25, 2025 20:48

add validate schema initial code

f03f6d4

temp commit

a8a75ee

gogetter update

bd098f6

[autofix.ci] apply automated fixes

07f768c

samtholiya force-pushed the feature/dev-3030-implement-schema-validation-for-vendoring-and-atmos branch from 250b633 to 07f768c Compare February 25, 2025 19:52

refactor the code

64e764d

mergify bot added the conflict This PR has conflicts label Feb 26, 2025

Merge branch 'main' into feature/dev-3030-implement-schema-validation…

806587b

…-for-vendoring-and-atmos

mergify bot removed the conflict This PR has conflicts label Feb 26, 2025

samtholiya and others added 7 commits February 27, 2025 09:10

Fix test case

435c4d8

[autofix.ci] apply automated fixes

7303a25

added unit testcase for validator

d79d19f

Merge branch 'feature/dev-3030-implement-schema-validation-for-vendor…

233b53f

…ing-and-atmos' of https://github.com/cloudposse/atmos into feature/dev-3030-implement-schema-validation-for-vendoring-and-atmos

[autofix.ci] apply automated fixes

56ac966

fixed golangci-lint

ffd0fb0

Merge branch 'feature/dev-3030-implement-schema-validation-for-vendor…

e9c9138

…ing-and-atmos' of https://github.com/cloudposse/atmos into feature/dev-3030-implement-schema-validation-for-vendoring-and-atmos

samtholiya force-pushed the feature/dev-3030-implement-schema-validation-for-vendoring-and-atmos branch 2 times, most recently from 4d3f699 to 248ad73 Compare February 27, 2025 09:05

fix golangci-lint

8e2067a

samtholiya force-pushed the feature/dev-3030-implement-schema-validation-for-vendoring-and-atmos branch from 248ad73 to 8e2067a Compare February 27, 2025 09:08

samtholiya and others added 2 commits February 27, 2025 23:33

used existing gogetter

bcba9fc

[autofix.ci] apply automated fixes

727affe

Merge branch 'main' into feature/dev-3030-implement-schema-validation…

bec10d6

…-for-vendoring-and-atmos

Merge branch 'main' into feature/dev-3030-implement-schema-validation…

5485e1e

…-for-vendoring-and-atmos

samtholiya changed the base branch from main to feature/dev-3056-refactor-gogetter-utility-for-better-testability-and March 4, 2025 20:23

mergify bot added conflict This PR has conflicts needs-cloudposse Needs Cloud Posse assistance labels Mar 4, 2025

Merge branch 'feature/dev-3056-refactor-gogetter-utility-for-better-t…

1b2d7a5

…estability-and' into feature/dev-3030-implement-schema-validation-for-vendoring-and-atmos

mergify bot removed the conflict This PR has conflicts label Mar 4, 2025

mergify bot added the conflict This PR has conflicts label Mar 4, 2025

cloudposse deleted a comment from github-actions bot Mar 4, 2025

coderabbitai bot approved these changes Mar 4, 2025

View reviewed changes

mergify bot removed the triage Needs triage label Mar 4, 2025

Merge branch 'feature/dev-3056-refactor-gogetter-utility-for-better-t…

c60280c

…estability-and' into feature/dev-3030-implement-schema-validation-for-vendoring-and-atmos

mergify bot removed the conflict This PR has conflicts label Mar 4, 2025

samtholiya and others added 4 commits March 5, 2025 22:55

more test cases

2dc68ad

[autofix.ci] apply automated fixes

2e34edd

Merge branch 'feature/dev-3056-refactor-gogetter-utility-for-better-t…

4f4f1fa

…estability-and' into feature/dev-3030-implement-schema-validation-for-vendoring-and-atmos

[autofix.ci] apply automated fixes

bbda1d9

samtholiya mentioned this pull request Mar 14, 2025

implement schema validation for vendoring and CLI schemas #1147

Open

samtholiya closed this Mar 14, 2025

mergify bot removed the needs-cloudposse Needs Cloud Posse assistance label Mar 14, 2025

samtholiya deleted the feature/dev-3030-implement-schema-validation-for-vendoring-and-atmos branch March 14, 2025 21:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support Multiple Schema Validations #1066

Support Multiple Schema Validations #1066

samtholiya commented Feb 14, 2025 •

edited

Loading

osterman Feb 15, 2025

mergify bot commented Feb 26, 2025

mergify bot commented Feb 28, 2025

This PR exceeds the recommended limit of 1,000 lines.

mergify bot commented Mar 4, 2025

mergify bot commented Mar 4, 2025

Cloud Posse Engineering Team Review Required

codecov bot commented Mar 4, 2025 •

edited

Loading

mergify bot commented Mar 4, 2025

samtholiya commented Mar 14, 2025

Support Multiple Schema Validations #1066

Support Multiple Schema Validations #1066

Conversation

samtholiya commented Feb 14, 2025 • edited Loading

Why Draft

what

why

references

osterman Feb 15, 2025

Choose a reason for hiding this comment

mergify bot commented Feb 26, 2025

mergify bot commented Feb 28, 2025

This PR exceeds the recommended limit of 1,000 lines.

mergify bot commented Mar 4, 2025

mergify bot commented Mar 4, 2025

Cloud Posse Engineering Team Review Required

codecov bot commented Mar 4, 2025 • edited Loading

Codecov Report

mergify bot commented Mar 4, 2025

samtholiya commented Mar 14, 2025

samtholiya commented Feb 14, 2025 •

edited

Loading

codecov bot commented Mar 4, 2025 •

edited

Loading