Introduce Atmos YAML functions #810
Conversation
# Conflicts: # website/docs/integrations/github-actions/setup-atmos.mdx
# Conflicts: # go.mod # go.sum
# Conflicts: # examples/quick-start-advanced/Dockerfile # go.mod # go.sum # internal/exec/help.go # website/docs/integrations/atlantis.mdx
# Conflicts: # examples/quick-start-advanced/Dockerfile # website/docs/integrations/atlantis.mdx
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (5)
website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx (3)
23-23: Consider using typographic quotes for better readability.Replace straight quotes with typographic quotes for better documentation aesthetics.
-Explicit tags are denoted by the exclamation point (__"!"__) symbol. +Explicit tags are denoted by the exclamation point (__"!"__) symbol.🧰 Tools
🪛 LanguageTool
[typographical] ~23-~23: Consider using a typographic close quote here.
Context: ...are denoted by the exclamation point ("!") symbol. Atmos detects the tags in ...(EN_QUOTES)
[typographical] ~23-~23: Consider using a typographic close quote here.
Context: ...e denoted by the exclamation point ("!") symbol. Atmos detects the tags in th...(EN_QUOTES)
34-44: Consider adding simple examples for each use case.While the descriptions are clear, adding a minimal example for each function would help users quickly understand their basic usage.
🧰 Tools
🪛 LanguageTool
[style] ~37-~37: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...YAML document, whether generated by theatmos.Componenttemplate function or any Go template. ...(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~42-~42: Using many exclamation marks might seem excessive (in this case: 4 exclamation marks for a text that’s 2361 characters long)
Context: ...ctions in Atmos stack manifests - The [!terraform.output](/core-concepts/stac...(EN_EXCESSIVE_EXCLAMATION)
52-88: Consider adding error handling examples.The examples demonstrate successful scenarios, but it would be valuable to show how to handle common errors:
- Component not found
- Output not available
- Permission issues
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (2)
132-137: Consider improving the Important tip formatting.The tip section could be more readable with better spacing and formatting.
-:::tip Important - By using the `printf "%s-%s-%s"` function, you are constructing stack names using the stack context variables/identifiers. - - For more information on Atmos stack names and how to define them, refer to `stacks.name_pattern` and `stacks.name_template` - sections in [`atmos.yaml` CLI config file](/cli/configuration/#stacks) -::: +:::tip Important +By using the `printf "%s-%s-%s"` function, you are constructing stack names using the stack context variables/identifiers. + +For more information on Atmos stack names and how to define them, refer to `stacks.name_pattern` and `stacks.name_template` +sections in [`atmos.yaml` CLI config file](/cli/configuration/#stacks) +:::🧰 Tools
🪛 LanguageTool
[typographical] ~133-~133: Consider using a typographic opening quote here.
Context: ... ``` :::tip Important By using theprintf "%s-%s-%s"function, you are constructin...(EN_QUOTES)
176-187: Add missing punctuation in recommendations.Each recommendation should end with a period for consistency.
-Using `!terraform.output` with secrets will leak secrets to standard output (stdout) in any commands that describe stacks or components +Using `!terraform.output` with secrets will leak secrets to standard output (stdout) in any commands that describe stacks or components. -If you have a narrow permissions scope (e.g. for `dev`) and use [`atmos describe affected`](/cli/commands/describe/affected) with stacks that reference production, it will fail +If you have a narrow permissions scope (e.g., for `dev`) and use [`atmos describe affected`](/cli/commands/describe/affected) with stacks that reference production, it will fail. -Overuse of the function within a stack for multiple components can significantly reduce performance +Overuse of the function within a stack for multiple components can significantly reduce performance. -Consider DR implications if using it across regions +Consider DR implications if using it across regions. -Consider cold-start implications, when the dependent component is not yet provisioned, and `terraform output` will fail +Consider cold-start implications, when the dependent component is not yet provisioned, and `terraform output` will fail.🧰 Tools
🪛 LanguageTool
[style] ~178-~178: Using many exclamation marks might seem excessive (in this case: 11 exclamation marks for a text that’s 4099 characters long)
Context: ...describe stacks or components - Using!terraform.outputwith [`atmos describe ...(EN_EXCESSIVE_EXCLAMATION)
[style] ~180-~180: A comma is missing here.
Context: ...If you have a narrow permissions scope (e.g. fordev) and use [`atmos describe aff...(EG_NO_COMMA)
[grammar] ~181-~181: Please add a punctuation mark at the end of paragraph.
Context: ...acks that reference production, it will fail - Overuse of the function within a st...(PUNCTUATION_PARAGRAPH_END)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (2)
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx(1 hunks)website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx(1 hunks)
🧰 Additional context used
📓 Learnings (2)
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (2)
Learnt from: aknysh
PR: cloudposse/atmos#810
File: website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx:0-0
Timestamp: 2024-12-03T04:01:16.446Z
Learning: In the `terraform.output.mdx` documentation file (`website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx`), the cache invalidation and cache scope behavior for the `!terraform.output` function are already described.
Learnt from: aknysh
PR: cloudposse/atmos#810
File: website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx:104-110
Timestamp: 2024-12-03T03:49:30.395Z
Learning: In the documentation for `!terraform.output`, warnings about template variable availability are already covered in other sections, so no need to suggest adding them here.
website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx (1)
Learnt from: aknysh
PR: cloudposse/atmos#810
File: examples/tests/stacks/catalog/terraform/template-functions-test2/defaults.yaml:28-32
Timestamp: 2024-12-01T00:33:20.298Z
Learning: In `examples/tests/stacks/catalog/terraform/template-functions-test2/defaults.yaml`, `!exec atmos terraform output` is used in examples to demonstrate its usage, even though `!terraform.output` is the recommended approach according to the documentation.
🪛 LanguageTool
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx
[typographical] ~133-~133: Consider using a typographic opening quote here.
Context: ... ``` :::tip Important By using the printf "%s-%s-%s" function, you are constructin...
(EN_QUOTES)
[style] ~144-~144: A comma is missing here.
Context: ...che is per Atmos CLI command execution, e.g. each new execution of a command like `a...
(EG_NO_COMMA)
[style] ~178-~178: Using many exclamation marks might seem excessive (in this case: 11 exclamation marks for a text that’s 4099 characters long)
Context: ...describe stacks or components - Using !terraform.output with [`atmos describe ...
(EN_EXCESSIVE_EXCLAMATION)
[style] ~180-~180: A comma is missing here.
Context: ...If you have a narrow permissions scope (e.g. for dev) and use [`atmos describe aff...
(EG_NO_COMMA)
[grammar] ~181-~181: Please add a punctuation mark at the end of paragraph.
Context: ...acks that reference production, it will fail - Overuse of the function within a st...
(PUNCTUATION_PARAGRAPH_END)
website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx
[typographical] ~23-~23: Consider using a typographic close quote here.
Context: ...are denoted by the exclamation point ("!") symbol. Atmos detects the tags in ...
(EN_QUOTES)
[typographical] ~23-~23: Consider using a typographic close quote here.
Context: ...e denoted by the exclamation point ("!") symbol. Atmos detects the tags in th...
(EN_QUOTES)
[style] ~37-~37: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...YAML document, whether generated by the atmos.Component template function or any Go template. ...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~42-~42: Using many exclamation marks might seem excessive (in this case: 4 exclamation marks for a text that’s 2361 characters long)
Context: ...ctions in Atmos stack manifests - The [!terraform.output](/core-concepts/stac...
(EN_EXCESSIVE_EXCLAMATION)
🔇 Additional comments (1)
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (1)
1-39: Documentation structure and content look great!
The introduction, usage, and arguments sections are well-organized and provide clear information.
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx
Outdated
Show resolved
Hide resolved
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx
Outdated
Show resolved
Hide resolved
…put.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]>
…put.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Outside diff range and nitpick comments (2)
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (2)
21-32: Consider adding validation details for arguments.The arguments section would benefit from additional information about:
- Required vs optional arguments
- Validation rules for each argument (e.g., allowed characters, length limits)
- Error messages users might encounter with invalid arguments
144-144: Add missing comma after "e.g."Add a comma after "e.g." to follow proper punctuation rules.
-The cache is per Atmos CLI command execution, e.g. each new execution +The cache is per Atmos CLI command execution, e.g., each new execution🧰 Tools
🪛 LanguageTool
[style] ~144-~144: A comma is missing here.
Context: ...che is per Atmos CLI command execution, e.g. each new execution of a command like `a...(EG_NO_COMMA)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (2)
website/docs/core-concepts/stacks/templates/templates.mdx(1 hunks)website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- website/docs/core-concepts/stacks/templates/templates.mdx
🧰 Additional context used
📓 Learnings (1)
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (2)
Learnt from: aknysh
PR: cloudposse/atmos#810
File: website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx:0-0
Timestamp: 2024-12-03T04:01:16.446Z
Learning: In the `terraform.output.mdx` documentation file (`website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx`), the cache invalidation and cache scope behavior for the `!terraform.output` function are already described.
Learnt from: aknysh
PR: cloudposse/atmos#810
File: website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx:104-110
Timestamp: 2024-12-03T03:49:30.395Z
Learning: In the documentation for `!terraform.output`, warnings about template variable availability are already covered in other sections, so no need to suggest adding them here.
🪛 LanguageTool
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx
[typographical] ~133-~133: Consider using a typographic opening quote here.
Context: ... ``` :::tip Important By using the printf "%s-%s-%s" function, you are constructin...
(EN_QUOTES)
[style] ~144-~144: A comma is missing here.
Context: ...che is per Atmos CLI command execution, e.g. each new execution of a command like `a...
(EG_NO_COMMA)
[style] ~178-~178: Using many exclamation marks might seem excessive (in this case: 11 exclamation marks for a text that’s 4081 characters long)
Context: ...e stacks or components. - When using !terraform.output with [`atmos describe ...
(EN_EXCESSIVE_EXCLAMATION)
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (3)
website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx (3)
14-32: Consider using typographic quotes for better readability.Replace the straight quotes with typographic quotes in the following line:
-Explicit tags are denoted by the exclamation point (__"!"__) symbol. +Explicit tags are denoted by the exclamation point (__"!"__) symbol.🧰 Tools
🪛 LanguageTool
[typographical] ~23-~23: Consider using a typographic close quote here.
Context: ...are denoted by the exclamation point ("!") symbol. Atmos detects the tags in ...(EN_QUOTES)
[typographical] ~23-~23: Consider using a typographic close quote here.
Context: ...e denoted by the exclamation point ("!") symbol. Atmos detects the tags in th...(EN_QUOTES)
34-44: Clarify the preferred approach for terraform outputs.The documentation should explicitly mention that
!terraform.outputis the recommended approach for accessing terraform outputs, even though!exec atmos terraform outputis shown in examples.🧰 Tools
🪛 LanguageTool
[style] ~37-~37: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...YAML document, whether generated by theatmos.Componenttemplate function or any Go template. ...(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~42-~42: Using many exclamation marks might seem excessive (in this case: 4 exclamation marks for a text that’s 2490 characters long)
Context: ...ctions in Atmos stack manifests - The [!terraform.output](/core-concepts/stac...(EN_EXCESSIVE_EXCLAMATION)
68-110: Consider updating examples to align with best practices.While the examples are comprehensive, consider:
- Replacing
!exec atmos terraform outputexamples (lines 90-96) with!terraform.outputfor consistency with recommended practices- Adding comments to explain when
!execis more appropriate than!terraform.output
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (1)
website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx(1 hunks)
🧰 Additional context used
📓 Learnings (1)
website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx (1)
Learnt from: aknysh
PR: cloudposse/atmos#810
File: examples/tests/stacks/catalog/terraform/template-functions-test2/defaults.yaml:28-32
Timestamp: 2024-12-01T00:33:20.298Z
Learning: In `examples/tests/stacks/catalog/terraform/template-functions-test2/defaults.yaml`, `!exec atmos terraform output` is used in examples to demonstrate its usage, even though `!terraform.output` is the recommended approach according to the documentation.
🪛 LanguageTool
website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx
[typographical] ~23-~23: Consider using a typographic close quote here.
Context: ...are denoted by the exclamation point ("!") symbol. Atmos detects the tags in ...
(EN_QUOTES)
[typographical] ~23-~23: Consider using a typographic close quote here.
Context: ...e denoted by the exclamation point ("!") symbol. Atmos detects the tags in th...
(EN_QUOTES)
[style] ~37-~37: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...YAML document, whether generated by the atmos.Component template function or any Go template. ...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~42-~42: Using many exclamation marks might seem excessive (in this case: 4 exclamation marks for a text that’s 2490 characters long)
Context: ...ctions in Atmos stack manifests - The [!terraform.output](/core-concepts/stac...
(EN_EXCESSIVE_EXCLAMATION)
🔇 Additional comments (3)
website/docs/core-concepts/stacks/yaml-functions/yaml-functions.mdx (3)
1-13: LGTM! Frontmatter and imports are well-structured.
The metadata and component imports follow MDX best practices.
46-67: LGTM! Clear explanation of processing order and supported sections.
The documentation effectively communicates how templates and YAML functions work together and provides a comprehensive list of supported sections.
112-116: LGTM! Good introduction to native functions.
The section effectively sets up the detailed documentation that follows.
…put.mdx Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (3)
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (3)
23-32: Consider enhancing argument descriptions with example values.While the arguments are well-documented, adding example values would make it even clearer:
<dl> <dt>`component`</dt> - <dd>Atmos component name</dd> + <dd>Atmos component name (e.g., `vpc`, `eks`, `rds`)</dd> <dt>`stack`</dt> - <dd>Atmos stack name</dd> + <dd>Atmos stack name (e.g., `plat-ue2-dev`, `plat-ue2-prod`)</dd> <dt>`output`</dt> - <dd>Terraform output</dd> + <dd>Terraform output (e.g., `vpc_id`, `subnet_ids`, `security_group_id`)</dd> </dl>
43-57: Consider adding example output formats.The examples show different output types but could be more helpful with sample outputs:
components: terraform: my_lambda_component: vars: vpc_config: # Output of type string - security_group_id: !terraform.output security-group/lambda {{ .stack }} id + # Returns: "sg-1234567890abcdef0" + security_group_id: !terraform.output security-group/lambda {{ .stack }} id # Output of type list - subnet_ids: !terraform.output vpc {{ .stack }} private_subnet_ids + # Returns: ["subnet-123...", "subnet-456..."] + subnet_ids: !terraform.output vpc {{ .stack }} private_subnet_ids # Output of type map - config_map: !terraform.output config {{ .stack }} config_map + # Returns: {"key1": "value1", "key2": "value2"} + config_map: !terraform.output config {{ .stack }} config_map
61-63: Consider adding a note about stack name validation.It would be helpful to mention how Atmos validates the stack names and what happens if an invalid or non-existent stack is specified.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (1)
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx(1 hunks)
🧰 Additional context used
📓 Learnings (1)
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (2)
Learnt from: aknysh
PR: cloudposse/atmos#810
File: website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx:0-0
Timestamp: 2024-12-03T04:01:16.446Z
Learning: In the `terraform.output.mdx` documentation file (`website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx`), the cache invalidation and cache scope behavior for the `!terraform.output` function are already described.
Learnt from: aknysh
PR: cloudposse/atmos#810
File: website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx:104-110
Timestamp: 2024-12-03T03:49:30.395Z
Learning: In the documentation for `!terraform.output`, warnings about template variable availability are already covered in other sections, so no need to suggest adding them here.
🪛 LanguageTool
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx
[typographical] ~133-~133: Consider using a typographic opening quote here.
Context: ... ``` :::tip Important By using the printf "%s-%s-%s" function, you are constructin...
(EN_QUOTES)
[style] ~178-~178: Using many exclamation marks might seem excessive (in this case: 11 exclamation marks for a text that’s 4082 characters long)
Context: ...e stacks or components. - When using !terraform.output with [`atmos describe ...
(EN_EXCESSIVE_EXCLAMATION)
🔇 Additional comments (4)
website/docs/core-concepts/stacks/yaml-functions/terraform.output.mdx (4)
1-13: LGTM! Clear and concise introduction.
The metadata and introduction effectively communicate the purpose of the !terraform.output function.
35-39: LGTM! Clear explanation of template processing order.
The tip effectively explains how template processing interacts with the !terraform.output function.
140-173: LGTM! Comprehensive caching explanation.
The caching behavior is well-documented with clear examples demonstrating the benefits.
174-185: LGTM! Thorough coverage of important considerations.
The considerations section effectively addresses security, performance, and operational aspects that users should be aware of.
🧰 Tools
🪛 LanguageTool
[style] ~178-~178: Using many exclamation marks might seem excessive (in this case: 11 exclamation marks for a text that’s 4082 characters long)
Context: ...e stacks or components. - When using !terraform.output with [`atmos describe ...
(EN_EXCESSIVE_EXCLAMATION)
what
why
Atmos YAML Functions are a crucial part of Atmos stack manifests. They allow you to manipulate data and perform operations on the data to customize the stack configurations.
Atmos YAML functions are based on YAML Explicit typing and user-defined Explicit Tags (local data types). Explicit tags are denoted by the exclamation point ("!") symbol. Atmos detects the tags in the stack manifests and executes the corresponding functions.
NOTE: YAML data types can be divided into three categories: core, defined, and user-defined. Core are ones expected to exist in any parser (e.g. floats, ints, strings, lists, maps). Many more advanced data types, such as binary data, are defined in the YAML specification but not supported in all implementations. Finally, YAML defines a way to extend the data type definitions locally to accommodate user-defined classes, structures, primitives, and functions.
Atmos YAML functions
The
!templateYAML function can be used to handle template outputs containing maps or lists returned from theatmos.Componenttemplate functionThe
!execYAML function is used to execute shell scripts and assign the results to the sections in Atmos stack manifestsThe
!terraform.outputYAML function is used to read the outputs (remote state) of components directly in Atmos stack manifestsNOTE; You can use Atmos Stack Manifest Templating and Atmos YAML functions in the same stack configurations at the same time. Atmos processes the templates first, and then executes the YAML functions, allowing you to provide the parameters to the YAML functions dynamically.
Examples
Summary by CodeRabbit
Release Notes
New Features
!exec,!template, and!terraform.outputfor enhanced stack manifest capabilities.Documentation Updates
atmos.Componentfunction and the new YAML functions.Dependency Updates