Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance Atmos CLI: Add Support for Custom Base Path and Config Paths #1091

Open
wants to merge 99 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
7f13ce6
refactor InitCliConfig
haitham911 Feb 17, 2025
9ed97f7
add readConfig function
haitham911 Feb 18, 2025
5389ffc
Merge branch 'main' into Viper-built-in-configuration
haitham911 Feb 18, 2025
1fe3e66
add load config
haitham911 Feb 18, 2025
1cfec36
[autofix.ci] apply automated fixes
autofix-ci[bot] Feb 18, 2025
3ad2d48
add default.go & update snapshots
haitham911 Feb 18, 2025
263d14a
resolve comments
haitham911 Feb 18, 2025
2532cfc
remove unused function processConfigFile
haitham911 Feb 19, 2025
86bf3cc
improve log
haitham911 Feb 19, 2025
9de9d7b
[autofix.ci] apply automated fixes
autofix-ci[bot] Feb 19, 2025
037560c
add imports
haitham911 Feb 21, 2025
5e134f5
update snapshots
haitham911 Feb 21, 2025
413f955
add import config test
haitham911 Feb 21, 2025
eae3f97
[autofix.ci] apply automated fixes
autofix-ci[bot] Feb 21, 2025
6c2f103
refactor LoadConfig into smaller helper functions
haitham911 Feb 21, 2025
0bc7b4a
update Valid_Log_Level snapshots
haitham911 Feb 21, 2025
f87f1e9
update log snapshots
haitham911 Feb 21, 2025
d660739
update snapshots
haitham911 Feb 22, 2025
1858c2b
update snapshots Valid_Log_Level
haitham911 Feb 22, 2025
36cd759
Merge branch 'main' into atmos-import-config
haitham911 Feb 22, 2025
daaa03c
remove //nolint:revive
haitham911 Feb 22, 2025
c0db066
Exclude any line containing "log."
haitham911 Feb 22, 2025
8920e56
fix lint revive.add-constant
haitham911 Feb 22, 2025
09fe32d
update add-constant to include "error","path"
haitham911 Feb 22, 2025
e4b6f24
improve code lint
haitham911 Feb 22, 2025
da8006a
improve log
haitham911 Feb 22, 2025
54bb994
improve log charmbracelet aliased
haitham911 Feb 23, 2025
36ab8e8
Preprocess Atmos config with custom functions
haitham911 Feb 24, 2025
e42aa94
[autofix.ci] apply automated fixes
autofix-ci[bot] Feb 24, 2025
19136b0
remove print line
haitham911 Feb 24, 2025
0c95ad9
improve logs
haitham911 Feb 24, 2025
b2669c7
refactor processScalarNode
haitham911 Feb 24, 2025
ab18aab
improve logs
haitham911 Feb 24, 2025
0a9353a
[autofix.ci] apply automated fixes
autofix-ci[bot] Feb 24, 2025
462b5bf
fix linter error
haitham911 Feb 24, 2025
55a9b05
fix linter errors
haitham911 Feb 24, 2025
115006b
update snapshots
haitham911 Feb 24, 2025
6180ff5
fix linter errors
haitham911 Feb 24, 2025
6f5d329
fix linter
haitham911 Feb 24, 2025
21ae927
add global --base-path ,--config ,--config-path
haitham911 Feb 25, 2025
982d33f
add embed atmos config
haitham911 Feb 25, 2025
d5c4373
add global --base-path ,--config ,--config-path
haitham911 Feb 25, 2025
6a09b65
enhance code
haitham911 Feb 25, 2025
0b4f71d
go format file
haitham911 Feb 25, 2025
8afa420
[autofix.ci] apply automated fixes
autofix-ci[bot] Feb 25, 2025
1cb9623
disable atmos_vendor_pull
haitham911 Feb 25, 2025
d4f7630
update snapshots Valid_Log_Level_in_Environment_Variable
haitham911 Feb 25, 2025
ea71b2f
update snapshots
haitham911 Feb 25, 2025
7ebb430
Merge branch 'atmos-import-config' into DEV-2790
haitham911 Feb 25, 2025
483782e
updates snapshots
haitham911 Feb 25, 2025
729e119
update snapshots
haitham911 Feb 25, 2025
7cda8a4
Merge branch 'main' into atmos-import-config
haitham911 Mar 4, 2025
2ee18b9
set config path dir
haitham911 Mar 4, 2025
25688c6
update snapshots
haitham911 Mar 5, 2025
351c73b
add processStackConfigs on config.go
haitham911 Mar 5, 2025
9ab9d3c
[autofix.ci] apply automated fixes
autofix-ci[bot] Mar 5, 2025
eff1fd7
fix linter errors config.go
haitham911 Mar 9, 2025
6c661ac
add nolint:gocritic for InitCliConfig
haitham911 Mar 9, 2025
3bceb0b
Merge branch 'atmos-import-config' into DEV-2790
haitham911 Mar 9, 2025
4a3b34a
Merge branch 'main' into atmos-import-config
osterman Mar 12, 2025
60bed99
fix readEnvAmosConfigPath
haitham911 Mar 12, 2025
2cdd2b9
[autofix.ci] apply automated fixes
autofix-ci[bot] Mar 12, 2025
2742bb9
improve log
haitham911 Mar 12, 2025
33f1aec
Merge branch 'main' into atmos-import-config
aknysh Mar 12, 2025
498baca
Merge branch 'main' into atmos-import-config
haitham911 Mar 12, 2025
c8accb4
fix import
haitham911 Mar 12, 2025
23423b3
Merge branch 'main' into atmos-import-config
haitham911 Mar 13, 2025
69269ff
Merge branch 'main' into atmos-import-config
osterman Mar 14, 2025
0043a45
Merge branch 'main' into atmos-import-config
haitham911 Mar 15, 2025
a7ff480
update snapshots
haitham911 Mar 16, 2025
3548783
fix linter error
haitham911 Mar 16, 2025
76b4971
Apply suggestions from code review
osterman Mar 17, 2025
a33f5ab
[autofix.ci] apply automated fixes
autofix-ci[bot] Mar 17, 2025
df9c5a6
remove log
haitham911 Mar 17, 2025
a0b0e45
update snap shots
haitham911 Mar 17, 2025
31251b1
Merge branch 'main' into atmos-import-config
haitham911 Mar 17, 2025
501a94b
add ProcessSchemas
haitham911 Mar 17, 2025
2040b9e
resolve comments
haitham911 Mar 17, 2025
f78b7b0
fix base path
haitham911 Mar 17, 2025
85e3d04
add doc
haitham911 Mar 17, 2025
c860c89
move functions to helper
haitham911 Mar 17, 2025
9ffb03d
Merge branch 'main' into atmos-import-config
haitham911 Mar 17, 2025
c936af3
update snapshots
haitham911 Mar 17, 2025
8871f80
[autofix.ci] apply automated fixes
autofix-ci[bot] Mar 17, 2025
29d78c2
update snapshot
haitham911 Mar 18, 2025
2bf442b
update snapshots
haitham911 Mar 18, 2025
4b6baad
update snapshots
haitham911 Mar 18, 2025
2bb9e3f
remove repository root configuration from CLI docs
haitham911 Mar 18, 2025
4ca552c
Merge branch 'atmos-import-config' into DEV-2790
haitham911 Mar 18, 2025
2444a40
update snapshots
haitham911 Mar 18, 2025
751240c
update doc
haitham911 Mar 18, 2025
ff38c59
update snapshot
haitham911 Mar 18, 2025
fe627ac
fix test TestExecuteVendorPullCommand
haitham911 Mar 18, 2025
4b3c1cb
fix tests TestExecuteWorkflowCmd,TestExecuteVendorPull
haitham911 Mar 18, 2025
bf346ea
add test cases
haitham911 Mar 18, 2025
9534bd4
[autofix.ci] apply automated fixes
autofix-ci[bot] Mar 18, 2025
d3195a0
Merge branch 'main' into DEV-2790
haitham911 Mar 22, 2025
ad754fe
remove comment
haitham911 Mar 27, 2025
3572a9b
Merge branch 'main' into DEV-2790
haitham911 Mar 27, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add load config
haitham911 committed Feb 18, 2025
commit 1fe3e6694fd0e78449adf23dd284eab31c5d12cb
156 changes: 2 additions & 154 deletions pkg/config/config.go
Original file line number Diff line number Diff line change
@@ -1,20 +1,15 @@
package config

import (
"bytes"
"encoding/json"
"fmt"
"os"
"path/filepath"
"runtime"

"github.com/mitchellh/go-homedir"
"github.com/pkg/errors"
"github.com/spf13/viper"

"github.com/cloudposse/atmos/internal/tui/templates"
"github.com/cloudposse/atmos/pkg/schema"
"github.com/cloudposse/atmos/pkg/ui/theme"
u "github.com/cloudposse/atmos/pkg/utils"
"github.com/cloudposse/atmos/pkg/version"
)
@@ -112,165 +107,18 @@ var (
// InitCliConfig finds and merges CLI configurations in the following order: system dir, home dir, current dir, ENV vars, command-line arguments
// https://dev.to/techschoolguru/load-config-from-file-environment-variables-in-golang-with-viper-2j2d
// https://medium.com/@bnprashanth256/reading-configuration-files-and-environment-variables-in-go-golang-c2607f912b63
func InitCliConfigOld(configAndStacksInfo schema.ConfigAndStacksInfo, processStacks bool) (schema.AtmosConfiguration, error) {
func InitCliConfig(configAndStacksInfo schema.ConfigAndStacksInfo, processStacks bool) (schema.AtmosConfiguration, error) {
// atmosConfig is loaded from the following locations (from lower to higher priority):
// system dir (`/usr/local/etc/atmos` on Linux, `%LOCALAPPDATA%/atmos` on Windows)
// home dir (~/.atmos)
// current directory
// ENV vars
// Command-line arguments

var atmosConfig schema.AtmosConfiguration
var err error
configFound := false
var found bool

v := viper.New()
v.SetConfigType("yaml")
v.SetTypeByDefaultValue(true)

// Default configuration values
v.SetDefault("components.helmfile.use_eks", true)
v.SetDefault("components.terraform.append_user_agent", fmt.Sprintf("Atmos/%s (Cloud Posse; +https://atmos.tools)", version.Version))
v.SetDefault("settings.inject_github_token", true)

v.SetDefault("logs.file", "/dev/stderr")
v.SetDefault("logs.level", "Info")

// Process config in system folder
configFilePath1 := ""
atmosConfigFilePath := ""
// https://pureinfotech.com/list-environment-variables-windows-10/
// https://docs.microsoft.com/en-us/windows/deployment/usmt/usmt-recognized-environment-variables
// https://softwareengineering.stackexchange.com/questions/299869/where-is-the-appropriate-place-to-put-application-configuration-files-for-each-p
// https://stackoverflow.com/questions/37946282/why-does-appdata-in-windows-7-seemingly-points-to-wrong-folder
if runtime.GOOS == "windows" {
appDataDir := os.Getenv(WindowsAppDataEnvVar)
if len(appDataDir) > 0 {
configFilePath1 = appDataDir
}
} else {
configFilePath1 = SystemDirConfigFilePath
}

if len(configFilePath1) > 0 {
configFile1 := filepath.Join(configFilePath1, CliConfigFileName)
found, err = processConfigFile(atmosConfig, configFile1, v)
if err != nil {
return atmosConfig, err
}
if found {
configFound = true
atmosConfigFilePath = configFile1
}
}

// Process config in user's HOME dir
configFilePath2, err := homedir.Dir()
if err != nil {
return atmosConfig, err
}
configFile2 := filepath.Join(configFilePath2, ".atmos", CliConfigFileName)
found, err = processConfigFile(atmosConfig, configFile2, v)
if err != nil {
return atmosConfig, err
}
if found {
configFound = true
atmosConfigFilePath = configFile2
}

// Process config in the current dir
configFilePath3, err := os.Getwd()
if err != nil {
return atmosConfig, err
}
configFile3 := filepath.Join(configFilePath3, CliConfigFileName)
found, err = processConfigFile(atmosConfig, configFile3, v)
if err != nil {
return atmosConfig, err
}
if found {
configFound = true
atmosConfigFilePath = configFile3
}

// Process config from the path in ENV var `ATMOS_CLI_CONFIG_PATH`
configFilePath4 := os.Getenv("ATMOS_CLI_CONFIG_PATH")
if len(configFilePath4) > 0 {
u.LogTrace(fmt.Sprintf("Found ENV var ATMOS_CLI_CONFIG_PATH=%s", configFilePath4))
configFile4 := filepath.Join(configFilePath4, CliConfigFileName)
found, err = processConfigFile(atmosConfig, configFile4, v)
if err != nil {
return atmosConfig, err
}
if found {
configFound = true
atmosConfigFilePath = configFile4
}
}

// Process config from the path specified in the Terraform provider (which calls into the atmos code)
if configAndStacksInfo.AtmosCliConfigPath != "" {
configFilePath5 := configAndStacksInfo.AtmosCliConfigPath
if len(configFilePath5) > 0 {
configFile5 := filepath.Join(configFilePath5, CliConfigFileName)
found, err = processConfigFile(atmosConfig, configFile5, v)
if err != nil {
return atmosConfig, err
}
if found {
configFound = true
atmosConfigFilePath = configFile5
}
}
}

if !configFound {
// If `atmos.yaml` not found, use the default config
// Set `ATMOS_LOGS_LEVEL` ENV var to "Debug" to see the message about Atmos using the default CLI config
logsLevelEnvVar := os.Getenv("ATMOS_LOGS_LEVEL")
if logsLevelEnvVar == u.LogLevelDebug || logsLevelEnvVar == u.LogLevelTrace {
u.PrintMessageInColor("'atmos.yaml' CLI config was not found in any of the searched paths: system dir, home dir, current dir, ENV vars.\n"+
"Refer to https://atmos.tools/cli/configuration for details on how to configure 'atmos.yaml'.\n"+
"Using the default CLI config:\n\n", theme.Colors.Info)

err = u.PrintAsYAMLToFileDescriptor(atmosConfig, defaultCliConfig)
if err != nil {
return atmosConfig, err
}
fmt.Println()
}

j, err := json.Marshal(defaultCliConfig)
if err != nil {
return atmosConfig, err
}

reader := bytes.NewReader(j)
err = v.MergeConfig(reader)
if err != nil {
return atmosConfig, err
}
}
// We want the editorconfig color by default to be true
atmosConfig.Validate.EditorConfig.Color = true
// https://gist.github.com/chazcheadle/45bf85b793dea2b71bd05ebaa3c28644
// https://sagikazarmark.hu/blog/decoding-custom-formats-with-viper/
err = v.Unmarshal(&atmosConfig)
atmosConfig, err := LoadConfig(configAndStacksInfo)
if err != nil {
return atmosConfig, err
}
// Set the CLI config path in the atmosConfig struct
if filepath.IsAbs(atmosConfigFilePath) {
atmosConfig.CliConfigPath = atmosConfigFilePath
} else {
absPath, err := filepath.Abs(atmosConfigFilePath)
if err != nil {
return atmosConfig, err
}
atmosConfig.CliConfigPath = absPath
}
// Process ENV vars
err = processEnvVars(&atmosConfig)
if err != nil {
259 changes: 0 additions & 259 deletions pkg/config/config_load.go

This file was deleted.

229 changes: 229 additions & 0 deletions pkg/config/load.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,229 @@
package config

import (
"bytes"
"encoding/json"
"fmt"
"os"
"path/filepath"
"runtime"

"github.com/cloudposse/atmos/pkg/schema"
"github.com/cloudposse/atmos/pkg/ui/theme"
u "github.com/cloudposse/atmos/pkg/utils"
"github.com/cloudposse/atmos/pkg/version"
"github.com/mitchellh/go-homedir"
"github.com/spf13/viper"
)

type ConfigSources struct {
paths string
atmosFileNames string
}

func LoadConfig(configAndStacksInfo schema.ConfigAndStacksInfo) (schema.AtmosConfiguration, error) {
v := viper.New()
var atmosConfig schema.AtmosConfiguration
v.SetConfigType("yaml")
v.SetTypeByDefaultValue(true)
setDefaultConfiguration(v)
err := readSystemConfig(v)
if err != nil {
return atmosConfig, err
}
err = readHomeConfig(v)
if err != nil {
return atmosConfig, err
}
err = readWorkDirConfig(v)
if err != nil {
return atmosConfig, err
}
err = readEnvAmosConfigPath(v)
if err != nil {
return atmosConfig, err
}
if configAndStacksInfo.AtmosCliConfigPath != "" {
configFilePath := configAndStacksInfo.AtmosCliConfigPath
if len(configFilePath) > 0 {
err := mergeConfig(v, configFilePath, CliConfigFileName)
switch err.(type) {
case viper.ConfigFileNotFoundError:
u.LogTrace(fmt.Sprintf("Not Found config file %s", configFilePath))
default:
return atmosConfig, err
}
}
}

atmosConfig.CliConfigPath = v.ConfigFileUsed()

if atmosConfig.CliConfigPath == "" {
// If `atmos.yaml` not found, use the default config
// Set `ATMOS_LOGS_LEVEL` ENV var to "Debug" to see the message about Atmos using the default CLI config
logsLevelEnvVar := os.Getenv("ATMOS_LOGS_LEVEL")
if logsLevelEnvVar == u.LogLevelDebug || logsLevelEnvVar == u.LogLevelTrace {
u.PrintMessageInColor("'atmos.yaml' CLI config was not found in any of the searched paths: system dir, home dir, current dir, ENV vars.\n"+
"Refer to https://atmos.tools/cli/configuration for details on how to configure 'atmos.yaml'.\n"+
"Using the default CLI config:\n\n", theme.Colors.Info)

err = u.PrintAsYAMLToFileDescriptor(atmosConfig, defaultCliConfig)
if err != nil {
return atmosConfig, err
}
fmt.Println()
}

j, err := json.Marshal(defaultCliConfig)
if err != nil {
return atmosConfig, err
}

reader := bytes.NewReader(j)
err = v.MergeConfig(reader)
if err != nil {
return atmosConfig, err
}
}
// Set the CLI config path in the atmosConfig struct
if !filepath.IsAbs(atmosConfig.CliConfigPath) {
absPath, err := filepath.Abs(atmosConfig.CliConfigPath)
if err != nil {
return atmosConfig, err
}
atmosConfig.CliConfigPath = absPath
}
// We want the editorconfig color by default to be true
atmosConfig.Validate.EditorConfig.Color = true
// https://gist.github.com/chazcheadle/45bf85b793dea2b71bd05ebaa3c28644
// https://sagikazarmark.hu/blog/decoding-custom-formats-with-viper/
err = v.Unmarshal(&atmosConfig)
if err != nil {
return atmosConfig, err
}
return atmosConfig, nil
}

// setDefaultConfiguration set default configuration for the viper instance.
func setDefaultConfiguration(v *viper.Viper) {
v.SetDefault("components.helmfile.use_eks", true)
v.SetDefault("components.terraform.append_user_agent",
fmt.Sprintf("Atmos/%s (Cloud Posse; +https://atmos.tools)", version.Version))
v.SetDefault("settings.inject_github_token", true)
v.SetDefault("logs.file", "/dev/stderr")
v.SetDefault("logs.level", "Info")

}

// readSystemConfig load config from system dir
func readSystemConfig(v *viper.Viper) error {
configFilePath := ""
if runtime.GOOS == "windows" {
appDataDir := os.Getenv(WindowsAppDataEnvVar)
if len(appDataDir) > 0 {
configFilePath = appDataDir
}
} else {
configFilePath = SystemDirConfigFilePath
}

if len(configFilePath) > 0 {
err := mergeConfig(v, configFilePath, CliConfigFileName)
switch err.(type) {
case viper.ConfigFileNotFoundError:
return nil
default:
return err
}
}
return nil

}

// readHomeConfig load config from user's HOME dir
func readHomeConfig(v *viper.Viper) error {
home, err := homedir.Dir()
if err != nil {
return err
}
configFilePath := filepath.Join(home, ".atmos")
err = mergeConfig(v, configFilePath, CliConfigFileName)
if err != nil {
switch err.(type) {
case viper.ConfigFileNotFoundError:
return nil
default:
return err
}
}
return nil
}

// readWorkDirConfig load config from current working directory
func readWorkDirConfig(v *viper.Viper) error {
wd, err := os.Getwd()
if err != nil {
return err
}
err = mergeConfig(v, wd, CliConfigFileName)
if err != nil {
switch err.(type) {
case viper.ConfigFileNotFoundError:
return nil
default:
return err
}
}
return nil
}

func readEnvAmosConfigPath(v *viper.Viper) error {
atmosPath := os.Getenv("ATMOS_CLI_CONFIG_PATH")
if atmosPath != "" {
configFilePath := filepath.Join(atmosPath, CliConfigFileName)
err := mergeConfig(v, configFilePath, CliConfigFileName)
if err != nil {
switch err.(type) {
case viper.ConfigFileNotFoundError:
u.LogTrace(fmt.Sprintf("Not Found ENV var ATMOS_CLI_CONFIG_PATH=%s", configFilePath))
return nil
default:
return err
}
}
u.LogTrace(fmt.Sprintf("Found ENV var ATMOS_CLI_CONFIG_PATH=%s", configFilePath))
}

return nil
}

// mergeConfig merge config from a specified path and file name.
func mergeConfig(v *viper.Viper, path string, fileName string) error {
v.AddConfigPath(path)
v.SetConfigName(fileName)
err := v.MergeInConfig()
if err != nil {
return nil
}

return nil
}

// applyDefaultConfiguration apply default configuration for the atmos config.
func applyDefaultConfiguration(v *viper.Viper) error {
logsLevel := os.Getenv("ATMOS_LOGS_LEVEL")
if logsLevel == u.LogLevelDebug || logsLevel == u.LogLevelTrace {
var atmosConfig schema.AtmosConfiguration
u.PrintMessageInColor("Using default configuration...\n", theme.Colors.Info)
err := u.PrintAsYAMLToFileDescriptor(atmosConfig, defaultCliConfig)
if err != nil {
return err
}
}

defaultConfig, err := json.Marshal(defaultCliConfig)
if err != nil {
return err
}
return v.MergeConfig(bytes.NewReader(defaultConfig))
}