Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Set default dbt_version to latest and add diff logic to treat versionless and latest as equivalents #324

Merged
merged 7 commits into from
Dec 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/guides/1_getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,15 +99,15 @@ resource "dbtcloud_project_repository" "my_project_repository" {
// here both are linked to the same Data Warehouse connection
// for Prod, we need to create a credential as well
resource "dbtcloud_environment" "my_dev" {
dbt_version = "versionless"
dbt_version = "latest"
name = "Dev"
project_id = dbtcloud_project.my_project.id
type = "development"
connection_id = dbtcloud_global_connection.my_connection.id
}

resource "dbtcloud_environment" "my_prod" {
dbt_version = "versionless"
dbt_version = "latest"
name = "Prod"
project_id = dbtcloud_project.my_project.id
type = "deployment"
Expand Down
8 changes: 4 additions & 4 deletions docs/resources/environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ This version of the provider has the `connection_id` as an optional field but it

```terraform
resource "dbtcloud_environment" "ci_environment" {
// the dbt_version is major.minor.0-latest , major.minor.0-pre or versionless (by default, it is set to versionless if not configured)
dbt_version = "versionless"
// the dbt_version is major.minor.0-latest , major.minor.0-pre or latest (by default, it is set to latest if not configured)
dbt_version = "latest"
name = "CI"
project_id = dbtcloud_project.dbt_project.id
type = "deployment"
Expand All @@ -44,7 +44,7 @@ resource "dbtcloud_environment" "prod_environment" {

// Creating a development environment
resource "dbtcloud_environment" "dev_environment" {
dbt_version = "versionless"
dbt_version = "latest"
name = "Dev"
project_id = dbtcloud_project.dbt_project.id
type = "development"
Expand All @@ -70,7 +70,7 @@ resource "dbtcloud_environment" "dev_environment" {
- To avoid Terraform state issues, when using this field, the `dbtcloud_project_connection` resource should be removed from the project or you need to make sure that the `connection_id` is the same in `dbtcloud_project_connection` and in the `connection_id` of the Development environment of the project
- `credential_id` (Number) Credential ID to create the environment with. A credential is not required for development environments but is required for deployment environments
- `custom_branch` (String) Which custom branch to use in this environment
- `dbt_version` (String) Version number of dbt to use in this environment. It needs to be in the format `major.minor.0-latest` (e.g. `1.5.0-latest`), `major.minor.0-pre` or `versionless`. Defaults to`versionless` if no version is provided
- `dbt_version` (String) Version number of dbt to use in this environment. It needs to be in the format `major.minor.0-latest` (e.g. `1.5.0-latest`), `major.minor.0-pre`, `versionless`, or `latest`. While `versionless` is still supported, using `latest` is recommended. Defaults to `latest` if no version is provided
- `deployment_type` (String) The type of environment. Only valid for environments of type 'deployment' and for now can only be 'production', 'staging' or left empty for generic environments
- `enable_model_query_history` (Boolean) Whether to enable model query history in this environment. As of Oct 2024, works only for Snowflake and BigQuery.
- `extended_attributes_id` (Number) ID of the extended attributes for the environment
Expand Down
2 changes: 1 addition & 1 deletion docs/resources/extended_attributes.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ resource "dbtcloud_extended_attributes" "my_attributes" {
}

resource "dbtcloud_environment" "issue_depl" {
dbt_version = "versionless"
dbt_version = "latest"
name = "My environment"
project_id = var.dbt_project.id
type = "deployment"
Expand Down
6 changes: 3 additions & 3 deletions examples/resources/dbtcloud_environment/resource.tf
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
resource "dbtcloud_environment" "ci_environment" {
// the dbt_version is major.minor.0-latest , major.minor.0-pre or versionless (by default, it is set to versionless if not configured)
dbt_version = "versionless"
// the dbt_version is major.minor.0-latest , major.minor.0-pre or latest (by default, it is set to latest if not configured)
dbt_version = "latest"
name = "CI"
project_id = dbtcloud_project.dbt_project.id
type = "deployment"
Expand All @@ -21,7 +21,7 @@ resource "dbtcloud_environment" "prod_environment" {

// Creating a development environment
resource "dbtcloud_environment" "dev_environment" {
dbt_version = "versionless"
dbt_version = "latest"
name = "Dev"
project_id = dbtcloud_project.dbt_project.id
type = "development"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ resource "dbtcloud_extended_attributes" "my_attributes" {
}

resource "dbtcloud_environment" "issue_depl" {
dbt_version = "versionless"
dbt_version = "latest"
name = "My environment"
project_id = var.dbt_project.id
type = "deployment"
Expand Down
2 changes: 1 addition & 1 deletion pkg/framework/acctest_helper/acctest_helper.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ func SharedClient() (*dbt_cloud.Client, error) {
}

const (
DBT_CLOUD_VERSION = "versionless"
DBT_CLOUD_VERSION = "latest"
)

var TestAccProtoV6ProviderFactories = map[string]func() (tfprotov6.ProviderServer, error){
Expand Down
2 changes: 1 addition & 1 deletion pkg/sdkv2/data_sources/helpers_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import (
)

const (
DBT_CLOUD_VERSION = "versionless"
DBT_CLOUD_VERSION = "latest"
)

func providers() map[string]*schema.Provider {
Expand Down
14 changes: 12 additions & 2 deletions pkg/sdkv2/resources/environment.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,18 @@ func ResourceEnvironment() *schema.Resource {
"dbt_version": {
Type: schema.TypeString,
Optional: true,
Default: "versionless",
Description: "Version number of dbt to use in this environment. It needs to be in the format `major.minor.0-latest` (e.g. `1.5.0-latest`), `major.minor.0-pre` or `versionless`. Defaults to`versionless` if no version is provided",
Default: "latest",
Description: "Version number of dbt to use in this environment. It needs to be in the format `major.minor.0-latest` (e.g. `1.5.0-latest`), `major.minor.0-pre`, `versionless`, or `latest`. While `versionless` is still supported, using `latest` is recommended. Defaults to `latest` if no version is provided",
DiffSuppressFunc: func(k, oldValue, newValue string, d *schema.ResourceData) bool {
switch oldValue {
case "versionless":
return newValue == "latest"
case "latest":
return newValue == "versionless"
default:
return false
}
},
},
"type": {
Type: schema.TypeString,
Expand Down
59 changes: 49 additions & 10 deletions pkg/sdkv2/resources/environment_acceptance_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import (
// testing for the historical use case where connection_id is not configured at the env level
func TestAccDbtCloudEnvironmentResourceNoConnection(t *testing.T) {

dbtVersionLatest := "latest"
environmentName := strings.ToUpper(acctest.RandStringFromCharSet(10, acctest.CharSetAlpha))
environmentName2 := strings.ToUpper(acctest.RandStringFromCharSet(10, acctest.CharSetAlpha))
projectName := strings.ToUpper(acctest.RandStringFromCharSet(10, acctest.CharSetAlpha))
Expand All @@ -30,6 +31,7 @@ func TestAccDbtCloudEnvironmentResourceNoConnection(t *testing.T) {
Config: testAccDbtCloudEnvironmentResourceNoConnectionBasicConfig(
projectName,
environmentName,
dbtVersionLatest,
),
Check: resource.ComposeTestCheckFunc(
testAccCheckDbtCloudEnvironmentExists("dbtcloud_environment.test_env"),
Expand All @@ -50,6 +52,7 @@ func TestAccDbtCloudEnvironmentResourceNoConnection(t *testing.T) {
Config: testAccDbtCloudEnvironmentResourceNoConnectionBasicConfig(
projectName,
environmentName2,
dbtVersionLatest,
),
Check: resource.ComposeTestCheckFunc(
testAccCheckDbtCloudEnvironmentExists("dbtcloud_environment.test_env"),
Expand Down Expand Up @@ -78,7 +81,7 @@ func TestAccDbtCloudEnvironmentResourceNoConnection(t *testing.T) {
resource.TestCheckResourceAttr(
"dbtcloud_environment.test_env",
"dbt_version",
DBT_CLOUD_VERSION,
dbtVersionLatest,
),
resource.TestCheckResourceAttr(
"dbtcloud_environment.test_env",
Expand Down Expand Up @@ -124,7 +127,7 @@ func TestAccDbtCloudEnvironmentResourceNoConnection(t *testing.T) {
resource.TestCheckResourceAttr(
"dbtcloud_environment.test_env",
"dbt_version",
DBT_CLOUD_VERSION,
dbtVersionLatest,
),
resource.TestCheckResourceAttr(
"dbtcloud_environment.test_env",
Expand Down Expand Up @@ -159,7 +162,7 @@ func TestAccDbtCloudEnvironmentResourceNoConnection(t *testing.T) {
}

func testAccDbtCloudEnvironmentResourceNoConnectionBasicConfig(
projectName, environmentName string,
projectName, environmentName, dbtVersion string,
) string {
return fmt.Sprintf(`
resource "dbtcloud_project" "test_project" {
Expand All @@ -173,7 +176,7 @@ resource "dbtcloud_environment" "test_env" {
project_id = dbtcloud_project.test_project.id
deployment_type = "production"
}
`, projectName, environmentName, DBT_CLOUD_VERSION)
`, projectName, environmentName, dbtVersion)
}

func testAccDbtCloudEnvironmentResourceNoConnectionModifiedConfig(
Expand Down Expand Up @@ -206,6 +209,7 @@ resource "dbtcloud_bigquery_credential" "test_credential" {

// testing for the global connection use case where connection_id is added at the env level
func TestAccDbtCloudEnvironmentResourceConnection(t *testing.T) {
dbtVersionLatest := "latest"
environmentName := strings.ToUpper(acctest.RandStringFromCharSet(10, acctest.CharSetAlpha))
environmentName2 := strings.ToUpper(acctest.RandStringFromCharSet(10, acctest.CharSetAlpha))
projectName := strings.ToUpper(acctest.RandStringFromCharSet(10, acctest.CharSetAlpha))
Expand All @@ -219,6 +223,7 @@ func TestAccDbtCloudEnvironmentResourceConnection(t *testing.T) {
Config: testAccDbtCloudEnvironmentResourceConnectionBasicConfig(
projectName,
environmentName,
dbtVersionLatest,
),
Check: resource.ComposeTestCheckFunc(
testAccCheckDbtCloudEnvironmentExists("dbtcloud_environment.test_env"),
Expand All @@ -239,6 +244,7 @@ func TestAccDbtCloudEnvironmentResourceConnection(t *testing.T) {
Config: testAccDbtCloudEnvironmentResourceConnectionBasicConfig(
projectName,
environmentName2,
dbtVersionLatest,
),
Check: resource.ComposeTestCheckFunc(
testAccCheckDbtCloudEnvironmentExists("dbtcloud_environment.test_env"),
Expand All @@ -256,6 +262,7 @@ func TestAccDbtCloudEnvironmentResourceConnection(t *testing.T) {
environmentName2,
"",
"false",
dbtVersionLatest,
),
Check: resource.ComposeTestCheckFunc(
testAccCheckDbtCloudEnvironmentExists("dbtcloud_environment.test_env"),
Expand All @@ -267,7 +274,7 @@ func TestAccDbtCloudEnvironmentResourceConnection(t *testing.T) {
resource.TestCheckResourceAttr(
"dbtcloud_environment.test_env",
"dbt_version",
DBT_CLOUD_VERSION,
dbtVersionLatest,
),
resource.TestCheckResourceAttr(
"dbtcloud_environment.test_env",
Expand Down Expand Up @@ -297,6 +304,7 @@ func TestAccDbtCloudEnvironmentResourceConnection(t *testing.T) {
environmentName2,
"main",
"true",
dbtVersionLatest,
),
Check: resource.ComposeTestCheckFunc(
testAccCheckDbtCloudEnvironmentExists("dbtcloud_environment.test_env"),
Expand All @@ -308,7 +316,7 @@ func TestAccDbtCloudEnvironmentResourceConnection(t *testing.T) {
resource.TestCheckResourceAttr(
"dbtcloud_environment.test_env",
"dbt_version",
DBT_CLOUD_VERSION,
dbtVersionLatest,
),
resource.TestCheckResourceAttr(
"dbtcloud_environment.test_env",
Expand Down Expand Up @@ -339,7 +347,7 @@ func TestAccDbtCloudEnvironmentResourceConnection(t *testing.T) {
}

func testAccDbtCloudEnvironmentResourceConnectionBasicConfig(
projectName, environmentName string,
projectName, environmentName, dbtVersion string,
) string {
return fmt.Sprintf(`
resource "dbtcloud_project" "test_project" {
Expand Down Expand Up @@ -367,11 +375,11 @@ resource "dbtcloud_environment" "test_env" {
connection_id = dbtcloud_global_connection.test.id
}

`, projectName, environmentName, DBT_CLOUD_VERSION)
`, projectName, environmentName, dbtVersion)
}

func testAccDbtCloudEnvironmentResourceConnectionModifiedConfig(
projectName, environmentName, customBranch, useCustomBranch string,
projectName, environmentName, customBranch, useCustomBranch, dbtVersion string,
) string {
return fmt.Sprintf(`
resource "dbtcloud_project" "test_project" {
Expand Down Expand Up @@ -419,7 +427,38 @@ resource "dbtcloud_bigquery_credential" "test_credential" {
num_threads = 16
}

`, projectName, environmentName, DBT_CLOUD_VERSION, customBranch, useCustomBranch)
`, projectName, environmentName, dbtVersion, customBranch, useCustomBranch)
}

// TestAccDbtCloudEnvironmentResourceVersionless tests the environment resource with dbt_version set to versionless
// This is a special case where if the dbt_version is set to `versionless`, the dbt Cloud API may return `latest`
func TestAccDbtCloudEnvironmentResourceVersionless(t *testing.T) {
dbtVersionless := "versionless"
environmentName := strings.ToUpper(acctest.RandStringFromCharSet(10, acctest.CharSetAlpha))
projectName := strings.ToUpper(acctest.RandStringFromCharSet(10, acctest.CharSetAlpha))

resource.Test(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t) },
ProtoV6ProviderFactories: acctest_helper.TestAccProtoV6ProviderFactories,
CheckDestroy: testAccCheckDbtCloudEnvironmentDestroy,
Steps: []resource.TestStep{
{
Config: testAccDbtCloudEnvironmentResourceNoConnectionBasicConfig(
projectName,
environmentName,
dbtVersionless,
),
Check: resource.ComposeTestCheckFunc(
testAccCheckDbtCloudEnvironmentExists("dbtcloud_environment.test_env"),
resource.TestMatchResourceAttr(
"dbtcloud_environment.test_env",
"dbt_version",
regexp.MustCompile("^versionless|latest$"),
),
),
},
},
})
}

func testAccCheckDbtCloudEnvironmentExists(resource string) resource.TestCheckFunc {
Expand Down
2 changes: 1 addition & 1 deletion pkg/sdkv2/resources/helpers_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (
)

const (
DBT_CLOUD_VERSION = "versionless"
DBT_CLOUD_VERSION = "latest"
)

var testAccProviders map[string]*schema.Provider
Expand Down
4 changes: 2 additions & 2 deletions templates/guides/1_getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,15 +99,15 @@ resource "dbtcloud_project_repository" "my_project_repository" {
// here both are linked to the same Data Warehouse connection
// for Prod, we need to create a credential as well
resource "dbtcloud_environment" "my_dev" {
dbt_version = "versionless"
dbt_version = "latest"
name = "Dev"
project_id = dbtcloud_project.my_project.id
type = "development"
connection_id = dbtcloud_global_connection.my_connection.id
}

resource "dbtcloud_environment" "my_prod" {
dbt_version = "versionless"
dbt_version = "latest"
name = "Prod"
project_id = dbtcloud_project.my_project.id
type = "deployment"
Expand Down
Loading