[ISSUE] Issue with databricks_sql_permissions resource

[forma-wolf]

Configuration

# Copy-paste your Terraform configuration here
resource "databricks_sql_permissions" "hive_metastore_databricks_sql_permissions" {
  ...
  catalog = true
  dynamic "privilege_assignments" {
    ...
  }
  ...
}

resource "databricks_sql_permissions" "files_databricks_sql_permissions" {
  ...
  any_file   = true
  dynamic "privilege_assignments" {
    ...
  }
  ...
}

Expected Behavior

It should not remove existing permissions everytime we add new groups.

Actual Behavior

Everytime this resource gets applied it to hive_metastore it removes all permissions and then adds all permissions in configuration. This will result in outages due to permissioning of hive metastore being completely removed and and readded during this time period. Also the time period it takes to apply this is directly proportional to the number of schemas. We are up to having to apply terraform twice (because 1st apply fails out), took a period of 90 minutes.

Steps to Reproduce

1. terraform apply this resource with an initial group.
2. terraform apply this resource adding permissions for the second group (you will see that it says it is only adding the permissions for the second group which it should be)
3. In reality, you are removing the permissions for all groups and re-adding the initial group, and second group's permissions.

Terraform and provider versions

• Latest (issue has always existed)

Is it a regression?

• No it has always worked like this and needs to be fixed. It's causing outages for customers using hive_metastore.

Debug Output

Important Factoids

• Nope, we do have a large amount of schemas.

Would you like to implement a fix?

• No