Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate resource databricks_catalog to Go SDK #2357

Merged
merged 3 commits into from
Jun 5, 2023
Merged

Migrate resource databricks_catalog to Go SDK #2357

merged 3 commits into from
Jun 5, 2023

Conversation

nkvuong
Copy link
Contributor

@nkvuong nkvuong commented May 31, 2023

Changes

Migrate databricks_catalog resource to Go SDK.

Tests

  • make test run locally
  • relevant change in docs/ folder
  • covered with integration tests in internal/acceptance
  • relevant acceptance tests are passing
  • using Go SDK

// We need to update the resource data because Name is updatable
// So if we don't update the field then the requests would be made to old Name which doesn't exists.
d.SetId(ci.Name)
if err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return error after setting the ID, otherwise we're tainted

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed the code to only set the new Id after a successful update

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch. This would need to be done for volumes as well. Created: #2374

@codecov-commenter
Copy link

codecov-commenter commented Jun 2, 2023

Codecov Report

Merging #2357 (b031b5b) into master (48e7be8) will decrease coverage by 0.07%.
The diff coverage is 72.22%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2357      +/-   ##
==========================================
- Coverage   88.53%   88.47%   -0.07%     
==========================================
  Files         141      141              
  Lines       11620    11628       +8     
==========================================
  Hits        10288    10288              
- Misses        886      890       +4     
- Partials      446      450       +4     
Impacted Files Coverage Δ
catalog/resource_schema.go 95.00% <ø> (-0.46%) ⬇️
catalog/resource_catalog.go 83.33% <72.22%> (-12.50%) ⬇️

@nkvuong nkvuong marked this pull request as ready for review June 5, 2023 09:57
@nkvuong nkvuong requested a review from a team June 5, 2023 09:57
}

return a.client.Delete(a.context, "/unity-catalog/catalogs/"+name, nil)
Name string `json:"name"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't the tf:"force_new" missing here?

Copy link
Contributor Author

@nkvuong nkvuong Jun 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

catalog name is updatable through our API but we force recreating the resource because the provider was not able to handle the change in name. Same explanation as the below comment from Alex

Comment on lines +97 to +99
var updateCatalogRequest catalog.UpdateCatalog
common.DataToStructPointer(d, catalogSchema, &updateCatalogRequest)
ci, err := w.Catalogs.Update(ctx, updateCatalogRequest)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How it will work when the name is already updated in resource - as I see in the docs, we address catalog by name: https://docs.databricks.com/api/workspace/catalogs/update, so if the name in resource is updated, it won't find an old one to change it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the change happened out of bound, then there is not much we can do. what happens with the update operation here is that we call the Update API, update the resource Id to the new name, so that the Get API will use the new name and succeed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

credit to @tanmay-db for getting this to worked with the Volume resource, so I am just backporting the fix

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we allow name change, then SDK needs to support that pattern. Inside provider we can find if the name field has changed, and then use previous name (the d.Id()) in the URL

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to do the same for all the UC resources then, as they are all addressed by names.

currently volume handles it like this https://github.com/databricks/terraform-provider-databricks/blob/master/catalog/resource_volume.go#L85

func NewCatalogsAPI(ctx context.Context, m any) CatalogsAPI {
return CatalogsAPI{m.(*common.DatabricksClient), context.WithValue(ctx, common.Api, common.API_2_1)}
}

type CatalogInfo struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nkvuong did you try using UpdateCatalog for the schema this way we won't have to worry about the new fields?...

Comment string `json:"comment,omitempty"`
StorageRoot string `json:"storage_root,omitempty" tf:"force_new"`
ProviderName string `json:"provider_name,omitempty" tf:"force_new,conflicts:storage_root"`
ShareName string `json:"share_name,omitempty" tf:"force_new,conflicts:storage_root"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can go into schema callback.

@nfx nfx enabled auto-merge (squash) June 5, 2023 17:35
@nfx nfx merged commit 035b47d into master Jun 5, 2023
@nfx nfx mentioned this pull request Jun 14, 2023
@nkvuong nkvuong deleted the sdk/catalog branch July 3, 2023 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants