Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for Jupyter (.ipynb) files in databricks_notebook resource #1801

Merged
merged 2 commits into from
Dec 1, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/resources/notebook.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This resource allows you to manage [Databricks Notebooks](https://docs.databrick

## Example Usage

You can declare Terraform-managed notebook by specifying `source` attribute of corresponding local file. Only `.scala`, `.py`, `.sql` and `.r` extensions are supported, if you would like to omit the `language` attribute.
You can declare Terraform-managed notebook by specifying `source` attribute of corresponding local file. Only `.scala`, `.py`, `.sql`, `.r`, and `.ipynb` extensions are supported, if you would like to omit the `language` attribute.

```hcl
data "databricks_current_user" "me" {
Expand Down Expand Up @@ -48,7 +48,7 @@ resource "databricks_notebook" "lesson" {

The size of a notebook source code must not exceed a few megabytes. The following arguments are supported:

* `path` - (Required) The absolute path of the notebook or directory, beginning with "/", e.g. "/Demo".
* `path` - (Required) The absolute path of the notebook or directory, beginning with "/", e.g. "/Demo".
* `source` - Path to notebook in source code format on local filesystem. Conflicts with `content_base64`.
* `content_base64` - The base64-encoded notebook source code. Conflicts with `source`. Use of `content_base64` is discouraged, as it's increasing memory footprint of Terraform state and should only be used in exceptional circumstances, like creating a notebook with configuration properties for a data pipeline.
* `language` - (required with `content_base64`) One of `SCALA`, `PYTHON`, `SQL`, `R`.
Expand Down Expand Up @@ -83,7 +83,7 @@ The following resources are often used in the same context:
* [databricks_job](job.md) to manage [Databricks Jobs](https://docs.databricks.com/jobs.html) to run non-interactive code in a [databricks_cluster](cluster.md).
* [databricks_notebook](../data-sources/notebook.md) data to export a notebook from Databricks Workspace.
* [databricks_notebook_paths](../data-sources/notebook_paths.md) data to list notebooks in Databricks Workspace.
* [databricks_pipeline](pipeline.md) to deploy [Delta Live Tables](https://docs.databricks.com/data-engineering/delta-live-tables/index.html).
* [databricks_pipeline](pipeline.md) to deploy [Delta Live Tables](https://docs.databricks.com/data-engineering/delta-live-tables/index.html).
* [databricks_repo](repo.md) to manage [Databricks Repos](https://docs.databricks.com/repos.html).
* [databricks_secret](secret.md) to manage [secrets](https://docs.databricks.com/security/secrets/index.html#secrets-user-guide) in Databricks workspace.
* [databricks_secret_acl](secret_acl.md) to manage access to [secrets](https://docs.databricks.com/security/secrets/index.html#secrets-user-guide) in Databricks workspace.
Expand Down
3 changes: 3 additions & 0 deletions workspace/resource_notebook.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ const (
Python string = "PYTHON"
SQL string = "SQL"
R string = "R"
Jupyter string = "JUPYTER"
)

type notebookLanguageFormat struct {
Expand All @@ -34,6 +35,7 @@ var extMap = map[string]notebookLanguageFormat{
".py": {"PYTHON", "SOURCE", true},
".sql": {"SQL", "SOURCE", true},
".r": {"R", "SOURCE", true},
".ipynb": {"", "JUPYTER", true},
".dbc": {"", "DBC", false},
}

Expand Down Expand Up @@ -209,6 +211,7 @@ func ResourceNotebook() *schema.Resource {
ValidateFunc: validation.StringInSlice([]string{
"SOURCE",
"DBC",
"JUPYTER",
}, false),
},
"url": {
Expand Down
54 changes: 54 additions & 0 deletions workspace/resource_notebook_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,60 @@ func TestResourceNotebookCreate(t *testing.T) {
assert.Equal(t, "/foo/path.py", d.Id())
}

func TestResourceNotebookCreateSource_Jupyter(t *testing.T) {
d, err := qa.ResourceFixture{
Fixtures: []qa.HTTPFixture{
{
Method: http.MethodPost,
Resource: "/api/2.0/workspace/import",
ExpectedRequest: ImportPath{
Content: "eyJjZWxscyI6W3siY2VsbF90eXBlIjoiY29kZSIsInNvdXJjZSI6WyJwc" +
"mludChcImhlbGxvIHdvcmxkXCIpIl0sIm1ldGFkYXRhIjp7fSwib3V0cHV" +
"0cyI6W10sImV4ZWN1dGlvbl9jb3VudCI6MX0seyJjZWxsX3R5cGUiOiJjb" +
"2RlIiwic291cmNlIjpbInByaW50KFwiaG93IGFyZSB5b3VcIikiXSwibWV" +
"0YWRhdGEiOnt9LCJvdXRwdXRzIjpbeyJtZXRhZGF0YSI6e30sIm91dHB1d" +
"F90eXBlIjoiZGlzcGxheV9kYXRhIiwiZGF0YSI6eyJ0ZXh0L2h0bWwiOls" +
"iPHN0eWxlIHNjb3BlZD5cbiAgLmFuc2lvdXQge1xuICAgIGRpc3BsYXk6I" +
"GJsb2NrO1xuICAgIHVuaWNvZGUtYmlkaTogZW1iZWQ7XG4gICAgd2hpdGU" +
"tc3BhY2U6IHByZS13cmFwO1xuICAgIHdvcmQtd3JhcDogYnJlYWstd29yZ" +
"DtcbiAgICB3b3JkLWJyZWFrOiBicmVhay1hbGw7XG4gICAgZm9udC1mYW1" +
"pbHk6IFwiU291cmNlIENvZGUgUHJvXCIsIFwiTWVubG9cIiwgbW9ub3NwY" +
"WNlOztcbiAgICBmb250LXNpemU6IDEzcHg7XG4gICAgY29sb3I6ICM1NTU" +
"7XG4gICAgbWFyZ2luLWxlZnQ6IDRweDtcbiAgICBsaW5lLWhlaWdodDogM" +
"TlweDtcbiAgfVxuPC9zdHlsZT5cbjxkaXYgY2xhc3M9XCJhbnNpb3V0XCI" +
"+aG93IGFyZSB5b3VcbjwvZGl2PiJdfX1dLCJleGVjdXRpb25fY291bnQiO" +
"jJ9LHsiY2VsbF90eXBlIjoiY29kZSIsInNvdXJjZSI6WyIiXSwibWV0YWR" +
"hdGEiOnt9LCJvdXRwdXRzIjpbXSwiZXhlY3V0aW9uX2NvdW50IjozfV0sI" +
"m1ldGFkYXRhIjp7Im5hbWUiOiJ0ZXN0X2p1cHl0ZXIiLCJub3RlYm9va0l" +
"kIjoxMjc1OTg0MjQzMjkzMDI4fSwibmJmb3JtYXQiOjQsIm5iZm9ybWF0X" +
"21pbm9yIjowfQo=",
Path: "/Mars",
Language: "",
Overwrite: true,
Format: "JUPYTER",
},
},
{
Method: http.MethodGet,
Resource: "/api/2.0/workspace/get-status?path=%2FMars",
Response: ObjectStatus{
ObjectID: 4567,
ObjectType: "NOTEBOOK",
Path: "/Mars",
},
},
},
Resource: ResourceNotebook(),
State: map[string]any{
"source": "acceptance/testdata/tf-test-jupyter.ipynb",
"path": "/Mars",
},
Create: true,
}.Apply(t)
assert.NoError(t, err, err)
assert.Equal(t, "/Mars", d.Id())
}

func TestResourceNotebookCreateSource(t *testing.T) {
d, err := qa.ResourceFixture{
Fixtures: []qa.HTTPFixture{
Expand Down