Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Initialise a empty default bundle if BUNDLE_ROOT and DATABRICKS_BUNDLE_INCLUDES env vars are present #604

Merged
merged 16 commits into from
Aug 2, 2023
Merged

Initialise a empty default bundle if BUNDLE_ROOT and DATABRICKS_BUNDLE_INCLUDES env vars are present #604

merged 16 commits into from
Aug 2, 2023

Conversation

kartikgupta-db
Copy link
Contributor

Changes

Tests

kartikgupta-db and others added 14 commits July 20, 2023 11:35
@kartikgupta-db kartikgupta-db changed the base branch from main to include-paths-env-var July 25, 2023 13:41
andrewnester
andrewnester reviewed Aug 2, 2023
View reviewed changes
bundle/bundle.go
@@ -43,10 +43,26 @@ type Bundle struct {
AutoApprove bool
}

const ExtraIncludePathsKey string = "DATABRICKS_BUNDLE_INCLUDES"

func Load(path string) (*Bundle, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly the behaviour is different from what is intended based on PR description

This will initialise it not only when BUNDLE_ROOT is set but also if there any bundle/databricks.yml in current or child dirs (https://github.com/databricks/cli/blob/main/bundle/root.go#L51-L56C9).

Is this correct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The assumption here is that configFile, err := config.FileNames.FindInPath(path) is reached only when we have established a bundle root (either through BUNDLE_ROOT or by finding a *.yml).
This functions returns an error only when there is no config file found (no *.yml). Which means BUNDLE_ROOT is set but there is no config file.

I will make it an explicit test though to make reasoning easier.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we be more explicit here? Our condition to have an empty config is:

  1. We have BUNDLE_ROOT env variable
  2. There is no config file in this BUNDLE_ROOT

Could you codify this condition? It would be somethin like

_, hasBundleRootEnv := os.LookupEnv(envBundleRoot)
configFile, noConfigFoundErr := config.FileNames.FindInPath(path)
if hasBundleRootEnv && noConfigFoundErr != nil {
  ...
}

bundle/root_test.go
@@ -102,3 +103,32 @@ func TestRootLookupError(t *testing.T) {
_, err := mustGetRoot()
require.ErrorContains(t, err, "unable to locate bundle root")
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a test case where you have bundle.yml in the temp dir and also have ExtraInclude set and make sure it loads the config correctly?

Base automatically changed from include-paths-env-var to main August 2, 2023 10:24
@kartikgupta-db kartikgupta-db added this pull request to the merge queue Aug 2, 2023
Merged via the queue into main with commit 3140a8f Aug 2, 2023
@kartikgupta-db kartikgupta-db deleted the empty-bundle branch August 2, 2023 17:32
pietern added a commit that referenced this pull request Aug 10, 2023
@pietern
Release v0.203.0
1c00d4f 
CLI:
 * Infer host from profile during `auth login` ([#629](#629)).

Bundles:
 * Extend deployment mode support ([#577](#577)).
 * Add validation for Git settings in bundles ([#578](#578)).
 * Only treat files with .tmpl extension as templates ([#594](#594)).
 * Add JSON schema validation for input template parameters ([#598](#598)).
 * Add DATABRICKS_BUNDLE_INCLUDE_PATHS to specify include paths through env vars ([#591](#591)).
 * Initialise a empty default bundle if BUNDLE_ROOT and DATABRICKS_BUNDLE_INCLUDES env vars are present ([#604](#604)).
 * Regenerate bundle resource structs from latest Terraform provider ([#633](#633)).
 * Fixed processing jobs libraries with remote path ([#638](#638)).
 * Add unit test for file name execution during rendering ([#640](#640)).
 * Add bundle init command and support for prompting user for input values ([#631](#631)).
 * Fix bundle git branch validation ([#645](#645)).

Internal:
 * Fix mkdir integration test on GCP ([#620](#620)).
 * Fix git clone integration test for non-existing repo ([#610](#610)).
 * Remove push to main trigger for build workflow ([#621](#621)).
 * Remove workflow to publish binaries to S3 ([#622](#622)).
 * Fix failing fs mkdir test on azure ([#627](#627)).
 * Print y/n options when displaying prompts using cmdio.Ask ([#650](#650)).

API Changes:
 * Changed `databricks account metastore-assignments create` command to not return anything.
 * Added `databricks account network-policy` command group.

OpenAPI commit 7b57ba3a53f4de3d049b6a24391fe5474212daf8 (2023-07-28)

Dependency updates:
 * Bump OpenAPI specification & Go SDK Version ([#624](#624)).
 * Bump golang.org/x/term from 0.10.0 to 0.11.0 ([#643](#643)).
 * Bump golang.org/x/text from 0.11.0 to 0.12.0 ([#642](#642)).
 * Bump golang.org/x/oauth2 from 0.10.0 to 0.11.0 ([#641](#641)).
@pietern pietern mentioned this pull request Aug 10, 2023
Merged
github-merge-queue bot pushed a commit that referenced this pull request Aug 10, 2023
@pietern
Release v0.203.0 (#653)
979b680 
CLI:
* Infer host from profile during `auth login`
([#629](#629)).

Bundles:
* Extend deployment mode support
([#577](#577)).
* Add validation for Git settings in bundles
([#578](#578)).
* Only treat files with .tmpl extension as templates
([#594](#594)).
* Add JSON schema validation for input template parameters
([#598](#598)).
* Add DATABRICKS_BUNDLE_INCLUDE_PATHS to specify include paths through
env vars ([#591](#591)).
* Initialise a empty default bundle if BUNDLE_ROOT and
DATABRICKS_BUNDLE_INCLUDES env vars are present
([#604](#604)).
* Regenerate bundle resource structs from latest Terraform provider
([#633](#633)).
* Fixed processing jobs libraries with remote path
([#638](#638)).
* Add unit test for file name execution during rendering
([#640](#640)).
* Add bundle init command and support for prompting user for input
values ([#631](#631)).
* Fix bundle git branch validation
([#645](#645)).

Internal:
* Fix mkdir integration test on GCP
([#620](#620)).
* Fix git clone integration test for non-existing repo
([#610](#610)).
* Remove push to main trigger for build workflow
([#621](#621)).
* Remove workflow to publish binaries to S3
([#622](#622)).
* Fix failing fs mkdir test on azure
([#627](#627)).
* Print y/n options when displaying prompts using cmdio.Ask
([#650](#650)).

API Changes:
* Changed `databricks account metastore-assignments create` command to
not return anything.
 * Added `databricks account network-policy` command group.

OpenAPI commit 7b57ba3a53f4de3d049b6a24391fe5474212daf8 (2023-07-28)

Dependency updates:
* Bump OpenAPI specification & Go SDK Version
([#624](#624)).
* Bump golang.org/x/term from 0.10.0 to 0.11.0
([#643](#643)).
* Bump golang.org/x/text from 0.11.0 to 0.12.0
([#642](#642)).
* Bump golang.org/x/oauth2 from 0.10.0 to 0.11.0
([#641](#641)).
pietern added a commit that referenced this pull request Mar 26, 2024
@pietern
Remove support for DATABRICKS_BUNDLE_INCLUDES
99f027e 
PR #604 added functionality to load a bundle without a `databricks.yml` if both
the `DATABRICKS_BUNDLE_ROOT` and `DATABRICKS_BUNDLE_INCLUDES` environment
variables were set. We never ended up using this in downstream tools so this
can be removed.
@pietern pietern mentioned this pull request Mar 26, 2024
Merged
github-merge-queue bot pushed a commit that referenced this pull request Mar 27, 2024
@pietern
Remove support for DATABRICKS_BUNDLE_INCLUDES (#1317)
f195b84 
## Changes

PR #604 added functionality to load a bundle without a `databricks.yml`
if both the `DATABRICKS_BUNDLE_ROOT` and `DATABRICKS_BUNDLE_INCLUDES`
environment variables were set. We never ended up using this in
downstream tools so this can be removed.

## Tests

Unit tests pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewnester andrewnester andrewnester approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kartikgupta-db @andrewnester