Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

profile flag seems to be ignored in databricks sync command #836

Closed
jmeidam opened this issue Oct 3, 2023 · 6 comments · Fixed by #837 or #867
Closed

profile flag seems to be ignored in databricks sync command #836

jmeidam opened this issue Oct 3, 2023 · 6 comments · Fixed by #837 or #867
Assignees
@andrewnester
Labels
Bug Something isn't working CLI CLI related issues

Comments

@jmeidam
Copy link

jmeidam commented Oct 3, 2023 •
edited
Loading

I noticed that when trying to use databricks sync, the --profile flag is being ignored.

When I run

databricks sync --profile my-profile . /Users/my.name@website.com/remote/folder

I get a "no such host" error, showing it is trying to connect to the host under the [default] section in .databrickscfg (Which in my case is an old host that does indeed not exist).

I expect to see that it either uses the host listed under [my-profile] or raise an error if my-profile is not defined. This is the case when I use the --profile flag in databricks fs ls for example.

Running on: Databricks CLI v0.206.0
@andrewnester andrewnester added Bug Something isn't working CLI CLI related issues labels Oct 3, 2023
@andrewnester andrewnester mentioned this issue Oct 3, 2023
Merged
@andrewnester andrewnester self-assigned this Oct 3, 2023
github-merge-queue bot pushed a commit that referenced this issue Oct 9, 2023
@andrewnester
Ensure profile flag is respected for sync command (#837)
ad4b476 
## Changes
Fixes #836 

## Tests
Manually running `sync` command with and without the flag

Integration tests pass as well

```
--- PASS: TestAccSyncFullFileSync (13.38s)
PASS
coverage: 39.1% of statements in ./...
ok      github.com/databricks/cli/internal      14.148s coverage: 39.1% of statements in ./...


--- PASS: TestAccSyncIncrementalFileSync (11.38s)
PASS
coverage: 39.1% of statements in ./...
ok      github.com/databricks/cli/internal      11.674s coverage: 39.1% of statements in ./...
```
@andrewnester
Copy link
Contributor

The fix has been merged and will be released in the next CLI release

@jmeidam
Copy link
Author

jmeidam commented Oct 9, 2023

That was very quick. Great work! :)

@jmeidam
Copy link
Author

jmeidam commented Oct 13, 2023

@andrewnester I just tried out the latest version:
Databricks CLI v0.207.1

I still get the error:
Error: Get "https://adb-123456789101112.0.azuredatabricks.net/api/2.0/preview/scim/v2/Me": dial tcp: lookup adb-123456789101112.0.azuredatabricks.net: no such host

when running
databricks sync --profile my-profile . /Users/my.name@website.com/remote/folder

I deliberately put a fake host in the default section of my .databrickscfg. It is still not respecting the --profile parameter.

@andrewnester andrewnester reopened this Oct 13, 2023
hectorcast-db pushed a commit that referenced this issue Oct 13, 2023
@andrewnester @hectorcast-db
Ensure profile flag is respected for sync command (#837)
9a59b04 
## Changes
Fixes #836 

## Tests
Manually running `sync` command with and without the flag

Integration tests pass as well

```
--- PASS: TestAccSyncFullFileSync (13.38s)
PASS
coverage: 39.1% of statements in ./...
ok      github.com/databricks/cli/internal      14.148s coverage: 39.1% of statements in ./...


--- PASS: TestAccSyncIncrementalFileSync (11.38s)
PASS
coverage: 39.1% of statements in ./...
ok      github.com/databricks/cli/internal      11.674s coverage: 39.1% of statements in ./...
```
@andrewnester andrewnester mentioned this issue Oct 13, 2023
Merged
github-merge-queue bot pushed a commit that referenced this issue Oct 13, 2023
@andrewnester
Use already instantiated WorkspaceClient in sync command (#867)
ff01898 
## Changes
Since we use `root.MustWorkspaceClient` now, we should use already
initialised version of WorkspaceClient instead of instantiating a new
one.

Fixes #836
@andrewnester
Copy link
Contributor

@jmeidam thanks for raising this! indeed, it seems to be not working due to a bug in CLI, it's now fixed in the main branch and will be released in the next CLI version.
In the meantime you can try out snapshot version of the CLI containing the fix from here:

@jmeidam
Copy link
Author

jmeidam commented Oct 16, 2023

Bytheway, I just noticed that the sync command is not in the docs linked to from the readme:
https://github.com/databricks/cli/blob/main/docs/commands.md

And the docs I can find online are missing information on how to define includes and excludes

@andrewnester
Copy link
Contributor

You can find docs on sync section here https://docs.databricks.com/en/dev-tools/bundles/settings.html#sync

And for the commands reference you linked, we have a followup to address this but thanks for calling this out
#829

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@andrewnester andrewnester

Labels
Bug Something isn't working CLI CLI related issues
Projects
None yet
Milestone
No milestone
2 participants
@andrewnester @jmeidam