Skip to content
This repository was archived by the owner on Feb 13, 2022. It is now read-only.

Commit f081bdf

Browse files
committed
📝 rewrite README.md for users and contributors
1 parent fe1bb52 commit f081bdf

File tree

2 files changed

+27
-23
lines changed

2 files changed

+27
-23
lines changed

README.md

+27-23
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,40 @@
1-
# azure-pipelines-task-template
1+
# Azure DevOps Databricks REST API
22

3-
![Azure Pipelines](./images/azure-pipelines.png)
3+
![Azure Databricks](./images/icon.png)
44

55
## Introduction
66

7-
This repository contains a quick start template for creating and publishning a custom azure pipelines task written in TypeScript. It uses [webpack](https://webpack.js.org) to bundle scripts and is configured to use [jest](https://jestjs.io/) to run tests.
7+
This repository contains an Azure DevOps extension for interacting with Azure Databricks via [REST API](https://docs.databricks.com/dev-tools/api/latest/index.html). It supports Databricks management on clusters, jobs, and instance pools.
88

9-
## Getting started
10-
11-
There are a few small steps that need to be carried out before you can start using this template for your purpose-
9+
You may find this extension **useful** when:
10+
- You are running Spark (structured) streaming jobs attached to automated clusters.
11+
- You are leveraging instance pools for [fast startup time](https://databricks.com/blog/2019/11/11/databricks-pools-speed-up-data-pipelines.html) and scalability.
1212

13-
1. Create two guids and replace `{{guid_production}}` and `{{guid_development}}` occurences with these values in webpack [config](webpack.config.js). You can use [this](https://www.guidgenerator.com) tool to generate guids. *(It helps to have different guids for different environments as it allows you to install multiple extension flavors in the same account)*
13+
You may find this extension **useless** when:
14+
- You are running Spark jobs on an interactive clusters, or running on scheduled manner.
15+
- You are running jobs via Jupyter notebooks.
1416

15-
1. Enter your publisher and author name in place of `{{publisher}}` and `{{author}}` strings. These files have references to these strings - [webpack config](webpack.config.js), [task.json](./src/custom-task/task.json) and [manifest.json](./src/manifest.json)
17+
## Getting started
1618

17-
1. Create a personal access token for the Azure Devops account where you would test run your task. Refer [this](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops) to learn how to create a personal access token.
19+
- Install an extension via <https://marketplace.visualstudio.com/items?itemName=raksit31667.azure-devops-databricks-rest-api>.
20+
- Add Azure DevOps task similar as a code snippet below:
1821

19-
1. Set following environment variables to enable inner loop commands-
20-
- `BUILD_ENV` = **development**
21-
- `ADO_PAT` - Your personal access token
22-
- `ADO_ACCOUNT_URI` - The default collection URI for your account *(eg. <https://dev.azure.com/{{name}})>*
23-
- `DEV_PUBLISHER` - Id of your publisher account
22+
```yaml
23+
- task: VerifyAllDatabricksJobRunning@1
24+
displayName: Verify Databricks jobs
25+
inputs:
26+
jobs: '<your-job-name-here>'
27+
region: 'your-region-here>'
28+
accessToken: '<your-databricks-access-token>'
29+
```
2430
25-
1. Run following commands to verify setup-
26-
1. `npm install` (Installs npm dependencies)
27-
1. `npm run dev` (Builds, packages and publishes the extension)
31+
Note that every task requires **region** and **access token**. You can find a region from your
32+
Databricks workspace URL (e.g. <https://southcentralus.azuredatabricks.net/?o=0000000>). For access token, you can generate via **User Settings**, see an instruction <https://docs.databricks.com/dev-tools/api/latest/authentication.html>.
2833
29-
1. Refer to [package.json](./package.json) to see the list of all commands.
34+
## Contributing
35+
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
3036
31-
## Typical inner loop commands
37+
Please make sure to update tests as appropriate.
3238
33-
| What changed? | Which NPM command to run? | What it does? |
34-
| ------------- |:-------------:|:----- |
35-
| Task implementation | `npm run dev:task` | Build, package and update policy task without updating extension |
36-
| Anything else | `npm run dev` | Build, package and publish full extension |
39+
## License
40+
[MIT](https://choosealicense.com/licenses/mit/)

images/azure-pipelines.png

-4.33 KB
Binary file not shown.

0 commit comments

Comments
 (0)