Adding Azure databricks workspace service

[promisinganuj]

Resolves #1725

What is being addressed

Adding the TRE workspace service for Azure Databricks.

How is this addressed

• Deploys Databricks workspace and firewall rules into a TRE workspace.
• Add private endpoints, private links for control plane, authentication and DBFS - https://github.com/asean-rssa/tf_azure_deployment/tree/master/adb-private-links-general
• Provide a way to configure public / private access to Databricks workspace
• Update documentation
• Update Change Log
• Use cases tested
• Configure Databricks clusters to install libraries using Nexus shared service - will get done in a separate story Allow researchers to install packages using the Nexus shared service on Azure Databricks service clusters #3120

[tamirkamara]

We currently don't use bicep and I think we should be thoughtful about adding more frameworks. What is the reason of using this verses keeping things in Terraform?

[tamirkamara]

We currently don't use bicep and I think we should be thoughtful about adding more frameworks. What is the reason of using this verses keeping things in Terraform?

[ross-p-smith]

/test-extended

[github-actions]

🤖 pr-bot 🤖

⚠️ When using /test-extended on external PRs, the SHA of the checked commit must be specified

(in response to this comment from @ross-p-smith)

[github-actions]

Unit Test Results

0 tests   0 ✔️  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit 254b589.

♻️ This comment has been updated with latest results.

[ross-p-smith]

/test-extended d5c9214

[github-actions]

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/2733686924 (with refid 39a3e9c8)

(in response to this comment from @ross-p-smith)

[marrobi]

We need this - #1846 completing to enable this work.