updated byovpc network config for GCP

[stikkireddy]

• [ ] create doc updates/guide updates
• test creating mws for aws with older version and then upgrade and see impact on previous state
• unit tests

Attempts to resolve #1393

byovpc example:

variable "databricks_account_id" {
  default = "myaccount-id"
}

variable "google_project" {
  default = "my-project"
}

terraform {
  required_providers {
    google = {
      source = "hashicorp/google"
      version = "4.37.0"
    }
    databricks = {
      source = "databricks/databricks"
      version = "1.6.5" # compile and install it as a new version
    }

  }
}


provider "google" {
  project     = "my-project"
  region      = "us-central1"
  credentials = ".../somecredfile.json"
}

provider "databricks" {
  host = "https://accounts.gcp.databricks.com"
  google_credentials = ".../somecredfile.json"
}

resource "random_string" "suffix" {
  special = false
  upper   = false
  length  = 6
}

resource "google_compute_network" "dbx_private_vpc" {
  project                 = var.google_project
  name                    = "tf-network-${random_string.suffix.result}"
  auto_create_subnetworks = false
#  mtu                     = 1460
}

resource "google_compute_subnetwork" "network-with-private-secondary-ip-ranges" {
  name          = "test-dbx-${random_string.suffix.result}"
  ip_cidr_range = "10.0.0.0/16"
  region        = "us-central1"
  network       = google_compute_network.dbx_private_vpc.id
  secondary_ip_range {
    range_name    = "pods"
    ip_cidr_range = "10.1.0.0/16"
  }
  secondary_ip_range {
    range_name    = "svc"
    ip_cidr_range = "10.2.0.0/20"
  }
  private_ip_google_access = true
}

resource "google_compute_router" "router" {
  name    = "my-router"
  region  = google_compute_subnetwork.network-with-private-secondary-ip-ranges.region
  network = google_compute_network.dbx_private_vpc.id
}

resource "google_compute_router_nat" "nat" {
  name                               = "my-router-nat"
  router                             = google_compute_router.router.name
  region                             = google_compute_router.router.region
  nat_ip_allocate_option             = "AUTO_ONLY"
  source_subnetwork_ip_ranges_to_nat = "ALL_SUBNETWORKS_ALL_IP_RANGES"
}

resource "databricks_mws_networks" "this" {
  account_id     = var.databricks_account_id
  network_name = "test-demo-${random_string.suffix.result}"
  gcp_network_info {
    network_project_id = var.google_project
    vpc_id = google_compute_network.dbx_private_vpc.name
    subnet_id = google_compute_subnetwork.network-with-private-secondary-ip-ranges.name
    subnet_region = google_compute_subnetwork.network-with-private-secondary-ip-ranges.region
    pod_ip_range_name = "pods"
    service_ip_range_name = "svc"
  }
}

resource "databricks_mws_workspaces" "this" {
 account_id     = var.databricks_account_id
 workspace_name = "tf-demo-test-${random_string.suffix.result}"
 location       = google_compute_subnetwork.network-with-private-secondary-ip-ranges.region
   cloud_resource_bucket {
     gcp {
       project_id = var.google_project
     }
   }
  network {
    network_id = databricks_mws_networks.this.network_id
    gcp_common_network_config {
      gke_connectivity_type = "PRIVATE_NODE_PUBLIC_MASTER"
      gke_cluster_master_ip_range = "10.3.0.0/28"
    }
  }
  depends_on = [google_compute_router_nat.nat]
}

[nfx]

nit: please do one space instead of two

[alexott]

code looks good

[nfx]

Following up - is this issue still relevant?

[nfx]

Following up - is this issue still relevant?

[martinzuern]

This feature is highly relevant for me. Is there anything I can do to move this forward? @nfx @stikkireddy

[yanw2]

We need this feature for my current client. Can you please advise when it is available to use? Thank you.

[nfx]

@martinzuern @yanw2 we cannot merge this without an acceptance test

[yanw2]

@nfx - Thank you for your reply. Can you please advise ETC if possible? Thanks.

[nfx]

etc?

[yanw2]

Sorry, I mean "Estimate to Complete" :)

[mahmuzicamel]

Is this pull request still a work in progress? Do you have any infos for us when it is available to use, since it is a kind of big relevance for my client? Thank you

[martinzuern]

@nfx what would you need for said acceptance test? Anything I can do to help here?

[codecov-commenter]

Codecov Report

Merging #1748 (d99015a) into master (1ee4ba8) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1748   +/-   ##
=======================================
  Coverage   90.15%   90.15%           
=======================================
  Files         142      142           
  Lines       11377    11382    +5     
=======================================
+ Hits        10257    10262    +5     
  Misses        711      711           
  Partials      409      409           

Impacted Files	Coverage Δ
mws/resource_mws_workspaces.go	90.64% <ø> (ø)
mws/resource_mws_networks.go	83.60% <100.00%> (+1.46%)	⬆️

[nfx]

acceptance tests passed

[yanw2]

Hi @nfx, great thanks for fixing this. Can you please advise when you can publish this feature as a new release? It looks like it isn't included in the v1.7.0 release. Thank you.