Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Azure integration #65

Merged
merged 106 commits into from
Aug 19, 2023
Merged
Show file tree
Hide file tree
Changes from 75 commits
Commits
Show all changes
106 commits
Select commit Hold shift + click to select a range
8d01a6b
add default integration folder structure
Tankilevitch Jul 24, 2023
a2df0fe
add blueprints.json and port-app-config.yaml basic example
Tankilevitch Jul 24, 2023
39d4e7e
override default port PortAppConfig to use AzurePortAppConfig
Tankilevitch Jul 24, 2023
d2cdbac
implement `on_resync` & `route`
Tankilevitch Jul 24, 2023
2f13dce
add import for azure integration ocean implementation
Tankilevitch Jul 24, 2023
75dc0c6
fix containerapp blueprint
Tankilevitch Jul 26, 2023
1841fb2
add query to port-app-config
Tankilevitch Jul 26, 2023
85935c9
set initializePortResources: true and configure Kafka as eventListener
Tankilevitch Jul 26, 2023
5d6e80b
add cloud event middleware handler
Tankilevitch Jul 26, 2023
eab56d5
add middleware to ocean app
Tankilevitch Jul 26, 2023
9a04780
add event handler and resync implementation
Tankilevitch Jul 26, 2023
b36f951
Merge branch 'main' of github.com:port-labs/Port-Ocean into add-azure…
Tankilevitch Jul 26, 2023
e4b780e
update .port files
Tankilevitch Jul 27, 2023
f69354a
add azure patch
Tankilevitch Jul 27, 2023
de63443
adjust logic to use the azure patch
Tankilevitch Jul 27, 2023
2d72174
add dependencies
Tankilevitch Jul 27, 2023
6a659be
handle events from port where entities are being deleted
Tankilevitch Jul 27, 2023
f6bc3f1
Merge branch 'main' of github.com:port-labs/Port-Ocean into add-azure…
Tankilevitch Jul 27, 2023
bac677e
fix to use snake_case
Tankilevitch Jul 27, 2023
8ef3ee9
comment out to support envs
Tankilevitch Jul 27, 2023
8261575
bump to rc2
Tankilevitch Jul 27, 2023
64abece
support old and new configuration
Tankilevitch Jul 27, 2023
b73c7fe
bump rc3
Tankilevitch Jul 27, 2023
c589d05
add logs
Tankilevitch Jul 27, 2023
1079371
rc4
Tankilevitch Jul 27, 2023
ce96829
don't log the whole request
Tankilevitch Jul 27, 2023
599f759
bump rc5
Tankilevitch Jul 27, 2023
d869c4a
update to debug logger
Tankilevitch Jul 28, 2023
6adb069
update query
Tankilevitch Jul 28, 2023
70b0cae
update requirements
Tankilevitch Jul 28, 2023
cbca072
fix middleware
Tankilevitch Jul 28, 2023
b433bdd
bump rc6
Tankilevitch Jul 28, 2023
7c7c4ed
Merge branch 'main' of github.com:port-labs/Port-Ocean into PORT-4351…
Tankilevitch Aug 6, 2023
7930f5e
fix
Tankilevitch Aug 6, 2023
683c48e
Merge branch 'main' of github.com:port-labs/Port-Ocean into PORT-4351…
Tankilevitch Aug 6, 2023
aa3110f
update requirements adjust config.yaml bumprc7
Tankilevitch Aug 7, 2023
cdcb09e
add terraform base implementation for azure
Tankilevitch Aug 8, 2023
4f46a8a
bump port ocean requirement 0.1.3
Tankilevitch Aug 8, 2023
3936447
bump rc8
Tankilevitch Aug 8, 2023
f9dc605
minor fixes
Tankilevitch Aug 8, 2023
9df3f00
update terraform modules
Tankilevitch Aug 9, 2023
3d38ecc
update blueprints and app config
Tankilevitch Aug 9, 2023
c0ac847
bump rc9
Tankilevitch Aug 9, 2023
02e6c0c
fix
Tankilevitch Aug 9, 2023
53c47a1
bump rc10
Tankilevitch Aug 9, 2023
bbce6b7
align blueprints and mapping
Tankilevitch Aug 9, 2023
8348bda
add support for resource groups
Tankilevitch Aug 9, 2023
adbb6d0
lint
Tankilevitch Aug 9, 2023
2bbf513
bump rc11
Tankilevitch Aug 9, 2023
59e216f
add virtual machine support
Tankilevitch Aug 10, 2023
4de6ecb
add comment
Tankilevitch Aug 10, 2023
2e514bb
add main example
Tankilevitch Aug 10, 2023
490dbba
remove comment
Tankilevitch Aug 10, 2023
c4879e5
add providers.tf to example
Tankilevitch Aug 10, 2023
621aead
oops adjust to virtualMachines
Tankilevitch Aug 10, 2023
e8f4784
Merge branch 'main' of github.com:port-labs/Port-Ocean into PORT-4351…
Tankilevitch Aug 10, 2023
0aa4992
add description to azure modules variables
Tankilevitch Aug 10, 2023
4fe2879
adjust
Tankilevitch Aug 10, 2023
11395e1
adjust endpoint
Tankilevitch Aug 10, 2023
6892fe0
lower case the resource group name as well
Tankilevitch Aug 10, 2023
c60a3d8
add support for batch response
Tankilevitch Aug 10, 2023
ee21a83
bump ocean version to 0.2.1
Tankilevitch Aug 10, 2023
8e4df92
bump rc12
Tankilevitch Aug 10, 2023
da7c122
rename directory from containerapp to container_app
Tankilevitch Aug 10, 2023
e493352
move main
Tankilevitch Aug 10, 2023
903bbb6
move authorization directory outside of container_app directory to en…
Tankilevitch Aug 10, 2023
fd8abd5
return debugging stuff
Tankilevitch Aug 10, 2023
40427d2
add empty line to end of files
Tankilevitch Aug 10, 2023
e747d2f
rename method
Tankilevitch Aug 10, 2023
9a9beea
missing __init__.py
Tankilevitch Aug 10, 2023
f858268
Merge branch 'main' of github.com:port-labs/Port-Ocean into PORT-4351…
Tankilevitch Aug 11, 2023
faac869
removed cors middleware
Tankilevitch Aug 11, 2023
47a8024
CR comments and added support for extension resources
Tankilevitch Aug 11, 2023
4004747
separate pr
Tankilevitch Aug 13, 2023
c28d3a0
accepts variables as envs
Tankilevitch Aug 13, 2023
652fd89
make the subscription creation more dynamic and add variables
Tankilevitch Aug 13, 2023
1cf9a7b
minor
Tankilevitch Aug 13, 2023
9d64b42
fix blueprints per morpaz CR
Tankilevitch Aug 13, 2023
5c00662
fix linting
Tankilevitch Aug 13, 2023
e11f632
add virtualMachine permission
Tankilevitch Aug 13, 2023
5e92a68
spaces
Tankilevitch Aug 13, 2023
7429db3
add README.md
Tankilevitch Aug 13, 2023
f27dae0
adjust resources
Tankilevitch Aug 13, 2023
fdaa82e
remove
Tankilevitch Aug 13, 2023
a86ee4f
add changelog
Tankilevitch Aug 13, 2023
bd5fb78
Merge branch 'main' of github.com:port-labs/Port-Ocean into PORT-4351…
Tankilevitch Aug 13, 2023
e25a7ab
remove redundant
Tankilevitch Aug 13, 2023
1ebc497
bump rc13
Tankilevitch Aug 14, 2023
92af7ea
oops
Tankilevitch Aug 14, 2023
6944128
fix virtualMachines
Tankilevitch Aug 14, 2023
a8050ef
fix cast
Tankilevitch Aug 14, 2023
111db1c
bump rc
Tankilevitch Aug 14, 2023
a5a629f
fix cr comments
Tankilevitch Aug 14, 2023
7a61e7d
Merge branch 'main' of github.com:port-labs/Port-Ocean into PORT-4351…
Tankilevitch Aug 14, 2023
061e7a1
minor
Tankilevitch Aug 14, 2023
d39880e
minor
Tankilevitch Aug 14, 2023
1bb39a7
bump rc17 and minor fixes
Tankilevitch Aug 15, 2023
ce679c4
fix
Tankilevitch Aug 15, 2023
fa3b5e6
rc18
Tankilevitch Aug 15, 2023
a36172c
bump to release
Tankilevitch Aug 17, 2023
75346cf
update docs
Tankilevitch Aug 17, 2023
4624832
bump to 0.1.0
Tankilevitch Aug 17, 2023
089cc48
Merge branch 'main' into PORT-4351-azure-integration-powered-by-ocean
Tankilevitch Aug 17, 2023
ecc71c3
CR fixes
Tankilevitch Aug 18, 2023
53c4e87
Merge branch 'main' of github.com:port-labs/Port-Ocean into PORT-4351…
Tankilevitch Aug 18, 2023
3f0a6f4
Merge branch 'PORT-4351-azure-integration-powered-by-ocean' of github…
Tankilevitch Aug 18, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
variable "port_client_id" {
Tankilevitch marked this conversation as resolved.
Show resolved Hide resolved
type = string
}
variable "port_client_secret" {
type = string
}
variable "port_base_url" {
type = string
default = ""
}

module "ocean_integration" {
source = "../.."

# required port parameters so that the integration could communicate with Port
port = {
client_id = var.port_client_id
client_secret = var.port_client_secret
base_url = var.port_base_url
}

initialize_port_resources = true

# required port integration parameters so Port could identify the integration
integration = {
type = "azure"
identifier = "az1"
config = {
}
}
# optional port integration parameters
subscription_id = "/subscriptions/xxxxxx"
location = "East US 2"

image = "ghcr.io/port-labs/port-ocean-azure:v0.1.0rc11"
Tankilevitch marked this conversation as resolved.
Show resolved Hide resolved

permissions = {
actions = [
"microsoft.app/containerapps/read",
"Microsoft.Storage/storageAccounts/read",
"Microsoft.ContainerService/managedClusters/read",
"Microsoft.Network/loadBalancers/read",
"Microsoft.Resources/subscriptions/resourceGroups/read",
"Microsoft.Resources/subscriptions/resources/read",
]
not_actions = []
data_actions = []
not_data_actions = []
}

additional_secrets = {
OCEAN__INTEGRATION__CONFIG__SUBSCRIPTION_ID = "xxxxxxxxx"
}
additional_environment_variables = {
OCEAN__INTEGRATION__CONFIG__SOME_ENV_VAR = "some-value"
}
}

resource "azurerm_eventgrid_system_topic" "subscription_event_grid_topic" {
name = "subscription-event-grid-topic"
resource_group_name = module.ocean_integration.resource_group_name
location = "Global"
topic_type = "Microsoft.Resources.Subscriptions"
source_arm_resource_id = module.ocean_integration.subscription_id
}


resource "azurerm_eventgrid_system_topic_event_subscription" "subscription_event_grid_topic_subscription" {
name = replace(replace("ocean-${module.ocean_integration.integration.type}-${module.ocean_integration.integration.identifier}-subscription","_", "-"),".","-")
resource_group_name = azurerm_eventgrid_system_topic.subscription_event_grid_topic.resource_group_name
system_topic = azurerm_eventgrid_system_topic.subscription_event_grid_topic.name

included_event_types = [
Tankilevitch marked this conversation as resolved.
Show resolved Hide resolved
"Microsoft.Resources.ResourceWriteSuccess",
"Microsoft.Resources.ResourceWriteFailure",
"Microsoft.Resources.ResourceDeleteSuccess",
"Microsoft.Resources.ResourceDeleteFailure",
]
event_delivery_schema = "CloudEventSchemaV1_0"
webhook_endpoint {
url = "https://${module.ocean_integration.container_app_latest_fqdn}/integration/events"
}
advanced_filtering_on_arrays_enabled = true
advanced_filter {
string_contains {
key = "data.operationName"
values = [
"microsoft.app/containerapps",
"Microsoft.Storage/storageAccounts",
"Microsoft.ContainerService/managedClusters",
"Microsoft.Network/loadBalancers",
"Microsoft.Compute/virtualMachine",
"Microsoft.Resources/subscriptions/resourceGroups",
]
}
}
delivery_property {
header_name = "Access-Control-Request-Method"
type = "Static"
value = "POST"
}
delivery_property {
header_name = "Origin"
type = "Static"
value = "azure"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "3.68.0"
}
}
}
provider "azurerm" {
# The AzureRM Provider supports authenticating using via the Azure CLI, a Managed Identity
# and a Service Principal. More information on the authentication methods supported by
# the AzureRM Provider can be found here:
# https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs#authenticating-to-azure

# The features block allows changing the behaviour of the Azure Provider, more
# information can be found here:
# https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/features-block
features {}
}
36 changes: 36 additions & 0 deletions deployment/terraform/azure/container_app/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
locals {
prefix = "port-ocean"
}

resource "azurerm_resource_group" "ocean-rg" {
count = var.resource_group_name != null ? 0 : 1
name = "${local.prefix}-${var.integration.type}-${var.integration.identifier}-rg"
location = var.location
}

module "port_ocean_authorization" {
source = "../modules/authorization"
location = var.location
resource_group_name = var.resource_group_name != null ? var.resource_group_name : azurerm_resource_group.ocean-rg[0].name
integration = var.integration
permissions = var.permissions
subscription_id = var.subscription_id
}

module "port_ocean_container_app" {
source= "./modules/container_app"
integration = var.integration
port = var.port
initialize_port_resources = var.initialize_port_resources
location = var.location
resource_group_name = var.resource_group_name != null ? var.resource_group_name : azurerm_resource_group.ocean-rg[0].name
container_app_environment_id = var.container_app_environment_id
log_analytics_workspace_id = var.log_analytics_workspace_id
image = var.image
min_replicas = var.min_replicas
max_replicas = var.max_replicas
user_assigned_identity_ids = [module.port_ocean_authorization.user_assigned_identity_id]
user_assigned_client_id = module.port_ocean_authorization.user_assigned_identity_client_id
additional_secrets = var.additional_secrets
additional_environment_variables = var.additional_environment_variables
}
110 changes: 110 additions & 0 deletions deployment/terraform/azure/container_app/modules/container_app/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
locals {
prefix = "port-ocean"
env = [
{
name = upper("OCEAN__INITIALIZE_PORT_RESOURCES"),
value = var.initialize_port_resources ? "true" : "false"
},
{
name = upper("OCEAN__EVENT_LISTENER")
value = jsonencode({
for key, value in var.event_listener : key => value if value != null
})
},
{
name = upper("OCEAN__INTEGRATION")
value = jsonencode(var.integration)
}
]
port_credentials_secret_name = "ocean-port-credentials"
}

resource "azurerm_log_analytics_workspace" "ocean-log-analytics" {
count = var.log_analytics_workspace_id != null ? 0 : 1
name = "${local.prefix}-${var.integration.type}-${var.integration.identifier}-law"
location = var.location
resource_group_name = var.resource_group_name
sku = "PerGB2018"
retention_in_days = 30
}

resource "azurerm_container_app_environment" "ocean-container-app-env" {
count = var.container_app_environment_id != null ? 0 : 1
name = "${local.prefix}-${var.integration.type}-${var.integration.identifier}-env"
location = var.location
resource_group_name = var.resource_group_name
log_analytics_workspace_id = var.log_analytics_workspace_id != null ? var.log_analytics_workspace_id : azurerm_log_analytics_workspace.ocean-log-analytics[0].id
}


resource "azurerm_container_app" "ocean-container-app" {
name = "${local.prefix}-${var.integration.type}-${var.integration.identifier}"
container_app_environment_id = var.container_app_environment_id != null ? var.container_app_environment_id : azurerm_container_app_environment.ocean-container-app-env[0].id
resource_group_name = var.resource_group_name
revision_mode = "Single"
identity {
type = "UserAssigned"
identity_ids = var.user_assigned_identity_ids
}
ingress {
external_enabled = var.assign_public_ip
target_port = var.container_port
traffic_weight {
percentage = 100
latest_revision = true
}
}
template {
min_replicas = 1
max_replicas = 1
container {
name = "${local.prefix}-${var.integration.type}"
cpu = var.cpu
memory = var.memory
image = var.image != null ? var.image : "${var.image_registry}/port-ocean-${var.integration.type}:${var.integration_version}"
dynamic "env" {
for_each = local.env
content {
name = env.value.name
value = env.value.value
}
}
dynamic "env" {
for_each = var.additional_environment_variables
content {
name = env.key
value = env.value
}
}
env {
name = "AZURE_CLIENT_ID"
value = var.user_assigned_client_id
}
env {
name = "OCEAN__PORT"
secret_name = local.port_credentials_secret_name
}
dynamic "env" {
for_each = var.additional_secrets
content {
name = env.key
secret_name = replace("ocean-${lower(env.key)}", "_", "-")
}
}
}
}
secret {
name = local.port_credentials_secret_name
value = jsonencode({
for key, value in var.port : key => value if value != null
})
}
dynamic "secret" {
for_each = var.additional_secrets
content {
name = replace("ocean-${lower(secret.key)}", "_", "-")
value = secret.value
}
}
}

Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
output "container_app_latest_fqdn" {
value = azurerm_container_app.ocean-container-app.latest_revision_fqdn
}

output "container_app_outbound_ip_addresses" {
value = azurerm_container_app.ocean-container-app.outbound_ip_addresses
}

output "container_latest_revision_name" {
value = azurerm_container_app.ocean-container-app.latest_revision_name
}
Loading
Loading