(backport): Juju, charmcraft, and dependency upgrades

.github/workflows/integrate.yaml

@@ -73,9 +73,10 @@ jobs:
       uses: charmed-kubernetes/actions-operator@main
       with:
         provider: microk8s
-        channel: 1.29-strict/stable
-        juju-channel: 3.4/stable
+        channel: 1.31-strict/stable
+        juju-channel: 3.6/stable
         microk8s-addons: "dns storage rbac metallb:10.64.140.43-10.64.140.49"
+        charmcraft-channel: 3.x/stable
 
     - name: Run integration tests
       run: tox -vve integration -- --model testing

.github/workflows/publish.yaml

@@ -91,4 +91,4 @@ jobs:
           charm-path: ${{ matrix.charm-path }}
           channel: ${{ steps.parse-inputs.outputs.destination_channel }}
           tag-prefix: ${{ steps.parse-inputs.outputs.tag_prefix }}
-          charmcraft-channel: latest/candidate
+          charmcraft-channel: 3.x/stable

charmcraft.yaml

@@ -12,3 +12,9 @@ bases:
 parts:
   charm:
     charm-python-packages: [setuptools, pip]
+    # Following lines are needed due to https://github.com/canonical/charmcraft/issues/1722
+    build-snaps: [rustup]
+    build-packages: [pkg-config, libffi-dev, libssl-dev]
+    override-build: |
+      rustup default stable
+      craftctl default

lib/charms/observability_libs/v1/kubernetes_service_patch.py

@@ -1,138 +1,26 @@
 # Copyright 2021 Canonical Ltd.
 # See LICENSE file for licensing details.
 
-"""# KubernetesServicePatch Library.
-
-This library is designed to enable developers to more simply patch the Kubernetes Service created
-by Juju during the deployment of a sidecar charm. When sidecar charms are deployed, Juju creates a
-service named after the application in the namespace (named after the Juju model). This service by
-default contains a "placeholder" port, which is 65536/TCP.
-
-When modifying the default set of resources managed by Juju, one must consider the lifecycle of the
-charm. In this case, any modifications to the default service (created during deployment), will be
-overwritten during a charm upgrade.
-
-When initialised, this library binds a handler to the parent charm's `install` and `upgrade_charm`
-events which applies the patch to the cluster. This should ensure that the service ports are
-correct throughout the charm's life.
-
-The constructor simply takes a reference to the parent charm, and a list of
-[`lightkube`](https://github.com/gtsystem/lightkube) ServicePorts that each define a port for the
-service. For information regarding the `lightkube` `ServicePort` model, please visit the
-`lightkube` [docs](https://gtsystem.github.io/lightkube-models/1.23/models/core_v1/#serviceport).
-
-Optionally, a name of the service (in case service name needs to be patched as well), labels,
-selectors, and annotations can be provided as keyword arguments.
-
-## Getting Started
-
-To get started using the library, you just need to fetch the library using `charmcraft`. **Note
-that you also need to add `lightkube` and `lightkube-models` to your charm's `requirements.txt`.**
-
-```shell
-cd some-charm
-charmcraft fetch-lib charms.observability_libs.v1.kubernetes_service_patch
-cat << EOF >> requirements.txt
-lightkube
-lightkube-models
-EOF
-```
-
-Then, to initialise the library:
-
-For `ClusterIP` services:
-
-```python
-# ...
-from charms.observability_libs.v1.kubernetes_service_patch import KubernetesServicePatch
-from lightkube.models.core_v1 import ServicePort
-
-class SomeCharm(CharmBase):
-  def __init__(self, *args):
-    # ...
-    port = ServicePort(443, name=f"{self.app.name}")
-    self.service_patcher = KubernetesServicePatch(self, [port])
-    # ...
-```
-
-For `LoadBalancer`/`NodePort` services:
-
-```python
-# ...
-from charms.observability_libs.v1.kubernetes_service_patch import KubernetesServicePatch
-from lightkube.models.core_v1 import ServicePort
-
-class SomeCharm(CharmBase):
-  def __init__(self, *args):
-    # ...
-    port = ServicePort(443, name=f"{self.app.name}", targetPort=443, nodePort=30666)
-    self.service_patcher = KubernetesServicePatch(
-        self, [port], "LoadBalancer"
-    )
-    # ...
-```
-
-Port protocols can also be specified. Valid protocols are `"TCP"`, `"UDP"`, and `"SCTP"`
-
-```python
-# ...
-from charms.observability_libs.v1.kubernetes_service_patch import KubernetesServicePatch
-from lightkube.models.core_v1 import ServicePort
-
-class SomeCharm(CharmBase):
-  def __init__(self, *args):
-    # ...
-    tcp = ServicePort(443, name=f"{self.app.name}-tcp", protocol="TCP")
-    udp = ServicePort(443, name=f"{self.app.name}-udp", protocol="UDP")
-    sctp = ServicePort(443, name=f"{self.app.name}-sctp", protocol="SCTP")
-    self.service_patcher = KubernetesServicePatch(self, [tcp, udp, sctp])
-    # ...
-```
-
-Bound with custom events by providing `refresh_event` argument:
-For example, you would like to have a configurable port in your charm and want to apply
-service patch every time charm config is changed.
-
-```python
-from charms.observability_libs.v1.kubernetes_service_patch import KubernetesServicePatch
-from lightkube.models.core_v1 import ServicePort
-
-class SomeCharm(CharmBase):
-  def __init__(self, *args):
-    # ...
-    port = ServicePort(int(self.config["charm-config-port"]), name=f"{self.app.name}")
-    self.service_patcher = KubernetesServicePatch(
-        self,
-        [port],
-        refresh_event=self.on.config_changed
-    )
-    # ...
-```
-
-Additionally, you may wish to use mocks in your charm's unit testing to ensure that the library
-does not try to make any API calls, or open any files during testing that are unlikely to be
-present, and could break your tests. The easiest way to do this is during your test `setUp`:
-
-```python
-# ...
-
-@patch("charm.KubernetesServicePatch", lambda x, y: None)
-def setUp(self, *unused):
-    self.harness = Harness(SomeCharm)
-    # ...
-```
+"""# [DEPRECATED!] KubernetesServicePatch Library.
+
+The `kubernetes_service_patch` library is DEPRECATED and will be removed in October 2025.
+
+For patching the Kubernetes service created by Juju during the deployment of a charm,
+`ops.Unit.set_ports` functionality should be used instead.
+
 """
 
 import logging
 from types import MethodType
-from typing import List, Literal, Optional, Union
+from typing import Any, List, Literal, Optional, Union
 
-from lightkube import ApiError, Client
+from lightkube import ApiError, Client  # pyright: ignore
 from lightkube.core import exceptions
 from lightkube.models.core_v1 import ServicePort, ServiceSpec
 from lightkube.models.meta_v1 import ObjectMeta
 from lightkube.resources.core_v1 import Service
 from lightkube.types import PatchType
+from ops import UpgradeCharmEvent
 from ops.charm import CharmBase
 from ops.framework import BoundEvent, Object
 
@@ -146,7 +34,7 @@ def setUp(self, *unused):
 
 # Increment this PATCH version before using `charmcraft publish-lib` or reset
 # to 0 if you are raising the major API version
-LIBPATCH = 6
+LIBPATCH = 13
 
 ServiceType = Literal["ClusterIP", "LoadBalancer"]
 
@@ -184,12 +72,22 @@ def __init__(
                 will be observed to re-apply the patch (e.g. on port change).
                 The `install` and `upgrade-charm` events would be observed regardless.
         """
+        logger.warning(
+            "The ``kubernetes_service_patch v1`` library is DEPRECATED and will be removed "
+            "in October 2025. For patching the Kubernetes service created by Juju during "
+            "the deployment of a charm, ``ops.Unit.set_ports`` functionality should be used instead."
+        )
         super().__init__(charm, "kubernetes-service-patch")
         self.charm = charm
-        self.service_name = service_name if service_name else self._app
+        self.service_name = service_name or self._app
+        # To avoid conflicts with the default Juju service, append "-lb" to the service name.
+        # The Juju application name is retained for the default service created by Juju.
+        if self.service_name == self._app and service_type == "LoadBalancer":
+            self.service_name = f"{self._app}-lb"
+        self.service_type = service_type
         self.service = self._service_object(
             ports,
-            service_name,
+            self.service_name,
             service_type,
             additional_labels,
             additional_selectors,
@@ -200,8 +98,11 @@ def __init__(
         assert isinstance(self._patch, MethodType)
         # Ensure this patch is applied during the 'install' and 'upgrade-charm' events
         self.framework.observe(charm.on.install, self._patch)
-        self.framework.observe(charm.on.upgrade_charm, self._patch)
+        self.framework.observe(charm.on.upgrade_charm, self._on_upgrade_charm)
         self.framework.observe(charm.on.update_status, self._patch)
+        # Sometimes Juju doesn't clean-up a manually created LB service,
+        # so we clean it up ourselves just in case.
+        self.framework.observe(charm.on.remove, self._remove_service)
 
         # apply user defined events
         if refresh_event:
@@ -268,7 +169,7 @@ def _patch(self, _) -> None:
             PatchFailed: if patching fails due to lack of permissions, or otherwise.
         """
         try:
-            client = Client()
+            client = Client()  # pyright: ignore
         except exceptions.ConfigError as e:
             logger.warning("Error creating k8s client: %s", e)
             return
@@ -277,7 +178,10 @@ def _patch(self, _) -> None:
             if self._is_patched(client):
                 return
             if self.service_name != self._app:
-                self._delete_and_create_service(client)
+                if not self.service_type == "LoadBalancer":
+                    self._delete_and_create_service(client)
+                else:
+                    self._create_lb_service(client)
             client.patch(Service, self.service_name, self.service, patch_type=PatchType.MERGE)
         except ApiError as e:
             if e.status.code == 403:
@@ -294,13 +198,19 @@ def _delete_and_create_service(self, client: Client):
         client.delete(Service, self._app, namespace=self._namespace)
         client.create(service)
 
+    def _create_lb_service(self, client: Client):
+        try:
+            client.get(Service, self.service_name, namespace=self._namespace)
+        except ApiError:
+            client.create(self.service)
+
     def is_patched(self) -> bool:
         """Reports if the service patch has been applied.
 
         Returns:
             bool: A boolean indicating if the service patch has been applied.
         """
-        client = Client()
+        client = Client()  # pyright: ignore
         return self._is_patched(client)
 
     def _is_patched(self, client: Client) -> bool:
@@ -310,18 +220,71 @@ def _is_patched(self, client: Client) -> bool:
         except ApiError as e:
             if e.status.code == 404 and self.service_name != self._app:
                 return False
-            else:
-                logger.error("Kubernetes service get failed: %s", str(e))
-                raise
+            logger.error("Kubernetes service get failed: %s", str(e))
+            raise
 
         # Construct a list of expected ports, should the patch be applied
-        expected_ports = [(p.port, p.targetPort) for p in self.service.spec.ports]
+        expected_ports = [(p.port, p.targetPort) for p in self.service.spec.ports]  # type: ignore[attr-defined]
         # Construct a list in the same manner, using the fetched service
         fetched_ports = [
             (p.port, p.targetPort) for p in service.spec.ports  # type: ignore[attr-defined]
         ]  # noqa: E501
         return expected_ports == fetched_ports
 
+    def _on_upgrade_charm(self, event: UpgradeCharmEvent):
+        """Handle the upgrade charm event."""
+        # If a charm author changed the service type from LB to ClusterIP across an upgrade, we need to delete the previous LB.
+        if self.service_type == "ClusterIP":
+
+            client = Client()  # pyright: ignore
+
+            # Define a label selector to find services related to the app
+            selector: dict[str, Any] = {"app.kubernetes.io/name": self._app}
+
+            # Check if any service of type LoadBalancer exists
+            services = client.list(Service, namespace=self._namespace, labels=selector)
+            for service in services:
+                if (
+                    not service.metadata
+                    or not service.metadata.name
+                    or not service.spec
+                    or not service.spec.type
+                ):
+                    logger.warning(
+                        "Service patch: skipping resource with incomplete metadata: %s.", service
+                    )
+                    continue
+                if service.spec.type == "LoadBalancer":
+                    client.delete(Service, service.metadata.name, namespace=self._namespace)
+                    logger.info(f"LoadBalancer service {service.metadata.name} deleted.")
+
+        # Continue the upgrade flow normally
+        self._patch(event)
+
+    def _remove_service(self, _):
+        """Remove a Kubernetes service associated with this charm.
+
+        Specifically designed to delete the load balancer service created by the charm, since Juju only deletes the
+        default ClusterIP service and not custom services.
+
+        Returns:
+            None
+
+        Raises:
+            ApiError: for deletion errors, excluding when the service is not found (404 Not Found).
+        """
+        client = Client()  # pyright: ignore
+
+        try:
+            client.delete(Service, self.service_name, namespace=self._namespace)
+            logger.info("The patched k8s service '%s' was deleted.", self.service_name)
+        except ApiError as e:
+            if e.status.code == 404:
+                # Service not found, so no action needed
+                return
+            # Re-raise for other statuses
+            raise
+
     @property
     def _app(self) -> str:
         """Name of the current Juju application.

requirements-fmt.txt

@@ -4,21 +4,21 @@
 #
 #    pip-compile requirements-fmt.in
 #
-black==23.3.0
+black==24.8.0
     # via -r requirements-fmt.in
-click==8.1.3
+click==8.1.7
     # via black
-isort==5.12.0
+isort==5.13.2
     # via -r requirements-fmt.in
 mypy-extensions==1.0.0
     # via black
-packaging==23.1
+packaging==24.2
     # via black
-pathspec==0.11.1
+pathspec==0.12.1
     # via black
-platformdirs==3.5.3
+platformdirs==4.3.6
     # via black
-tomli==2.0.1
+tomli==2.2.1
     # via black
-typing-extensions==4.6.3
+typing-extensions==4.12.2
     # via black