Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Hot Reloading: Kubernetes Component hot reloading (dapr#7260)
* Hot Reloading: SelfHosted Componenthot reloading Part of dapr#1172 Adds hot reloading functionality to Daprd, which is gated behind the HotReload preview feature. If enabled, Daprd in Selfhosted mode will watch for resource updates from watching the resource directory(s). If a resource is detected as changed (spec is different to what is currently loaded), then Dapr will close/create/update the resource. Supports Component _except_ HTTP middlware which requires further changes to the HTTP server handler machinery. A warning is thrown if hot reloading is enabled and a HTTP middleware is updated. The hot reloader reconciler is made generic to enable other resource types to be added in future. --- When running in standalone mode, the disk loader watches the directories passed by --resources-path for updates to yaml files containing Components. When an event occurs, the reconciler and differ are responsible for determining whether any resources have been created, updated, or deleted, by comparing the local store specs with that of the remote. If any have changed, the resource is closed (if it exists), and then re-initialized (if it exists). A resource will only be closed if it has been deleted, only initialized if it has been created, and closed & initialized if it has been updated. We consider a resource to have changed generally if anything apart from its Kubernetes metadata or type meta object meta has changed, and therefore needs some kind of reloading. --- Currently, if a reloading component errors and `spec.ignoreErrors=false` then Daprd will gracefully close, like a component loaded at startup today. It is intended that in future the component will be re-inited on a backoff queue in future in the case of errors, even if `spec.ignoreErros=true`. HTTP middleware component hot reloading is not supported as it requires further changes to the HTTP server handler machinery, and I didn't want to grow this PR further. To use the HotReloading feature, which is currently only available as an alpha feature in Selfhosted mode, users need to add the following Configuration to the target Daprd; ```yaml apiVersion: dapr.io/v1alpha1 kind: Configuration metadata: name: hotreloading spec: features: - name: HotReload enabled: true ``` Signed-off-by: joshvanl <me@joshvanl.dev> * Adds app ready checks for subscribed input bindings and pubsub Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Updates reconciler to wait for processor queue to be empty after each event Signed-off-by: joshvanl <me@joshvanl.dev> * Revert processor pending resource queue to 0 channel Signed-off-by: joshvanl <me@joshvanl.dev> * Review comments Signed-off-by: joshvanl <me@joshvanl.dev> * Linitng Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Adds review comments Signed-off-by: joshvanl <me@joshvanl.dev> * Hot Reloading: SelfHosted Componenthot reloading Part of dapr#1172 Adds hot reloading functionality to Daprd, which is gated behind the HotReload preview feature. If enabled, Daprd in Selfhosted mode will watch for resource updates from watching the resource directory(s). If a resource is detected as changed (spec is different to what is currently loaded), then Dapr will close/create/update the resource. Supports Component _except_ HTTP middlware which requires further changes to the HTTP server handler machinery. A warning is thrown if hot reloading is enabled and a HTTP middleware is updated. The hot reloader reconciler is made generic to enable other resource types to be added in future. --- When running in standalone mode, the disk loader watches the directories passed by --resources-path for updates to yaml files containing Components. When an event occurs, the reconciler and differ are responsible for determining whether any resources have been created, updated, or deleted, by comparing the local store specs with that of the remote. If any have changed, the resource is closed (if it exists), and then re-initialized (if it exists). A resource will only be closed if it has been deleted, only initialized if it has been created, and closed & initialized if it has been updated. We consider a resource to have changed generally if anything apart from its Kubernetes metadata or type meta object meta has changed, and therefore needs some kind of reloading. --- Currently, if a reloading component errors and `spec.ignoreErrors=false` then Daprd will gracefully close, like a component loaded at startup today. It is intended that in future the component will be re-inited on a backoff queue in future in the case of errors, even if `spec.ignoreErros=true`. HTTP middleware component hot reloading is not supported as it requires further changes to the HTTP server handler machinery, and I didn't want to grow this PR further. To use the HotReloading feature, which is currently only available as an alpha feature in Selfhosted mode, users need to add the following Configuration to the target Daprd; ```yaml apiVersion: dapr.io/v1alpha1 kind: Configuration metadata: name: hotreloading spec: features: - name: HotReload enabled: true ``` Signed-off-by: joshvanl <me@joshvanl.dev> * Adds app ready checks for subscribed input bindings and pubsub Signed-off-by: joshvanl <me@joshvanl.dev> * Hot Reloading: Operator Componenthot reloading Branched from dapr#7239 Part of dapr#1172 Adds hot reloading functionality to Daprd, which is gated behind the HotReload preview feature. If enabled, Daprd in Kubernetes mode will watch for Component resource updates from the Operator. If a resource is detected as changed (spec is different to what is currently loaded), then Dapr will close/create/update the resource. Supports Component _except_ HTTP middlware which requires further changes to the HTTP server handler machinery. A warning is thrown if hot reloading is enabled and a HTTP middleware is updated. --- When running in Kubernetes mode, the operator loader streams updates from the Operator API. When an event occurs, the event type will trigger the resource to be created, updated, or deleted. Updated components will be closed, and then re-initialized. We consider a resource to have changed generally if anything apart from its Kubernetes metadata or type meta object meta has changed, and therefore needs some kind of reloading. --- Currently, if a reloading component errors and `spec.ignoreErrors=false` then Daprd will gracefully close, like a component loaded at startup today. It is intended that in future the component will be re-inited on a backoff queue in future in the case of errors, even if `spec.ignoreErros=true`. HTTP middleware component hot reloading is not supported as it requires further changes to the HTTP server handler machinery, and I didn't want to grow this PR further. To use the HotReloading feature, which is currently only available as an alpha feature in Selfhosted mode, users need to add the following Configuration to the target Daprd; ```yaml apiVersion: dapr.io/v1alpha1 kind: Configuration metadata: name: hotreloading spec: features: - name: HotReload enabled: true ``` Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Adds hotreloading e2e config Signed-off-by: joshvanl <me@joshvanl.dev> * Use correct URL endpoints for hotreloading e2e save Signed-off-by: joshvanl <me@joshvanl.dev> * Revert integration test case timeout to 180 seconds Signed-off-by: joshvanl <me@joshvanl.dev> * Fix hotreloading_test.go Signed-off-by: joshvanl <me@joshvanl.dev> * Use correct external URL target Signed-off-by: joshvanl <me@joshvanl.dev> * Lint code Signed-off-by: joshvanl <me@joshvanl.dev> * Hot Reloading: SelfHosted Componenthot reloading Part of dapr#1172 Adds hot reloading functionality to Daprd, which is gated behind the HotReload preview feature. If enabled, Daprd in Selfhosted mode will watch for resource updates from watching the resource directory(s). If a resource is detected as changed (spec is different to what is currently loaded), then Dapr will close/create/update the resource. Supports Component _except_ HTTP middlware which requires further changes to the HTTP server handler machinery. A warning is thrown if hot reloading is enabled and a HTTP middleware is updated. The hot reloader reconciler is made generic to enable other resource types to be added in future. --- When running in standalone mode, the disk loader watches the directories passed by --resources-path for updates to yaml files containing Components. When an event occurs, the reconciler and differ are responsible for determining whether any resources have been created, updated, or deleted, by comparing the local store specs with that of the remote. If any have changed, the resource is closed (if it exists), and then re-initialized (if it exists). A resource will only be closed if it has been deleted, only initialized if it has been created, and closed & initialized if it has been updated. We consider a resource to have changed generally if anything apart from its Kubernetes metadata or type meta object meta has changed, and therefore needs some kind of reloading. --- Currently, if a reloading component errors and `spec.ignoreErrors=false` then Daprd will gracefully close, like a component loaded at startup today. It is intended that in future the component will be re-inited on a backoff queue in future in the case of errors, even if `spec.ignoreErros=true`. HTTP middleware component hot reloading is not supported as it requires further changes to the HTTP server handler machinery, and I didn't want to grow this PR further. To use the HotReloading feature, which is currently only available as an alpha feature in Selfhosted mode, users need to add the following Configuration to the target Daprd; ```yaml apiVersion: dapr.io/v1alpha1 kind: Configuration metadata: name: hotreloading spec: features: - name: HotReload enabled: true ``` Signed-off-by: joshvanl <me@joshvanl.dev> * Adds app ready checks for subscribed input bindings and pubsub Signed-off-by: joshvanl <me@joshvanl.dev> * Linitng Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Linting Signed-off-by: joshvanl <me@joshvanl.dev> * Review comments Signed-off-by: joshvanl <me@joshvanl.dev> * Review comments Signed-off-by: joshvanl <me@joshvanl.dev> * Set eventually in crypto test to 5 secs Signed-off-by: joshvanl <me@joshvanl.dev> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Signed-off-by: Cassandra Coyle <cassie@diagrid.io>
- Loading branch information