Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Weight changes Dynamic Reload #5212

Merged
merged 41 commits into from
Mar 19, 2024
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
4229818
generate split clients tests
Mar 7, 2024
edd08f1
Merge branch 'main' into weight-changes-without-reload
Mar 7, 2024
2db6149
go mod tidy
Mar 7, 2024
382001e
pre-commit had told me to remove this before ¯\_(ツ)_/¯
Mar 7, 2024
65843d4
capitalise VariableNamer for linter
Mar 7, 2024
de0ee07
fix helm value
Mar 7, 2024
37d209f
remove unnecessary printlns
Mar 7, 2024
f73ea63
add python test for weight changes without reload flag
Mar 8, 2024
ae43a02
Merge branch 'main' into weight-changes-without-reload
Mar 8, 2024
e7e1e0c
add vsr python test
Mar 11, 2024
e8d5d98
delete unnecessary logs
Mar 11, 2024
2bcf130
remove magic number
Mar 11, 2024
1e296c2
Merge branch 'main' into weight-changes-without-reload
Mar 11, 2024
b673f8b
move functionality from handlers.go to controller.go
Mar 13, 2024
381dd7e
Merge branch 'main' into weight-changes-without-reload
Mar 13, 2024
6dee37d
feature off by default
Mar 13, 2024
ecfac67
Merge branch 'main' into weight-changes-without-reload
Mar 13, 2024
106b30c
use go design reflect instead
Mar 13, 2024
1b530fb
remove redundant variable, add v3 logging
Mar 13, 2024
901fdbd
add variable
Mar 14, 2024
3fb9e5a
fix state_files folder permissions and add copier back
Mar 14, 2024
81b1ac7
Merge main
Mar 14, 2024
0049706
Revert "Merge main"
Mar 14, 2024
ad1ce82
Merge branch 'main' into weight-changes-without-reload
Mar 14, 2024
6272254
Merge branch 'main' into weight-changes-without-reload
Mar 14, 2024
bc4ea35
improvement of description of feature, disable v3 logging in python t…
Mar 15, 2024
e3703bd
description of feature
Mar 15, 2024
e752bfa
Test using many splits when using larger map bucket and variable buck…
Mar 15, 2024
900df19
wait before test
Mar 15, 2024
231ad88
isolate tests
Mar 15, 2024
526f4a8
further isolate tests
Mar 15, 2024
4094efe
Merge branch 'main' into weight-changes-without-reload
Mar 15, 2024
87b4311
fix virtual_server_test.go
Mar 15, 2024
14fc319
fix status and event processing
Mar 19, 2024
4f09779
Merge branch 'main' into weight-changes-without-reload
Mar 19, 2024
e149fb5
change to without-reload to dynamic reload
Mar 19, 2024
7523d4a
Merge branch 'main' into weight-changes-without-reload
Mar 19, 2024
00084c5
add warning log
Mar 19, 2024
e5ff2ea
Add link to configmap and improve logs
Mar 19, 2024
68a9cdf
Merge branch 'main' into weight-changes-without-reload
Mar 19, 2024
d396110
Merge branch 'main' into weight-changes-without-reload
Mar 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions build/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -375,6 +375,7 @@ RUN --mount=type=bind,target=/tmp mkdir -p /var/lib/nginx /etc/nginx/secrets /et
&& setcap -v 'cap_net_bind_service=+eip' /usr/sbin/nginx 'cap_net_bind_service=+eip' /usr/sbin/nginx-debug \
&& [ -z "${BUILD_OS##*plus*}" ] && PLUS=-plus; cp -a /tmp/internal/configs/version1/nginx$PLUS.ingress.tmpl /tmp/internal/configs/version1/nginx$PLUS.tmpl \
/tmp/internal/configs/version2/nginx$PLUS.virtualserver.tmpl /tmp/internal/configs/version2/nginx$PLUS.transportserver.tmpl / \
&& if [ -z "${BUILD_OS##*plus*}" ]; then mkdir -p /etc/nginx/state_files/; fi \
&& chown -R 101:0 /etc/nginx /var/cache/nginx /var/lib/nginx /var/log/nginx /*.tmpl \
&& chmod -R g=u /etc/nginx /var/cache/nginx /var/lib/nginx /var/log/nginx /*.tmpl \
&& rm -f /etc/nginx/conf.d/*
Expand Down
1 change: 1 addition & 0 deletions charts/nginx-ingress/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -471,6 +471,7 @@ The following tables lists the configurable parameters of the NGINX Ingress Cont
|`controller.readOnlyRootFilesystem` | Configure root filesystem as read-only and add volumes for temporary data. Three major releases after 3.5.x this argument will be moved permanently to the `controller.securityContext` section. | false |
|`controller.enableSSLDynamicReload` | Enable lazy loading for SSL Certificates. | true |
|`controller.telemetryReporting.enable` | Enable telemetry reporting. | true |
|`controller.enableWeightChangesWithoutReload` | Enable weight changes without reloading the NGINX configuration. | false |
j1m-ryan marked this conversation as resolved.
Show resolved Hide resolved
|`rbac.create` | Configures RBAC. | true |
|`prometheus.create` | Expose NGINX or NGINX Plus metrics in the Prometheus format. | true |
|`prometheus.port` | Configures the port to scrape the metrics. | 9113 |
Expand Down
1 change: 1 addition & 0 deletions charts/nginx-ingress/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,7 @@ Build the args for the service binary.
- -enable-latency-metrics={{ .Values.controller.enableLatencyMetrics }}
- -ssl-dynamic-reload={{ .Values.controller.enableSSLDynamicReload }}
- -enable-telemetry-reporting={{ .Values.controller.telemetryReporting.enable}}
- -weight-changes-without-reload={{ .Values.controller.enableWeightChangesWithoutReload}}
{{- if .Values.nginxAgent.enable }}
- -agent=true
- -agent-instance-group={{ default (include "nginx-ingress.controller.fullname" .) .Values.nginxAgent.instanceGroup }}
Expand Down
8 changes: 8 additions & 0 deletions charts/nginx-ingress/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -1432,6 +1432,14 @@
]
}
}
},
"enableWeightChangesWithoutReload": {
"type": "boolean",
"default": false,
"title": "Enables weight changes without reloading for NGINX Plus",
"examples": [
false
]
}
},
"examples": [
Expand Down
5 changes: 5 additions & 0 deletions charts/nginx-ingress/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -492,6 +492,11 @@ controller:
## Enable telemetry reporting
enable: true

## Enables weight changes without reloading the NGINX Configuration for two way splits in NGINX Plus.
## Enabling this feature may require increasing map_hash_bucket_size, map_hash_max_size,
## variable_hash_bucket_size and variable_hash_max_size in the ConfigMap, depending on the number of two way splits.
enableWeightChangesWithoutReload: false

rbac:
## Configures RBAC.
create: true
Expand Down
9 changes: 8 additions & 1 deletion cmd/nginx-ingress/flags.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ import (
)

const (
dynamicSSLReloadParam = "ssl-dynamic-reload"
dynamicSSLReloadParam = "ssl-dynamic-reload"
weightChangesWithoutReloadParam = "weight-changes-without-reload"
j1m-ryan marked this conversation as resolved.
Show resolved Hide resolved
)

var (
Expand Down Expand Up @@ -206,6 +207,8 @@ var (

enableTelemetryReporting = flag.Bool("enable-telemetry-reporting", true, "Enable gathering and reporting of product related telemetry.")

enableWeightChangesWithoutReload = flag.Bool(weightChangesWithoutReloadParam, false, "Enable changing weights of split clients without reloading NGINX. Requires -nginx-plus")

startupCheckFn func() error
)

Expand Down Expand Up @@ -281,6 +284,10 @@ func parseFlags() {
glog.Fatal("ingresslink and external-service cannot both be set")
}

if *enableWeightChangesWithoutReload && !*nginxPlus {
*enableWeightChangesWithoutReload = false
}

if *agent && !*appProtect {
glog.Fatal("NGINX Agent is used to enable the Security Monitoring dashboard and requires NGINX App Protect to be enabled")
}
Expand Down
29 changes: 16 additions & 13 deletions cmd/nginx-ingress/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,7 @@ func main() {
SSLRejectHandshake: sslRejectHandshake,
EnableCertManager: *enableCertManager,
DynamicSSLReload: *enableDynamicSSLReload,
WeightChangesWithoutReload: *enableWeightChangesWithoutReload,
StaticSSLPath: nginxManager.GetSecretsDir(),
NginxVersion: nginxVersion,
}
Expand All @@ -140,19 +141,20 @@ func main() {

plusCollector, syslogListener, latencyCollector := createPlusAndLatencyCollectors(registry, constLabels, kubeClient, plusClient, staticCfgParams.NginxServiceMesh)
cnf := configs.NewConfigurator(configs.ConfiguratorParams{
NginxManager: nginxManager,
StaticCfgParams: staticCfgParams,
Config: cfgParams,
TemplateExecutor: templateExecutor,
TemplateExecutorV2: templateExecutorV2,
LatencyCollector: latencyCollector,
LabelUpdater: plusCollector,
IsPlus: *nginxPlus,
IsWildcardEnabled: isWildcardEnabled,
IsPrometheusEnabled: *enablePrometheusMetrics,
IsLatencyMetricsEnabled: *enableLatencyMetrics,
IsDynamicSSLReloadEnabled: *enableDynamicSSLReload,
NginxVersion: nginxVersion,
NginxManager: nginxManager,
StaticCfgParams: staticCfgParams,
Config: cfgParams,
TemplateExecutor: templateExecutor,
TemplateExecutorV2: templateExecutorV2,
LatencyCollector: latencyCollector,
LabelUpdater: plusCollector,
IsPlus: *nginxPlus,
IsWildcardEnabled: isWildcardEnabled,
IsPrometheusEnabled: *enablePrometheusMetrics,
IsLatencyMetricsEnabled: *enableLatencyMetrics,
IsDynamicSSLReloadEnabled: *enableDynamicSSLReload,
IsWeightChangesWithoutReloadEnabled: *enableWeightChangesWithoutReload,
NginxVersion: nginxVersion,
})

controllerNamespace := os.Getenv("POD_NAMESPACE")
Expand Down Expand Up @@ -211,6 +213,7 @@ func main() {
WatchNamespaceLabel: *watchNamespaceLabel,
EnableTelemetryReporting: *enableTelemetryReporting,
NICVersion: version,
WeightChangesWithoutReload: *enableWeightChangesWithoutReload,
}

lbc := k8s.NewLoadBalancerController(lbcInput)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -536,6 +536,18 @@ The default value is `true`.

<a name="cmdoption-ssl-dynamic-reload"></a>

### -weight-changes-without-reload

Enables the ability to change the weight of split clients with two splits without reloading NGINX.

Requires [-nginx-plus](#cmdoption-nginx-plus).

The default value is `false`.

- If the argument is set, but `nginx-plus` is set to false, NGINX Ingress Controller will ignore the flag.

<a name="cmdoption-weight-changes-without-reload"></a>

### -enable-telemetry-reporting

Enable gathering and reporting of software telemetry.
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ require (
github.com/golang-jwt/jwt/v4 v4.5.0
github.com/golang/glog v1.1.2
github.com/google/go-cmp v0.6.0
github.com/jinzhu/copier v0.4.0
github.com/kr/pretty v0.3.1
github.com/nginxinc/nginx-plus-go-client v1.2.0
github.com/nginxinc/nginx-prometheus-exporter v1.1.0
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,8 @@ github.com/imdario/mergo v0.3.16 h1:wwQJbIsHYGMUyLSPrEq1CT16AhnhNJQ51+4fdHUnCl4=
github.com/imdario/mergo v0.3.16/go.mod h1:WBLT9ZmE3lPoWsEzCh9LPo3TiwVN+ZKEjmz+hD27ysY=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/jinzhu/copier v0.4.0 h1:w3ciUoD19shMCRargcpm0cm91ytaBhDvuRpz1ODO/U8=
github.com/jinzhu/copier v0.4.0/go.mod h1:DfbEm0FYsaqBcKcFuvmOZb218JkPGtvSHsKg8S8hyyg=
github.com/jonboulle/clockwork v0.2.2 h1:UOGuzwb1PwsrDAObMuhUnj0p5ULPj8V/xJ7Kx9qUBdQ=
github.com/jonboulle/clockwork v0.2.2/go.mod h1:Pkfl5aHPm1nk2H9h0bjmnJD/BcgbGXUBGnn1kMkgxc8=
github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
Expand Down
1 change: 1 addition & 0 deletions internal/configs/config_params.go
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,7 @@ type StaticConfigParams struct {
EnableCertManager bool
DynamicSSLReload bool
StaticSSLPath string
WeightChangesWithoutReload bool
NginxVersion nginx.Version
}

Expand Down
Loading
Loading