Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/develop' into feature/XRDDEV-2728
Browse files Browse the repository at this point in the history
  • Loading branch information
@figueroarvictor
figueroarvictor committed Dec 11, 2024
2 parents 8e1c600 + 5813777 commit eccd7e3
Show file tree
Hide file tree
Showing 27 changed files with 1,086 additions and 457 deletions.
28 changes: 28 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,34 @@
# Change Log

## 7.6.0 - UNRELEASED
- XRDDEV-2185: As a Security Server Administrator I want the information system TLS certificate table under the subsystem internal server tab to show more information regarding the certificate so that I can differentiate them better
- XRDDEV-2536: As a Security Server Administrator I want the software to automatically renew signing and authentication certificates issued by trusted CA-s that support ACME so that the renewal would happen automatically
- XRDDEV-2542: As a Security Server Administrator I would like to be able to log the CN field of the client certificate that is used to send a query to the Security Server so that I can better monitor the connections
- XRDDEV-2567: As an X-Road user I would like to get operational metrics on an endpoint level not just on a service level so that more granular statistics could be gathered
- XRDDEV-2627: As a Developer I want to bump our components JAVA version to 21 so that we run the latest LTS release
- XRDDEV-2652: As a Central Server Administrator I want to be able to unregister a Member from a Security Server so that I can remove an association between a Member and a Security Server if needed.
- XRDDEV-2656: Central Server tries to re-initialize when the signer module is stopped
- XRDDEV-2659: Disabled subsystem can be deleted on the Security Server without unregistering first, causing a faulty state
- XRDDEV-2665: Signer port 5558 must be removed source code and configuration files, because it's not used anymore.
- XRDDEV-2667: As a Security Server Administrator I want to be notified if automatic certificate renewal using ACME fails or succeeds so that I know what the certificate renewal status is.
- XRDDEV-2682: As a System Administrator, I want backups to work in Kubernetes when a volume is added for /var/lib/xroad/ so that automatic backups work as expected
- XRDDEV-2683: As a Developer I want to ensure the configuration files override-securityserver-ee.ini and override-docker.ini have consistent and appropriate settings for Docker environments
- XRDDEV-2686: As a Developer I want to redirect the Security Server API UI access logs in the sidecar image to standard output instead of files within the container to align with container best practices
- XRDDEV-2692: Re-ordering keys with multiple CSR-s and certificates causes duplicates to appear in the table
- XRDDEV-2693: As a Developer I want the Security Server encryption and verification code to be refactored so that it would better support adding new crypto algorithms
- XRDDEV-2694: As a Developer I want to add EC key support to the Security Server and test backwards compatibility so that we can use it in the product
- XRDDEV-2695: As a Developer I want to add support for using EC keys to sign the global configuration so that it can use the more secure algorithm
- XRDDEV-2697: Security Server minor UI issues
- XRDDEV-2706: verify_external_configuration.sh script fails on Central Server
- XRDDEV-2714: As a Developer I want the sidecar to correctly output ConfigurationVerifier logs for the confclient so that troubleshooting issues with global conf in sidecar is easier
- XRDDEV-2718: Issues with database custom database configuration when using backup/restore
- XRDDEV-2723: As a Developer I want to update our PKCS11 library so that we have the latest fixes
- XRDDEV-2729: Unable to delete a subsystem that is both a consumer and producer on the same Security Server
- XRDDEV-2731: As a Developer I want the potential CSPT to be fixed so that we are hardened against future issues
- XRDDEV-2732: As a Developer I want to disable port 4000 for ACME challenges so that we don't expose it needlessly
- XRDDEV-2733: As a Developer I want to harden the way we handle use input in ACME challenges so that we don't leave a potential opening in the future
- XRDDEV-2759: Fix an issue on the Security Server that caused the proxy-ui-api service not being started automatically after a fresh metapackage install
- XRDDEV-2773: Broken notifications breaks Certificate status change flow

## 7.5.1 - 2024-09-02
- XRDDEV-2669: As an Administrator I would like to be able to override the database host in the environments file as well so that it can be used in scripts
Expand Down
8 changes: 4 additions & 4 deletions doc/EnvironmentalMonitoring/Monitoring-architecture.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ xmlns:prod="http://vrk-test.x-road.fi/producer">
</SOAP-ENV:Envelope>
```

For monitoring queries this is not enough. In a clustered security server configuration, one service can be served from multiple security servers. When X-Road routes the message, it picks one candidate based on which one answers the quickest. When executing monitoring queries, we need to be able to fetch monitoring data from a specific security server in a cluster. To make this possible the Security server targeting extension for the X-Road message protocol \[[PR-TARGETSS](#12-references)\] is used, which adds a new SOAP header element `securityServer`. Using this element, the caller identifies which security server should respond with the monitoring data (`servercode` = `fdev-ss1.i.palveluvayla.com`). To execute a query, we call service `getSecurityServerMetrics`:
For monitoring queries this is not enough. In a clustered security server configuration, one service can be served from multiple security servers. When X-Road routes the message, it picks one candidate based on which one answers the quickest. When executing monitoring queries, we need to be able to fetch monitoring data from a specific security server in a cluster. To make this possible the Security server targeting extension for the X-Road message protocol \[[PR-TARGETSS](#12-references)\] is used, which adds a new SOAP header element `securityServer`. Using this element, the caller identifies which security server should respond with the monitoring data (`servercode` = `fdev-ss1.i.x-road.global`). To execute a query, we call service `getSecurityServerMetrics`:

```xml
<SOAP-ENV:Envelope
Expand All @@ -258,7 +258,7 @@ For monitoring queries this is not enough. In a clustered security server config
<id:xRoadInstance>fdev</id:xRoadInstance>
<id:memberClass>GOV</id:memberClass>
<id:memberCode>1710128-9</id:memberCode>
<id:serverCode>fdev-ss1.i.palveluvayla.com</id:serverCode>
<id:serverCode>fdev-ss1.i.x-road.global</id:serverCode>
</xrd:securityServer>
<xrd:id>ID11234</xrd:id>
<xrd:protocolVersion>4.0</xrd:protocolVersion>
Expand Down Expand Up @@ -293,7 +293,7 @@ The response looks like:
<id:xRoadInstance>fdev</id:xRoadInstance>
<id:memberClass>GOV</id:memberClass>
<id:memberCode>1710128-9</id:memberCode>
<id:serverCode>fdev-ss1.i.palveluvayla.com</id:serverCode>
<id:serverCode>fdev-ss1.i.x-road.global</id:serverCode>
</xrd:securityServer>
<xrd:id>ID11234</xrd:id>
<xrd:protocolVersion>4.0</xrd:protocolVersion>
Expand All @@ -302,7 +302,7 @@ The response looks like:
<SOAP-ENV:Body>
<m:getSecurityServerMetricsResponse>
<m:metricSet>
<m:name>SERVER:fdev/GOV/1710128-9/fdev-ss1.i.palveluvayla.com</m:name>
<m:name>SERVER:fdev/GOV/1710128-9/fdev-ss1.i.x-road.global</m:name>
<m:stringMetric>
<m:name>proxyVersion</m:name>
<m:value>6.7.7-1.20151201075839gitb72b28e</m:value>
Expand Down
6 changes: 3 additions & 3 deletions doc/EnvironmentalMonitoring/Monitoring-messages.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ An optional `outputSpec` child element can be used to request a subset of the me
<id:xRoadInstance>fdev</id:xRoadInstance>
<id:memberClass>GOV</id:memberClass>
<id:memberCode>1710128-9</id:memberCode>
<id:serverCode>fdev-ss1.i.palveluvayla.com</id:serverCode>
<id:serverCode>fdev-ss1.i.x-road.global</id:serverCode>
</xrd:securityServer>

<xrd:id>ID11234</xrd:id>
Expand Down Expand Up @@ -128,7 +128,7 @@ The response `Body` contains one `getSecurityServerMetricsResponse` element whic
<id:xRoadInstance>fdev</id:xRoadInstance>
<id:memberClass>GOV</id:memberClass>
<id:memberCode>1710128-9</id:memberCode>
<id:serverCode>fdev-ss1.i.palveluvayla.com</id:serverCode>
<id:serverCode>fdev-ss1.i.x-road.global</id:serverCode>
</xrd:securityServer>
<xrd:id>ID11234</xrd:id>
<xrd:protocolVersion>4.0</xrd:protocolVersion>
Expand All @@ -137,7 +137,7 @@ The response `Body` contains one `getSecurityServerMetricsResponse` element whic
<SOAP-ENV:Body>
<m:getSecurityServerMetricsResponse>
<m:metricSet>
<m:name>SERVER:fdev/GOV/1710128-9/fdev-ss1.i.palveluvayla.com</m:name>
<m:name>SERVER:fdev/GOV/1710128-9/fdev-ss1.i.x-road.global</m:name>
<m:stringMetric>
<m:name>proxyVersion</m:name>
<m:value>6.7.7-1.20151201075839gitb72b28e</m:value>
Expand Down
7 changes: 4 additions & 3 deletions doc/Manuals/ug-syspar_x-road_v6_system_parameters.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# X-Road: System Parameters User Guide

Version: 2.90
Version: 2.91
Doc. ID: UG-SYSPAR


Expand Down Expand Up @@ -101,6 +101,7 @@ Doc. ID: UG-SYSPAR
| 19.08.2024 | 2.88 | Added parameters for management requests sender | Justas Samuoslis |
| 20.09.2024 | 2.89 | Acme automatic certificate renewal job related parameters | Mikk-Erik Bachmann |
| 08.11.2024 | 2.90 | Added new parameters *key-named-curve*, *soft-token-pin-keystore-algorithm*, *authentication-key-algorithm* and *signing-key-algorithm* to add ECDSA support for Authentication/Signing certificates | Ovidijus Narkevicius |
| 09.12.2024 | 2.91 | Rename parameters *global_conf_tls_cert_verification* -> *global-conf-tls-cert-verification*, *global_conf_hostname_verification* -> *global-conf-hostname-verification* | Eneli Reimets |


## Table of Contents
Expand Down Expand Up @@ -355,8 +356,8 @@ Proxy-ui has been removed in version 6.24 and it's parameters are not used anymo
| admin-port | 5675 | TCP port on which the configuration client process listens for admin commands. |
| allowed-federations | none | A comma-separated list of case-insensitive X-Road instances that fetching configuration anchors is allowed for. This enables federation with the listed instances if the X-Road instance is already federated at the central server level . Special value *none*, if present, disables all federation (the default value), while *all* allows all federations if *none* is not present. Example: *allowed-federations=ee,sv* allows federation with example instances *EE* and *Sv* while *allowed-federations=all,none* disables federation. X-Road services `xroad-confclient` and `xroad-proxy` need to be restarted (in that order) for the setting change to take effect. |
| proxy-configuration-backup-cron | 0 15 3 * * ? | Cron expression for proxy configuration automatic backup job |
| global_conf_tls_cert_verification | true | It is possible to disable the verification of the global configuration download TLS certificate. Should be `true` in production environment |
| global_conf_hostname_verification | true | It is possible to disable the hostname verification. Does the hostname specified in the URL match the hostname specified in the Common Name (CN) of the Central Server’s TLS certificate. Should be `true` in production environment |
| global-conf-tls-cert-verification | true | It is possible to disable the verification of the global configuration download TLS certificate. Should be `true` in production environment |
| global-conf-hostname-verification | true | It is possible to disable the hostname verification. Does the hostname specified in the URL match the hostname specified in the Common Name (CN) of the Central Server’s TLS certificate. Should be `true` in production environment |

### 3.7 Message log add-on parameters: `[message-log]`

Expand Down
39 changes: 36 additions & 3 deletions ...ecture/arc-opmond_x-road_operational_monitoring_daemon_architecture_Y-1096-1.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# X-Road: Operational Monitoring Daemon Architecture <!-- omit in toc -->

Version: 1.3
Version: 1.4
Document ID: ARC-OPMOND

| Date | Version | Description | Author |
Expand All @@ -14,6 +14,7 @@ Document ID: ARC-OPMOND
| 25.06.2020 | 1.1 | Update section 3.3 with the instructions how to enable JMX | Petteri Kivimäki |
| 01.06.2023 | 1.2 | Update references | Petteri Kivimäki |
| 02.10.2024 | 1.3 | Update schema file locations | Justas Samuolis |
| 05.12.2024 | 1.4 | Add endpoint level statistics gathering support | Eneli Reimets |

## Table of Contents <!-- omit in toc -->

Expand Down Expand Up @@ -49,7 +50,7 @@ This document is licensed under the Creative Commons Attribution-ShareAlike 3.0

## 1 Introduction

The X-Road monitoring solution is conceptually split into two parts: environmental and operational monitoring. The operational monitoring processes operational statistics (such as which services have been called, how many times, what was the size of the response, etc.) of the security servers.
The X-Road monitoring solution is conceptually split into two parts: environmental and operational monitoring. The operational monitoring processes operational statistics (such as which services or endpoints have been called, how many times, what was the size of the response, etc.) of the security servers.

This document describes the architecture of the X-Road operational monitoring daemon. It presents an overview of the components of the monitoring daemon and its interfaces.

Expand Down Expand Up @@ -169,7 +170,7 @@ The schema is located in the file *src/op-monitor-daemon/core/src/main/resources

### A.2 Example Store Operational Monitoring Data Request

The first record of the store request reflects successfully mediated request, the second one unsuccessfully mediated request.
The first record of the store request reflects successfully mediated SOAP request, the second one successfully mediated REST request and the third one unsuccessfully mediated request.

```json
{
Expand Down Expand Up @@ -206,6 +207,38 @@ The first record of the store request reflects successfully mediated request, th
"xRequestId": "d4490e7f-305e-44c3-b869-beaaeda694e7",
"serviceType": "WSDL"
},
{
"monitoringDataTs": 1733404603,
"securityServerInternalIp": "fd42:2642:2cb3:31ac:216:3eff:fedf:85c%eth0",
"securityServerType": "Client",
"requestInTs": 1733404602876,
"requestOutTs": 1733404602884,
"responseInTs": 1733404602970,
"responseOutTs": 1733404603005,
"clientXRoadInstance": "FI",
"clientMemberClass": "COM",
"clientMemberCode": "111",
"clientSubsystemCode": "CLIENT",
"serviceXRoadInstance": "FI",
"serviceMemberClass": "COM",
"serviceMemberCode": "111",
"serviceSubsystemCode": "SERVICE",
"serviceCode": "pets",
"restMethod": "GET",
"restPath": "/cat",
"messageId": "1234",
"messageProtocolVersion": "1",
"clientSecurityServerAddress": "ss1",
"serviceSecurityServerAddress": "ss1",
"requestSize": 214,
"responseSize": 462,
"requestAttachmentCount": 0,
"responseAttachmentCount": 0,
"succeeded": true,
"statusCode": 200,
"xRequestId": "1244d018-9300-4f1b-8c2b-9b7f2bc4e933",
"serviceType": "REST"
},
{
"monitoringDataTs": 1576134508,
"securityServerInternalIp": "fd42:2642:2cb3:31ac:216:3eff:fedf:85c%eth0",
Expand Down
Loading

0 comments on commit eccd7e3

Please sign in to comment.