Skip to content

de4a-eu/de4a

Repository files navigation

DE4A - CONNECTOR Iteration 2

Connector component. Check out following instructions and descriptions.

de4a-connector

package

Checkout the technical documentation on the Wiki page

Public REST API

Requests

  • /request/im - As DR, take an IM request (RequestExtractMultiEvidenceIMType) and send it with AS4 to DT. Returns a generic synchronous response (ResponseErrorType).
  • /request/usi - As DR, take a USI request (RequestExtractMultiEvidenceUSIType) and send it with AS4 to DT. Returns a generic synchronous response (ResponseErrorType).
  • /request/lu - As DR, take a USI request (RequestExtractMultiEvidenceLUType) and send it with AS4 to DT. Returns a generic synchronous response (ResponseErrorType).
  • /request/subscription - As DR, take a USI request (RequestEventSubscriptionType) and send it with AS4 to DT. Returns a generic synchronous response (ResponseErrorType).
  • /requestTransferEvidenceIM - This is the backwards compatibility layer for Iteration 1. As DR, take an Iteration 1 IM request (RequestTransferEvidenceUSIIMDRType) and send it with AS4 to DT. Wait synchronously until the DR receives a matching response from DT. Return an Iteration 1 IM response (ResponseTransferEvidenceType). It times out after 60 seconds.

Responses

  • /response/usi/redirectUser - As DT, takes a Redirect User response (RedirectUserType) and send it with AS4 to DR. Returns a generic synchronous response (ResponseErrorType).
  • /response/evidence - As DT, takes an Evidence response (ResponseExtractMultiEvidenceType) and send it with AS4 to DR. Returns a generic synchronous response (ResponseErrorType).
  • /response/subscription - As DT, takes an Subscription response (ResponseEventSubscriptionType) and send it with AS4 to DR. Returns a generic synchronous response (ResponseErrorType).

Events

  • /event/notification - Takes an event notification (EventNotificationType). Returns a generic synchronous response (ResponseErrorType).

Service APIs

  • /service/ial/{cot} - Query the IAL for provided Canonical Object Types (COTs). Multiple COTs can be separated by ,. No ATU restrictions are applied.
    • Optional query parameter environment - The runtime environment for which the Connector will filter the result entries from IAL. The following environments are allowed.
      • The playground environment only allows 9999:.+mock-it2 identifier values
      • The test environment only allows 99\d\d:.+test-it2 identifier values
      • The pilot environment allows for all other identifiers (so the ones that are neither playground nor test)
  • /service/ial/{cot}/{atu} - Query the IAL for provided Canonical Object Types (COTs) but only for the ones in the specified ATU. Multiple COTs can be separated by ,.
    • Optional query parameter environment - The runtime environment for which the Connector will filter the result entries from IAL. See above for details.
  • /service/reload-addresses - Reload the internal address list for forwarding to DE and DO from the backend file.

Note: the MOR data cannot be queried via the Connector. Please query the IAL directly. Example URL: https://de4a.simplegob.com/ial/mor/en.json

Configuration

Application properties

Global application properties definitions (including AS4 properties):

~/de4a/de4a-connector/src/main/resources/application.properties

External addresses configuration

Configuration meant to define the DE/DOs endpoints where the components expect the delivered messages from the Connector. The structure is divided in two principal groups and in turn into different types according with the component nature regarding the interactions patterns:

  • Owner Addresses
    • im: endpoint where the Connector will deliver an IM message
    • usi: endpoint where the Connector will deliver an USI message
    • lu: endpoint where the Connector will deliver an LookUp message
    • sn: endpoint where the Connector will deliver an Subscription message
  • Evaluator Addresses
    • response: endpoint where the Connector will deliver a response message
    • subscription_resp: endpoint where the Connector will deliver a subscription response message
    • redirect: endpoint where the Connector will deliver a user redirection message
    • notification: endpoint where the Connector will deliver a notification message

The configuration file is located at:

~/de4a/de4a-connector/src/main/resources/de-do.json

Logging config

Supported by the logging framework Log4J2, multiple parameters can be configured by editing the following config file:

~/de4a/de4a-connector/src/main/resources/log4j2.xml

Installation

You should be able to compile entire packages from the parent POM file:

mvn clean install

It is also possible to compile each package separately by browsing to the folder and running the command above. It requires Java 11 and Maven 3.x (latest).

Maven packages

The Connector is deployed to Maven Central and is available via the following coordinates, replacing x.y.z with the real version:

<dependency>
  <groupId>eu.de4a</groupId>
  <artifactId>de4a-connector</artifactId>
  <version>x.y.z</version>
  <type>war</type>
</dependency>

Docker

The Docker images are not yet ready. See DOCKER.md for details

Package

The compilation process will packaging the project into a .war file located on /target/ path, which should be deployable on any applications server.

This project leverages other DE4A applications and libraries:

Connector configuration guide

For a correct configuration of the Connector, three main property files must be considered:

  • reference.properties: Part of the system configuration that should not be changed.
  • application.properties: main system configuration. Properties configured here, have precedence over the ones in reference.properties.
  • de-do.json: DE/DOs addresses for delivering messages
  • log4j2.xml: logging configuration

Lets review relevant aspects of the overall configuration:

Kafka configuration application.properties

In order to send log messages to a kafka server, configure the following parameters:

de4a.kafka.enabled = true
# Enables the standard logging separately of the Kafka messages. It is necessary for print metrics messages - (default: true)
de4a.kafka.logging.enabled = true
# Enables Kafka connection via HTTP (Only enable HTTP mode if outbound TCP connections are blocked from your internal network)
de4a.kafka.http.enabled = false
# Kafka server address
de4a.kafka.url = de4a.simplegob.com:9092
# Uncomment the following property and remove the above one if HTTP mode is enabled
#de4a.kafka.url=https://de4a.simplegob.com/kafka-rest/
# Establish a topic on kafka tracker - Pattern: de4a-<country-code>-<partner-name> - Eg.: de4a-es-sgad - (default: de4a-connector)
de4a.kafka.topic=de4a-connector
# Logging metrics messages prefix - Default: DE4A METRICS
log.metrics.prefix=DE4A METRICS

IMPORTANT - If your server has no access to external domains, the HTTP kafka and proxy configuration should be enabled. To enable HTTP kafka log producer, you only need to set the property to true de4a.kafka.http.enabled=true - Also configure the proper endpoint in order to use HTTP connections

It is important to mention the property de4a.kafka.logging.enabled, used to enable the file log printing for each kafka message sent, that property could be enabled even when the de4a.kafka.enabled=false, just for write the log at the different appenders configured in the log4j2 configuration file.

SMP/SML properties reference.properties

To establish which SMP server will provide the Connector with metadata services, the following properties must be used:

# SMP stuff is always the same for the pilots
de4a.smp.http.useglobalsettings = true
de4a.smp.usedns = true
de4a.smp.tls.trustall = true
de4a.smp.sml.name = SMK DE4A
de4a.smp.sml.dnszone = de4a.acc.edelivery.tech.ec.europa.eu.
de4a.smp.sml.serviceurl = https://acc.edelivery.tech.ec.europa.eu/edelivery-sml
de4a.smp.sml.clientcert = true

# SMP truststore for validating messages
smpclient.truststore.type = jks
smpclient.truststore.path = truststore/de4a-truststore-smp-v3-pw-de4a.jks
smpclient.truststore.password = de4a

AS4 properties application.properties and reference.properties

# What AS4 implementation to use?
de4a.me.implementation = phase4

Parameters used by the built-in Phase4 module of the Connector. Set up the properties above following the commented indications. Some of them are filled in to clarify the content -- Important to consider if each property is optional or not (check out the the in-line comments).

# The from party ID to be used for outgoing messages
phase4.send.fromparty.id = de4a-test
# To party ID to be used for outgoing messages. Configure it IN CASE OF USING Domibus without dynamic participant 
# discovery or in the phase4 side in a mixed AS4 implementations phase4 <-> Domibus
# phase4.send.toparty.id=
# The AS4 To/PartyId/@type value. Default: urn:oasis:names:tc:ebcore:partyid-type:unregistered
phase4.send.toparty.id.type=urn:oasis:names:tc:ebcore:partyid-type:unregistered
# The AS4 From/PartyId/@type value. Default: urn:oasis:names:tc:ebcore:partyid-type:unregistered
phase4.send.fromparty.id.type=urn:oasis:names:tc:ebcore:partyid-type:unregistered
# An OPTIONAL folder, where sent responses should be stored. If this property is not provided, they are not stored
phase4.send.response.folder = 

# The absolute path to a local directory to store data
phase4.datapath = 
# Enable or disable HTTP debugging for AS4 transmissions. The default value is false.
phase4.debug.http = false
# Enable or disable debug logging for incoming AS4 transmissions. The default value is false.
phase4.debug.incoming = false
# An OPTIONAL absolute directory path where the incoming AS4 messages should be dumped to. Disabled by default.
phase4.dump.incoming.path = 
# An OPTIONAL absolute directory path where the outgoing AS4 messages should be dumped to. Disabled by default.
phase4.dump.outgoing.path = 

# AS4 keystore for signing/decrypting
phase4.keystore.type = 
phase4.keystore.path = 
phase4.keystore.password = 
phase4.keystore.key-alias = 
phase4.keystore.key-password = 

# AS4 truststore for validating
phase4.truststore.type = jks
phase4.truststore.path = truststore/de4a-truststore-as4-v3-pw-de4a.jks
phase4.truststore.password = de4a

Logging configuration log4j2.xml

The configuration file bellow maintains the logging configuration where you can set the level of each appender, set up log file path, or even include more appenders or configuration. Important - to correctly configure the path of log file. By default it is a relative path to catalina.base (Tomcat server) ${sys:catalina.base}/logs/connector.log

<RollingFile name="rollingFile"
	fileName="${sys:catalina.base}/logs/connector.log"
	filePattern="${sys:catalina.base}/logs/history/connector.%d{dd-MMM}.log.gz"
	ignoreExceptions="false">
	<PatternLayout>
		<Pattern>>$${ctx:metrics:-}[%date{ISO8601}][%-5level][$${ctx:logcode:-}][DE4A-CONNECTOR][$${ctx:origin:-}][$${ctx:destiny:-}] %msg -- %location [%thread]%n</Pattern>
	</PatternLayout>
	<Policies>
		<OnStartupTriggeringPolicy />
		<SizeBasedTriggeringPolicy size="30 MB" />
	</Policies>
	<DefaultRolloverStrategy max="5" />
</RollingFile>

Also, in the application.properties there is another property related with the logging.

# Logging metrics messages prefix - Default: DE4A METRICS
log.metrics.prefix=DE4A METRICS

It is used to include a prefix on each logging line written by the Kafka logging that could be useful to parse and filter the lines with metrics information.

Starting up Connector

Once you have all configuration parameters well configured (if not, check the logs to find out the problem), it is time to deploy the component into an applications server. Once you have deployed the war file, there are several checks to ensure that the deployment was successful:

  • Connector index page will be at root path: http://host:port/

News and Noteworthy

  • v0.3.5 - 2023-04-10
    • Updated the license information to the final version
  • v0.3.4 - 2023-01-10
    • Updated to DCNG 0.2.12 fixing the value of DcngIdentifierFactory.DOCTYPE_SCHEME_CANONICAL_EVENT_CATALOGUE (Type was missing)
  • v0.3.3 - 2022-12-21
    • Fixed an error in determining the EMessageServiceType if the source element used an explicit namespace prefix
  • v0.3.2 - 2022-12-15
    • Fixed an error in /response/subscription that Data Evaluator and Data Owner were not swapped correctly
    • Fixed NullPointerException in the DBA legacy layer, if error messages are returned
    • Fixed the missing "Error handling" part in the DBA legacy layer for all kind of responses
  • v0.3.1 - 2022-11-16
    • Updated default application.properties for easier Tracker usage
  • v0.3.0 - 2022-10-21
    • Added support for the new V4 truststore for CommisSign2 PKI
    • Using the new CommisSign2 PKI as the default
  • v0.2.6 - 2022-09-07
    • Really fixed the Spring problem with the /service/ial APIs - thx to @zigc
  • v0.2.5 - 2022-08-24
    • Extended the /service/ial queries with an optional environment request parameter that can limit the result DOs
  • v0.2.4 - 2022-08-19
    • Fixed a Spring problem with the /service/ial APIs when run in a Docker container (same name of resolved method)
  • v0.2.3 - 2022-08-18
    • The /service/ial queries no longer require the application/xml Content Type
    • Updated the Iteration 1 code bindings to avoid conflicting paths (de4a-iem 0.1.13)
    • Fixed bugs in the default page of the Connector - now all properties are shown
    • Made the IAL remote query more resilient
  • v0.2.2 - 2022-08-16
    • Updated to de4a-commons 0.2.12
    • Updated to de4a-connector-ng 0.2.7 with improved configurability
    • Using a consistent logging towards Kafka Tracker
  • v0.2.1 - 2022-07-06
    • Updated to de4a-commons 0.2.8
    • Improved the Iteration 1 backwards compatibility interface for the DO side as well
  • v0.2.0 - 2022-05-20
    • This is the first version of the Iteration 2 Connector. It's not the final version, but a good starting point