[advise] winlogbai 7.12.1 + dataprepper + opensearch implementation

[Lmaquaire92]

Hello Team,

I need some advise to implement a new environments with winlogbait agent (7.12.1) sending event logs to opensearch throught dataprepper.

i used this settings:
logging.to_files: true
logging.files:
path: C:\ProgramData\winlogbeat\Logs
logging.level: info
output.elasticsearch:
hosts: ["localhost:9200"]
enabled: false
ssl.certificate: "/etc/pki/client/cert.pem"
ssl.key: "/etc/pki/client/cert.key"
output.logstash:
hosts: ["192.168.107.1:7104"]
enabled: true
ssl.enabled: true

rootca is installed in ROOTCA OS envirronement

but i have this issue in logs:
2022-12-15T17:20:25.594+0100 ERROR [publisher_pipeline_output] pipeline/output.go:180 failed to publish events: write tcp 192.168.0.66:59140->192.168.107.1:7104: wsasend: Une connexion existante a dû être fermée par l’hôte distant.
2022-12-15T17:20:25.594+0100 INFO [publisher_pipeline_output] pipeline/output.go:143 Connecting to backoff(async(tcp://192.168.107.1:7104))
2022-12-15T17:20:25.594+0100 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer

also i tried to create index in opensearch by using winlogbeat.template.json by this method:

curl --insecure -s -H 'Content-Type: application/json' -XPUT https://opensearch-node1:9200/_index_template/winlogbeat-7.12.1 -u 'admin:XXXXX' --data-binary "@winlogbeat.template.json";

but i have this issue :
{"error":{"root_cause":[{"type":"invalid_index_template_exception","reason":"index_template [winlogbeat-8.5.3] invalid, cause [Validation Failed: 1: unknown setting [index.lifecycle.name] please check that any required plugins are installed, or check the breaking changes documentation for removed settings;2: expected [index.lifecycle.name] to be private but it was not;]"}],"type":"invalid_index_template_exception","reason":"index_template [winlogbeat-8.5.3] invalid, cause [Validation Failed: 1: unknown setting [index.lifecycle.name] please check that any required plugins are installed, or check the breaking changes documentation for removed settings;2: expected [index.lifecycle.name] to be private but it was not;]"},"status":400}

thanks for your help

[dlvenable]

@Lmaquaire92 , I understand that you are trying to configure Winlogbeat to send events to Data Prepper and then OpenSearch.

Currently Data Prepper does not support Beats as an input. We have two related issues to add this support. #248 is an issue to allow OpenSearch _bulk API requests. This may not work with Beats directly though. We also have #950 to accept data from Filebeat, which should also support Winlogbeat.