Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

update logstash-oss container to 8.9.0 #234

Closed
mmguero opened this issue Aug 8, 2023 · 0 comments
Closed

update logstash-oss container to 8.9.0 #234

mmguero opened this issue Aug 8, 2023 · 0 comments
Assignees
@mmguero
Labels
external Depends on a bug or feature external to this project logstash Relating to Malcolm's use of Logstash
Milestone
v23.08.0

Comments

@mmguero
Copy link
Collaborator

mmguero commented Aug 8, 2023

To update the logstash container, we're going to change the logstash.Dockerfile so that it has:

FROM docker.elastic.co/logstash/logstash-oss:8.9.0

and then, add logstash-output-opensearch to the logstash-plugin install section. This saves us from being tied to logstash-oss-with-opensearch-output-plugin which is infrequently updated.

However, there is an issue that needs to be resolved dealing with loading large YAML files in the ./logstash/ruby/mac_lookup.rb and ./logstas/ruby/netbox_enrich.rb scripts.

Here are the relevant issues I'll need to look at to figure it out:
@mmguero mmguero added external Depends on a bug or feature external to this project logstash Relating to Malcolm's use of Logstash labels Aug 8, 2023
@mmguero mmguero added this to the v23.08.0 milestone Aug 8, 2023
@mmguero mmguero self-assigned this Aug 8, 2023
@mmguero mmguero added this to Malcolm Aug 8, 2023
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Aug 8, 2023
@mmguero
idaholab#234, update logstash-oss container to 8.9.0 and handle loadi…
53c0625 
…ng large YAML documents in mac_lookup.rb and netbox_enrich.rb
@mmguero mmguero moved this to Done in Malcolm Aug 8, 2023
@mmguero mmguero closed this as completed Aug 8, 2023
This was referenced Aug 14, 2023
Merged
Merged
mmguero added a commit that referenced this issue Aug 15, 2023
@mmguero
Merge pull request #238 from idaholab/v23.08.0_merge_idaholab
a5678f7 
Malcolm v23.08.0 is a minor release with a few improvements, bug fixes and component updates.

v23.07.1...v23.08.0

* Features and enhancements
    + Rewrote the [Network Traffic Artifact Upload](https://idaholab.github.io/Malcolm/docs/upload.html#Upload) interface and backend, replacing the defunct [jQuery-File-Upload](https://github.com/blueimp/jQuery-File-Upload) with [FilePond](https://pqina.nl/filepond/). This was mainly due to jQuery-File-Upload no longer receiving security fixes and having some known vulnerabilities. see #235
    + Use [netbox-initializers](https://github.com/tobiasge/netbox-initializers) plugin, adding the ability to drop YAML files for various NetBox obects to be [preloaded](https://idaholab.github.io/Malcolm/docs/asset-interaction-analysis.html#NetBoxPreload) at startup. see #228
    + handle changes to ICSNPP parsers with source_ip/destination_ip fields (#233 and #226)

* Bug fixes
    + Fixed extracting Malcolm version during ISO build
    + Workaround for wireshark no longer publishing raw manuf (OUI) list (#230)
    + Remove news feed from default NetBox dashboard (as it would try to reach out to the web for RSS updates)

* Component version updates
    + Rebased Docker and ISO images to Debian 12 (bookworm)
    + live-build tool for building ISO images to debian/1%20230131
    + Arkime to [v4.4.0](https://github.com/arkime/arkime/blob/6f667600596e8a2252555640933f424730c258d5/CHANGELOG#L33-L55)
    + supercronic to [v0.2.26](https://github.com/aptible/supercronic/releases/tag/v0.2.26)
    + FileBeat to [v8.9.0](https://www.elastic.co/guide/en/beats/libbeat/current/release-notes-8.9.0.html)
    + LogStash to [v8.9.0](https://www.elastic.co/guide/en/logstash/8.9/logstash-8-9-0.html) (#234)
    + NetBox to [v3.5.7](https://github.com/netbox-community/netbox/releases/tag/v3.5.7)
    + PostgreSQL (used by NetBox) to [v15](https://www.postgresql.org/docs/release/15.0/)
    + opensearch-py to [v2.3.0](https://github.com/opensearch-project/opensearch-py/releases/tag/v2.3.0)
    + PHP (as used by Upload interface) to [v8.2](https://www.php.net/ChangeLog-8.php#PHP_8_2)
    + Fluent Bit to [v2.1.8](https://github.com/fluent/fluent-bit/releases/tag/v2.1.8)
    + certifi to [v2023.7.22](https://github.com/certifi/python-certifi/releases/tag/2023.07.22) (#229)
mmguero added a commit to cisagov/Malcolm that referenced this issue Aug 15, 2023
@mmguero
Merge pull request #274 from cisagov/v23.08.0_merge_cisagov
d5ee03e 
Malcolm v23.08.0 is a minor release with a few improvements, bug fixes and component updates.

v23.07.1...v23.08.0

* Features and enhancements
    + Rewrote the [Network Traffic Artifact Upload](https://cisagov.github.io/Malcolm/docs/upload.html#Upload) interface and backend, replacing the defunct [jQuery-File-Upload](https://github.com/blueimp/jQuery-File-Upload) with [FilePond](https://pqina.nl/filepond/). This was mainly due to jQuery-File-Upload no longer receiving security fixes and having some known vulnerabilities. see idaholab#235
    + Use [netbox-initializers](https://github.com/tobiasge/netbox-initializers) plugin, adding the ability to drop YAML files for various NetBox obects to be [preloaded](https://cisagov.github.io/Malcolm/docs/asset-interaction-analysis.html#NetBoxPreload) at startup. see idaholab#228
    + handle changes to ICSNPP parsers with source_ip/destination_ip fields (idaholab#233 and idaholab#226)

* Bug fixes
    + Fixed extracting Malcolm version during ISO build
    + Workaround for wireshark no longer publishing raw manuf (OUI) list (idaholab#230)
    + Remove news feed from default NetBox dashboard (as it would try to reach out to the web for RSS updates)

* Component version updates
    + Rebased Docker and ISO images to Debian 12 (bookworm)
    + live-build tool for building ISO images to debian/1%20230131
    + Arkime to [v4.4.0](https://github.com/arkime/arkime/blob/6f667600596e8a2252555640933f424730c258d5/CHANGELOG#L33-L55)
    + supercronic to [v0.2.26](https://github.com/aptible/supercronic/releases/tag/v0.2.26)
    + FileBeat to [v8.9.0](https://www.elastic.co/guide/en/beats/libbeat/current/release-notes-8.9.0.html)
    + LogStash to [v8.9.0](https://www.elastic.co/guide/en/logstash/8.9/logstash-8-9-0.html) (idaholab#234)
    + NetBox to [v3.5.7](https://github.com/netbox-community/netbox/releases/tag/v3.5.7)
    + PostgreSQL (used by NetBox) to [v15](https://www.postgresql.org/docs/release/15.0/)
    + opensearch-py to [v2.3.0](https://github.com/opensearch-project/opensearch-py/releases/tag/v2.3.0)
    + PHP (as used by Upload interface) to [v8.2](https://www.php.net/ChangeLog-8.php#PHP_8_2)
    + Fluent Bit to [v2.1.8](https://github.com/fluent/fluent-bit/releases/tag/v2.1.8)
    + certifi to [v2023.7.22](https://github.com/certifi/python-certifi/releases/tag/2023.07.22) (idaholab#229)
@mmguero mmguero moved this from Done to Released in Malcolm Aug 15, 2023
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jan 17, 2024
@mmguero
Merge pull request idaholab#234 from Njinx/feature/tail-logs
18af350 
Added -n argument to script/logs akin to 'tail -n ...'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mmguero mmguero

Labels
external Depends on a bug or feature external to this project logstash Relating to Malcolm's use of Logstash
Projects
Malcolm
Status: Released
Milestone
v23.08.0
Development

No branches or pull requests
1 participant
@mmguero