Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Malcolm v23.12.0 #307

Merged
merged 84 commits into from
Dec 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
fa17a78
bump for v23.11.0 development
mmguero Oct 26, 2023
308a1b0
add option to auto-create catch-all netbox IPAM prefixes for private …
mmguero Oct 26, 2023
e20983c
add option to auto-create catch-all netbox IPAM prefixes for private …
mmguero Oct 26, 2023
edd49a1
reduce verbosity of netbox
mmguero Oct 26, 2023
94e8f76
add option to auto-create catch-all netbox IPAM prefixes for private …
mmguero Oct 26, 2023
cab66bd
address issues with NetBox database and Logstash's NetBox cache (idah…
mmguero Oct 26, 2023
8831476
work in pgoress for address issues with NetBox database and Logstash'…
mmguero Oct 27, 2023
20a89d5
work in pgoress for address issues with NetBox database and Logstash'…
mmguero Oct 27, 2023
16d5053
specify lru_redux in the gemspec
mmguero Oct 27, 2023
92b4b34
address issues with NetBox database and Logstash's NetBox cache (idah…
mmguero Oct 27, 2023
12b4f57
bump Werkzeug to v3.0.1 as patch for DoS 'High resource usage when pa…
mmguero Oct 27, 2023
86a41d0
allow specifying capture parameters when orchmode is kubernetes
mmguero Oct 27, 2023
b4f8d44
Update Zeek to v6.1.0
mmguero Oct 30, 2023
ed99af5
point downstream to mmguero-dev/icsnpp-modbus until https://github.co…
mmguero Oct 30, 2023
da45848
Working on zeek v6.1.0 build
mmguero Oct 31, 2023
70c7773
update some of the documentation for docker compose (v2) as a plugin …
mmguero Oct 31, 2023
c9742bb
update some of the documentation for docker compose (v2) as a plugin …
mmguero Oct 31, 2023
be6a9df
fix zeek build
mmguero Oct 31, 2023
3aa72fc
use spicy disable protocol analyzer for LDAP
mmguero Oct 31, 2023
30a05d9
point back upstream for a few of the plugins
mmguero Oct 31, 2023
2a804c6
use prefix.description instead of VRF for identifying subnets in NetB…
mmguero Oct 31, 2023
69ce77c
use docker compose as a plugin rather than docker-compose when possible
mmguero Nov 1, 2023
4be0d17
fix issue with logs
mmguero Nov 1, 2023
fd69cad
fix issue parsing dns.ip
mmguero Nov 1, 2023
e518880
make sure preloaded prefixes get populated with default site name (id…
mmguero Nov 1, 2023
c2f043e
fix start not stopping log display
mmguero Nov 1, 2023
570fce9
fix issue with prefix name not being used for segments correctly (ida…
mmguero Nov 1, 2023
326b8f6
update logstash (https://www.elastic.co/guide/en/logstash/current/log…
mmguero Nov 8, 2023
5dbb346
Fluent-bit to v2.2.0 (https://github.com/fluent/fluent-bit/releases/t…
mmguero Nov 9, 2023
0a6565d
various updates for v23.11.0 development:
mmguero Nov 10, 2023
61e3b6e
Fix logstash parser issues with ldap (idaholab/Malcolm#289)
mmguero Nov 10, 2023
3934eae
elasticsearch python libraries to 8.11.0
mmguero Nov 13, 2023
baebc49
idaholab/Malcolm#275, integrate suricata version of nsacyber ELITEWOL…
mmguero Nov 13, 2023
b0826dd
added https://github.com/reversinglabs/reversinglabs-yara-rules yara …
mmguero Nov 13, 2023
99afed6
update logstash and beats to 8.11.1
mmguero Nov 13, 2023
9b1253a
Work in progress for idaholab/Malcolm#287
mmguero Nov 13, 2023
de41962
better startup for netbox restore
mmguero Nov 13, 2023
48bca91
fix idaholab/Malcolm#287; fix issue with SUPERUSER_API_TOKEN not bein…
mmguero Nov 13, 2023
fd6d050
idaholab/Malcolm#286, strip out broken Arkime and NetBox links from d…
mmguero Nov 14, 2023
b7133e3
idaholab/Malcolm#285, allow customizing Arkime's freeSpaceG setting (…
mmguero Nov 14, 2023
684d69f
fix file type validation not working for upload from (some?) windows …
mmguero Nov 14, 2023
f3c9872
improve error messages for PCAP/artifact processing beyond just icons…
mmguero Nov 14, 2023
a0dc39a
idaholab/Malcolm#147, replace kbn_sankey with vega
mmguero Nov 14, 2023
1ceddc7
allow netbox to restore database from preload directory if it exists
mmguero Nov 15, 2023
ea76ffb
documentation tweak for idaholab/Malcolm#294
mmguero Nov 15, 2023
d111631
idaholab/Malcolm#294, put netbox restore database functionality insid…
mmguero Nov 15, 2023
cdaf056
bump opensearch-py to v2.4.0
mmguero Nov 15, 2023
ecf5599
Revert "bump opensearch-py to v2.4.0"
mmguero Nov 15, 2023
9d8861c
opensearch-py to 2.4.1
mmguero Nov 16, 2023
cba9dc9
idaholab/Malcolm#294, fix a minor bug when preload gz is not specified
mmguero Nov 16, 2023
c2e6008
for idaholab/Malcolm#280, on restore of an older database migrate ipa…
mmguero Nov 16, 2023
b22ea96
for idaholab/Malcolm#280, on restore of an older database migrate ipa…
mmguero Nov 16, 2023
379054f
bump opensearch-py to v2.4.2
mmguero Nov 20, 2023
54d7c15
idaholab/Malcolm#295, specify local networks via ZEEK_LOCAL_NETS envi…
mmguero Nov 20, 2023
301b2a7
fix idaholab/Malcolm#294, put netbox restore database functionality i…
mmguero Nov 21, 2023
95b3c2a
restore kubernetes image names in prep for release
mmguero Nov 21, 2023
103da34
for idaholab/Malcolm#299, fix a vim and libx11 vulnerability in a few…
mmguero Nov 21, 2023
e5405bb
idaholab/Malcolm#280, update existing prefixes to VRF on startup for …
mmguero Nov 22, 2023
2bd9578
rework y axis of results visualization
mmguero Nov 22, 2023
2808618
update supercronic to v0.2.28 (https://github.com/aptible/supercronic…
mmguero Nov 27, 2023
ff81092
update supercronic to v0.2.28 (https://github.com/aptible/supercronic…
mmguero Nov 27, 2023
0a1bf36
flesh out automatic config for virter
mmguero Nov 28, 2023
9059cf2
one minor tweak to idaholab/Malcolm#147, allow vega visualizations to…
mmguero Nov 28, 2023
2331293
idaholab/Malcolm#301, allow configuration of docker's logging driver …
mmguero Nov 28, 2023
70684fb
idaholab/Malcolm#302, allow user to include other suricata config YML…
mmguero Nov 29, 2023
7338264
idaholab/Malcolm#303, allow user to include custom zeek config
mmguero Nov 29, 2023
1a199dc
idaholab/Malcolm#303, allow user to include custom zeek config
mmguero Nov 29, 2023
3684c75
idaholab/Malcolm#303, allow user to include custom zeek config
mmguero Nov 29, 2023
6861469
httpupdate netbox to v3.6.6 (https://github.com/netbox-community/netb…
mmguero Nov 30, 2023
b499390
as it's Nov 30th and the fix for idaholab/Malcolm#298 isn't out yet, …
mmguero Nov 30, 2023
e46b8d6
documentation updates for idaholab/Malcolm#302 and idaholab/Malcolm#3…
mmguero Nov 30, 2023
33c655c
allow suricata config to tune max-pending-packets with SURICATA_MAX_P…
mmguero Nov 30, 2023
ad937ed
idaholab/Malcolm#298, bump opensearch to v2.11.1
mmguero Dec 1, 2023
f96a922
update branding/logos
mmguero Dec 4, 2023
5ae973e
update/fix branding for v2.11.1
mmguero Dec 4, 2023
36af1c1
update documentation SHA sums
mmguero Dec 4, 2023
da0886d
fix issue with image name for kubernetes
mmguero Dec 4, 2023
73f95a7
Merge branch 'development' of https://github.com/mmguero-dev/Malcolm …
mmguero Dec 4, 2023
14970bd
issue with K8s with zeek custom/intel
mmguero Dec 4, 2023
7cc9e10
fix issue loading zeek intel on startup
mmguero Dec 4, 2023
acbe2d3
fix issue loading zeek intel on startup
mmguero Dec 4, 2023
39bdbd5
fix issue loading zeek intel on startup
mmguero Dec 4, 2023
eca0c86
sha1sum update
mmguero Dec 4, 2023
64cba5e
Merge branch 'development' of https://github.com/mmguero-dev/Malcolm …
mmguero Dec 4, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Dockerfiles/dashboards-helper.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,10 @@ ENV DASHBOARDS_URL $DASHBOARDS_URL
ENV DASHBOARDS_DARKMODE $DASHBOARDS_DARKMODE
ENV PATH="/data:${PATH}"

ENV SUPERCRONIC_VERSION "0.2.27"
ENV SUPERCRONIC_VERSION "0.2.28"
ENV SUPERCRONIC_URL "https://github.com/aptible/supercronic/releases/download/v$SUPERCRONIC_VERSION/supercronic-linux-amd64"
ENV SUPERCRONIC "supercronic-linux-amd64"
ENV SUPERCRONIC_SHA1SUM "7dadd4ac827e7bd60b386414dfefc898ae5b6c63"
ENV SUPERCRONIC_SHA1SUM "fe1a81a8a5809deebebbd7a209a3b97e542e2bcd"
ENV SUPERCRONIC_CRONTAB "/etc/crontab"

ENV ECS_RELEASES_URL "https://api.github.com/repos/elastic/ecs/releases/latest"
Expand Down
126 changes: 34 additions & 92 deletions Dockerfiles/dashboards.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,77 +1,4 @@
# build ####################################################################
FROM amazonlinux:2 AS build

# Copyright (c) 2023 Battelle Energy Alliance, LLC. All rights reserved.

# set up build environment for dashboard plugins built from source

ARG DEFAULT_UID=1000
ARG DEFAULT_GID=1000
ENV DEFAULT_UID $DEFAULT_UID
ENV DEFAULT_GID $DEFAULT_GID
ENV PUSER "dashboarder"
ENV PGROUP "dashboarder"

ENV TERM xterm

ARG OPENSEARCH_VERSION="2.8.0"
ENV OPENSEARCH_VERSION $OPENSEARCH_VERSION

ARG OPENSEARCH_DASHBOARDS_VERSION="2.8.0"
ENV OPENSEARCH_DASHBOARDS_VERSION $OPENSEARCH_DASHBOARDS_VERSION

# base system dependencies for checking out and building plugins

USER root

RUN amazon-linux-extras install -y epel && \
yum upgrade -y && \
yum install -y curl patch procps psmisc tar zip unzip gcc-c++ make moreutils jq git && \
amazon-linux-extras install -y python3.8 && \
ln -s -r -f /usr/bin/python3.8 /usr/bin/python3 && \
ln -s -r -f /usr/bin/pip3.8 /usr/bin/pip3 && \
groupadd -g ${DEFAULT_GID} ${PGROUP} && \
adduser -u ${DEFAULT_UID} -d /home/${PUSER} -s /bin/bash -G ${PGROUP} -g ${PUSER} ${PUSER} && \
mkdir -p /usr/share && \
git clone --depth 1 --recurse-submodules --shallow-submodules --single-branch --branch "${OPENSEARCH_VERSION}" https://github.com/opensearch-project/OpenSearch /usr/share/opensearch && \
git clone --depth 1 --recurse-submodules --shallow-submodules --single-branch --branch "${OPENSEARCH_DASHBOARDS_VERSION}" https://github.com/opensearch-project/OpenSearch-Dashboards /usr/share/opensearch-dashboards && \
chown -R ${DEFAULT_UID}:${DEFAULT_GID} /usr/share/opensearch-dashboards /usr/share/opensearch

# build plugins as non-root

USER ${PUSER}

# use nodenv (https://github.com/nodenv/nodenv) to manage nodejs/yarn

ENV PATH "/home/${PUSER}/.nodenv/bin:${PATH}"

RUN git clone --single-branch --depth=1 --recurse-submodules --shallow-submodules https://github.com/nodenv/nodenv.git /home/${PUSER}/.nodenv && \
cd /home/${PUSER}/.nodenv && \
./src/configure && \
make -C src && \
cd /tmp && \
eval "$(nodenv init -)" && \
mkdir -p "$(nodenv root)"/plugins && \
git clone --depth 1 --recurse-submodules --shallow-submodules --single-branch https://github.com/nodenv/node-build.git "$(nodenv root)"/plugins/node-build && \
git clone --depth 1 --recurse-submodules --shallow-submodules --single-branch https://github.com/nodenv/nodenv-update.git "$(nodenv root)"/plugins/nodenv-update && \
git clone --depth 1 --recurse-submodules --shallow-submodules --single-branch https://github.com/pine/nodenv-yarn-install.git "$(nodenv root)"/plugins/nodenv-yarn-install && \
nodenv install "$(cat /usr/share/opensearch-dashboards/.node-version)" && \
nodenv global "$(cat /usr/share/opensearch-dashboards/.node-version)"

# check out and build plugins

RUN eval "$(nodenv init -)" && \
mkdir -p /usr/share/opensearch-dashboards/plugins && \
git clone --depth 1 --recurse-submodules --shallow-submodules --single-branch --branch opensearch-v2-dashboards-compatibility https://github.com/mmguero-dev/osd_sankey_vis.git /usr/share/opensearch-dashboards/plugins/sankey_vis && \
cd /usr/share/opensearch-dashboards/plugins/sankey_vis && \
yarn osd bootstrap && \
yarn install && \
yarn build --opensearch-dashboards-version "${OPENSEARCH_DASHBOARDS_VERSION}" && \
mv ./build/kbnSankeyVis-"${OPENSEARCH_DASHBOARDS_VERSION}".zip ./build/kbnSankeyVis.zip

# runtime ##################################################################

FROM opensearchproject/opensearch-dashboards:2.8.0
FROM opensearchproject/opensearch-dashboards:2.11.1

LABEL maintainer="malcolm@inl.gov"
LABEL org.opencontainers.image.authors='malcolm@inl.gov'
Expand All @@ -93,7 +20,7 @@ ENV PUSER_PRIV_DROP true
ENV TERM xterm

ENV TINI_VERSION v0.19.0
ENV OSD_TRANSFORM_VIS_VERSION 2.8.0
ENV OSD_TRANSFORM_VIS_VERSION 2.11.0

ARG OPENSEARCH_URL="http://opensearch:9200"
ARG OPENSEARCH_PRIMARY="opensearch-local"
Expand All @@ -115,22 +42,20 @@ ENV NODE_OPTIONS $NODE_OPTIONS

USER root

COPY --from=build /usr/share/opensearch-dashboards/plugins/sankey_vis/build/kbnSankeyVis.zip /tmp/kbnSankeyVis.zip
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /usr/bin/tini
ADD https://github.com/lguillaud/osd_transform_vis/releases/download/$OSD_TRANSFORM_VIS_VERSION/transformVis-$OSD_TRANSFORM_VIS_VERSION.zip /tmp/transformVis.zip

RUN yum upgrade -y && \
yum install -y curl psmisc util-linux openssl rsync python3 zip unzip && \
yum install -y curl-minimal psmisc findutils util-linux openssl rsync python3 zip unzip && \
yum remove -y vim-* && \
usermod -a -G tty ${PUSER} && \
# Malcolm manages authentication and encryption via NGINX reverse proxy
/usr/share/opensearch-dashboards/bin/opensearch-dashboards-plugin remove securityDashboards --allow-root && \
cd /usr/share/opensearch-dashboards/plugins && \
/usr/share/opensearch-dashboards/bin/opensearch-dashboards-plugin install file:///tmp/kbnSankeyVis.zip --allow-root && \
cd /tmp && \
# unzip transformVis.zip opensearch-dashboards/transformVis/opensearch_dashboards.json opensearch-dashboards/transformVis/package.json && \
# sed -i "s/2\.9\.0/2\.9\.0/g" opensearch-dashboards/transformVis/opensearch_dashboards.json && \
# sed -i "s/2\.9\.0/2\.9\.0/g" opensearch-dashboards/transformVis/package.json && \
# zip transformVis.zip opensearch-dashboards/transformVis/opensearch_dashboards.json opensearch-dashboards/transformVis/package.json && \
unzip transformVis.zip opensearch-dashboards/transformVis/opensearch_dashboards.json opensearch-dashboards/transformVis/package.json && \
sed -i "s/2\.11\.0/2\.11\.1/g" opensearch-dashboards/transformVis/opensearch_dashboards.json && \
sed -i "s/2\.11\.0/2\.11\.1/g" opensearch-dashboards/transformVis/package.json && \
zip transformVis.zip opensearch-dashboards/transformVis/opensearch_dashboards.json opensearch-dashboards/transformVis/package.json && \
cd /usr/share/opensearch-dashboards/plugins && \
/usr/share/opensearch-dashboards/bin/opensearch-dashboards-plugin install file:///tmp/transformVis.zip --allow-root && \
rm -rf /tmp/transformVis /tmp/opensearch-dashboards && \
Expand All @@ -150,15 +75,32 @@ ADD scripts/malcolm_utils.py /usr/local/bin/
# Yeah, I know about https://opensearch.org/docs/latest/dashboards/branding ... but I can't figure out a way
# to specify the entries in the opensearch_dashboards.yml such that they are valid BOTH from the
# internal opensearch code validating them AND the web browser retrieving them. So we're going scorched earth instead.
ADD docs/images/logo/malcolm_logo.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/default_branding/opensearch_logo.svg
ADD docs/images/logo/malcolm_logo.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/default_branding/opensearch_logo_dark_mode.svg
ADD docs/images/logo/malcolm_logo.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/default_branding/opensearch_logo_default_mode.svg
ADD docs/images/icon/malcolm_mark_dashboards.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/default_branding/opensearch_mark_dark_mode.svg
ADD docs/images/icon/malcolm_mark_dashboards.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/default_branding/opensearch_mark_default_mode.svg
ADD docs/images/favicon/favicon.ico /usr/share/opensearch-dashboards/src/core/server/core_app/assets/favicons/favicon.ico
ADD docs/images/favicon/favicon16.png /usr/share/opensearch-dashboards/src/core/server/core_app/assets/favicons/favicon-16x16.png
ADD docs/images/favicon/favicon32.png /usr/share/opensearch-dashboards/src/core/server/core_app/assets/favicons/favicon-32x32.png
ADD docs/images/favicon/apple-touch-icon-precomposed.png /usr/share/opensearch-dashboards/src/core/server/core_app/assets/favicons/apple-touch-icon.png

COPY --chmod=644 docs/images/favicon/favicon192.png /usr/share/opensearch-dashboards/src/core/server/core_app/assets/favicons/android-chrome-192x192.png
COPY --chmod=644 docs/images/favicon/favicon512.png /usr/share/opensearch-dashboards/src/core/server/core_app/assets/favicons/android-chrome-512x512.png
COPY --chmod=644 docs/images/favicon/apple-touch-icon-precomposed.png /usr/share/opensearch-dashboards/src/core/server/core_app/assets/favicons/apple-touch-icon.png
COPY --chmod=644 docs/images/favicon/favicon16.png /usr/share/opensearch-dashboards/src/core/server/core_app/assets/favicons/favicon-16x16.png
COPY --chmod=644 docs/images/favicon/favicon32.png /usr/share/opensearch-dashboards/src/core/server/core_app/assets/favicons/favicon-32x32.png
COPY --chmod=644 docs/images/favicon/favicon.ico /usr/share/opensearch-dashboards/src/core/server/core_app/assets/favicons/favicon.ico
COPY --chmod=644 docs/images/favicon/favicon144.png /usr/share/opensearch-dashboards/src/core/server/core_app/assets/favicons/mstile-144x144.png
COPY --chmod=644 docs/images/favicon/favicon150.png /usr/share/opensearch-dashboards/src/core/server/core_app/assets/favicons/mstile-150x150.png
COPY --chmod=644 docs/images/favicon/favicon310.png /usr/share/opensearch-dashboards/src/core/server/core_app/assets/favicons/mstile-310x310.png
COPY --chmod=644 docs/images/favicon/favicon70.png /usr/share/opensearch-dashboards/src/core/server/core_app/assets/favicons/mstile-70x70.png
COPY --chmod=644 docs/images/logo/Malcolm.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch.svg
COPY --chmod=644 docs/images/icon/malcolm_mark_dashboards.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch_center_mark.svg
COPY --chmod=644 docs/images/icon/malcolm_mark_dashboards.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch_center_mark_on_dark.svg
COPY --chmod=644 docs/images/icon/malcolm_mark_dashboards.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch_center_mark_on_light.svg
COPY --chmod=644 docs/images/logo/Malcolm.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch_dashboards.svg
COPY --chmod=644 docs/images/logo/malcolm_logo.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch_dashboards_on_dark.svg
COPY --chmod=644 docs/images/logo/Malcolm.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch_dashboards_on_light.svg
COPY --chmod=644 docs/images/icon/malcolm_mark_dashboards.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch_mark.svg
COPY --chmod=644 docs/images/icon/malcolm_mark_dashboards.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch_mark_on_dark.svg
COPY --chmod=644 docs/images/icon/malcolm_mark_dashboards.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch_mark_on_light.svg
COPY --chmod=644 docs/images/logo/malcolm_logo.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch_on_dark.svg
COPY --chmod=644 docs/images/logo/Malcolm.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch_on_light.svg
COPY --chmod=644 docs/images/icon/malcolm_mark_dashboards.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch_spinner.svg
COPY --chmod=644 docs/images/icon/malcolm_mark_dashboards.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch_spinner_on_dark.svg
COPY --chmod=644 docs/images/icon/malcolm_mark_dashboards.svg /usr/share/opensearch-dashboards/src/core/server/core_app/assets/logos/opensearch_spinner_on_light.svg


ENTRYPOINT ["/usr/bin/tini", \
Expand Down
4 changes: 2 additions & 2 deletions Dockerfiles/file-monitor.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -93,10 +93,10 @@ ENV EXTRACTED_FILE_HTTP_SERVER_ENCRYPT $EXTRACTED_FILE_HTTP_SERVER_ENCRYPT
ENV EXTRACTED_FILE_HTTP_SERVER_KEY $EXTRACTED_FILE_HTTP_SERVER_KEY
ENV EXTRACTED_FILE_HTTP_SERVER_PORT $EXTRACTED_FILE_HTTP_SERVER_PORT

ENV SUPERCRONIC_VERSION "0.2.27"
ENV SUPERCRONIC_VERSION "0.2.28"
ENV SUPERCRONIC_URL "https://github.com/aptible/supercronic/releases/download/v$SUPERCRONIC_VERSION/supercronic-linux-amd64"
ENV SUPERCRONIC "supercronic-linux-amd64"
ENV SUPERCRONIC_SHA1SUM "7dadd4ac827e7bd60b386414dfefc898ae5b6c63"
ENV SUPERCRONIC_SHA1SUM "fe1a81a8a5809deebebbd7a209a3b97e542e2bcd"
ENV SUPERCRONIC_CRONTAB "/etc/crontab"

COPY --chmod=755 shared/bin/yara_rules_setup.sh /usr/local/bin/
Expand Down
4 changes: 2 additions & 2 deletions Dockerfiles/file-upload.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,10 @@ ENV FILEPOND_SERVER_BRANCH $FILEPOND_SERVER_BRANCH
ARG STALE_UPLOAD_DELETE_MIN=360
ENV STALE_UPLOAD_DELETE_MIN $STALE_UPLOAD_DELETE_MIN

ENV SUPERCRONIC_VERSION "0.2.27"
ENV SUPERCRONIC_VERSION "0.2.28"
ENV SUPERCRONIC_URL "https://github.com/aptible/supercronic/releases/download/v$SUPERCRONIC_VERSION/supercronic-linux-amd64"
ENV SUPERCRONIC "supercronic-linux-amd64"
ENV SUPERCRONIC_SHA1SUM "7dadd4ac827e7bd60b386414dfefc898ae5b6c63"
ENV SUPERCRONIC_SHA1SUM "fe1a81a8a5809deebebbd7a209a3b97e542e2bcd"
ENV SUPERCRONIC_CRONTAB "/etc/crontab"

COPY --from=npmget /usr/local/lib/node_modules/filepond /var/www/upload/filepond
Expand Down
6 changes: 3 additions & 3 deletions Dockerfiles/filebeat.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM docker.elastic.co/beats/filebeat-oss:8.10.4
FROM docker.elastic.co/beats/filebeat-oss:8.11.1

# Copyright (c) 2023 Battelle Energy Alliance, LLC. All rights reserved.
LABEL maintainer="malcolm@inl.gov"
Expand Down Expand Up @@ -61,10 +61,10 @@ ARG FILEBEAT_TCP_PARSE_TARGET_FIELD=""
ARG FILEBEAT_TCP_PARSE_DROP_FIELD=""
ARG FILEBEAT_TCP_TAG="_malcolm_beats"

ENV SUPERCRONIC_VERSION "0.2.27"
ENV SUPERCRONIC_VERSION "0.2.28"
ENV SUPERCRONIC_URL "https://github.com/aptible/supercronic/releases/download/v$SUPERCRONIC_VERSION/supercronic-linux-amd64"
ENV SUPERCRONIC "supercronic-linux-amd64"
ENV SUPERCRONIC_SHA1SUM "7dadd4ac827e7bd60b386414dfefc898ae5b6c63"
ENV SUPERCRONIC_SHA1SUM "fe1a81a8a5809deebebbd7a209a3b97e542e2bcd"
ENV SUPERCRONIC_CRONTAB "/etc/crontab"

ENV TINI_VERSION v0.19.0
Expand Down
11 changes: 8 additions & 3 deletions Dockerfiles/logstash.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM docker.elastic.co/logstash/logstash-oss:8.10.4
FROM docker.elastic.co/logstash/logstash-oss:8.11.1

LABEL maintainer="malcolm@inl.gov"
LABEL org.opencontainers.image.authors='malcolm@inl.gov'
Expand Down Expand Up @@ -32,6 +32,8 @@ ARG LOGSTASH_NETBOX_ENRICHMENT=false
ARG LOGSTASH_NETBOX_ENRICHMENT_VERBOSE=false
ARG LOGSTASH_NETBOX_ENRICHMENT_LOOKUP_SERVICE=true
ARG LOGSTASH_NETBOX_AUTO_POPULATE=false
ARG LOGSTASH_NETBOX_CACHE_SIZE=1000
ARG LOGSTASH_NETBOX_CACHE_TTL=30

ENV LOGSTASH_ENRICHMENT_PIPELINE $LOGSTASH_ENRICHMENT_PIPELINE
ENV LOGSTASH_PARSE_PIPELINE_ADDRESSES $LOGSTASH_PARSE_PIPELINE_ADDRESSES
Expand All @@ -42,6 +44,8 @@ ENV LOGSTASH_NETBOX_ENRICHMENT $LOGSTASH_NETBOX_ENRICHMENT
ENV LOGSTASH_NETBOX_ENRICHMENT_VERBOSE $LOGSTASH_NETBOX_ENRICHMENT_VERBOSE
ENV LOGSTASH_NETBOX_ENRICHMENT_LOOKUP_SERVICE $LOGSTASH_NETBOX_ENRICHMENT_LOOKUP_SERVICE
ENV LOGSTASH_NETBOX_AUTO_POPULATE $LOGSTASH_NETBOX_AUTO_POPULATE
ENV LOGSTASH_NETBOX_CACHE_SIZE $LOGSTASH_NETBOX_CACHE_SIZE
ENV LOGSTASH_NETBOX_CACHE_TTL $LOGSTASH_NETBOX_CACHE_TTL

USER root

Expand All @@ -63,11 +67,12 @@ RUN set -x && \
pip3 install ipaddress supervisor manuf pyyaml && \
export JAVA_HOME=/usr/share/logstash/jdk && \
/usr/share/logstash/vendor/jruby/bin/jruby -S gem install bundler && \
echo "gem 'lru_cache'" >> /usr/share/logstash/Gemfile && \
echo "gem 'concurrent-ruby'" >> /usr/share/logstash/Gemfile && \
echo "gem 'deep_merge'" >> /usr/share/logstash/Gemfile && \
echo "gem 'fuzzy-string-match'" >> /usr/share/logstash/Gemfile && \
echo "gem 'stringex'" >> /usr/share/logstash/Gemfile && \
echo "gem 'lru_redux'" >> /usr/share/logstash/Gemfile && \
echo "gem 'psych'" >> /usr/share/logstash/Gemfile && \
echo "gem 'stringex'" >> /usr/share/logstash/Gemfile && \
/usr/share/logstash/bin/ruby -S bundle install && \
logstash-plugin install --preserve logstash-filter-translate logstash-filter-cidr logstash-filter-dns \
logstash-filter-json logstash-filter-prune logstash-filter-http \
Expand Down
13 changes: 10 additions & 3 deletions Dockerfiles/netbox.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM netboxcommunity/netbox:v3.6.4
FROM netboxcommunity/netbox:v3.6.6

# Copyright (c) 2023 Battelle Energy Alliance, LLC. All rights reserved.
LABEL maintainer="malcolm@inl.gov"
Expand All @@ -24,10 +24,10 @@ ENV PUSER "ubuntu"
ENV PGROUP "ubuntu"
ENV PUSER_PRIV_DROP true

ENV SUPERCRONIC_VERSION "0.2.27"
ENV SUPERCRONIC_VERSION "0.2.28"
ENV SUPERCRONIC_URL "https://github.com/aptible/supercronic/releases/download/v$SUPERCRONIC_VERSION/supercronic-linux-amd64"
ENV SUPERCRONIC "supercronic-linux-amd64"
ENV SUPERCRONIC_SHA1SUM "7dadd4ac827e7bd60b386414dfefc898ae5b6c63"
ENV SUPERCRONIC_SHA1SUM "fe1a81a8a5809deebebbd7a209a3b97e542e2bcd"
ENV SUPERCRONIC_CRONTAB "/etc/crontab"

ENV YQ_VERSION "4.33.3"
Expand All @@ -39,25 +39,31 @@ ARG NETBOX_DEVICETYPE_LIBRARY_PATH="/opt/netbox-devicetype-library"
ARG NETBOX_DEFAULT_SITE=Malcolm
ARG NETBOX_CRON=true
ARG NETBOX_PRELOAD_PATH="/opt/netbox-preload"
ARG NETBOX_PRELOAD_PREFIXES=false

ENV NETBOX_PATH /opt/netbox
ENV BASE_PATH netbox
ENV NETBOX_DEVICETYPE_LIBRARY_PATH $NETBOX_DEVICETYPE_LIBRARY_PATH
ENV NETBOX_DEFAULT_SITE $NETBOX_DEFAULT_SITE
ENV NETBOX_CRON $NETBOX_CRON
ENV NETBOX_PRELOAD_PATH $NETBOX_PRELOAD_PATH
ENV NETBOX_PRELOAD_PREFIXES $NETBOX_PRELOAD_PREFIXES

ADD netbox/patch/* /tmp/netbox-patches/

RUN apt-get -q update && \
apt-get -y -q --no-install-recommends upgrade && \
apt-get install -q -y --no-install-recommends \
gcc \
file \
git \
jq \
libmagic-dev \
libmagic1 \
libpq-dev \
libpq5 \
patch \
postgresql-client \
procps \
psmisc \
python3-dev \
Expand All @@ -68,6 +74,7 @@ RUN apt-get -q update && \
'git+https://github.com/tobiasge/netbox-initializers' \
psycopg2 \
pynetbox \
python-magic \
python-slugify \
randomcolor && \
cd "${NETBOX_PATH}" && \
Expand Down
Loading