Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PCAP File with no - in pcapng Fails to Upload #265

Closed
jaredthecoder opened this issue Jul 6, 2023 · 1 comment
Closed

PCAP File with no - in pcapng Fails to Upload #265

jaredthecoder opened this issue Jul 6, 2023 · 1 comment

Comments

@jaredthecoder
Copy link

jaredthecoder commented Jul 6, 2023 •
edited
Loading

Hi team!

The line below causes PCAPNG files with the file type (application/octet-stream/pcapng capture file - version 1.0) to fail on upload with the following message: pcap-monitor_1 | 2023-07-06 18:08:37 WARNING: watch-pcap-uploads-folder.py: ���� /pcap/upload/AUTOSURICATA,AUTOZEEK,AUTOCARVEall,sourcehttpsgithub.comEmreEkinICS-Pcaps,USERTAG,TheUltimate.pcap (application/octet-stream/pcapng capture file - version 1.0)

This PCAP is an open source one from https://www.netresec.com/?page=PcapFiles.

The issue is the if check looks for pcap-ng exclusively and not also pcapng.

On top of this, the warning is not apparent to the real issue and without looking at the source code, appears as if the upload silently fails. My terminal also is not rendering the Unicode characters (I am SSH'd in to a server that doesn't have it setup).
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Jul 7, 2023
@mmguero
for cisagov#265, be more flexible in recognizing PCAPNG file types
afad9d4
@mmguero
Copy link
Collaborator

mmguero commented Jul 7, 2023

Thanks, I've adjusted the code to be more flexible in how it recognizes those file types. It will be included in the next Malcolm release.

FWIW, Malcolm only has partial support for PCAP-NG files, as Arkime doesn't support it very well. Your best bet is to convert to PCAP format first.

@mmguero mmguero closed this as completed Jul 7, 2023
This was referenced Jul 18, 2023
Merged
Closed
Merged
mmguero added a commit that referenced this issue Jul 19, 2023
@mmguero
Merge pull request #267 from cisagov/v23.07.0_merge_cisagov
432a90a 
Malcolm v23.07.0 is a feature release with a number of improvements, bux fixes and component updates.

v23.05.1...v23.07.0

* New features
    - scan docker images built via GitHub actions for vulnerabilities using Trivy (idaholab#218)
    - document building and deplolying Malcolm with an AWS AMI image (idaholab#205)
    - handle Arkime field actions (idaholab#200)
    - kubernetes: document how to get running on Amazon EKS (idaholab#194)
    - Populate NetBox inventory via passively-gathered network traffic metadata (basic functionality, work in progress) (idaholab#135)

* Enhancements
    - use .tar.xz instead of .tar.gz for packaging Malcolm docker images for better compression (and smaller ISO file size)
    - Malcolm documentation edits (idaholab#204)
    - add option to enable SSH via password in hedgehog's configure-interfaces.py script (idaholab#158)
    - updated "Network Traffic Analysis with Malcolm" slides
    - use an init container in Kubernetes container startup to ensure necessary directories get created under PersistentVolume objects before startup
    - improvements to identifying source of third-party logs sent via fluent bit
    - don't do unnecessary clone of Zeek plugins, just install using URL
    - parse [bacnet_device_control.log](https://github.com/cisagov/icsnpp-bacnet/#device-control-log-bacnet_device_controllog) produced by the icsnpp-bacnet parser for Zeek

* Bug fixes
    - maxlogins value includes tmux sessions, can lock user out of SSH (idaholab#214)
    - curl rc file for connecting to external OpenSearch without auth enabled causes logstash startup to fail (idaholab#209)
    - failure to parse some suricata alerts due to integer type which should be indexed as long (idaholab#206)
    - netbox-restore doesn't work in Kubernetes (idaholab#202)
    - PCAP File with no `-` in pcapng Fails to Upload (#265)
    - disable NetBox telemetry

* Component version updates
    - Alpine (docker container image base) to [v3.18.0](https://www.alpinelinux.org/posts/Alpine-3.18.0-released.html)
    - Arkime to [v4.3.2](https://github.com/arkime/arkime/blob/8bd9d1ccaf3214eeb07da910c45d6172f9ff4ca8/CHANGELOG#L40-L55)
    - capa to [v6.0.0](https://github.com/mandiant/capa/releases/tag/v6.0.0)
    - filebeat to [v8.8.2](https://www.elastic.co/guide/en/beats/libbeat/current/release-notes-8.8.2.html)
    - NetBox to [v3.5.4](https://github.com/netbox-community/netbox/releases/tag/v3.5.4)
    - OpenSearch and OpenSearch Dashboards to [v2.8.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.8.0.md)
    - Supercronic to [v0.2.25](https://github.com/aptible/supercronic/releases/tag/v0.2.25)
    - YARA to [v4.3.2](https://github.com/VirusTotal/yara/releases/tag/v4.3.2)
    - Zeek to [v5.2.2](https://github.com/zeek/zeek/releases/tag/v5.2.2)

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from [https://malcolm.fyi/](https://malcolm.fyi/docs/download.html).
mmguero added a commit to idaholab/Malcolm that referenced this issue Jul 19, 2023
@mmguero
Merge pull request #224 from idaholab/v23.07.0_merge_idaholab
924431d 
Malcolm v23.07.0 is a feature release with a number of improvements, bux fixes and component updates.

v23.05.1...v23.07.0

* New features
    - scan docker images built via GitHub actions for vulnerabilities using Trivy (#218)
    - document building and deplolying Malcolm with an AWS AMI image (#205)
    - handle Arkime field actions (#200)
    - kubernetes: document how to get running on Amazon EKS (#194)
    - Populate NetBox inventory via passively-gathered network traffic metadata (basic functionality, work in progress) (#135)

* Enhancements
    - use .tar.xz instead of .tar.gz for packaging Malcolm docker images for better compression (and smaller ISO file size)
    - Malcolm documentation edits (#204)
    - add option to enable SSH via password in hedgehog's configure-interfaces.py script (#158)
    - updated "Network Traffic Analysis with Malcolm" slides
    - use an init container in Kubernetes container startup to ensure necessary directories get created under PersistentVolume objects before startup
    - improvements to identifying source of third-party logs sent via fluent bit
    - don't do unnecessary clone of Zeek plugins, just install using URL
    - parse [bacnet_device_control.log](https://github.com/cisagov/icsnpp-bacnet/#device-control-log-bacnet_device_controllog) produced by the icsnpp-bacnet parser for Zeek

* Bug fixes
    - maxlogins value includes tmux sessions, can lock user out of SSH (#214)
    - curl rc file for connecting to external OpenSearch without auth enabled causes logstash startup to fail (#209)
    - failure to parse some suricata alerts due to integer type which should be indexed as long (#206)
    - netbox-restore doesn't work in Kubernetes (#202)
    - PCAP File with no `-` in pcapng Fails to Upload (cisagov#265)
    - disable NetBox telemetry

* Component version updates
    - Alpine (docker container image base) to [v3.18.0](https://www.alpinelinux.org/posts/Alpine-3.18.0-released.html)
    - Arkime to [v4.3.2](https://github.com/arkime/arkime/blob/8bd9d1ccaf3214eeb07da910c45d6172f9ff4ca8/CHANGELOG#L40-L55)
    - capa to [v6.0.0](https://github.com/mandiant/capa/releases/tag/v6.0.0)
    - filebeat to [v8.8.2](https://www.elastic.co/guide/en/beats/libbeat/current/release-notes-8.8.2.html)
    - NetBox to [v3.5.4](https://github.com/netbox-community/netbox/releases/tag/v3.5.4)
    - OpenSearch and OpenSearch Dashboards to [v2.8.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.8.0.md)
    - Supercronic to [v0.2.25](https://github.com/aptible/supercronic/releases/tag/v0.2.25)
    - YARA to [v4.3.2](https://github.com/VirusTotal/yara/releases/tag/v4.3.2)
    - Zeek to [v5.2.2](https://github.com/zeek/zeek/releases/tag/v5.2.2)

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from [https://malcolm.fyi/](https://malcolm.fyi/docs/download.html).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jaredthecoder @mmguero