Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Prepare Changelog for 8.8.0 #35524

Merged
merged 7 commits into from
May 24, 2023
Merged

docs: Prepare Changelog for 8.8.0 #35524

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ee24f93
docs: Close changelog for 8.8.0
elasticmachine May 22, 2023
a88a92a
adjust changelog
AndersonQ May 22, 2023
72c4d66
fix changelog 'View commits' link
AndersonQ May 22, 2023
877a131
delete empty sections
AndersonQ May 23, 2023
5342b72
adjust changelog
AndersonQ May 24, 2023
cfa02d3
Apply suggestions from code review
AndersonQ May 24, 2023
682d95f
Merge branch '8.8' into prepare-changelog-8.8.0
AndersonQ May 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 89 additions & 0 deletions CHANGELOG.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,95 @@
:issue: https://github.com/elastic/beats/issues/
:pull: https://github.com/elastic/beats/pull/

[[release-notes-8.8.0]]
=== Beats version 8.8.0
https://github.com/elastic/beats/compare/v8.7.1...v8.8.0[View commits]


==== Bugfixes

*Affecting all Beats*
- Fix namespacing on self-monitoring {pull}32336[32336]
- Fix race condition when stopping runners {pull}32433[32433]
- Fix concurrent map writes when system/process code called from reporter code {pull}32491[32491]
- Fix namespacing on self-monitoring {pull}32336[32336]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are also old PRs that were revived by merge conflicts. The newest one in this group is from July 2022 and has already been released: #32491

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These can be removed entirely. Look at the PR number for an idea of how old it is. Anything that isn't at least 34xxx is suspect for this.

- Fix concurrent map writes when system/process code called from reporter code {pull}32491[32491]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mentioned two times

- The Elasticsearch output now splits large requests instead of dropping them when it receives a StatusRequestEntityTooLarge error. {pull}34911[34911]
- In cases where the matcher detects a non-string type in a match statement, report the error as a debug statement, and not a warning statement. {pull}35119[35119]
- 'add_cloud_metadata' processor - add cloud.region field for GCE cloud provider
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- 'add_cloud_metadata' processor - update azure metadata api version to get missing `cloud.account.id` field
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved

*Filebeat*
- [GCS Input] - Added missing locks for safe concurrency {pull}34914[34914]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Fix the ignore_inactive option being ignored in Filebeat's filestream input {pull}34770[34770]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Add input instance id to request trace filename for httpjson and cel inputs {pull}35024[35024]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Sanitize filenames for request tracer in httpjson input. {pull}35143[35143]
- Sanitize filenames for request tracer in cel input. {pull}35154[35154]
- Fixing the grok expression outputs of log files {pull}35221[35221]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Move repeated Windows event channel not found errors in winlog input to debug level. {issue}35314[35314] {pull}35317[35317]
- Fix crash when processing forwarded logs missing a message. {issue}34705[34705] {pull}34865[34865]
- Fix crash when loading azurewebstorage cursor with no partially processed data. {pull}35433[35433]

*Heartbeat*

- Fix panics when parsing dereferencing invalid parsed url. {pull}34702[34702]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Fix broken state ID location naming. {pull}35336[35336]
- Fix project monitor temp directories permission to include group access. {pull}35398[35398]
- Fix output pipeline exit on run_once. {pull}35376[35376]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Fix formatting issue with socket trace timeout. {pull}35434[35434]

*Metricbeat*

- Make generic SQL GA {pull}34637[34637]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Collect missing remote_cluster in elasticsearch ccr metricset {pull}34957[34957]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Add context with timeout in AWS API calls {pull}35425[35425]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved

*Osquerybeat*

- Adds the `elastic_file_analysis` table to the Osquery extension for macOS builds. {pull}35056[35056]

*Packetbeat*

- Fix BPF filter setting not being applied to sniffers. {issue}35363[35363] {pull}35484[35484]

*Winlogbeat*

- Move repeated channel not found errors to debug level. {issue}35314[35314] {pull}35317[35317]
- Fix panic due to misrepresented buffer use. {pull}35437[35437]
- Allow program termination when attempting to open an absent channel. {pull}35474[35474]

==== Added

*Filebeat*

- Added metric `sqs_messages_waiting_gauge` for aws-s3 input. {pull}34488[34488]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Add support for Okta debug attributes, `risk_reasons`, `risk_behaviors` and `factor`. {issue}33677[33677] {pull}34508[34508]
- Add nginx.ingress_controller.upstream.ip to related.ip {issue}34645[34645] {pull}34672[34672]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Include NAT and firewall IPs in `related.ip` in Fortinet Firewall module. {issue}34640[34640] {pull}34673[34673]
- Add unix socket log parsing for nginx ingress_controller {pull}34732[34732]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Added metric `sqs_worker_utilization` for aws-s3 input. {pull}34793[34793]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Register MIME handlers for CSV types in CEL input. {pull}34934[34934]
- Add MySQL authentication message parsing and `related.ip` and `related.user` fields {pull}34810[34810]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Mention `mito` CEL tool in CEL input docs. {pull}34959[34959]
- Add nginx ingress_controller parsing if one of upstreams fails to return response {pull}34787[34787]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Allow neflow v9 and ipfix templates to be shared between source addresses. {pull}35036[35036]
- Add support for collecting IPv6 metrics. {pull}35123[35123]
- Add oracle authentication messages parsing {pull}35127[35127]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved

*Heartbeat*
- Added status to monitor run log report.
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved
- Removed beta label for browser monitors. {pull}35424[35424].
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved

*Metricbeat*

- Add GCP Carbon Footprint metricbeat data {pull}34820[34820]
- Add event loop utilization metric to Kibana module {pull}35020[35020]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved

*Winlogbeat*

- Add "event.category" and "event.type" to Sysmon module for EventIDs 8, 9, 19, 20, 27, 28, 255 {pull}35193[35193]
AndersonQ marked this conversation as resolved.
Show resolved Hide resolved


[[release-notes-8.6.2]]
=== Beats version 8.6.2
https://github.com/elastic/beats/compare/v8.6.1\...v8.6.2[View commits]
Expand Down
115 changes: 5 additions & 110 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,7 @@ https://github.com/elastic/beats/compare/v8.2.0\...main[Check the HEAD diff]

*Filebeat*

- Fixed error spam from `add_kubernetes_metadata` processor when running on AKS. {pull}33697[33697]
- Metrics hosted by the HTTP monitoring endpoint for the `aws-cloudwatch`, `aws-s3`, `cel`, and `lumberjack` inputs are now available under `/inputs/` instead of `/dataset`.
- The `close.on_state_change.inactive` default value is now set to 5 minutes, matching the documentation.

*Heartbeat*

Expand All @@ -31,42 +29,18 @@ https://github.com/elastic/beats/compare/v8.2.0\...main[Check the HEAD diff]

*Winlogbeat*

- Corrects issue with security events with source IP of "LOCAL" or "Unknown" failing to ingest {issue}19627[19627] {pull}34295[34295]
- Added processing for Windows Event ID's 4797, 5379, 5380, 5381, and 5382 for the Security Ingest Pipeline {issue}34293[34293] {pull}34294[34294]
- Added processing for Windows Event ID's 5140 and 5145 for the Security Ingest Pipeline {pull}34352[34352]
- Add "event.category" and "event.type" to Sysmon module for EventIDs 8, 9, 19, 20, 27, 28, 255 {pull}35193[35193]

*Functionbeat*


==== Bugfixes

*Affecting all Beats*
- Fix Windows service install/uninstall when Win32_Service returns error, add logic to wait until the Windows Service is stopped before proceeding. {pull}33322[33322]
- Support for multiline zookeeper logs {issue}2496[2496]
- Allow `clock_nanosleep` in the default seccomp profiles for amd64 and 386. Newer versions of glibc (e.g. 2.31) require it. {issue}33792[33792]
- Disable lockfile when running under elastic-agent. {pull}33988[33988]
- Fix lockfile logic, retry locking {pull}34194[34194]
- Add checks to ensure reloading of units if the configuration actually changed. {pull}34346[34346]
- Fix namespacing on self-monitoring {pull}32336[32336]
- Fix race condition when stopping runners {pull}32433[32433]
- Fix concurrent map writes when system/process code called from reporter code {pull}32491[32491]
- Log errors from the Elastic Agent V2 client errors channel. Avoids blocking when error occurs communicating with the Elastic Agent. {pull}34392[34392]
- Only log publish event messages in trace log level under elastic-agent. {pull}34391[34391]
- Fix issue where updating a single Elastic Agent configuration unit results in other units being turned off. {pull}34504[34504]
- Fix dropped events when monitor a beat under the agent and send its `Host info` log entry. {pull}34599[34599]

- Fix namespacing on self-monitoring {pull}32336[32336]
- Fix race condition when stopping runners {pull}32433[32433]
- Fix concurrent map writes when system/process code called from reporter code {pull}32491[32491]
- Fix panics when a processor is closed twice {pull}34647[34647]
- Update elastic-agent-system-metrics to v0.4.6 to allow builds on mips platforms. {pull}34674[34674]
- The Elasticsearch output now splits large requests instead of dropping them when it receives a StatusRequestEntityTooLarge error. {pull}34911[34911]
- Fix Beats started by agent do not respect the allow_older_versions: true configuration flag {issue}34227[34227] {pull}34964[34964]
- Fix performance issues when we have a lot of inputs starting and stopping by allowing to disable global processors under fleet. {issue}35000[35000] {pull}35031[35031]
- In cases where the matcher detects a non-string type in a match statement, report the error as a debug statement, and not a warning statement. {pull}35119[35119]
- 'add_cloud_metadata' processor - add cloud.region field for GCE cloud provider
- 'add_cloud_metadata' processor - update azure metadata api version to get missing `cloud.account.id` field



*Auditbeat*
Expand Down Expand Up @@ -128,24 +102,6 @@ https://github.com/elastic/beats/compare/v8.2.0\...main[Check the HEAD diff]

*Heartbeat*

- Fix panics when parsing dereferencing invalid parsed url. {pull}34702[34702]
- Fix broken zip URL monitors. NOTE: Zip URL Monitors will be removed in version 8.7 and replaced with project monitors. {pull}33723[33723]
- Fix integration hashing to prevent reloading all when updated. {pull}34697[34697]
- Fix release of job limit semaphore when context is cancelled. {pull}34697[34697]
- Fix bug where states.duration_ms was incorrect type. {pull}33563[33563]
- Fix handling of long UDP messages in UDP input. {issue}33836[33836] {pull}33837[33837]
- Fix browser monitor summary reporting as up when monitor is down. {issue}33374[33374] {pull}33819[33819]
- Fix beat capabilities on Docker image. {pull}33584[33584]
- Fix serialization of state duration to avoid scientific notation. {pull}34280[34280]
- Enable nodejs engine strict validation when bundling synthetics. {pull}34470[34470]
with the ecs field name `container`. {pull}34403[34403]
automatic splitting at root level, if root level element is an array. {pull}34155[34155]
- Fix broken mapping for state.ends field. {pull}34891[34891]
- Fix issue using projects in airgapped environments by disabling npm audit. {pull}34936[34936]
- Fix broken state ID location naming. {pull}35336[35336]
- Fix project monitor temp directories permission to include group access. {pull}35398[35398]
- Fix output pipeline exit on run_once. {pull}35376[35376]
- Fix formatting issue with socket trace timeout. {pull}35434[35434]

*Heartbeat*

Expand All @@ -159,9 +115,6 @@ automatic splitting at root level, if root level element is an array. {pull}3415
*Filebeat*

- Allow the `misp` fileset in the Filebeat `threatintel` module to ignore CIDR ranges for an IP field. {issue}29949[29949] {pull}34195[34195]
- Remove incorrect reference to CEL ext extensions package. {issue}34610[34610] {pull}34620[34620]
- Fix handling of RFC5988 links' relation parameters by `getRFC5988Link` in HTTPJSON. {issue}34603[34603] {pull}34622[34622]
- Drop empty API response events for Microsoft module. {issue}34786[34786] {pull}34893[34893]

*Auditbeat*

Expand All @@ -181,23 +134,12 @@ automatic splitting at root level, if root level element is an array. {pull}3415
- Fix logstash cgroup mappings {pull}33131[33131]
- Remove unused `elasticsearch.node_stats.indices.bulk.avg_time.bytes` mapping {pull}33263[33263]
- Fix kafka dashboard field names {pull}33555[33555]
- Add tags to events based on parsed identifier. {pull}33472[33472]
- Support Oracle-specific connection strings in SQL module {issue}32089[32089] {pull}32293[32293]
- Remove deprecated metrics from controller manager, scheduler and proxy {pull}34161[34161]
- Fix metrics split through different events and metadata not matching for aws cloudwatch. {pull}34483[34483]
- Fix metadata enricher with correct container ids for pods with multiple containers in container metricset. Align `kubernetes.container.id` and `container.id` fields for state_container metricset. {pull}34516[34516]
- Make generic SQL GA {pull}34637[34637]
- Collect missing remote_cluster in elasticsearch ccr metricset {pull}34957[34957]
- Add context with timeout in AWS API calls {pull}35425[35425]

*Osquerybeat*

- Adds the `elastic_file_analysis` table to the Osquery extension for macOS builds. {pull}35056[35056]

*Packetbeat*

- Fix documentation for `flows.period` related to flow reporting. {pull}35009[35009]
- Fix BPF filter setting not being applied to sniffers. {issue}35363[35363] {pull}35484[35484]

*Winlogbeat*

Expand Down Expand Up @@ -235,25 +177,9 @@ automatic splitting at root level, if root level element is an array. {pull}3415
- httpjson input: Add request tracing logger. {issue}32402[32402] {pull}32412[32412]
- Add cloudflare R2 to provider list in AWS S3 input. {pull}32620[32620]
- Add support for single string containing multiple relation-types in getRFC5988Link. {pull}32811[32811]
- Fix handling of invalid UserIP and LocalIP values. {pull}32896[32896]
- Allow http_endpoint instances to share ports. {issue}32578[32578] {pull}33377[33377]
- Improve httpjson documentation for split processor. {pull}33473[33473]
- Added separation of transform context object inside httpjson. Introduced new clause `.parent_last_response.*` {pull}33499[33499]
- Cloud Foundry input uses server-side filtering when retrieving logs. {pull}33456[33456]
- Add `parse_aws_vpc_flow_log` processor. {pull}33656[33656]
- Update `aws.vpcflow` dataset in AWS module have a configurable log `format` and to produce ECS 8.x fields. {pull}33699[33699]
- Modified `aws-s3` input to reduce mutex contention when multiple SQS message are being processed concurrently. {pull}33658[33658]
- Disable "event normalization" processing for the aws-s3 input to reduce allocations. {pull}33673[33673]
- Add Common Expression Language input. {pull}31233[31233]
- Add support for http+unix and http+npipe schemes in httpjson input. {issue}33571[33571] {pull}33610[33610]
- Add support for http+unix and http+npipe schemes in cel input. {issue}33571[33571] {pull}33712[33712]
- Add `decode_duration`, `move_fields` processors. {pull}31301[31301]
- Add backup to bucket and delete functionality for the `aws-s3` input. {issue}30696[30696] {pull}33559[33559]
- Add metrics for UDP packet processing. {pull}33870[33870]
- Convert UDP input to v2 input. {pull}33930[33930]
- Improve collection of risk information from Okta debug data. {issue}33677[33677] {pull}34030[34030]
- Adding filename details from zip to response for httpjson {issue}33952[33952] {pull}34044[34044]
- Allow user configuration of keep-alive behaviour for HTTPJSON and CEL inputs. {issue}33951[33951] {pull}34014[34014]
- Add support for polling system UDP stats for UDP input metrics. {pull}34070[34070]
- Add support for recognizing the log level in Elasticsearch JVM logs {pull}34159[34159]
- Add new Entity Analytics input with Azure Active Directory support. {pull}34305[34305]
Expand All @@ -263,69 +189,37 @@ automatic splitting at root level, if root level element is an array. {pull}3415
- Add beta `take over` mode for `filestream` for simple migration from `log` inputs {pull}34292[34292]
- Add pagination support for Salesforce module. {issue}34057[34057] {pull}34065[34065]
- Allow users to redact sensitive data from CEL input debug logs. {pull}34302[34302]
- Added support for HTTP destination override to Google Cloud Storage input. {pull}34413[34413]
- Added metric `sqs_messages_waiting_gauge` for aws-s3 input. {pull}34488[34488]
- Add support for new Rabbitmq timestamp format for logs {pull}34211[34211]
- Allow user configuration of timezone offset in Cisco ASA and FTD modules. {pull}34436[34436]
- Allow user configuration of timezone offset in Checkpoint module. {pull}34472[34472]
- Add support for Okta debug attributes, `risk_reasons`, `risk_behaviors` and `factor`. {issue}33677[33677] {pull}34508[34508]
- Fill okta.request.ip_chain.* as a flattened object in Okta module. {pull}34621[34621]
- Fixed GCS log format issues. {pull}34659[34659]
- Add nginx.ingress_controller.upstream.ip to related.ip {issue}34645[34645] {pull}34672[34672]
- Include NAT and firewall IPs in `related.ip` in Fortinet Firewall module. {issue}34640[34640] {pull}34673[34673]
- Add Basic Authentication support on constructed requests to CEL input {issue}34609[34609] {pull}34689[34689]
- Add string manipulation extensions to CEL input {issue}34610[34610] {pull}34689[34689]
- Add unix socket log parsing for nginx ingress_controller {pull}34732[34732]
- Added metric `sqs_worker_utilization` for aws-s3 input. {pull}34793[34793]
- Improve CEL input documentation {pull}34831[34831]
- Add metrics documentation for CEL and AWS CloudWatch inputs. {issue}34887[34887] {pull}34889[34889]
- Register MIME handlers for CSV types in CEL input. {pull}34934[34934]
- Add MySQL authentication message parsing and `related.ip` and `related.user` fields {pull}34810[34810]
- Mention `mito` CEL tool in CEL input docs. {pull}34959[34959]
- Add nginx ingress_controller parsing if one of upstreams fails to return response {pull}34787[34787]
- Allow neflow v9 and ipfix templates to be shared between source addresses. {pull}35036[35036]
- Add support for collecting IPv6 metrics. {pull}35123[35123]
- Add oracle authentication messages parsing {pull}35127[35127]

*Auditbeat*
- Migration of system/package module storage from gob encoding to flatbuffer encoding in bolt db. {pull}34817[34817]
- Migration of system/package module storage from gob encoding to flatbuffer encoding in bolt db. {pull}34817[34817]

*Filebeat*


*Heartbeat*
- Users can now configure max scheduler job limits per monitor type via env var. {pull}34307[34307]
- Added status to monitor run log report.
- Removed beta label for browser monitors. {pull}35424[35424].

- Remove host and port matching restrictions on hint-generated monitors. {pull}34376[34376]

*Metricbeat*

- Add Data Granularity option to AWS module to allow for for fewer API calls of longer periods and keep small intervals. {issue}33133[33133] {pull}33166[33166]
- Update README file on how to run Metricbeat on Kubernetes. {pull}33308[33308]
- Add per-thread metrics to system_summary {pull}33614[33614]
- Add GCP CloudSQL metadata {pull}33066[33066]
- Remove GCP Compute metadata cache {pull}33655[33655]
- Add support for multiple regions in GCP {pull}32964[32964]
- Add GCP Redis regions support {pull}33728[33728]
- Add namespace metadata to all namespaced kubernetes resources. {pull}33763[33763]
- Changed cloudwatch module to call ListMetrics API only once per region, instead of per AWS namespace {pull}34055[34055]
- Add beta ingest_pipeline metricset to Elasticsearch module for ingest pipeline monitoring {pull}34012[34012]
- Handle duplicated TYPE line for prometheus metrics {issue}18813[18813] {pull}33865[33865]
- Add GCP Carbon Footprint metricbeat data {pull}34820[34820]
- Add event loop utilization metric to Kibana module {pull}35020[35020]

*Packetbeat*

- Add option to allow sniffer to change device when default route changes. {issue}31905[31905] {pull}32681[32681]
- Add option to allow sniffing multiple interface devices. {issue}31905[31905] {pull}32933[32933]
- Bump Windows Npcap version to v1.71. {issue}33164[33164] {pull}33172[33172]
- Add fragmented IPv4 packet reassembly. {issue}33012[33012] {pull}33296[33296]
- Reduce logging level for ENOENT to WARN when mapping sockets to processes. {issue}33793[33793] {pull}33854[33854]
- Add metrics for TCP and UDP packet processing. {pull}33833[33833] {pull}34353[34353]
- Allow user to prevent Npcap library installation on Windows. {issue}34420[34420] {pull}34428[34428]
- Add metrics documentation for TCP and UDP protocols. {issue}34887[34887] {pull}34889[34889]

*Packetbeat*

Expand All @@ -336,8 +230,6 @@ automatic splitting at root level, if root level element is an array. {pull}3415
*Winlogbeat*

- Add metrics for log event processing. {pull}33922[33922]
- Add metrics documentation for event processing. {issue}34887[34887] {pull}34889[34889]
- Add note in documentation about 21 event ID clause limit {issue}35048[35048] {pull}35049[35049]

*Elastic Log Driver*

Expand Down Expand Up @@ -373,3 +265,6 @@ automatic splitting at root level, if root level element is an array. {pull}3415






1 change: 1 addition & 0 deletions libbeat/docs/release.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ This section summarizes the changes in each release. Also read
<<breaking-changes>> for more detail about changes that affect
upgrade.

* <<release-notes-8.8.0>>
* <<release-notes-8.6.2>>
* <<release-notes-8.6.1>>
* <<release-notes-8.6.0>>
Expand Down