Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Merge next-patch into next-minor #1305

Merged
merged 33 commits into from
Dec 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
f8d2f6d
changelog
ashnwade Dec 16, 2024
39cd507
add note about timestamp truncation. Remove erroneous statement about
david-fritz-gravwell Dec 16, 2024
40e7983
update release date
ashnwade Dec 16, 2024
1a5bb35
Update changelog/5.6.8.md
ashnwade Dec 16, 2024
a6cd09f
Merge pull request #1303 from ashnwade/changelog
ashnwade Dec 16, 2024
5f29e8a
Merge next-minor into merge/next-patch_to_next-minor
github-actions[bot] Dec 16, 2024
d120370
document start/end constraints in grav.y spec
david-fritz-gravwell Dec 16, 2024
19e6ed9
Merge branch 'next-patch' of github.com:gravwell/wiki into issue-1296
david-fritz-gravwell Dec 16, 2024
89b6d85
address comments
david-fritz-gravwell Dec 16, 2024
44c5f1b
Merge pull request #1306 from david-fritz-gravwell/issue-1208
ashnwade Dec 16, 2024
a1117e1
Merge next-patch into merge/next-patch_to_next-minor
github-actions[bot] Dec 16, 2024
f238652
address comments
david-fritz-gravwell Dec 16, 2024
fa098d9
Merge pull request #1304 from david-fritz-gravwell/issue-1296
ashnwade Dec 16, 2024
51f7ce7
Merge next-patch into merge/next-patch_to_next-minor
github-actions[bot] Dec 16, 2024
e353c87
add more context and links to intrinsic EV documentation
david-fritz-gravwell Dec 16, 2024
c384588
Merge pull request #1307 from david-fritz-gravwell/issue-1086
ashnwade Dec 16, 2024
e7366fb
Merge next-patch into merge/next-patch_to_next-minor
github-actions[bot] Dec 16, 2024
0cd1f9f
fix links in regex docs
ashnwade Dec 16, 2024
f0d65a8
Make replication storage limit considerations more clear.
david-fritz-gravwell Dec 16, 2024
859f60a
fix language in parameters
david-fritz-gravwell Dec 16, 2024
20ba03e
Update configuration/parameters.md
david-fritz-gravwell Dec 16, 2024
3969469
Merge pull request #1309 from david-fritz-gravwell/issue-1222
ashnwade Dec 16, 2024
e94361c
Merge next-patch into merge/next-patch_to_next-minor
github-actions[bot] Dec 16, 2024
8228632
Merge pull request #1308 from ashnwade/regex-links
kris-watts-gravwell Dec 16, 2024
85e9ae1
Merge next-patch into merge/next-patch_to_next-minor
github-actions[bot] Dec 16, 2024
e262c0d
Merge pull request #1310 from ashnwade/patch-to-main
kris-watts-gravwell Dec 16, 2024
ac52721
Merge pull request #1311 from gravwell/merge/main_to_next-patch
kris-watts-gravwell Dec 16, 2024
2e39f04
Merge next-patch into merge/next-patch_to_next-minor
github-actions[bot] Dec 16, 2024
0c9af38
updating Hashes for release 5.6.8-1
invalid-email-address Dec 16, 2024
8b6bef2
Merge pull request #1312 from gravwell/actions/5.6.8-1
ashnwade Dec 16, 2024
328e35a
Merge next-patch into merge/main_to_next-patch
github-actions[bot] Dec 16, 2024
607fa15
Merge pull request #1313 from gravwell/merge/main_to_next-patch
ashnwade Dec 16, 2024
47e7a68
Merge next-patch into merge/next-patch_to_next-minor
github-actions[bot] Dec 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions _static/versions.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
[
{
"name": "v5.6.7 (latest)",
"version": "v5.6.7",
"name": "v5.6.8 (latest)",
"version": "v5.6.8",
"url": "/",
"preferred": true
},
{
"version": "v5.6.7",
"url": "/v5.6.7/"
},
{
"version": "v5.6.6",
"url": "/v5.6.6/"
Expand Down
16 changes: 16 additions & 0 deletions changelog/5.6.8.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Changelog for version 5.6.8

## Released 16 December 2024

## Gravwell

### Additions
* Added hotkeys to support auto-closing pairs for `"`, `(`, `[`, and `{` in the query editor.
* Added strict transport security header when running in TLS mode.
* Added support for start/end constraints in inner queries when using compound queries.

### Bug Fixes

* Fixed an issue where JavaScript returning `undefined` could improperly halt execution of a Flow.
* Fixed an issue with macro expansion with invalid macros.
* Improved timestamp processing to truncate subsecond precision when using start/end constraints.
3 changes: 2 additions & 1 deletion changelog/list.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
maxdepth: 1
caption: Current Release
---
5.6.7 <5.6.7>
5.6.8 <5.6.8>
```

## Previous Versions
Expand All @@ -18,6 +18,7 @@ maxdepth: 1
caption: Previous Releases
---

5.6.7 <5.6.7>
5.6.6 <5.6.6>
5.6.5 <5.6.5>
5.6.4 <5.6.4>
Expand Down
2 changes: 1 addition & 1 deletion conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
project = "Gravwell"
copyright = f"Gravwell, Inc. {date.today().year}"
author = "Gravwell, Inc."
release = "v5.6.7"
release = "v5.6.8"

# Default to localhost:8000, so the version switcher looks OK on livehtml
version_list_url = os.environ.get(
Expand Down
6 changes: 5 additions & 1 deletion configuration/accelerators.md
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,11 @@ Note that the tag `zeekconn` can be matched against both accelerators, however t
Tags=foo*
```

(intrinsic-acceleration-target)=
## Acceleration with Intrinsic Enumerated Values

When acceleration is enabled, [intrinsic enumerated values](#attach-target) will always be accelerated with the fulltext engine. This enables queries using the [intrinsic](/search/intrinsic/intrinsic) module to be accelerated. No specific configuration is required for acceleration with intrinsic EVs other than having acceleration enabled.

## Fulltext

The fulltext accelerator is designed to index words within text logs and is considered the most flexible acceleration option. Many of the other search modules support invoking the fulltext accelerator when executing queries. However, the primary search module for engaging with the fulltext accelerator is the [grep](/search/grep/grep) module with the `-w` flag. Much like the Unix grep utility, `grep -w` specifies that the provided filter is expected to a word, rather than a subset of bytes. Running a search with `words foo bar baz` will look for the words foo, bar, and baz and engage the fulltext accelerator.
Expand Down Expand Up @@ -686,7 +691,6 @@ The results show why fulltext may often be worth the storage and ingest penalty:
| fulltextindex | 2.99s | 12.49X |
| fulltextbloom | 3.40s | 12.49X |


#### Query AX modules

The AX definition file for all four tags is below, see the [AX](/configuration/autoextractors) documentation for more information:
Expand Down
2 changes: 1 addition & 1 deletion configuration/parameters.md
Original file line number Diff line number Diff line change
Expand Up @@ -902,7 +902,7 @@ Description: Sets the storage location for data replicated from other Gravwell i
### **Max-Replicated-Data-GB**
Default Value:
Example: `Max-Replicated-Data-GB=100`
Description: Sets, in gigabytes, the maximum amount of replicated data to store. When this is exceeded, the indexer will begin walking the replicated data to clean up; it will first remove any shards which have been deleted on the original indexer, then it will begin deleting the oldest shards. Once the storage size is below the limit, deletion will stop.
Description: Sets, in gigabytes, the maximum amount of replicated data to store. When this is exceeded, the indexer will begin walking the replicated data to clean up; it will first remove any shards which have been deleted on the original indexer, then cold shards, then by oldest date. Once the storage size is below the limit, deletion will stop.

### **Replication-Secret-Override**
Default Value:
Expand Down
2 changes: 1 addition & 1 deletion configuration/replication.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The replication system is logically separated into "Clients" and "Peers", with e

Replication connections are encrypted by default and require that indexers have functioning X509 certificates. If the certificates are not signed by a valid certificate authority (CA) then `Insecure-Skip-TLS-Verify=true` must be added to the Replication configuration section.

Replication storage nodes (nodes which receive replicated data) are allotted a specific amount of storage and will not delete data until that storage is exhausted. If a remote client node deletes data as part of normal ageout, the data shard is marked as deleted and prioritized for deletion when the replication node hits its storage limit. The replication system prioritizes deleted shards first, cold shards second, and oldest shards last. All replicated data is compressed; if a cold storage location is provided it is usually recommended that the replication storage location have the same storage capacity as the cold and hot storage combined.
Replication storage nodes (nodes which receive replicated data) are allotted a specific amount of storage and will not delete data unless the `Max-Replicated-Data-GB` parameter is set. Even with `Max-Replicated-Data-GB` set, the replication system will not delete replicated shards until the storage limit has been reached. If a remote client node deletes data as part of normal ageout, the data shard is marked as deleted and prioritized for deletion when the replication node hits its storage limit. The replication system prioritizes deleted shards first, cold shards second, and oldest shards last. All replicated data is compressed; if a cold storage location is provided it is usually recommended that the replication storage location have the same storage capacity as the cold and hot storage combined.

```{note}
By default, the replication engine uses port 9406.
Expand Down
4 changes: 4 additions & 0 deletions gui/queries/queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,10 @@ The queries stored in the query library are also available through the right-han
(timeframe_selector)=
## Selecting a Timeframe

```{note}
Timeframes are always aligned to one second boundaries. Sub-second timeframes will be automatically rounded down to the second.
```

By default, queries run over the last hour of data. This is easily changed by clicking on the calendar icon or timeframe above the query and selecting a timeframe from the dropdown:

![](timeframe-icon.png)
Expand Down
1 change: 1 addition & 0 deletions ingesters/ingesters.md
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,7 @@ Log-Source-Override=DEAD:BEEF::FEED:FEBE
Log-Source-Override=::1
```

(attach-target)=
### Attach

All ingesters support the `Attach` global configuration stanza, which allows [intrinsic enumerated values](intrinsic_enumerated_values) to be attached to entries during ingest. Intrinsic enumerated values can later be accessed with the [intrinsic](/search/intrinsic/intrinsic) search module.
Expand Down
2 changes: 1 addition & 1 deletion ingesters/win_file_follow.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Download the Gravwell Windows File Follower installer:

| Ingester Name | Installer | More Info |
| :------------ | :----------- | :-------- |
| Windows File Follower | <a data-bs-custom-class="hash-popover" href="https://update.gravwell.io/archive/5.6.7/installers/gravwell_file_follow_5.6.7.1.msi">Download <i class="fa-solid fa-download"></i></a>&nbsp;&nbsp;&nbsp;<a data-bs-custom-class="hash-popover" href="javascript:void(0);" data-bs-toggle="popover" data-bs-placement="bottom" data-bs-html="true" data-bs-content='<code class="docutils literal notranslate"><span class="pre">ef27478ee05cd92c682386d35cc6d471c662114d2635893904a3fdf7df0efb1c</span></code>'>(SHA256)</a> | [Documentation](/ingesters/win_file_follow) |
| Windows File Follower | <a data-bs-custom-class="hash-popover" href="https://update.gravwell.io/archive/5.6.8/installers/gravwell_file_follow_5.6.8.1.msi">Download <i class="fa-solid fa-download"></i></a>&nbsp;&nbsp;&nbsp;<a data-bs-custom-class="hash-popover" href="javascript:void(0);" data-bs-toggle="popover" data-bs-placement="bottom" data-bs-html="true" data-bs-content='<code class="docutils literal notranslate"><span class="pre">392c1400f0d4fae3410fea6fcc510c9fe641b73445d19bb4b0726faa523ee7b9</span></code>'>(SHA256)</a> | [Documentation](/ingesters/win_file_follow) |

The Gravwell Windows file follower is installed using a signed MSI package. Gravwell signs both the Windows executable and MSI installer with our private key pairs, but depending on download volumes, you may see a warning about the MSI being untrusted. This is due to the way Microsoft "weighs" files. Basically, as they see more people download and install a given package, it becomes more trustworthy. Don't worry though, we have a well audited build pipeline and we sign every package.

Expand Down
2 changes: 1 addition & 1 deletion ingesters/winevent.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ Download the Gravwell Windows Events installer:

| Ingester Name | Installer | More Info |
| :------------ | :----------- | :-------- |
| Windows Events | <a data-bs-custom-class="hash-popover" href="https://update.gravwell.io/archive/5.6.7/installers/gravwell_win_events_5.6.7.1.msi">Download <i class="fa-solid fa-download"></i></a>&nbsp;&nbsp;&nbsp;<a data-bs-custom-class="hash-popover" href="javascript:void(0);" data-bs-toggle="popover" data-bs-placement="bottom" data-bs-html="true" data-bs-content='<code class="docutils literal notranslate"><span class="pre">14242932f36dbc4a726db4156b5ebed2cdaf5e01c5e49239e79d3344bd4b4a73</span></code>'>(SHA256)</a> | [Documentation](/ingesters/winevent) |
| Windows Events | <a data-bs-custom-class="hash-popover" href="https://update.gravwell.io/archive/5.6.8/installers/gravwell_win_events_5.6.8.1.msi">Download <i class="fa-solid fa-download"></i></a>&nbsp;&nbsp;&nbsp;<a data-bs-custom-class="hash-popover" href="javascript:void(0);" data-bs-toggle="popover" data-bs-placement="bottom" data-bs-html="true" data-bs-content='<code class="docutils literal notranslate"><span class="pre">2b664f995fb404929f01e1e5b1b525b3f0d0a05a0e51c339aba7f90d9968a60e</span></code>'>(SHA256)</a> | [Documentation](/ingesters/winevent) |

Run the .msi installation wizard to install the Gravwell events service. On first installation the installation wizard will prompt to configure the indexer endpoint and ingest secret. Subsequent installations and/or upgrades will identify a resident configuration file and will not prompt.

Expand Down
Loading