Update calculated fields and add new extraction doc

config/_default/menus/main.en.yaml

@@ -5875,11 +5875,16 @@ menu:
       url: logs/explorer/calculated_fields/
       parent: log_explorer
       weight: 506
-    - name: Expression Language
+    - name: Formula Expression Language
       identifier: log_explorer_calculated_fields_expression_language
       url: logs/explorer/calculated_fields/expression_language/
       parent: log_explorer_calculated_fields
       weight: 5061
+    - name: Extraction Grok Parsing
+      identifier: log_explorer_calculated_fields_extraction
+      url: logs/explorer/calculated_fields/extraction/
+      parent: log_explorer_calculated_fields
+      weight: 5062
     - name: Analytics
       identifier: log_explorer_analytics
       url: logs/explorer/analytics/

content/en/logs/explorer/calculated_fields/_index.md

@@ -5,103 +5,118 @@ further_reading:
 - link: "/logs/explorer/calculated_fields/expression_language"
   tag: "Documentation"
   text: "Calculated Fields expression language"
+- link: "/logs/explorer/calculated_fields/extraction"
+  tag: "Documentation"
+  text: "Extraction Grok Parsing"
 - link: "/logs/explorer/"
   tag: "Documentation"
   text: "Log Explorer"
 - link: "https://www.datadoghq.com/blog/calculated-fields-log-management-datadog/"
   tag: "Blog"
-  text: "Transform and enrich your logs at query time with Calculated Fields"  
+  text: "Transform and enrich your logs at query time with Calculated Fields"
 ---
 
 
 <div class="alert alert-info">For syntax, operators, and functions, see <a href="/logs/explorer/calculated_fields/expression_language">Expression Language</a></div>
 
 ## Overview
 
-Use calculated fields to transform and enrich your log data at query time. Define [formulas](#formula) to:
-- [Manipulate text][1]
-- [Perform arithmetic][2]
-- [Evaluate conditional logic][3]
+Calculated Fields lets you transform and enrich your log data at **query time**. It is defined directly in the Log Explorer and is available only for the duration of your session. Once created, a calculated field behaves like any other [log attribute][1] and can be used for search, aggregation, visualization, or even defining additional calculated fields.
 
-Once defined, a calculated field can be used like any [log attribute][5] for search, aggregation, visualization, and even defining other calculated fields.
+There are two types of calculated fields: **Extraction** and **Formula**. Both share the following properties:
 
-**Notes**:
-- You can define up to five calculated fields at a time.
-- Calculated fields are temporary and do not persist beyond a given Log Explorer session. If a calculated field may be repeatedly useful, update your [log pipelines][6] to encode the information in your logs when they are ingested and processed.
+- They are **temporary** and do not persist beyond your Log Explorer session.
+- They are **user-scoped** and visible only to you.
+- They are ideal for **retroactive analysis**, since they can be applied to already indexed logs.
+- They must be referenced with the `#` prefix when used in queries, aggregations, or other calculated fields.
+- You can define up to **five** calculated fields at a time.
 
-## Create a calculated field
+## When to use calculated fields
 
-There are two entry points for creating a calculated field in the Log Explorer: from the **Add** menu or from within a specific log event or attribute.
-### Choose a starting point for your calculated field
+Calculated fields are not a replacement for log pipelines and processors, which handle ingest-time parsing, normalization, and enrichment of logs. Instead, use calculated fields in the following scenarios:
 
-#### From the Add menu
+- When you need to perform a short-term investigation or analysis that requires a field that you don't need to keep in the long-term.
+- When you need to retroactively analyze indexed logs (pipelines changes only affect logs ingested after the update).
+- When you don't have the permission or expertise to modify log pipelines quickly.
+- When you want a calculated field visible only to you, useful for quick exploration and low-risk experimentation.
 
-1. Navigate to [Log Explorer][7].
-1. Click the **Add** button next to the search bar.
-1. Select **Calculated field**.
+If you find that a calculated field is valuable in the long-term, update your [log pipelines][2] so the entireyour team benefits from automated processing.
+
+## Types of calculated fields
 
-This is a quick way to create a calculated field when you are already familiar with the structure and content of the logs you are interested in.
+### Formula
 
-#### From a specific log event or attribute
+Formula fields use the calculated fields expression language to compute new values from existing attributes. You can:
+- Manipulate text values.
+- Perform arithmetic on numeric attributes.
+- Evaluate conditional logic.
 
-1. Navigate to [Log Explorer][7].
-1. Click on a log event of interest to open the side panel.
-1. Click on a specific JSON attribute to open the context menu.
-1. Select **Create calculated from...**.
+For example:
+```
+#latency_gap = @client_latency - @server_latency
+```
 
+For a complete list of supported syntax, operators, and functions, see the [Expression Language Reference][3].
 
-{{< img src="logs/explorer/calculated_fields/create_field.png" alt="The log panel duration attribute with the option to create a calculated field for it" style="width:80%;" >}}
+### Extraction
 
-This approach allows you to adapt quickly during an investigation or explore unfamiliar logs. For example, you may want to multiply or concatenate two values and store the result in a single field to simplify a graph or answer a specific question.
+Extraction uses Grok parsing rules to capture values from raw log messages or attributes. You can use Grok rules to:
+- Capture values from raw log messages.
+- Retroactively extract attributes from already indexed logs without editing pipelines.
+- Test against sample logs
 
-### Define a calculated field
+For example, you can extract the first three words of a message into separate fields:
+```
+%{WORD:first} %{WORD:second} %{WORD:third}
+```
 
-{{< img src="logs/explorer/calculated_fields/define_a_calculated_field.png" alt="A calculated field for throughput, with a formula that concatenates firstName and lastName attributes " style="width:70%;" >}}
+Extraction rules are evaluated globally across all logs in your session. For more details and syntax example, see the [Extraction Grok Parsing reference][4].
 
-#### Name
+## Create a calculated field
+
+You can create a calculated field from two entry points in the Log Explorer: from the **Add** menu or from within a specific log event or attribute.
+
+### From the Add menu
 
-Set a descriptive name that clearly indicates the purpose of the calculated field. For example, if the goal is to combine users' first and last names into one field, you might name your calculated field `fullName`. 
+1. Navigate to the [Log Explorer][5].
+1. Click the **Add** button next to the search bar.
+1. Select **Calculated field**.
 
-To filter logs from a user named `Pinkie Smith`, include the calculated field name in your query: `#fullName:"Pinkie Smith"`. **Note:** You must use the `#` prefix to refer to calculated fields in searches, aggregation, or other calculated field definitions.
+This is useful when you are already familiar with the structure and content of the logs and want to quickly define formula or parsing rule.
 
-#### Formula
+### From a specific log event or attribute
 
-The formula (or expression) determines the result to be computed and stored as the value of the calculated field for each log event. Valid constructs include log attributes, other calculated fields, and a set of supported functions and operators. Relevant fields, functions, and operators are automatically suggested as you write or edit the formula.
+1. Navigate to the [Log Explorer][5].
+1. Click on a log event to open the side panel.
+1. Select a JSON attribute to open the context menu.
+1. Choose **Create calculated from...**.
 
-See [Calculated Fields Expression Language][4] for the available functions and operators.
+{{< img src="/logs/explorer/calculated_fields/add_calculated_field_side_panel.png" alt="Creating a calculated field from the log side panel in the Log Explorer" style="width:70%;" >}}
 
-## Use a calculated field
+This approach is useful for extractions, since it provides a concrete log sample for building a parsing rule.
 
-After successful creation of a calculated field, the Log Explorer updates to:
-- Display active calculated fields in a new row directly under the search bar.
-    - Hover over a field to view its definition, and use quick actions to edit, filter by, or group by the field.
-- Include a column for the calculated field in the **[List][8]** visualization. The title includes the # prefix.
-- Show calculated fields in a separate section inside the log side panel.
+## Using calculated fields
 
-Calculated fields function like log attributes and can be used for search, aggregation, visualization, and defining other calculated fields. Remember to use the `#` prefix when referencing calculated field names.
+After you create a calculated field, the Log Explorer updates instantly to show you the new data and give you tools to interact with it.
+- **Header row**: A new row appears under the search bar, showing all active calculated fields. Hover to view the full definition, or use quick actions to edit, filter by, or group by the field.
+- **List visualization**: In [List][6] view, a column for the calculated field is automatically added.
+- **Log side panel**: Calculated fields are grouped into a dedicated section when you inspect a log.
 
 {{< img src="logs/explorer/calculated_fields/calculated_field.png" alt="A calculated field called request_duration used to filter results in the Log Explorer" style="width:100%;" >}}
 
-### Use cases
+### Key usage rules
 
-Calculated fields are not a replacement for log pipelines and processors for ingest-time parsing, normalization, and enrichment of logs. Use calculated fields in the following scenarios:
+Calculated fields function like log attributes and can be used for search, aggregation, visualization, or defining other calculated fields. Always use the `#` prefix when referencing a calculated field.
 
-- You need to perform a one-off investigation or ad-hoc analysis which requires a field that you don't need to reuse in the long-term.
-- You need to retroactively update indexed logs to answer a certain question (pipelines changes only apply to logs ingested after a pipeline update).
-- You lack the permission (or knowledge) to modify log pipelines in a timely manner.
-  - The calculated fields you create are visible only to you, which makes them ideal for quick exploration and worry-free experimentation.
 
-If you realize that a calculated field may be valuable in the long-term, update your log pipelines so you and the rest of your team can benefit from automated processing.
 
 ## Further reading
 
 {{< partial name="whats-next/whats-next.html" >}}
 
-[1]: /logs/explorer/calculated_fields/expression_language/#string
-[2]: /logs/explorer/calculated_fields/expression_language/#arithmetic
-[3]: /logs/explorer/calculated_fields/expression_language/#logical
-[4]: /logs/explorer/calculated_fields/expression_language/
-[5]: /logs/log_configuration/attributes_naming_convention/
-[6]: /logs/log_configuration/pipelines/?tab=source
-[7]: https://app.datadoghq.com/logs
-[8]: /logs/explorer/visualize/#lists
+[1]: /logs/log_configuration/attributes_naming_convention/
+[2]: /logs/log_configuration/pipelines/?tab=source
+[3]: /logs/explorer/calculated_fields/expression_language/
+[4]: /logs/explorer/calculated_fields/extraction
+[5]: https://app.datadoghq.com/logs
+[6]: /logs/explorer/visualize/#lists

content/en/logs/explorer/calculated_fields/expression_language.md

@@ -1,12 +1,16 @@
 ---
-title: Calculated Fields Expression Language
+title: Formula Expression Language
 disable_toc: false
 further_reading:
 - link: "/logs/explorer/calculated_fields/"
   tag: "Documentation"
   text: "Calculated Fields"
 ---
 
+## Overview
+
+The formula (or expression) defines the value of the value of the calculated field for each log event. You can reference log attributes, other calculated fields, and supported functions and operators. As you write or edit a formula, the editor automatically suggestion relevant fields, functions, and operators.
+
 ## Basic syntax and language constructs
 
 | Construct                                                                 | Syntax and Notation                                                                                                                  |

content/en/logs/explorer/calculated_fields/extraction.md

@@ -0,0 +1,139 @@
+---
+title: Extraction Grok Parsing
+further_reading:
+- link: "/logs/explorer/calculated_fields/"
+  tag: "Documentation"
+  text: "Learn more about Calculated Fields"
+---
+
+## Overview
+
+Calculated Fields Extraction lets you apply Grok parsing rules at query time in the Log Explorer. This makes it possible to extract values from raw log messages or attributes without modifying pipelines or re-ingesting data.
+
+## Syntax
+
+Extraction fields use Grok patterns to identify and capture values from a log attribute. A Grok pattern is composed of one or more tokens in the form:
+```
+%{PATTERN_NAME:field_name}
+```
+- `PATTERN_NAME`: A Grok matcher.
+- `field_name`: The name of the extracted calculated field.
+
+You can chain multiple patterns together to parse complex log messages.
+
+## Supported matchers and filters at query time
+
+<div class="alert alert-warning">Grok parsing features available at <em>query-time</em> (in the <a href="/logs/explorer/calculated_fields/">Log Explorer</a>) support a limited subset of matchers (<strong>data</strong>, <strong>integer</strong>, <strong>notSpace</strong>, <strong>number</strong>, and <strong>word</strong>) and filters (<strong>number</strong> and <strong>integer</strong>) For long-term parsing needs, define a log pipeline..</div>
+
+Query-time Grok parsing in the Log Explorer supports a limited subset of matchers and filters. Each matcher or filter is used in a Grok pattern with the format:
+
+```
+%{MATCHER:field_name}
+```
+
+### Matchers
+
+<table>
+  <thead>
+    <tr>
+      <th style="width: 150px;">Matcher</th>
+      <th style="width: 40%;">Example Grok Pattern</th>
+      <th style="width: 35%;">Example Result</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="width: 150px;">
+        <code>DATA</code><br>
+        <em>Any sequence of characters (non-greedy)</em>
+      </td>
+      <td><code>status=%{DATA:status}</code></td>
+      <td><code>#status = 200 OK</code></td>
+    </tr>
+    <tr>
+      <td style="width: 150px;">
+        <code>WORD</code><br>
+        <em>Alphanumeric characters</em>
+      </td>
+      <td><code>country=%{WORD:country}</code></td>
+      <td><code>#country = Brazil</code></td>
+    </tr>
+    <tr>
+      <td style="width: 150px;">
+        <code>NUMBER</code><br>
+        <em>Floating-point numbers</em>
+      </td>
+      <td><code>value=%{NUMBER:float_val}</code></td>
+      <td><code>#float_val = 3.14</code></td>
+    </tr>
+    <tr>
+      <td style="width: 150px;">
+        <code>INTEGER</code><br>
+        <em>Integer values</em>
+      </td>
+      <td><code>count=%{INTEGER:count}</code></td>
+      <td><code>#count = 42</code></td>
+    </tr>
+    <tr>
+      <td style="width: 150px;">
+        <code>NOTSPACE</code><br>
+        <em>Non-whitespace characters</em>
+      </td>
+      <td><code>path=%{NOTSPACE:request_path}</code></td>
+      <td><code>#request_path = /index.html</code></td>
+    </tr>
+  </tbody>
+</table>
+
+### Filters
+Filters can be applied in combination with matchers to cast values into numeric types.
+
+<table>
+  <thead>
+    <tr>
+      <th style="width: 150px;">Filter</th>
+      <th style="width: 40%;">Example Grok Pattern</th>
+      <th style="width: 35%;">Example Result</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="width: 150px;">
+        <code>NUMBER</code><br>
+        <em>Parses numeric strings as numbers</em>
+      </td>
+      <td><code>latency=%{NUMBER:lat}</code></td>
+      <td><code>#lat = 1.23</code> (number)</td>
+    </tr>
+    <tr>
+      <td style="width: 150px;">
+        <code>INTEGER</code><br>
+        <em>Parses numeric strings as integers</em>
+      </td>
+      <td><code>users=%{INTEGER:user_count}</code></td>
+      <td><code>#user_count = 100</code> (integer)</td>
+    </tr>
+  </tbody>
+</table>
+
+### Example
+
+**Log line**:
+
+```
+country=Brazil duration=123ms path=/index.html status=200 OK
+```
+
+**Extraction grok rule**:
+```
+country=%{WORD:country} duration=%{INTEGER:duration} path=%{NOTSPACE:request_path} status=%{DATA:status}
+```
+**Resulting calculated fields**:
+- `#country = Brazil`
+- `#duration = 123`
+- `#request_path = /index.html`
+- `#status = 200 OK`
+
+## Further reading
+
+{{< partial name="whats-next/whats-next.html" >}}