Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[APM] Renaming helper functions about transaction aggegations #143299

Merged
merged 7 commits into from
Oct 14, 2022
105 changes: 79 additions & 26 deletions x-pack/plugins/apm/dev_docs/apm_queries.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,20 @@
### Table of Contents
- [Transactions](#transactions)
- [System metrics](#system-metrics)
- [Transaction breakdown metrics](#transaction-breakdown-metrics)
- [Span breakdown metrics](#span-breakdown-metrics)
- [Service destination metrics](#service-destination-metrics)
- [Common filters](#common-filters)

- [Transactions](#transactions)
- [System metrics](#system-metrics)
- [Transaction breakdown metrics](#transaction-breakdown-metrics)
- [Span breakdown metrics](#span-breakdown-metrics)
- [Service destination metrics](#service-destination-metrics)
- [Common filters](#common-filters)

---

### Data model

Elastic APM agents capture different types of information from within their instrumented applications. These are known as events, and can be spans, transactions, errors, or metrics. You can find more information [here](https://www.elastic.co/guide/en/apm/get-started/current/apm-data-model.html).

### Running examples

You can run the example queries on the [edge cluster](https://edge-oblt.elastic.dev/) or any another cluster that contains APM data.

# Transactions
Expand All @@ -34,7 +37,8 @@ A single transaction event where `transaction.duration.us` is the latency.
or

#### Aggregated (metric) document
A pre-aggregated document where `_doc_count` is the number of transaction events, and `transaction.duration.histogram` is the latency distribution.

A pre-aggregated document where `_doc_count` is the number of transaction events, and `transaction.duration.histogram` is the latency distribution.

```json
{
Expand All @@ -52,7 +56,7 @@ A pre-aggregated document where `_doc_count` is the number of transaction events

You can find all the APM transaction fields [here](https://www.elastic.co/guide/en/apm/server/current/exported-fields-apm-transaction.html).

The decision to use aggregated transactions or not is determined in [`getSearchAggregatedTransactions`](https://github.com/elastic/kibana/blob/a2ac439f56313b7a3fc4708f54a4deebf2615136/x-pack/plugins/apm/server/lib/helpers/aggregated_transactions/index.ts#L53-L79) and then used to specify [the transaction index](https://github.com/elastic/kibana/blob/a2ac439f56313b7a3fc4708f54a4deebf2615136/x-pack/plugins/apm/server/lib/suggestions/get_suggestions.ts#L30-L32) and [the latency field](https://github.com/elastic/kibana/blob/a2ac439f56313b7a3fc4708f54a4deebf2615136/x-pack/plugins/apm/server/lib/alerts/chart_preview/get_transaction_duration.ts#L62-L65)
The decision to use aggregated transactions or not is determined in [`getSearchTransactionsEvents`](https://github.com/elastic/kibana/blob/a2ac439f56313b7a3fc4708f54a4deebf2615136/x-pack/plugins/apm/server/lib/helpers/aggregated_transactions/index.ts#L53-L79) and then used to specify [the transaction index](https://github.com/elastic/kibana/blob/a2ac439f56313b7a3fc4708f54a4deebf2615136/x-pack/plugins/apm/server/lib/suggestions/get_suggestions.ts#L30-L32) and [the latency field](https://github.com/elastic/kibana/blob/a2ac439f56313b7a3fc4708f54a4deebf2615136/x-pack/plugins/apm/server/lib/alerts/chart_preview/get_transaction_duration.ts#L62-L65)

### Latency

Expand Down Expand Up @@ -134,7 +138,6 @@ GET apm-*-transaction-*,traces-apm*/_search?terminate_after=1000
}
```


#### Metric-based throughput

```json
Expand Down Expand Up @@ -174,23 +177,23 @@ Noteworthy fields: `event.outcome`

#### Transaction-based failed transaction rate

```json
```json
GET apm-*-transaction-*,traces-apm*/_search?terminate_after=1000
{
"size": 0,
"query": {
"bool": {
"filter": [{ "terms": { "processor.event": ["transaction"] } }]
}
},
"aggs": {
"outcomes": {
"terms": {
"field": "event.outcome",
"include": ["failure", "success"]
}
}
}
"size": 0,
"query": {
"bool": {
"filter": [{ "terms": { "processor.event": ["transaction"] } }]
}
},
"aggs": {
"outcomes": {
"terms": {
"field": "event.outcome",
"include": ["failure", "success"]
}
}
}
}
```

Expand Down Expand Up @@ -219,6 +222,57 @@ GET apm-*-metric-*,metrics-apm*/_search?terminate_after=1000
}
```

# Transactions in service inventory page

A new aggregated metric document introduced and holds latency and throughput metrics pivoted by `service.name+service.environment+transaction.type`
kpatticha marked this conversation as resolved.
Show resolved Hide resolved

The decision to use service metrics aggregation or not is determined in [getServiceInventorySearchSource](https://github.com/elastic/kibana/blob/5d585ea375be551a169a0bea49b011819b9ac669/x-pack/plugins/apm/server/lib/helpers/get_service_inventory_search_source.ts#L12) and [getSearchServiceMetrics](https://github.com/elastic/kibana/blob/5d585ea375be551a169a0bea49b011819b9ac669/x-pack/plugins/apm/server/lib/helpers/service_metrics/index.ts#L38)

A pre-aggregated document where `_doc_count` is the number of transaction events

```
{
"_doc_count": 2,
"@timestamp": "2021-09-01T10:00:00.000Z",
"processor.event": "metric",
"metricset.name": "service",
"service": {
"environment": "production",
"name": "web-go"
},
"transaction": {
"duration.summary": {
"sum": 376492831,
"value_count": 775
Copy link
Member

@sorenlouv sorenlouv Oct 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should value_count and _doc_count be the same?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

YES! my fault copy-paste issue, should update it

ref: https://github.com/elastic/apm-server/pull/9143/files

},
"success_count": 476,
"failure_count": 151,
"type": "request"
}
}
```

- `_doc_count` is the number of bucket counts
- `transaction.duration.summary` is an [aggregate_metric_double](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/aggregate-metric-double.html) field and holds an aggregated transaction duration summary, for service metrics
- `failure_count` holds an aggregated count of transactions with the outcome "failure"
- `success_count` holds an aggregated count of transactions with the outcome "success"

### Latency

```
{
"size": 0,
"query": {
"bool": {
"filter": [{ "term": { "metricset.name": "service" } }]
}
},
"aggs": {
"latency": { "avg": { "field": "transaction.duration.summary" }}
}
}
```

# System metrics

System metrics are captured periodically (every 60 seconds by default). You can find all the System Metrics fields [here](https://www.elastic.co/guide/en/apm/server/current/exported-fields-system.html).
Expand Down Expand Up @@ -317,7 +371,7 @@ The above example is overly simplified. In reality [we do a bit more](https://gi

# Span breakdown metrics

A pre-aggregations of span documents where `span.self_time.count` is the number of original spans. Measures the "self-time" for a span type, and optional subtype, within a transaction group.
A pre-aggregations of span documents where `span.self_time.count` is the number of original spans. Measures the "self-time" for a span type, and optional subtype, within a transaction group.

Span breakdown metrics are used to power the "Time spent by span type" graph. Agents collect summarized metrics about the timings of spans, broken down by `span.type`.

Expand Down Expand Up @@ -436,7 +490,6 @@ GET apm-*-metric-*,metrics-apm*/_search?terminate_after=1000

Captures the number of requests made from a service to an (external) endpoint


#### Query

```json
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
* 2.0.
*/
import { APMEventClient } from './create_es_client/create_apm_event_client';
import { getSearchAggregatedTransactions } from './transactions';
import { getSearchAggregatedServiceMetrics } from './service_metrics';
import { getSearchTransactionsEvents } from './transactions';
import { getSearchServiceMetrics } from './service_metrics';
import { APMConfig } from '../..';

export async function getServiceInventorySearchSource({
Expand Down Expand Up @@ -35,8 +35,8 @@ export async function getServiceInventorySearchSource({
};
const [searchAggregatedTransactions, searchAggregatedServiceMetrics] =
await Promise.all([
getSearchAggregatedTransactions({ ...commonProps, config }),
getSearchAggregatedServiceMetrics({
getSearchTransactionsEvents({ ...commonProps, config }),
getSearchServiceMetrics({
...commonProps,
serviceMetricsEnabled,
}),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import { ProcessorEvent } from '@kbn/observability-plugin/common';
import { METRICSET_NAME } from '../../../../common/elasticsearch_fieldnames';
import { APMEventClient } from '../create_es_client/create_apm_event_client';

export async function getSearchAggregatedServiceMetrics({
export async function getSearchServiceMetrics({
serviceMetricsEnabled,
start,
end,
Expand All @@ -24,7 +24,7 @@ export async function getSearchAggregatedServiceMetrics({
kuery: string;
}): Promise<boolean> {
if (serviceMetricsEnabled) {
return getHasAggregatedServicesMetrics({
return getHasServicesMetrics({
start,
end,
apmEventClient,
Expand All @@ -35,7 +35,7 @@ export async function getSearchAggregatedServiceMetrics({
return false;
}

export async function getHasAggregatedServicesMetrics({
export async function getHasServicesMetrics({
start,
end,
apmEventClient,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

import { kqlQuery, rangeQuery } from '@kbn/observability-plugin/server';
import { ProcessorEvent } from '@kbn/observability-plugin/common';
import { getSearchAggregatedTransactions } from '.';
import { getSearchTransactionsEvents } from '.';
import { Setup } from '../setup_request';
import { APMEventClient } from '../create_es_client/create_apm_event_client';
import { SearchAggregatedTransactionSetting } from '../../../../common/aggregated_transactions';
Expand All @@ -23,7 +23,7 @@ export async function getIsUsingTransactionEvents({
start?: number;
end?: number;
}): Promise<boolean> {
const searchesAggregatedTransactions = await getSearchAggregatedTransactions({
const searchesAggregatedTransactions = await getSearchTransactionsEvents({
config,
start,
end,
Expand Down
8 changes: 4 additions & 4 deletions x-pack/plugins/apm/server/lib/helpers/transactions/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import {
import { APMConfig } from '../../..';
import { APMEventClient } from '../create_es_client/create_apm_event_client';

export async function getHasAggregatedTransactions({
export async function getHasTransactionsEvents({
start,
end,
apmEventClient,
Expand Down Expand Up @@ -54,7 +54,7 @@ export async function getHasAggregatedTransactions({
return response.hits.total.value > 0;
}

export async function getSearchAggregatedTransactions({
export async function getSearchTransactionsEvents({
config,
start,
end,
Expand All @@ -70,11 +70,11 @@ export async function getSearchAggregatedTransactions({
switch (config.searchAggregatedTransactions) {
case SearchAggregatedTransactionSetting.always:
return kuery
? getHasAggregatedTransactions({ start, end, apmEventClient, kuery })
? getHasTransactionsEvents({ start, end, apmEventClient, kuery })
: true;

case SearchAggregatedTransactionSetting.auto:
return getHasAggregatedTransactions({
return getHasTransactionsEvents({
start,
end,
apmEventClient,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import {
import { environmentQuery } from '../../../../common/utils/environment_query';
import { AlertParams } from '../route';
import {
getSearchAggregatedTransactions,
getSearchTransactionsEvents,
getDocumentTypeFilterForTransactions,
getDurationFieldForTransactions,
getProcessorEventForTransactions,
Expand All @@ -38,7 +38,7 @@ export async function getTransactionDurationChartPreview({
start,
end,
} = alertParams;
const searchAggregatedTransactions = await getSearchAggregatedTransactions({
const searchAggregatedTransactions = await getSearchTransactionsEvents({
...setup,
kuery: '',
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import {
import { environmentQuery } from '../../../../common/utils/environment_query';
import { AlertParams } from '../route';
import {
getSearchAggregatedTransactions,
getSearchTransactionsEvents,
getDocumentTypeFilterForTransactions,
getProcessorEventForTransactions,
} from '../../../lib/helpers/transactions';
Expand All @@ -34,7 +34,7 @@ export async function getTransactionErrorRateChartPreview({
const { serviceName, environment, transactionType, interval, start, end } =
alertParams;

const searchAggregatedTransactions = await getSearchAggregatedTransactions({
const searchAggregatedTransactions = await getSearchTransactionsEvents({
...setup,
kuery: '',
start,
Expand Down
4 changes: 2 additions & 2 deletions x-pack/plugins/apm/server/routes/environments/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

import * as t from 'io-ts';
import { maxSuggestions } from '@kbn/observability-plugin/common';
import { getSearchAggregatedTransactions } from '../../lib/helpers/transactions';
import { getSearchTransactionsEvents } from '../../lib/helpers/transactions';
import { setupRequest } from '../../lib/helpers/setup_request';
import { getEnvironments } from './get_environments';
import { rangeRt } from '../default_api_types';
Expand Down Expand Up @@ -39,7 +39,7 @@ const environmentsRoute = createApmServerRoute({
const setup = await setupRequest(resources);
const { context, params } = resources;
const { serviceName, start, end } = params.query;
const searchAggregatedTransactions = await getSearchAggregatedTransactions({
const searchAggregatedTransactions = await getSearchTransactionsEvents({
apmEventClient: setup.apmEventClient,
config: setup.config,
start,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import { termQuery } from '@kbn/observability-plugin/server';
import { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
import { getOverallLatencyDistribution } from './get_overall_latency_distribution';
import { setupRequest } from '../../lib/helpers/setup_request';
import { getSearchAggregatedTransactions } from '../../lib/helpers/transactions';
import { getSearchTransactionsEvents } from '../../lib/helpers/transactions';
import { createApmServerRoute } from '../apm_routes/create_apm_server_route';
import { environmentRt, kueryRt, rangeRt } from '../default_api_types';
import {
Expand Down Expand Up @@ -74,7 +74,7 @@ const latencyOverallTransactionDistributionRoute = createApmServerRoute({
// only the transaction latency distribution chart can use metrics data
const searchAggregatedTransactions =
chartType === LatencyDistributionChartType.transactionLatency
? await getSearchAggregatedTransactions({
? await getSearchTransactionsEvents({
...setup,
kuery,
start,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import { getMemoryChartData } from '../shared/memory';
import { getComputeUsage } from './compute_usage';
import { getActiveInstances } from './active_instances';
import { getColdStartCount } from './cold_start_count';
import { getSearchAggregatedTransactions } from '../../../../lib/helpers/transactions';
import { getSearchTransactionsEvents } from '../../../../lib/helpers/transactions';

export function getServerlessAgentMetricCharts({
environment,
Expand All @@ -31,7 +31,7 @@ export function getServerlessAgentMetricCharts({
end: number;
}) {
return withApmSpan('get_serverless_agent_metric_charts', async () => {
const searchAggregatedTransactions = await getSearchAggregatedTransactions({
const searchAggregatedTransactions = await getSearchTransactionsEvents({
...setup,
kuery,
start,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import { getServiceCount } from './get_service_count';
import { getTransactionsPerMinute } from './get_transactions_per_minute';
import { getHasData } from './has_data';
import { rangeRt } from '../default_api_types';
import { getSearchAggregatedTransactions } from '../../lib/helpers/transactions';
import { getSearchTransactionsEvents } from '../../lib/helpers/transactions';
import { withApmSpan } from '../../utils/with_apm_span';
import { createApmServerRoute } from '../apm_routes/create_apm_server_route';

Expand Down Expand Up @@ -50,7 +50,7 @@ const observabilityOverviewRoute = createApmServerRoute({
const setup = await setupRequest(resources);
const { bucketSize, intervalString, start, end } = resources.params.query;

const searchAggregatedTransactions = await getSearchAggregatedTransactions({
const searchAggregatedTransactions = await getSearchTransactionsEvents({
apmEventClient: setup.apmEventClient,
config: setup.config,
start,
Expand Down
Loading