Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use client_geoip.location for the GeoIP location of the client_ip #2795

Merged
merged 1 commit into from
Oct 24, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@ https://github.com/elastic/beats/compare/v5.0.0-rc1...master[Check the HEAD diff
- Add experimental docker module. Provided by Ingensi and @douaejeouit based on dockbeat.

*Packetbeat*
- Define `client_geoip.location` as geo_point in the mappings to be used by the GeoIP processor in the Ingest Node pipeline.
{pull}2795[2795]

*Topbeat*

Expand Down
18 changes: 17 additions & 1 deletion packetbeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -1121,7 +1121,23 @@ type: geo_point

example: 40.715, -74.011

The GeoIP location of the `real_ip` IP address or of the `client_ip` address if the `real_ip` is disabled. The field is a string containing the latitude and longitude separated by a comma.
DEPRECATED. Please use `client_geoip` instead. The GeoIP location of the `real_ip` IP address or of the `client_ip` address if the `real_ip` is disabled. The field is a string containing the latitude and longitude separated by a comma.


[float]
== client_geoip Fields

The GeoIP information of the client.


[float]
=== client_geoip.location

type: geo_point

example: {'lat': 51, 'lon': 9}

The GeoIP location of the `client_ip` address. This field is available only if you define a https://www.elastic.co/guide/en/elasticsearch/plugins/master/using-ingest-geoip.html[GeoIP Processor] as a pipeline in the https://www.elastic.co/guide/en/elasticsearch/plugins/master/ingest-geoip.html[Ingest GeoIP processor plugin] or using Logstash.


[float]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -465,8 +465,7 @@ The default is false.
The header field to extract the real IP from. This setting is useful when
you want to capture traffic behind a reverse proxy, but you want to get the geo-location
information. If this header is present and contains a valid IP addresses, the
information is used for the `real_ip` and `client_location` indexed
fields.
information is used for the `real_ip` field.

===== max_message_size

Expand Down
21 changes: 17 additions & 4 deletions packetbeat/etc/fields.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,24 @@

- name: client_location
type: geo_point
example: "40.715, -74.011"
example: 40.715, -74.011
description: >
The GeoIP location of the `real_ip` IP address or of the
`client_ip` address if the `real_ip` is disabled. The field is a string
containing the latitude and longitude separated by a comma.
DEPRECATED. Please use `client_geoip` instead.
The GeoIP location of the `real_ip` IP address or of the `client_ip` address if the `real_ip` is
disabled. The field is a string containing the latitude and longitude separated by a comma.

- name: client_geoip
description: The GeoIP information of the client.
type: group
fields:
- name: location
type: geo_point
example: {lat: 51, lon: 9}
description: >
The GeoIP location of the `client_ip` address. This field is available
only if you define a
https://www.elastic.co/guide/en/elasticsearch/plugins/master/using-ingest-geoip.html[GeoIP Processor] as a pipeline in the
https://www.elastic.co/guide/en/elasticsearch/plugins/master/ingest-geoip.html[Ingest GeoIP processor plugin] or using Logstash.

- name: client_port
description: >
Expand Down
2 changes: 1 addition & 1 deletion packetbeat/etc/kibana/index-pattern/packetbeat.json

Large diffs are not rendered by default.

7 changes: 3 additions & 4 deletions packetbeat/etc/kibana/visualization/Client-locations.json
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
{
"visState": "{\"title\":\"New Visualization\",\"type\":\"tile_map\",\"params\":{\"mapType\":\"Scaled Circle Markers\",\"isDesaturated\":true,\"addTooltip\":true,\"heatMaxZoom\":16,\"heatMinOpacity\":0.1,\"heatRadius\":25,\"heatBlur\":15,\"heatNormalizeData\":true,\"wms\":{\"enabled\":false,\"url\":\"https://basemap.nationalmap.gov/arcgis/services/USGSTopo/MapServer/WMSServer\",\"options\":{\"version\":\"1.3.0\",\"layers\":\"0\",\"format\":\"image/png\",\"transparent\":true,\"attribution\":\"Maps provided by USGS\",\"styles\":\"\"}}},\"aggs\":[{\"id\":\"1\",\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"type\":\"geohash_grid\",\"schema\":\"segment\",\"params\":{\"field\":\"client_location\",\"autoPrecision\":true,\"precision\":2}}],\"listeners\":{}}",
"visState": "{\"aggs\":[{\"enabled\":true,\"id\":\"1\",\"params\":{},\"schema\":\"metric\",\"type\":\"count\"},{\"enabled\":true,\"id\":\"2\",\"params\":{\"autoPrecision\":true,\"field\":\"client_geoip.location\"},\"schema\":\"segment\",\"type\":\"geohash_grid\"}],\"listeners\":{},\"params\":{\"addTooltip\":true,\"heatBlur\":15,\"heatMaxZoom\":16,\"heatMinOpacity\":0.1,\"heatNormalizeData\":true,\"heatRadius\":25,\"isDesaturated\":true,\"mapCenter\":[15,5],\"mapType\":\"Scaled Circle Markers\",\"mapZoom\":2,\"wms\":{\"enabled\":false,\"options\":{\"attribution\":\"Maps provided by USGS\",\"format\":\"image/png\",\"layers\":\"0\",\"styles\":\"\",\"transparent\":true,\"version\":\"1.3.0\"},\"url\":\"https://basemap.nationalmap.gov/arcgis/services/USGSTopo/MapServer/WMSServer\"}},\"title\":\"Client locations\",\"type\":\"tile_map\"}",
"description": "",
"title": "Client locations",
"uiStateJSON": "{}",
"uiStateJSON": "{\"mapCenter\":[0,-0.17578125]}",
"version": 1,
"savedSearchId": "Packetbeat-Search",
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"filter\":[]}"
"searchSourceJSON": "{\"index\":\"packetbeat-*\",\"query\":{\"query_string\":{\"analyze_wildcard\":true,\"query\":\"*\"}},\"filter\":[]}"
}
}
7 changes: 7 additions & 0 deletions packetbeat/packetbeat.template-es2x.json
Original file line number Diff line number Diff line change
Expand Up @@ -573,6 +573,13 @@
}
}
},
"client_geoip": {
"properties": {
"location": {
"type": "geo_point"
}
}
},
"client_ip": {
"ignore_above": 1024,
"index": "not_analyzed",
Expand Down
7 changes: 7 additions & 0 deletions packetbeat/packetbeat.template.json
Original file line number Diff line number Diff line change
Expand Up @@ -499,6 +499,13 @@
}
}
},
"client_geoip": {
"properties": {
"location": {
"type": "geo_point"
}
}
},
"client_ip": {
"ignore_above": 1024,
"type": "keyword"
Expand Down