Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ingest: move Dot Expander Processor doc to correct alpha sorted position #31743

Merged
merged 2 commits into from
Aug 3, 2018
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
238 changes: 119 additions & 119 deletions docs/reference/ingest/ingest-node.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -1022,6 +1022,125 @@ understands this to mean `2016-04-01` as is explained in the <<date-math-index-n
| `index_name_format` | no | yyyy-MM-dd | The format to be used when printing the parsed date into the index name. An valid Joda pattern is expected here.
|======

[[dot-expand-processor]]
=== Dot Expander Processor

Expands a field with dots into an object field. This processor allows fields
with dots in the name to be accessible by other processors in the pipeline.
Otherwise these <<accessing-data-in-pipelines,fields>> can't be accessed by any processor.

[[dot-expender-options]]
.Dot Expand Options
[options="header"]
|======
| Name | Required | Default | Description
| `field` | yes | - | The field to expand into an object field
| `path` | no | - | The field that contains the field to expand. Only required if the field to expand is part another object field, because the `field` option can only understand leaf fields.
|======

[source,js]
--------------------------------------------------
{
"dot_expander": {
"field": "foo.bar"
}
}
--------------------------------------------------
// NOTCONSOLE

For example the dot expand processor would turn this document:

[source,js]
--------------------------------------------------
{
"foo.bar" : "value"
}
--------------------------------------------------
// NOTCONSOLE

into:

[source,js]
--------------------------------------------------
{
"foo" : {
"bar" : "value"
}
}
--------------------------------------------------
// NOTCONSOLE

If there is already a `bar` field nested under `foo` then
this processor merges the `foo.bar` field into it. If the field is
a scalar value then it will turn that field into an array field.

For example, the following document:

[source,js]
--------------------------------------------------
{
"foo.bar" : "value2",
"foo" : {
"bar" : "value1"
}
}
--------------------------------------------------
// NOTCONSOLE

is transformed by the `dot_expander` processor into:

[source,js]
--------------------------------------------------
{
"foo" : {
"bar" : ["value1", "value2"]
}
}
--------------------------------------------------
// NOTCONSOLE

If any field outside of the leaf field conflicts with a pre-existing field of the same name,
then that field needs to be renamed first.

Consider the following document:

[source,js]
--------------------------------------------------
{
"foo": "value1",
"foo.bar": "value2"
}
--------------------------------------------------
// NOTCONSOLE

Then the `foo` needs to be renamed first before the `dot_expander`
processor is applied. So in order for the `foo.bar` field to properly
be expanded into the `bar` field under the `foo` field the following
pipeline should be used:

[source,js]
--------------------------------------------------
{
"processors" : [
{
"rename" : {
"field" : "foo",
"target_field" : "foo.bar""
}
},
{
"dot_expander": {
"field": "foo.bar"
}
}
]
}
--------------------------------------------------
// NOTCONSOLE

The reason for this is that Ingest doesn't know how to automatically cast
a scalar field to an object field.

[[fail-processor]]
=== Fail Processor
Raises an exception. This is useful for when
Expand Down Expand Up @@ -2026,125 +2145,6 @@ Converts a string to its uppercase equivalent.
--------------------------------------------------
// NOTCONSOLE

[[dot-expand-processor]]
=== Dot Expander Processor

Expands a field with dots into an object field. This processor allows fields
with dots in the name to be accessible by other processors in the pipeline.
Otherwise these <<accessing-data-in-pipelines,fields>> can't be accessed by any processor.

[[dot-expender-options]]
.Dot Expand Options
[options="header"]
|======
| Name | Required | Default | Description
| `field` | yes | - | The field to expand into an object field
| `path` | no | - | The field that contains the field to expand. Only required if the field to expand is part another object field, because the `field` option can only understand leaf fields.
|======

[source,js]
--------------------------------------------------
{
"dot_expander": {
"field": "foo.bar"
}
}
--------------------------------------------------
// NOTCONSOLE

For example the dot expand processor would turn this document:

[source,js]
--------------------------------------------------
{
"foo.bar" : "value"
}
--------------------------------------------------
// NOTCONSOLE

into:

[source,js]
--------------------------------------------------
{
"foo" : {
"bar" : "value"
}
}
--------------------------------------------------
// NOTCONSOLE

If there is already a `bar` field nested under `foo` then
this processor merges the `foo.bar` field into it. If the field is
a scalar value then it will turn that field into an array field.

For example, the following document:

[source,js]
--------------------------------------------------
{
"foo.bar" : "value2",
"foo" : {
"bar" : "value1"
}
}
--------------------------------------------------
// NOTCONSOLE

is transformed by the `dot_expander` processor into:

[source,js]
--------------------------------------------------
{
"foo" : {
"bar" : ["value1", "value2"]
}
}
--------------------------------------------------
// NOTCONSOLE

If any field outside of the leaf field conflicts with a pre-existing field of the same name,
then that field needs to be renamed first.

Consider the following document:

[source,js]
--------------------------------------------------
{
"foo": "value1",
"foo.bar": "value2"
}
--------------------------------------------------
// NOTCONSOLE

Then the `foo` needs to be renamed first before the `dot_expander`
processor is applied. So in order for the `foo.bar` field to properly
be expanded into the `bar` field under the `foo` field the following
pipeline should be used:

[source,js]
--------------------------------------------------
{
"processors" : [
{
"rename" : {
"field" : "foo",
"target_field" : "foo.bar""
}
},
{
"dot_expander": {
"field": "foo.bar"
}
}
]
}
--------------------------------------------------
// NOTCONSOLE

The reason for this is that Ingest doesn't know how to automatically cast
a scalar field to an object field.

[[urldecode-processor]]
=== URL Decode Processor
URL-decodes a string
Expand Down