[Feature Request] Convert ingest processor supports ip data type

[gaobinlong]

Is your feature request related to a problem? Please describe

The ip data type was introduced in OpenSearch 1.x, but the convert ingest processor has not support that type yet, we need to add the ip type to the supported types in the processor.
Request:

PUT _ingest/pipeline/convert
{
  "processors": [
    {
      "convert": {
        "field": "ip_field_raw",
        "type": "ip",
        "target_field": "ip"
      }
    }
  ]
}

Response:

{
  "error": {
    "root_cause": [
      {
        "type": "parse_exception",
        "reason": "[type] type [ip] not supported, cannot convert field.",
        "processor_type": "convert",
        "property_name": "type"
      }
    ],
    "type": "parse_exception",
    "reason": "[type] type [ip] not supported, cannot convert field.",
    "processor_type": "convert",
    "property_name": "type"
  },
  "status": 400
}

Describe the solution you'd like

The convert ingest processor supports the ip data type.

Related component

Indexing

Describe alternatives you've considered

No

Additional context

No response

[peternied]

[Triage - attendees 1 2 3 4 5]
@gaobinlong Thanks for filing this issue, please feel free to submit a pull request.

[gaobinlong]

By my investigation, I found that we cannot change the mapping of an index when executing pipelines, so if we support ip type for the convert processor, what we can do is to validate the specified field to make sure it can be indexed into an ip type field. One use case is like this:

PUT _ingest/pipeline/ip
{
  "processors": [
      {
        "convert": {
          "field": "raw_ip",
          "type": "ip",
          "target_field": "ip_field",
          "ignore_failure": true
        }
      }
    ]
}

, if the field raw_ip is a valid ip address, then we set the target field ip_type which has been mapped to IP data type, if the ip address is not valid, because ignore_failure is true, the target field ip_type will not be set. @peternied @reta ,do you think this is a valid use case?

[reta]

@peternied @reta ,do you think this is a valid use case?

Thanks @gaobinlong , I don't think the convert processor should have any support for ignore_failure: if it cannot convert, it should use the raw value (I believe AUTO mode does that). The mapping field may decide to set ignore_failure or not, but to me too many levels of ignore_failure is confusing at least.

[gaobinlong]

@peternied @reta ,do you think this is a valid use case?

Thanks @gaobinlong , I don't think the convert processor should have any support for ignore_failure: if it cannot convert, it should use the raw value (I believe AUTO mode does that). The mapping field may decide to set ignore_failure or not, but to me too many levels of ignore_failure is confusing at least.

ignore_failure is a common parameter in all processors, it's not easy to only remove the parameter from convert processor, and it will be a breaking change if we remove it. The current implementation makes users can control the behavior they'd like to see, throw error or exit quietly when converting fails, but if ignore_failure is removed, users have no other options.

[reta]

ignore_failure is a common parameter in all processors, it's not easy to only remove the parameter from convert processor, and it will be a breaking change if we remove it.

Ah I see, I should have checked that for sure, thanks for explaining. So back to your question: I think this is a valid case than (and it seems like also followed by Elasticsearch).

[msfroh]

There was a little bit of discussion about convert and ip field type on #6646

[gaobinlong]

There was a little bit of discussion about convert and ip field type on #6646

The user's case is same to the above case, if convert processor supports ip type, it can do some validation for the specified field and then populate the target field if the value is a valid ipv4/ipv6 address, if not throw an error or exit quietly(with ignore_failure setting to true). string type in convert processor cannot do any ip address validation, so it makes sense that the convert processor supports ip type.