Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parse XML docs #6822

Merged
merged 7 commits into from
Apr 2, 2024
Merged

Add parse XML docs #6822

merged 7 commits into from
Apr 2, 2024

Conversation

lizsnyder
Copy link
Member

@lizsnyder lizsnyder commented Apr 1, 2024

Description

Adds docs for parse_xml processor:

Issues Resolved

N/A
Closes #6530

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Liz Snyder <31932630+lizsnyder@users.noreply.github.com>
Signed-off-by: Liz Snyder <31932630+lizsnyder@users.noreply.github.com>
Copy link
Collaborator

@kolchfa-aws kolchfa-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Left a couple of comments.

lizsnyder and others added 4 commits April 1, 2024 13:39
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Liz Snyder <31932630+lizsnyder@users.noreply.github.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Liz Snyder <31932630+lizsnyder@users.noreply.github.com>
Signed-off-by: Liz Snyder <31932630+lizsnyder@users.noreply.github.com>
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lizsnyder @kolchfa-aws Just a few comments/changes. Thanks!


| Option | Required | Type | Description |
| :--- | :--- | :--- | :--- |
| `source` | No | String | The field in the `event` object that is parsed. Default value is `message`. |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As written, it's not clear whether the field or the object is parsed.

| `source` | No | String | The field in the `event` object that is parsed. Default value is `message`. |
| `destination` | No | String | The destination field of the parsed XML. Defaults to the root of the `event`. Cannot be `""`, `/`, or any white-space-only string because these are not valid `event` fields. |
| `pointer` | No | String | A JSON pointer to the field to be parsed. The value is null by default, meaning that the entire `source` is parsed. The `pointer` can access JSON array indexes as well. If the JSON pointer is invalid, then the entire `source` data is parsed into the outgoing `event` object. If the key that is pointed to already exists in the `event` object and the `destination` is the root, then the pointer uses the entire path of the key. |
| `parse_when` | No | String | Specifies under what conditions the processor should perform parsing. Default is no condition. Accepts a Data Prepper expression string following the [Data Prepper Expression Syntax]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/expression-syntax/). |
Copy link
Collaborator

@natebower natebower Apr 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `parse_when` | No | String | Specifies under what conditions the processor should perform parsing. Default is no condition. Accepts a Data Prepper expression string following the [Data Prepper Expression Syntax]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/expression-syntax/). |
| `parse_when` | No | String | Specifies under what conditions the processor should perform parsing using the [Data Prepper expression syntax]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/expression-syntax/). Default is no condition. |

| `source` | No | String | The field in the `event` object that is parsed. Default value is `message`. |
| `destination` | No | String | The destination field of the parsed XML. Defaults to the root of the `event`. Cannot be `""`, `/`, or any white-space-only string because these are not valid `event` fields. |
| `pointer` | No | String | A JSON pointer to the field to be parsed. The value is null by default, meaning that the entire `source` is parsed. The `pointer` can access JSON array indexes as well. If the JSON pointer is invalid, then the entire `source` data is parsed into the outgoing `event` object. If the key that is pointed to already exists in the `event` object and the `destination` is the root, then the pointer uses the entire path of the key. |
| `parse_when` | No | String | Specifies under what conditions the processor should perform parsing. Default is no condition. Accepts a Data Prepper expression string following the [Data Prepper Expression Syntax]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/expression-syntax/). |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Following" can be read as either "after" or "in accordance with" here.

@Naarcha-AWS Naarcha-AWS added 5 - Editorial review PR: Editorial review in progress backport 2.12 PR: Backport label for 2.12 data-prepper 2.7 labels Apr 2, 2024
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
@Naarcha-AWS Naarcha-AWS merged commit 7bbea74 into opensearch-project:main Apr 2, 2024
3 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Apr 2, 2024
* Add parse XML docs

Signed-off-by: Liz Snyder <31932630+lizsnyder@users.noreply.github.com>

* Add parse_when

Signed-off-by: Liz Snyder <31932630+lizsnyder@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/parse-xml.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Liz Snyder <31932630+lizsnyder@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/parse-xml.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Liz Snyder <31932630+lizsnyder@users.noreply.github.com>

* Tweaks

Signed-off-by: Liz Snyder <31932630+lizsnyder@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Liz Snyder <31932630+lizsnyder@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
(cherry picked from commit 7bbea74)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Naarcha-AWS added a commit that referenced this pull request Apr 2, 2024
* Add parse XML docs



* Add parse_when



* Update _data-prepper/pipelines/configuration/processors/parse-xml.md




* Update _data-prepper/pipelines/configuration/processors/parse-xml.md




* Tweaks



* Apply suggestions from code review



---------





(cherry picked from commit 7bbea74)

Signed-off-by: Liz Snyder <31932630+lizsnyder@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Editorial review PR: Editorial review in progress backport 2.12 PR: Backport label for 2.12 data-prepper 2.7
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DOC] New parse_xml processor in Data Prepper
5 participants