Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for LinkedHashMap in record value #218

Merged
merged 2 commits into from
May 27, 2024

Conversation

AbdelkaderR
Copy link
Contributor

@AbdelkaderR AbdelkaderR commented Dec 29, 2023

Description

This pull request addresses an issue where the HTTP Connector for Apache Kafka does not support LinkedHashMap as the record value, leading to errors when used in conjunction with Single Message Transforms (SMTs) that return LinkedHashMap. The specific error message is: "Record value must be String, Schema Struct, or HashMap, but LinkedHashMap is given."
This issue has specifically been encountered with certain SMTs from Confluent, such as io.confluent.connect.transforms.Filter, which can utilize LinkedHashMap in ways that the HTTP Connector previously could not handle.

Changes Made

  • Modified the relevant code to support LinkedHashMap as a valid record value.

Testing

  • Added test cases to ensure that the HTTP Connector functions correctly with record values of type LinkedHashMap.
  • Ran existing tests to ensure backward compatibility.

How to Verify

  1. Build and test the modified connector.
  2. Ensure that the connector now accepts and handles LinkedHashMap as the record value without generating errors.

Checklist

  • [ x ] Code follows the project's coding conventions.
  • Tests have been added to cover the changes.
  • Documentation has been updated to reflect the changes.
  • [ x ] All existing tests pass.

@AbdelkaderR AbdelkaderR requested review from a team as code owners December 29, 2023 17:08
@jeqo jeqo self-assigned this Jan 22, 2024
@jeqo
Copy link
Contributor

jeqo commented Jan 23, 2024

@AbdelkaderR thanks for opening this PR!

Overall, if there are some SMTs using this type, I think it makes sense to add this change.
Could you share a bit more context on what SMT uses LinkedHashMap to keep as context?

Also, would you mind adding some tests to validate this change? RecordValueConverterTest should be a good place to start.

@AbdelkaderR
Copy link
Contributor Author

Hello @jeqo,

Thank you for reviewing the PR.

LinkedHashMap are indeed utilized by several Single Message Transforms (SMTs) provided by Confluent. For example in io.confluent.connect.transforms.Filter$Value among others within the Confluent library...
As requested, I've added the test case to RecordValueConverterTest to ensure that LinkedHashMap handling is properly verified.

@AbdelkaderR
Copy link
Contributor Author

Hello @jeqo,

I wanted to follow up on the pull request. I have made the changes you requested regarding the LinkedHashMap usage and added the necessary tests.

Could you please take a look when you get a chance? Your feedback would be greatly appreciated to move this forward.

Thank you for your time and assistance.

@eliax1996
Copy link
Contributor

Hi @AbdelkaderR!
Thanks again for the contribution, the changes looks fine to me.

I think the description has been truncated:

Certain SMTs from Confluent, such as io.confluent.connect.transforms.Filter$Value.

Can you update it? After this I'm open to merge it

@AbdelkaderR
Copy link
Contributor Author

Hi @eliax1996 ,

Thank you for your feedback. I have updated the description as you requested.
Looking forward to your confirmation for merging.

Best regards

@eliax1996 eliax1996 merged commit a9a9a6e into Aiven-Open:main May 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants